Using JMP® for Group
Segmentation and Predictive
Modeling for Indicators of
High Utilization in a Rural
North Carolina Hospital
Jason Brinkley, PhD - Senior Researcher and Biostatistician
Elizabeth Horner, PhD – Senior Researcher
September 2016
Copyright © 2016 American Institutes for Research. All rights reserved.
2016 - Sept
AMERICAN INSTITUTES FOR RESEARCH
The Current Problem
• Potentially avoidable hospitalizations for acute and
chronic conditions substantially contribute to excess
hospital expenditures.
• Unnecessary and excessive treatment inflate health care
costs leading to waste and inefficiencies that critically
affect the health care system as a whole.
• Continued efforts are needed to advance understanding of
the root causes of persistent high hospital utilization and
to develop innovative, responsive strategies to improve
care delivery processes, particularly, clinically-directed,
patient-centered policies that can have significant
hospital-level impact.
2
AMERICAN INSTITUTES FOR RESEARCH
The Next Problem
• New payment models such as bundled payments
and upside shared savings programs that
incentivize the provision of efficient, high-quality
care, prioritizing value over volume.
• This is a major shift from traditional fee-for-
service reimbursement.
• In this new reality, controlling utilization is gaining
increasing attention.
• Same problem, increased importance for hospital
administrators and decision makers.
3
AMERICAN INSTITUTES FOR RESEARCH
Using Data
• Currently, the logic has been that identifying and
predicting high utilization patients is the first step in
tackling the problem.
• Hospitals are having to become more aware of the data
they are collecting and are thinking through how to use
administrative data to tackle this problem.
• A traditional approach to explore the problem is to use
regression models on outcomes such as number of
visits or length of stay with a set of predictors that
include patient characteristics and administrative
records.
4
AMERICAN INSTITUTES FOR RESEARCH
Problem 1 - Outcomes
• Examining traditional outcomes such as number of
visits or length of stay is problematic.
• Patterns of hospital utilization are multifaceted with
causal mechanisms as diverse as patient populations.
• That is to say that what predicts a high number of visits
for mental health patients is very different than what
we might see for cancer care or for issues in aging.
• So while these metrics are good for deciding on
payment reforms or evaluating quality, they are not a
good choice for understanding utilization patterns.
5
AMERICAN INSTITUTES FOR RESEARCH
Problem 2 – The Data
• Hospitals are collecting an increasingly large set of
administrative, diagnostic, and patient data.
• Some of that data is being under-utilized while other
factors tend to ‘drive’ statistical modeling.
• The current strategy identifies largely immutable risk
factors (e.g., age, gender, and race)—an approach that
contributes to general public health and health disparities
research, but provides little guidance on policy changes for
rural or community hospitals.
• Hospital decision makers are being routinely given a set of
factors that ‘predict’ higher than expected utilization but are
not factors that they can directly influence.
6
AMERICAN INSTITUTES FOR RESEARCH
Problem 3 - Regression
• Traditional regression is a time tested method for
exploring the associations between an outcome and
multiple potential predictors simultaneously.
• We start to see issues in traditional regression when we
have a high number of predictors and we believe those
predictors may interact with one another.
• Hospital diagnostic data (i.e. ICD-9 or ICD-10 codes)
provide potentially thousands of unique predictors that
can seemingly overwhelm traditional methods and may
lead to models with poor predictive capacity or fit.
7
AMERICAN INSTITUTES FOR RESEARCH
Utility
• “All models are wrong, but some are useful”
– George Box
• The problem here, simply put, is that the
models being generated in this framework
are not useful to some.
• Not for those who set local hospital policy
and have a need to understand or explore
potential predictors for which they can
move the dial and have direct impact.
8
AMERICAN INSTITUTES FOR RESEARCH
Reframing the Question
• Instead of “What people put the
most utilization and cost burden
on the health system?”
• Ask“What conditions put the
most utilization and cost burden
on the health system?”
9
AMERICAN INSTITUTES FOR RESEARCH
Market Segmentation
• Our alternative approach draws upon the market-
segmentation literature.
• Marketing experts have long supported a technique of
first identifying one or more subgroups of specific
individuals that represent an area of interest in terms of
product utilization.
• Market segmentation provides a way for firms to more
effectively concentrate their resources and respond to
consumers’ needs. These firms are able to divide and
conquer by matching their strengths to specific groups
of customers.
10
AMERICAN INSTITUTES FOR RESEARCH
Our Approach
• We propose using a similar framework to identify a high
utilization subgroup of patients based on like utilization
behavior and then applying predictive modeling with
diagnosis codes or admin data to determine how well we
can predict whether patients would end up in that
subgroup.– Defining a Subgroup: utilizing a combination of clinical wit and data driven analytics
to explore electronic health records, claims or other administrative data to identify key
high utilization and/or high-cost patient subgroups that exhibit similar utilization
behavior;
– Predictive Modeling: creating models with diagnosis codes (e.g., The International
Classification of Diseases, 9th Revision, Clinical Modification (ICD-9-CM)) or other
data to identify clinical predictors of belonging to the high utilizer subgroups
– Data Translation: using the identified clinical predictors to develop targeted policy
interventions for patients presenting to the hospital with those diagnoses to help
prevent or reduce unnecessary hospital utilization.
11
AMERICAN INSTITUTES FOR RESEARCH
Rural Hospitals
• Rural hospitals are slowly coming around to a data
driven point of view. But the lack of resources makes
utilizing data to it’s full potential challenging.
• Therefore the rural settings are increasingly relying on
the study and knowledge of larger systems to help
direct their efforts.
• This has seen mixed results as some issues are more
‘rural’ or ‘urban’ specific while some solutions do not
scale in the rural setting well.
12
AMERICAN INSTITUTES FOR RESEARCH
Nash County, NC
• Nash County has a population of approximately 93,919 as of
2015; its estimated population density is 177.3 people per
square mile (2010).
• The Robert Wood Johnson Foundation ranks the health of
nearly all counties in the US from 1 (best) to 100 (worst),
including a 2015 rating of Nash County. Nash County has
several poor results indicating a generally unhealthy
population, including length of life (72), health behaviors (76),
and socioeconomic factors (76).
• This suggests, as Nash County representatives noted in a
posting on their website, that “not only do our citizens not
practice healthy behaviors, but that our overall environment
and infrastructure in Nash County may not be as conducive to
living healthy as it could be.”
13
AMERICAN INSTITUTES FOR RESEARCH 14
Characteristic Nash County
North
Carolina
USA
Median age (2010) 38.4 37.4 37.2
Percent female (2010) 52% 51% 51%
Percent White (2010) 56% 70% 72%
Percent Black (2010) 37% 23% 13%
Percent Hispanic, any race (2010) 6% 8% 16%
Percent with high school degree or higher (2014) 84% 85% 86%
Percent with college degree or higher (2014) 18% 27% 29%
Median household income (2014) $43,341 $46,693 $53,482
Percent households below poverty line (2014) 13% 13% 12%
Percent households w/ children and below poverty line (2014)
21% 21% 18%
AMERICAN INSTITUTES FOR RESEARCH
Nash County Hospital In-Patient High Utilization 2010-2013
(2 or more visits per year)
15
Variables Utilization
Count Percent Mean St Dev Sex
Female 2,198 56% 2.96 1.47
Male 1,731 44% 2.98 1.46 Race
Black 2,094 53% 3.09 1.63
White 1,725 44% 2.70 1.02
Other 110 3% 2.83 1.24
Age
0 to 24 194 5% 2.69 1.58
25 - 44 498 12% 2.91 1.61
45 - 64 408 10% 3.08 1.67
65 - 84 2,415 60% 2.94 1.25
85 and up 491 12% 2.91 1.21
Overall Utilization Characteristics 2.97 1.5
AMERICAN INSTITUTES FOR RESEARCH
Creating a Segment – Transition in Care
16
AMERICAN INSTITUTES FOR RESEARCH
Wrangling ICD Codes
• Clinicians from our source hospital entered International
Classification of Disease version 9 (ICD-9) codes for
these inpatient visits to indicate the health problems
experienced by patients. ICD-9 has over 14,000 codes.
• Clinical Classification Software (CCS) developed by the
Healthcare Cost and Utilization Project (HCUP) groups
ICD-9 codes into groups.
• Specifically, this sample of patients had 3,352 distinct
ICD-9 codes that were regrouped into 240 distinct
diagnosis groupings.
17
AMERICAN INSTITUTES FOR RESEARCH
Results
18
• Notice that the patients who transition have a higher
number of visits compared to those either continuously
released to home care or continuously released to
LTCF.
• We also see more variability in that segment of
patients, note the minimum is 2 for all.
• Also note the median age differential.
AMERICAN INSTITUTES FOR RESEARCH
Bootstrap Forest in JMP – All Code Groups
19
• So here is a very well fitting
forest model to this data. I
show only a subset of the 240
code groups we input into the
model.
• No patient characteristics,
only ICD-9 code groups.
AMERICAN INSTITUTES FOR RESEARCH
Only some predictors are really useful
• The major conditions found in the bootstrap forest were
examined more closely by the analytic and clinical group.
• The table below represents the most useful set of
diagnostic codes for early warning purposes.
20
AMERICAN INSTITUTES FOR RESEARCH
Bootstrap Forest – Key Predictors
21
AMERICAN INSTITUTES FOR RESEARCH
Logistic Regression - Comparisons
22
Key Factors Key Factors + R/G/A
AMERICAN INSTITUTES FOR RESEARCH
Key Points
• The goal may be to have the model with the highest
predictive capacity from all ICD-9 records. We tend to
think of that as less useful because while predictive it
does not leave the door open to actionable clinical
thought.
• We also do not want to stress a set of specific techniques.
For example, JUST using cluster analysis for
segmentation or JUST using random forests for
prediction.
• The most useful output will likely come from a
combination of analytics and clinical insight.
• The most useful models may not actually have best fit.
23
AMERICAN INSTITUTES FOR RESEARCH
Concluding Remarks
• The traditional model for understanding hospital utilization
relies on 3 focal points: Regression modeling on a mix of
characteristics and health indicators to directly predict
utilization or cost outcomes.
• Our approach is to segment the problem into distinct
areas (similar to triaging patients) then find targeted
health indicators that well predict the segment of interest.
• This approach is aided by clinical input and creates more
useful models from which to explore admission data.
• Ideally, one should think of this process as a cycle of
continuous improvement where the patient population is
broken up and dealt with in targeted chunks.
24
AMERICAN INSTITUTES FOR RESEARCH
References• Berwick, D.M. & Hackbarth, A.D. Eliminating Waste in US Health Care. Journal of the American Medical Association,
307(4), 1513-1526.
• Bottle, A. (2006). Identifying patients at high risk of emergency hospital admissions: A logistic regression analysis.
Journal of the Royal Society of Medicine, 99(8), 406-414. doi:10.1258/jrsm.99.8.406
• Joynt, K. E. (2011). Thirty-Day Readmission Rates for Medicare Beneficiaries by Race and Site of Care. Jama, 305(7),
675. doi:10.1001/jama.2011.123
• Wu, J., Grannis, S. J., Xu, H., & Finnell, J. T. (2016). A practical method for predicting frequent use of emergency
department care using routinely available electronic registration data. BMC Emergency Medicine BMC Emerg Med,
16(1). doi:10.1186/s12873-016-0076-3
• Frank, R. E., Massy, W. F., & Wind, Y. (1972). Market segmentation. Englewood Cliffs, NJ: Prentice-Hall.
• United States Census Bureau. Quick Facts. Nash County, North Carolina.2015. Available at:
http://www.census.gov/quickfacts/table/PST045215/37127
• County Health Rankings & Roadmaps. 2015. Available at: http://www.countyhealthrankings.org/app/north-
carolina/2015/rankings/nash/county/outcomes/overall/snapshot
• Update on County Health Rankings Nash County.2014. Available at:
http://www.co.nash.nc.us/DocumentCenter/View/808
• Healthcare Cost and Utilization Project.2015. Clinical Classifications Software (CCS) for ICD-9-CM. Available at:
https://www.hcup-us.ahrq.gov/toolssoftware/ccs/ccs.jsp
25
Jason Brinkley
919-918-2318
100 Europa Drive, Suite 315
Chapel Hill, NC 27517-2310
General Information: 919-918-2324
www.air.org
26
AMERICAN INSTITUTES FOR RESEARCH
Appendix – Example Raw Data
(Not Real Patient)
27
ReferenceNumber of Visits Visit Date Dischg Date
Visit Number
Diagnosis Number Admit Origin Admit from Visit Type Discharge Status Age Race Ethnicity Sex Language
Translator Indicator
Mode of Arrival
Primary Insurance
Secondary Insurance
Employent Status Admitting DX Diagnosis Description
111111 3 5/17/2017 5/20/2017 1 1 Emergency Room EMERGENCY DISCH TO HOME (ROUTINE) 56 Black NON-HISPANIC F English NO Stretcher BCBSNC SELF PAY On Disabil CHEST PAIN NOS PRIM CARDIOMYOPATHY NEC
111111 3 5/17/2017 5/20/2017 1 2 Emergency Room EMERGENCY DISCH TO HOME (ROUTINE) 56 Black NON-HISPANIC F English NO Stretcher BCBSNC SELF PAY On Disabil CHEST PAIN NOS MAL HYPERT HRT WO FAIL
111111 3 5/17/2017 5/20/2017 1 3 Emergency Room EMERGENCY DISCH TO HOME (ROUTINE) 56 Black NON-HISPANIC F English NO Stretcher BCBSNC SELF PAY On Disabil CHEST PAIN NOS INTERMED CORONARY SYND
111111 3 5/17/2017 5/20/2017 1 4 Emergency Room EMERGENCY DISCH TO HOME (ROUTINE) 56 Black NON-HISPANIC F English NO Stretcher BCBSNC SELF PAY On Disabil CHEST PAIN NOS CORONARY ATHEROSCLEROSIS
111111 3 5/17/2017 5/20/2017 1 5 Emergency Room EMERGENCY DISCH TO HOME (ROUTINE) 56 Black NON-HISPANIC F English NO Stretcher BCBSNC SELF PAY On Disabil CHEST PAIN NOS DYSTHYMIC DISORDER
111111 3 5/17/2017 5/20/2017 1 6 Emergency Room EMERGENCY DISCH TO HOME (ROUTINE) 56 Black NON-HISPANIC F English NO Stretcher BCBSNC SELF PAY On Disabil CHEST PAIN NOS HYPERLIPIDEMIA NEC/NOS
111111 3 5/17/2017 5/20/2017 1 7 Emergency Room EMERGENCY DISCH TO HOME (ROUTINE) 56 Black NON-HISPANIC F English NO Stretcher BCBSNC SELF PAY On Disabil CHEST PAIN NOS DM UNCOMP TYP II UNCNTRD
111111 3 5/17/2017 5/20/2017 1 8 Emergency Room EMERGENCY DISCH TO HOME (ROUTINE) 56 Black NON-HISPANIC F English NO Stretcher BCBSNC SELF PAY On Disabil CHEST PAIN NOS LNG-TERM CURR USE OT MED
111111 3 5/17/2017 5/20/2017 1 9 Emergency Room EMERGENCY DISCH TO HOME (ROUTINE) 56 Black NON-HISPANIC F English NO Stretcher BCBSNC SELF PAY On Disabil CHEST PAIN NOS LONG-TERM USE INSULIN
111111 3 5/22/2017 5/24/2017 2 1 Emergency Room EMERGENCY DISCH TO HOME (ROUTINE) 56 Black NON-HISPANIC F English NO Stretcher BCBSNC SELF PAY On Disabil MUSCLE WEAKNESS GENERALZ IATROGEN CV INFARC/HMRHG
111111 3 5/22/2017 5/24/2017 2 2 Emergency Room EMERGENCY DISCH TO HOME (ROUTINE) 56 Black NON-HISPANIC F English NO Stretcher BCBSNC SELF PAY On Disabil MUSCLE WEAKNESS GENERALZ CEREBRAL EMBOLISM W CI
111111 3 5/22/2017 5/24/2017 2 3 Emergency Room EMERGENCY DISCH TO HOME (ROUTINE) 56 Black NON-HISPANIC F English NO Stretcher BCBSNC SELF PAY On Disabil MUSCLE WEAKNESS GENERALZ APHASIA
111111 3 5/22/2017 5/24/2017 2 4 Emergency Room EMERGENCY DISCH TO HOME (ROUTINE) 56 Black NON-HISPANIC F English NO Stretcher BCBSNC SELF PAY On Disabil MUSCLE WEAKNESS GENERALZ UNSPEC HEMIPLEG/HEMIPRES
111111 3 5/22/2017 5/24/2017 2 5 Emergency Room EMERGENCY DISCH TO HOME (ROUTINE) 56 Black NON-HISPANIC F English NO Stretcher BCBSNC SELF PAY On Disabil MUSCLE WEAKNESS GENERALZ PRIM CARDIOMYOPATHY NEC
111111 3 5/22/2017 5/24/2017 2 6 Emergency Room EMERGENCY DISCH TO HOME (ROUTINE) 56 Black NON-HISPANIC F English NO Stretcher BCBSNC SELF PAY On Disabil MUSCLE WEAKNESS GENERALZ CORONARY ATHEROSCLEROSIS
111111 3 5/22/2017 5/24/2017 2 7 Emergency Room EMERGENCY DISCH TO HOME (ROUTINE) 56 Black NON-HISPANIC F English NO Stretcher BCBSNC SELF PAY On Disabil MUSCLE WEAKNESS GENERALZ HYPERTENSION NOS
111111 3 5/22/2017 5/24/2017 2 8 Emergency Room EMERGENCY DISCH TO HOME (ROUTINE) 56 Black NON-HISPANIC F English NO Stretcher BCBSNC SELF PAY On Disabil MUSCLE WEAKNESS GENERALZ DM UNCOMP TYP II UNCNTRD
111111 3 5/22/2017 5/24/2017 2 9 Emergency Room EMERGENCY DISCH TO HOME (ROUTINE) 56 Black NON-HISPANIC F English NO Stretcher BCBSNC SELF PAY On Disabil MUSCLE WEAKNESS GENERALZ HYPERLIPIDEMIA NEC/NOS
111111 3 5/22/2017 5/24/2017 2 10 Emergency Room EMERGENCY DISCH TO HOME (ROUTINE) 56 Black NON-HISPANIC F English NO Stretcher BCBSNC SELF PAY On Disabil MUSCLE WEAKNESS GENERALZ DYSTHYMIC DISORDER
111111 3 5/22/2017 5/24/2017 2 11 Emergency Room EMERGENCY DISCH TO HOME (ROUTINE) 56 Black NON-HISPANIC F English NO Stretcher BCBSNC SELF PAY On Disabil MUSCLE WEAKNESS GENERALZ ANEMIA NOS
111111 3 5/22/2017 5/24/2017 2 12 Emergency Room EMERGENCY DISCH TO HOME (ROUTINE) 56 Black NON-HISPANIC F English NO Stretcher BCBSNC SELF PAY On Disabil MUSCLE WEAKNESS GENERALZ OBESITY NOS
111111 3 5/22/2017 5/24/2017 2 13 Emergency Room EMERGENCY DISCH TO HOME (ROUTINE) 56 Black NON-HISPANIC F English NO Stretcher BCBSNC SELF PAY On Disabil MUSCLE WEAKNESS GENERALZ BD MS INDX 38.0-38.9 ADL
111111 3 5/22/2017 5/24/2017 2 14 Emergency Room EMERGENCY DISCH TO HOME (ROUTINE) 56 Black NON-HISPANIC F English NO Stretcher BCBSNC SELF PAY On Disabil MUSCLE WEAKNESS GENERALZ LNG-TERM CURR USE OT MED
111111 3 5/22/2017 5/24/2017 2 15 Emergency Room EMERGENCY DISCH TO HOME (ROUTINE) 56 Black NON-HISPANIC F English NO Stretcher BCBSNC SELF PAY On Disabil MUSCLE WEAKNESS GENERALZ ABN REACT-CARDIAC CATH
111111 3 11/7/2017 11/12/2017 3 1 Phy Referrral EMERGENCY DISCH TO HOME (ROUTINE) 57 Black NON-HISPANIC F English NO Stretcher BCBSNC SELF PAY On Disabil HYPOTENSION NOS LA EFF CEREB,HEMIPLEGIA
111111 3 11/7/2017 11/12/2017 3 2 Phy Referrral EMERGENCY DISCH TO HOME (ROUTINE) 57 Black NON-HISPANIC F English NO Stretcher BCBSNC SELF PAY On Disabil HYPOTENSION NOS PRIM CARDIOMYOPATHY NEC
111111 3 11/7/2017 11/12/2017 3 3 Phy Referrral EMERGENCY DISCH TO HOME (ROUTINE) 57 Black NON-HISPANIC F English NO Stretcher BCBSNC SELF PAY On Disabil HYPOTENSION NOS OCC/STEN CAR ART W/O CI
111111 3 11/7/2017 11/12/2017 3 4 Phy Referrral EMERGENCY DISCH TO HOME (ROUTINE) 57 Black NON-HISPANIC F English NO Stretcher BCBSNC SELF PAY On Disabil HYPOTENSION NOS HYPERTENSION NOS
111111 3 11/7/2017 11/12/2017 3 5 Phy Referrral EMERGENCY DISCH TO HOME (ROUTINE) 57 Black NON-HISPANIC F English NO Stretcher BCBSNC SELF PAY On Disabil HYPOTENSION NOS DYSTHYMIC DISORDER
111111 3 11/7/2017 11/12/2017 3 6 Phy Referrral EMERGENCY DISCH TO HOME (ROUTINE) 57 Black NON-HISPANIC F English NO Stretcher BCBSNC SELF PAY On Disabil HYPOTENSION NOS HYPERLIPIDEMIA NEC/NOS
111111 3 11/7/2017 11/12/2017 3 7 Phy Referrral EMERGENCY DISCH TO HOME (ROUTINE) 57 Black NON-HISPANIC F English NO Stretcher BCBSNC SELF PAY On Disabil HYPOTENSION NOS ANEMIA NOS
111111 3 11/7/2017 11/12/2017 3 8 Phy Referrral EMERGENCY DISCH TO HOME (ROUTINE) 57 Black NON-HISPANIC F English NO Stretcher BCBSNC SELF PAY On Disabil HYPOTENSION NOS MORBID OBESITY
111111 3 11/7/2017 11/12/2017 3 9 Phy Referrral EMERGENCY DISCH TO HOME (ROUTINE) 57 Black NON-HISPANIC F English NO Stretcher BCBSNC SELF PAY On Disabil HYPOTENSION NOS ADHESIVE CAPSULIT SHLDER
111111 3 11/7/2017 11/12/2017 3 10 Phy Referrral EMERGENCY DISCH TO HOME (ROUTINE) 57 Black NON-HISPANIC F English NO Stretcher BCBSNC SELF PAY On Disabil HYPOTENSION NOS OBSTRUCTIVE SLEEP APNEA
111111 3 11/7/2017 11/12/2017 3 11 Phy Referrral EMERGENCY DISCH TO HOME (ROUTINE) 57 Black NON-HISPANIC F English NO Stretcher BCBSNC SELF PAY On Disabil HYPOTENSION NOS DIAB W MANIF NEC TYPE II
111111 3 11/7/2017 11/12/2017 3 12 Phy Referrral EMERGENCY DISCH TO HOME (ROUTINE) 57 Black NON-HISPANIC F English NO Stretcher BCBSNC SELF PAY On Disabil HYPOTENSION NOS HYPOTENSION NOS
111111 3 11/7/2017 11/12/2017 3 13 Phy Referrral EMERGENCY DISCH TO HOME (ROUTINE) 57 Black NON-HISPANIC F English NO Stretcher BCBSNC SELF PAY On Disabil HYPOTENSION NOS LONG-TERM USE ANTICOAGUL
111111 3 11/7/2017 11/12/2017 3 14 Phy Referrral EMERGENCY DISCH TO HOME (ROUTINE) 57 Black NON-HISPANIC F English NO Stretcher BCBSNC SELF PAY On Disabil HYPOTENSION NOS LNG-TERM CURR USE OT MED