Moving the field from ‘no wrong door’ to the ‘best door’: An actuarial estimate of expected outcomes by level of care among adolescents presenting for substance abuse treatment Michael L. Dennis, Rodney R. Funk, and Laverne Hanes- Stevens, Chestnut Health Systems, Bloomington, IL Panel at the Joint Meeting on Adolescent Treatment Effectiveness, March 25-27, 2008, Washington, DC. This presentation supported by Center for Substance Abuse Treatment (CSAT), Substance Abuse and Mental Health Services Administration (SAMHSA) contracts 270-2003- 00006 and 270-07-0191, as well as several individual CSAT, NIAAA, NIDA and private foundation grants. The opinions are those of the author and do not reflect official
Transcript
Moving the field from ‘no wrong door’ to the ‘best door’: An actuarial estimate of expected outcomes by level of care amongadolescents presenting for substance abuse treatmentMichael L. Dennis, Rodney R. Funk, and Laverne Hanes-Stevens, Chestnut Health Systems, Bloomington, IL
Panel at the Joint Meeting on Adolescent Treatment Effectiveness, March 25-27, 2008, Washington, DC. This presentation supported by Center for Substance Abuse Treatment (CSAT), Substance Abuse and Mental Health Services Administration (SAMHSA) contracts 270-2003-00006 and 270-07-0191, as well as several individual CSAT, NIAAA, NIDA and private foundation grants. The opinions are those of the author and do not reflect official positions of the consortium or government. Available on line at www.chestnut.org/LI/Posters or by contacting Joan Unsicker at 720 West Chestnut, Bloomington, IL 61701, phone: (309) 827-6026, fax: (309) 829-4661, e-Mail: [email protected]
Background In practice, programs primarily refer people to the limited
range of services they have readily available. Knowing nothing about the person other than what door they
walked through we can correctly predict 75% (kappa=.51) of the adolescent level of care placements.
The American Society for Addiction Medicine (ASAM) has tried to recommend placement rules for deciding what level of care an adolescent should receive based on expert opinion, but run into many problems including - difficulty synthesizing multiple pieces of information- inconsistencies between competing rules, - the lack of the full continuum of care to refer people to, - having to negotiate with the participant, families and
funders over what they will do or pay for- there is virtually no actual data on the expected outcomes
by level of care to inform decision making related to placement
Objectives
This presentation uses data from intake to 12 months collected with the Global Appraisal of Individual Needs (GAIN) with the ASAM statements and clusters just discussed by Hanes-Stevens and Funk in the preceding presentations.
The goal is to make an actuarial estimate of the expected outcomes for each individual for each potential level of care to inform clinical decision making related to placement.
Method Started with the 8301 people in the Funk et al cluster analysis. Dropped 2156 of the 8301 people in the cluster analysis who were not due
or did not have at least one follow-up yet Analysis done on 6,145 adolescents with 1 or more follow-ups (83% of
those due) from 203 level of care x site combinations Examined the actual level of care within each cluster, collapsing any that
had less than 50 adolescents with follow-ups. Used logistic regression on individual outcomes and linear regression to
predict counts of positive outcomes based on actual level of care Used coefficients from above analysis to compute predicted outcomes
within each cluster based on each level of care within that cluster Compared levels of care based on the predicted outcomes Cohen’s f (.1=small, .2=moderate, .4= large) Odds Ratio (0.8/1.2 – small, 0.5/2.0- large)
Simplified Levels of Care
0% 20% 40% 60% 80% 100%
A Low-Low
B Low-Mod
C Mod-Mod
D Hi-Low
E Hi-Mod
F Hi-Hi (CC)
G Hi-Mod (E/P)
H Hi-Hi (I/P/M)
Outpatient (OP) Intensive Outpatient (IOP)
Outpatient Continuing Care (OPCC) Short Term Residential (STR)
Long Term Residential (LTR)
All higher levels
STR & LTR
IOP/OPCC
Outcomes
Washington Circle Group and National Commission on Quality Assurance (NCQA) from private insurance
National Outcome Monitoring System from States (NOMS), including SAMHSA Cost Bands
Government Performance and Results Act (GPRA)
Group 1. Cohen’s Effect Size f on Treatment received based on records
\1 Past month \2 Past 90 days Cohen’s f > .1 in bold
Group 3. Variance Explained in Tx Outcomes*
\1 Past month \2 Past 90 days *All statistically Significant
26%
24%
11%
25%
15%
33%
26%
18%
14%
8%
24%
0% 5% 10% 15% 20% 25% 30% 35%
No AOD Use \1
No AOD related Prob.\1
No Health Problems \2
No Mental Health Prob.\2
No Illegal Activity \2
No JJ System Involve. \1
Living in Community \1
No Family Prob. \2
Vocationally Engaged \1
Social Support \2
Count of above
Percent of Variance Explained
Key Predictors of Outcomes: Baseline Characteristics (1 of 3)
0% 20% 40% 60% 80% 100%
Female
African American (vs Mixed/Other)
Caucasian (vs Mixed/Other)
Hispanic (vs Mixed/Other)
Age (per year)
Alcohol Primary (vs. Cannabis)
Other Drug Primary (vs. Cannabis)
% of 18 Odds Ratio
LT .5 .5 to .8 .8 to 1.2 1.2 to 2.0 GT 2.0
Key Predictors of Outcomes: Baseline Characteristics (2 of 3)
0% 20% 40% 60% 80% 100%
Substance Frequency Scale
Emotional Problems Scale
Illegal Activity Scale
Recovery Environmental Risk Index
Past month abstinence
No Substance Problems in past month
no major health problems
% of 18 Odds Ratio
LT .5 .5 to .8 nsd 1.2 to 2.0 GT 2.0LT .5 .5 to .8 .8 to 1.2 1.2 to 2.0 GT 2.0
Key Predictors of Outcomes: Baseline Characteristics (3 of 3)
0% 20% 40% 60% 80% 100%
No major Mental Health Problems
No Illegal Activity
No Past Month JJ Involvement
Livining in the community
No Family Problems
Vocationally Engaged
Any Social Support
% of 18 Odds Ratio
LT .5 .5 to .8 nsd 1.2 to 2.0 GT 2.0LT .5 .5 to .8 .8 to 1.2 1.2 to 2.0 GT 2.0
Key Predictors of Outcomes: ASAM Tx Planning Cluster
0% 20% 40% 60% 80% 100%
Cluster B (ref=A)
Cluster C (ref=A)
Cluster D (ref=A)
Cluster E (ref=A)
Cluster F (ref=A)
Cluster G (ref=A)
Cluster H (ref=A)
% of 18 Odds Ratio
LT .5 .5 to .8 nsd 1.2 to 2.0 GT 2.0LT .5 .5 to .8 .8 to 1.2 1.2 to 2.0 GT 2.0
Key Predictors of Outcomes: Level of Care within Tx Planning Cluster A to C
0% 20% 40% 60% 80% 100%
Higher LOC vs. OP in A
IOP vs. OP in B
OPCC vs. OP in B
Residential vs. OP in B
IOP vs. OP in C
OPCC vs OP in C
Residential vs. OP in C
% of 18 Odds Ratio
LT .5 .5 to .8 nsd 1.2 to 2.0 GT 2.0LT .5 .5 to .8 .8 to 1.2 1.2 to 2.0 GT 2.0
0% 20% 40% 60% 80% 100%
IOP/OPCC vs OP in D
Residential vs. OP in D
IOP vs. OP in E
OPCC vs. OP in E
Residential vs. OP in E
IOP vs. OP in F
OPCC vs. OP in F
Residential vs. OP in F
% of 18 Odds Ratio
LT .5 .5 to .8 nsd 1.2 to 2.0 GT 2.0
Key Predictors of Outcomes: Level of Care within Tx Planning Cluster D to F
LT .5 .5 to .8 .8 to 1.2 1.2 to 2.0 GT 2.0
Key Predictors of Outcomes: Level of Care within Tx Planning Cluster G to H
0% 20% 40% 60% 80% 100%
IOP/OPCC vs. OP in G
Residential vs. OP in G
IOP vs. OP in H
OPCC vs. OP in H
Residential vs. OP in H
% of 18 Odds Ratio
LT .5 .5 to .8 nsd 1.2 to 2.0 GT 2.0LT .5 .5 to .8 .8 to 1.2 1.2 to 2.0 GT 2.0
Key Predictors of Outcomes: Group 1 Treatment Received from Records
0% 20% 40% 60% 80% 100%
Initiation
Evidence Based Treatment
Engagement
Continuing Care
% of 14 Odds Ratio
LT .5 .5 to .8 .8 to 1.2 1.2 to 2.0 GT 2.0
Key Predictors of Outcomes: Group 2 Treatment Received from Self Report
0% 20% 40% 60% 80% 100%
Early Treatment Satisfaction
TX Satisfaction at 3 months
No/Reduce AOD at 3 months
Within SAMSHA Tx Cost Bands
% of 10 Odds Ratio
LT .5 .5 to .8 nsd 1.2 to 2.0 GT 2.0LT .5 .5 to .8 .8 to 1.2 1.2 to 2.0 GT 2.0
Predicted Count of Positive Outcomes by Level of Care: Cluster A Low - Low (n=1,025)
2
3
4
5
6
7
8
9
10
Outpatient Higher LOC
2
3
4
5
6
7
8
9
10
Predicted Count of Positive Outcomes by Level of Care: Cluster A Low - Low (n=1,025)
Best Level of Care*: Cluster A Low - Low (n=1,025)Best Level of Care*:
Cluster A Low - Low (n=1,025)
99.6%
0.4%0%
20%
40%
60%
80%
100%
120%
Outpatient Higher LOC
% B
est P
redi
cted
Out
com
es
* Based on Maximum Predicted Count of Positive Outcomes
Predicted Count of Positive Outcomes by Level of Care: Cluster C Mod-Mod (n=1209)
2
3
4
5
6
7
8
9
10
Outpatient Intensive Outpatient
Outpatient -Continuing Care
Residential
2
3
4
5
6
7
8
9
10
Best Level of Care*: Cluster C Mod-Mod (n=1209)
30.2%
7.6%
23.6%
38.6%
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
Outpatient IOP OPCC Residential
% B
est P
redi
cted
Out
com
es
* Based on Maximum Predicted Count of Positive Outcomes
Predicted Count of Positive Outcomes by Level of Care: Cluster F Hi-Hi (CC) (n=968)
2
3
4
5
6
7
8
9
10
2
3
4
5
6
7
8
9
10
Outpatient Intensive Outpatient
Outpatient -Continuing Care
Residential
Best Level of Care*: Cluster F Hi-Hi (CC) (n=968)
81.5%
8.6%
0.0%
9.9%
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
Outpatient IOP OPCC Residential
% B
est P
redi
cted
Out
com
es
* Based on Maximum Predicted Count of Positive Outcomes
Predicted Count of Positive Outcomes by Level of Care: Cluster G. Hi-Mod (Env/PH) (n=749)
2
3
4
5
6
7
8
9
10
Outpatient IOP/OPCC Residential
2
3
4
5
6
7
8
9
10
Predicted Count of Positive Outcomes by Level of Care: Cluster Hi-Mod (Env/PH) (n=749)
Best Level of Care*: Cluster G Hi-Mod (Env/PH) (n=749)Best Level of Care*:
Cluster G Hi-Mod (Env/PH) (n=749)
94.1%
5.9%0.0%
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Outpatient IOP/OPCC Residential
* Based on Maximum Predicted Count of Positive Outcomes
Planned use
Best fit will be used to recommend a level of care at the end of the GAIN Recommendation and Referral Summary
Table of 18 outcomes by level of care for the predicted cluster will be available to consider other options if a given recommendation is not available or there is a need to negotiate
If staff change the cluster type (may be relevant if there is new information or they are between two), the above can be recalculated
Limitations
Data limited to self report, thus it is important to inform (not control) clinical decision making
Not a representative sample
Not available yet for subtypes of a level of care (e.g., a specific evidenced based approach to treatment), young adults or adults
Ideally it needs to be tested prospectively
Conclusions
The relationship between multiple variables and outcomes is complex and not easily done by clinicians.
The 8 cluster groups based on ASAM treatment planning cells can help to predict outcome
It is feasible to make an actuarial estimate of treatment outcomes that has the potential to improve treatment outcomes
While there often is an advantage to one particular level of care placement, there is also a fair amount of overlap – suggesting the value of informed decisions (not a fixed rule).
The above presentation was supported by the Substance Abuse and Mental Health Services Administration’s (SAMHSA) Center for Substance Abuse Treatment (CSAT) under contracts 207-98-7047, 277-00-6500, 270-2003-00006, and 270-07-0191 using data provided by the following grantees: CSAT TI-13190, TI-13305, TI-13308, TI-13309, TI-13313, TI-13322, TI-13323, TI-13340, TI-13344, TI-13345, TI-13354, TI-13356, TI-13601, TI-14090, TI-14103, TI-14188, TI-14189, TI-14196, TI-14214, TI-14252, TI-14254, TI-14261, TI-14267, TI-14271, TI-14272, TI-14283, TI-14311, TI-14315, TI-14355, TI-14376, TI-15348, TI-15413, TI-15415, TI-15421, TI-15433, TI-15446, TI-15447, TI-15458, TI-15461, TI-15466, TI-15467, TI-15469, TI-15475, TI-15478, TI-15479, TI-15481, TI-15483, TI-15485, TI-15486, TI-15489, TI-15511, TI-15514, TI-15524, TI-15527, TI-15545, TI-15562, TI-15577, TI-15584, TI-15586, TI-15670, TI-15671, TI-15672, TI-15674, TI-15677, TI-15678, TI-15682, TI-15686, TI-16386, TI-16400, TI-16414, TI-16904, TI-16915, TI-16928, TI-16939, TI-16961, TI-16984, TI-16992, TI-17046, TI-17055, TI-17070, TI-17071, TI-17334, TI-17433, TI-17434, TI-17475, TI-17484). Any opinions about these data are those of the authors and do not reflect official positions of the government or individual grantees. Suggestions, comments, and questions can be sent to Dr. Michael Dennis, Chestnut Health Systems, 720 West Chestnut, Bloomington, IL 61701, [email protected] .
