Post on 15-Dec-2015
transcript
Practice-Based Evidence Research Model
Predictors of Clinician PerformanceRisk Adjustment
Comparative Effectiveness Research
Focus on Therapeutic Outcomes, Inc.Interactive Health AnalysisTM
AAOMPT ConferenceAnaheim, CA 2011
Dennis L. Hart, PT, PhDDirector, Consulting and Research
Focus On Therapeutic Outcomes, Inc.
Knoxville, TN
Faculty Affiliation & Disclosure
• Affiliation– Director of Consulting and Research for FOTO.– Develops outcomes and other measures used in FOTO’s
software to assist clinician’s ability to manage patients designed to improve their outcomes.
– Publishes or assists in the publication of scientific endeavors related to analyses of the FOTO database.
• Disclosure– Employee and investor in FOTO, a proprietary international
medical/rehabilitation database management company.
2
Focus on Therapeutic Outcomes, Inc.Interactive Health Analysis TM
The First Presentation
• Describe predictors of physical therapy clinician performance identified using Practice-Based-Evidence (PBE) research designs.
• PBE methods/designs allow development of comparative effectiveness research (CER) models.
• CER uses advanced risk-adjustment to control for confounders and diverse patient case-mix for meaningful interpretation of results to identify which treatments are associated with better outcomes for which patients.
3
Focus on Therapeutic Outcomes, Inc.Interactive Health Analysis TM
Background
• FOTO is combining data from the U.S. and Israel looking for best practices.
• Both countries are interested in public reporting, value-based purchasing and finding best practices.
• Research teams are analyzing data from evidence-based practice from MDT and EIM trained clinicians.
• As these data become risk-adjusted, they become the foundation for value-based purchasing and public reporting and become the foundation for comparative effectiveness research.
4
Focus on Therapeutic Outcomes, Inc.Interactive Health Analysis TM
Predictors of Clinician Performance
• To predict clinician performance, one needs data describing patients, clinicians and outcomes (etc!).
• We collect and analyze real life clinical practice data.• Use of real life clinical data and their analyses are
“Practice-Based Evidence for Clinical Practice Improvement Study Design” (PBE): Susan Horn, PhD.
• PBE analyzes content and timing of individual steps of a health care process to determine how to achieve superior outcomes for the least necessary cost over the continuum of a patient’s care.
5
Focus on Therapeutic Outcomes, Inc.Interactive Health Analysis TM
Predictors of Clinician Performance
• Let’s look at some data from MDT trained clinicians.• N=743 patients with lumbar impairments.• Age 52 (17), 18 to 93: 57% female: 17% had surgery• 52% chronic symptoms• 40% had 4 or more comorbid conditions• 53% PPO or HMO (11% military, 3% WC)• 45% elevated somatization: 47% elevated depression• 27% elevated FABQ-PA• 41% CEN
6
Focus on Therapeutic Outcomes, Inc.Interactive Health Analysis TM
Clinician Performance: Raw Data
PT1 PT2 PT3 PT4 PT5 PT8 PT 1150
55
60
65
70
75
80
85
90
Discharge FS By Clinician
7
Focus on Therapeutic Outcomes, Inc.Interactive Health Analysis TM
Dis
char
ge F
S
Therapists
N=743, mean 106/therapist, range 31 - 288
Clinician Performance: Risk-Adjusted Predictions for Discharge FS
PT1 PT2 PT3 PT4 PT5 PT8 PT1150
55
60
65
70
75
80
85
90
RA Predicted Discharge FS by Clinician
8
Focus on Therapeutic Outcomes, Inc.Interactive Health Analysis TM
RA
Pre
dic
ted
Dis
cha
rge
FS
Therapists
N=448, R2 = 0.50, P<0.001
Clinician Performance: Raw & Risk-Adjusted Predicted Discharge FS
9
Focus on Therapeutic Outcomes, Inc.Interactive Health Analysis TM
Ra
w &
RA
Pre
dic
ted
Dis
char
ge
FS
Therapists
N=448, R2 = 0.50, P<0.001
PT1 PT2 PT3 PT4 PT5 PT8 PT1150
55
60
65
70
75
80
85
90
Raw FSRA Pred FS
Clinician Performance: Risk-Adjusted Residuals for Discharge FS
PT1 PT2 PT3 PT4 PT5 PT8 PT11
-3
-2
-1
0
1
2
3
RA Residuals Discharge FS by Clinician
10
Focus on Therapeutic Outcomes, Inc.Interactive Health Analysis TM
RA
Re
sid
uals
Dis
char
ge F
S
Therapists
N=448, R2 = 0.50, P<0.001
Clinician Performance: Ratings by Raw vs. Risk-Adjusted Residuals for Discharge FS
11
Focus on Therapeutic Outcomes, Inc.Interactive Health Analysis TM
Therapist Rating Using Raw FS
Rating Using RA Residual
PT1 5 6
PT2 3 2
PT3 2 7
PT4 6 1
PT5 7 4
PT8 4 3
PT11 1 5
Risk Adjustment
• Risk adjustment, also called ‘case-mix’ adjustment, is a statistical method used to control effects of confounding variables seen in patient populations.
• Risk adjustment considers factors other than the health care intervention or processes of care that help explain variation in patient outcomes.
• Without clinically appropriate and statistically sound risk adjustment, you cannot compare outcomes amongst clinicians in a meaningful manner.
12
Focus on Therapeutic Outcomes, Inc.Interactive Health Analysis TM
Comparative Effectiveness Research
• CER: which treatments are associated with better outcomes for which patients.
• Deutscher, Horn, Dickstein, Hart et al, 2009 is the only paper I am aware of that describes a CER data analysis plan in outpatient rehab.
• Observational study where all data are included.• Bias is present to some extent in all observational
studies (systematic error from factors that influence the decision to treat and the outcomes of interest, like patient selection, confounding, misclassification, timing of measurements, loss of follow-up data).
13
Focus on Therapeutic Outcomes, Inc.Interactive Health Analysis TM
Dreyer NA et al Am J Man Care 2010 Dreyer NA et al Health Affairs 2010
Controlling for Bias
• Large, complete, diverse data sets.• Statistical risk-adjustment to reduce effects of bias.• Many variables (patient demographic, patient
classification, clinic/clinician details, process, outcomes, timing of measurements) quantified.
• Exacting operational definitions for each variable.• Variability in treatment.• Groups being compared should have a reasonable
amount of overlap.
14
Focus on Therapeutic Outcomes, Inc.Interactive Health Analysis TM
Dreyer NA et al Health Affairs 2010
Deutscher et al Results: Quick Review
• Data set large: 54 outpatient facilities in Israel, of 57,008 patients, 22,019 (39%) had both intake and discharge data, diverse patients.
• Many variables: patient characteristics, interventions, and outcomes.
• Variability in treatments.• Operational definitions: working on it!• Groups had reasonable overlap in needs for
treatment.• Multivariable linear regression models.
15
Focus on Therapeutic Outcomes, Inc.Interactive Health Analysis TM
Deutscher D et al Arch Phys Med Rehabil 2009
Deutscher et al Results: Quick Review
• Exploratory power of models: 30% to 39%.• Patient variables explained most variation in
outcomes.• Better outcomes were associated with: patient
compliance with self-exercise and therapy attendance, use of therapeutic exercise and manual therapy, completion of 3 or more outcomes surveys.
• Worse outcomes were associated with: women, electrotherapy for pain, ultrasound for shoulder impairments, use of depression medication, longer waiting times before starting therapy.
16
Focus on Therapeutic Outcomes, Inc.Interactive Health Analysis TM
Deutscher D et al Arch Phys Med Rehabil 2009
The MDT and PBE Studies
• To improve patient outcomes:– Collect all data – No missing data.– Data must be risk adjusted.– Add variables to the risk-adjustment models.– Define everything you do, like treatments
provided and methods of clinical examination.– Classify patients in meaningful ways.– Little difference between patients U.S. vs. Israel.– Follow the data, not the guru.– Integrate results into clinical practice.
17
Focus on Therapeutic Outcomes, Inc.Interactive Health Analysis TM
EBP and Risk-Adjusted Outcomes
• Evidence-Based Practice provides structure to patient examination and management.
• EBP is the use of the current best science to help understand the patient for improved patient management and outcomes.
• EBP has led to improved patient examination, which has led to better ways of classifying/grouping patients, and better ways of treating patients, for improved statistical modeling of outcomes.
• Adding patient classification brings predictive power of FS outcomes models to 50%.
18
Focus on Therapeutic Outcomes, Inc.Interactive Health Analysis TM
Werneke MW et al JOSPT 2011
Extraneous Variables
• Observational studies have bias to some extent.• The trick is to describe and quantify the bias, so you
can control the effect of the bias, which improves interpretation of results.
• Outcome Confounder: Using MDT or TBC approaches direct treatment to specific subgroups of patients, which confounds outcomes.
• Treatment Mediator: Clinical finding like elevated fear-avoidance that helps direct treatment but might not be a good predictor of FS outcomes.
19
Focus on Therapeutic Outcomes, Inc.Interactive Health Analysis TM
George SZ et al Spine 2003Edmond SL et al JOSPT 2011
Extraneous Variables
• Observational studies have to some degree of bias.• The trick is to describe and quantify the bias, so you
can control the effect of the bias, which improves interpretation of results.
• Outcome Confounder: Using MDT or TBC approaches direct treatment to specific subgroups of patients, which confounds outcomes.
• Treatment Mediator: Clinical finding like elevated fear-avoidance that helps direct treatment but might not be a good predictor of FS outcomes.
20
Focus on Therapeutic Outcomes, Inc.Interactive Health Analysis TM
George SZ et al Spine 2003Edmond SL et al JOSPT 2011
Extraneous Variables
• Effect of treatment on outcomes.• Be prepared to recognize that treatment might not
affect FS outcomes as much as we would like!• As predictive models evolve, current risk-adjustment
variables control a large percent of FS outcomes data variance, so new variables add little to predictions.
• Comorbidities: <2% additional variance control.• Fear-avoidance: <2% additional variance control.• Classification: ~50% additional variance control.
21
Focus on Therapeutic Outcomes, Inc.Interactive Health Analysis TM
Resnik L et al JCE 2010Werneke MW et al JOSPT 2010Hart DL et al JOSPT 2011
Effect of Adding Variables:Comorbid Conditions & Fear
22
Focus on Therapeutic Outcomes, Inc.Interactive Health Analysis TM
R2
Val
ues
Orthopedic/Lumbar: Functional Status ChangeResnik L et al 2010; Hart DL et al 2011
No FCI or Fear No Fear Full0.24
0.242
0.244
0.246
0.248
0.25
0.252
0.254
0.2453
0.25170.2531
0.64% gained
0.14% gained
WithFCI
WithFCI&Fear
Effect of Adding Patient Classification:Pain Pattern Classification
23
Focus on Therapeutic Outcomes, Inc.Interactive Health Analysis TM
R2
Val
ues
Orthopedic/Lumbar: Functional Status ChangeWerneke MW et al 2010
Full PPC0
0.05
0.1
0.15
0.2
0.25
0.3
0.35
0.4
0.45
0.5
0.2531
0.5
MDT Project
Doubled!
Conclusions
• Have described predictors of physical therapy clinician performance using Practice-Based-Evidence (PBE) research designs.
• Introduced CER and risk-adjustment to control for confounders for meaningful interpretation of results.
• Demonstrated how we have identified which treatments are associated with better outcomes for which patients.
• Be prepared to change your understanding of many issues related to patients, treatments, outcomes and payment.
24
Focus on Therapeutic Outcomes, Inc.Interactive Health Analysis TM
Thank You
25
Focus on Therapeutic Outcomes, Inc.Interactive Health Analysis TM
hart@fotoinc.com