Understanding the Outcomes Research Toolbox
Justin B. Dimick, MD, MPH Henry King Ransom Professor of Surgery Director, Center for Healthcare Outcomes & Policy University of Michigan ACS Outcomes Research Course 2014
Agenda:
1. Conceptual overview – three major areas of research
2. For each area, explore major
problems and tools for fixing them
Comparative effectiveness
Randomized clinical trials
Quality of life
? Qualitative research
Patient centered outcomes
Decision analysis
Policy evaluation
Disparities
How can we make sense of these different research areas?
Groups of 3 to 4
5 minutes
Evaluating Disease Management (Patient-Level)
Healthcare Delivery System (Macro-systems)
Understanding Local Provision of Care (Micro-systems)
Patient-Centered Outcomes Comparative Effectiveness
Data Synthesis
Quality Improvement Patient Safety
Policy Evaluation Payment Reform
Work Force
Coverage Decisions Technology Adoption
Cost Effectiveness
Implementation Science
Quality Measurement
Problems & Solutions
Evaluating Disease Management(Patient-Level)
Patient-Centered OutcomesComparative Effectiveness
Data Synthesis
Major Problems?
VS.
VS.
Problem: Patients getting treatment A and B are different
Treatment (Endo vs. open
aneurysm)
Outcome (1-year death)
Confounding variables
(Illness levels)
Confounding
Selection bias & confounding
Toolbox: • Randomized clinical trials • Multivariate adjustment using
regression • Propensity scores • Instrumental variable analysis
Problem: External Validity
Randomized Clinical Trials – External Validity?
Toolbox: Pragmatic trials Cluster randomized trials Observational studies
CER Methodologies
Randomized trials
Observational Studies
Pragmatic Trials Cluster Randomized Trials Natural Experiments Cohort
*Lohr, Medical Care (2007) 45 (10); S5
Problems with external validity
Problems with confounding
What about the patient perspective?
Toolbox: Patient centered outcomes; Quality of life measures; Survey research
Dr: “Congratulations, your hernia if fixed” Patient: “Great, but why is my groin pain worse?”
What about the patient perspective?
Understanding Local Provision of Care (Micro-systems)
Quality ImprovementPatterns of Care Analyses
Patient Safety
Major problems?
“The mere existence of evidence-based practices, however, does not always result in the use of such practices because of the complexities inherent in translating evidence into practice.” How do we better understand these factors?
Toolbox: Qualitative research – Observation, key informant interviews, Focus groups
In-depth phone and in-person interviews were conducted between October 2006 and September 2007 with 86 participants (31 physicians). Active resistance to evidence-based practice change was pervasive. Successful efforts to overcome active resisters included benchmarking infection rates, identifying effective champions, and participating in collaborative efforts.
Complexity in frontline systems
• Complex assembly of people, information, resources, equipment and procedures working toward a common goal
• System vulnerability reflects exposure to
events and factors that can make the system less safe or more prone to adverse events
Toolbox: Human factors engineering, systems analysis
Human Factors Engineering: Design systems and environments for safer, more effective and more efficient use by humans
“There is a general consensus by intensivists and nonsurgical providers that surgeons hesitate to withdraw life-sustaining therapy on their operative patients despite a patient's or surrogate's request to do so.” How do you better understand this?
Healthcare Delivery System(Macro-system)
Policy EvaluationPayment Reform
Work Force
Major problems?
VS.
Problem: Patients getting treatment A and B are different
Evidence supporting policy change
What we know: Outcomes in Medicare patients improved after the national coverage decision
What we don’t know:
Improvements due to secular trends or the policy itself?
Change over time
Toolbox: Health economics, econometrics, natural experiments, health policy
Differences-in-differences technique
Isolates the impact of a policy change on outcomes above and beyond any changes to a control group
Policy implementation
After Before
Out
com
es
Time
Expected Improvement
A B
Impact of policy change
Control
Intervention
Toolbox: Health economics, econometrics, natural experiments, health policy
Difference-in-difference methods – Econometric techniques for “subtracting out” effect of time Instrumental variables analysis – Econometric technique for isolating causal effects of interventions
Evaluating Disease Management (Patient-Level)
Healthcare Delivery System (Macro-systems)
Understanding Local Provision of Care (Micro-systems)
Patient-Centered Outcomes Comparative Effectiveness
Data Synthesis
Quality Improvement Patterns of Care Analyses
Patient Safety Policy Evaluation Payment Reform
Work Force
Coverage Decisions Technology Adoption
Cost Effectiveness
Implementation Science
Quality Measurement
Major theme Research areas Research tools Resources
Evaluation of disease management (patient level)
• Comparative effectiveness
• Patient centered outcomes
• Data synthesis
• Randomized trials • Pragmatic trials • Advanced
observational study designs
Understanding local health care systems (micro-systems)
• Quality measurement and improvement
• Patterns of Care Analyses
• Patient Safety
• Qualitative research methods
• Human factors theory
• Process engineering
Understanding influence of healthcare delivery system (macro-systems)
• Policy Evaluation • Payment Reform • Surgical work force
• Large database analyses
• Econometric methods
• Health care policy