Applying Implementation Science to Enhance the Impact of Health Services Research

Brian S. Mittman, PhDSenior Social Scientist

VA/UCLA/RAND Center for the Study ofHealthcare Provider Behavior

June 5, 2007

Center for the Study of Healthcare Provider Behavior Quality

Enhancement Research Initiative

Outline

• What is implementation science?

– Definition, scope, goals

– Disciplinary foundations: theoretical, methodological

• How does IS relate to HSR? How can IS enhance HSR?

• What insights, conclusions and guidance does IS offer? How does one plan, conduct, document and use IS?

• What is needed to enhance IS (to further enhance HSR)?

Implementation science definition

• “Implementation research is the scientific study of methods to promote the systematic uptake of research findings and other evidence-based practices into routine practice, and, hence, to improve the quality and effectiveness of health services and care.”

“This relatively new field includes the study of influences on healthcare professional and organizational behavior.”

Implementation science goals

• Generate new insights and generalizable knowledge regarding dissemination/implementation processes and strategies – and their outcomes and determinants

• Apply, test and refine models, theories and hypotheses

• Improve health-related processes and outcomes within participating settings and groups

• Develop reliable strategies for improving health-related processes and outcomes in other settings and groups

Implementation science and the pipelineIoM Clinical Research Roundtable identified

two “translational roadblocks”

Crowley WF, et al. Clinical Research in the United States at a Crossroads. JAMA. 2004. 291: 1120-1126

IoM CRR translational roadblocks

Basic / Lab Research

Clinical Research

Improved Health

Basic / Lab Research

Clinical Research

Improved Health

Revised depiction of health-relatedresearch / implementation pipeline

Translational,Pre-ClinicalResearch

Basic Science

Improved HealthProcesses,Outcomes

Implemen-tationResearch

Clinical

Health Behavior

Health Services

Revised depiction of health-relatedresearch / implementation pipeline

Translational,Pre-ClinicalResearch

Basic Science

Improved HealthProcesses,Outcomes

Implemen-tationResearch

Clinical

Health Behavior

Basic/Lab Science

Clinical Science

Health Services Research

Health Services

Health Behavior/Promotion Research

Developing, implementing effective practices

• Effective (evidence-based) practices are developed via clinical research, health behavior/promotion research, health services research

• HSR also studies attributes, use and impacts of effective clinical and health behavior/promotion practices

• Implementation research studies strategies and processes related to adoption/update (dissemination, implementation) of effective practices

• Not all research findings are suitable for implementation

Implementation science subfields, labels• Knowledge utilization

• Knowledge translation

• Diffusion research

• Research utilization

• Research translation

• Technology transfer

• Taking to scale

• Knowledge brokering

• Translational research

• Dissemination/implementation research

Implementation science foundations: theory, conceptual frameworks

• Psychology, sociology, political science, economics

• Health psychology, organization/management theory

• Program evaluation, clinical research

Implementation science foundations: research approaches and frameworks

• Clinical research: RCT, PCT frameworks

• Program evaluation: logic models, designs, measurement

• Health behavior/health promotion research: PBT, etc.

• Pipeline and phase models: IoM CRR, drug trials, complex interventions

• CONSORT-like statements, templates, criteria

Implementation theories and principles

• Organization/management theory vs. psychological theories

– Systems, structures, policies/procedures, norms vs. attitudes and intentions

– Management principles and frameworks

• Multi-level framework (hypothesis)

– Multi-faceted portfolio addressing all levels

– Coordinated campaign

Implementation research frameworks

1. QUERI Six-Step Process

2. QUERI 4-phase implementation research framework

3. QUERI Service-Directed Project template: elements for an implementation research “CONSORT extension”

• To enhance the quality, outcomes and efficiency of VA health care by systematically implementing evidence-based clinical guidelines and innovations into routine clinical practice

QUERI’s mission

Nine QUERI coordinating centers

• Chronic Heart Failure

• Diabetes• HIV/AIDS• Ischemic Heart

Disease

• Mental Health• Polytrauma• Spinal Cord Injury• Stroke• Substance Use Disorder

Framework 1:The Six-Step QUERI Process

1. Identify high risk/high burden conditions

2. Identify best practices

3. Define existing practice patterns in VA and variations from best practices

4. Identify (or develop) and implement programs to promote best practices

5. Document outcome and system improvements

6. Document improvements in health related quality of life

QUERI’s research/implementation pipeline

Implementation Research

Steps 4/5/6

Clinical, Health Behavior Research / Guideline Development

Mainstream Health Services Research

Step 3

Step 1

Step M

Step 2Step C

Improved processes, outcomes

Document/diagnose current practices and quality gaps

Demonstration /evaluation projects

Clinical evidence (effective practices)

Expanded QUERI Six-Step Process

Step 1: Select Diseases/Conditions/Patient Populations1A. Identify and prioritize (via a formal ranking procedure) high

risk/high burden clinical conditions

1B. Identify high priority clinical practices/outcomes within a selected condition

Step 2: Identify Evidence-Based Guidelines/Recommendations 2A. Identify evidence-based clinical practice guidelines

2B. Identify evidence-based clinical recommendations

Expanded QUERI Six-Step Process

Step 3: Measure and Diagnose Quality/Performance Gaps 3A. Measure existing practice patterns and outcomes across VHA

and identify variations from evidence-based practices (quality, outcome and performance gaps)

3B. Identify determinants of current practices; diagnose quality gaps

3C. Identify barriers and facilitators to improvement

(Step 3 activities are performed first in representative sites and then in Step 4/5/6 sites.)

Expanded QUERI Six-Step ProcessStep 4: Implement Improvement Programs

4A. Identify quality improvement strategies, programs and program components or tools to address quality gaps(e.g., via literature reviews)

4B. Develop or adapt quality improvement strategies, programs, program components or tools (e.g., educational resources, decision support tools)

4C. Implement quality improvement strategies and programs

Step 5/6: Evaluate Improvement Programs 5. Assess improvement program feasibility, implementation and

impacts on patient, family and system outcomes

6. Assess improvement program impacts on health related quality of life (HRQOL)

Expanded QUERI Six-Step Process

Step M: Develop Measures, Methods and Data Resources M1. Develop and/or evaluate patient registries, cohort databases,

data warehouses

M2. Develop and/or evaluate case-finding or screening tools

M3. Develop and/or evaluate structure, process or outcome measures

M4. Develop and/or evaluate organizational structure/system, clinical practice, utilization or outcome databases

Step C: Develop Clinical Evidence, Effective PracticesC1. Develop evidence-based clinical interventions, recommendations

(clinical research)

C2. Develop evidence-based health promotion/prevention programs (health behavior/health promotion research)

C3. Develop evidence-based health services interventions (health services research)

QUERI’s research/implementation pipeline

Implementation Research

Steps 4/5/6

Clinical, Health Behavior Research / Guideline Development

Mainstream Health Services Research

Step 3

Step 1Step M

Step 2Step C

Improved processes, outcomes

Phase 1 Pilot Projects

Phase 1 Pilot Projects

Phase 2 Small-Scale

Demonstrations

Phase 2 Small-Scale

Demonstrations

Phase 3Regional

Demonstrations

Phase 3Regional

Demonstrations

Phase 4“National Rollout”

Framework 2: QUERI four-phase implementation research framework

Phase Study Type Form of Evaluation

Pre-trial Design Conceptual design of implementation program and underlying program (logic) model from theory, prior empirical research

Phase 1 Pilot / Pilot test, assess feasibility, formative evaluation and refinement,Formative develop intervention/evaluation protocols

Phase 2 Efficacy Small-scale rigorous trial in controlled settings with ongoingintervention support

Phase 3 Effectiveness Large-scale rigorous trial under routine conditions in variedsettings

Phase 4 Monitoring Ongoing monitoring and feedback

Framework 3:QUERI Service-Directed Project template

Motivation

• implementation projects are hybrid research/practice initiatives

• implementation projects require a unique set of design features, methods, skills and competencies

Goal

• provide guidance in designing, conducting and documenting implementation projects

QUERI Service-Directed Project Template

A. Specific aims– Implementation aims: short- and long-term (phases)– Science aims: theory/models, empirical insights– Hypotheses: intervention impacts, influences

B. Background and significance– Clinical issue: morbidity, mortality, burden– Clinical evidence (effective practice): strength,

acceptance, implementation gaps (magnitude, potential to close)

– Implementation processes, knowledge

QUERI SDP Template

C. Previous studies

– Current practices: determinants, barriers and facilitators to change

– Implementation strategy: theory, empirical evidence base and status (phase)

QUERI SDP Template

D. Design and methods– Intervention overview (conceptual framework, approach)– Intervention details (components, operational details):

• Legitimacy of evidence• Motivation for change, external expectations,

pressure• Norms (organizational, professional, consumer)• Clinician, staff, consumer education and skills• Financial, administrative, logistical facilitation• Organizational redesign, as needed• Tailoring and adaptation• Monitoring and refinement

QUERI SDP TemplateD. Design and methods (continued)

– Evaluation details

• Experimental design• Usual care condition• Randomization protocol

• Diagnostic analysis: practice determinants; barriers and facilitators to change

• Formative evaluation: overall plan, details• Impact (summative) evaluation: overall plan• Process evaluation

• Project sites, site recruitment• Subjects, subject recruitment

QUERI SDP Template

D. Design and methods (continued)– Evaluation details, impact evaluation

• Outcomes (clinical practices, patient outcomes, system outcomes): variables, measures, data collection protocols

• Contextual factors: variables, measures, data collection protocols

• Analysis plans and methods; interpretation (research, policy, practice)

QUERI SDP Template

D. Design and methods (continued)

– Evaluation details, process evaluation

• Identify mechanisms of impact and measures: variables, measures, data collection protocols

• Influences on mechanisms: variables, measures, data collection protocols

• Analysis plans and methods

– Evaluation details, other• Sustainability (phase 3)• Cost (phase 3)

QUERI SDP Template

D. Design and methods (continued)– Management plan

• Intervention management plan, team• Evaluation management plan, team• Staff qualifications: intervention, evaluation

Implementation research resources

• Northstar, RE-AIM, UK MRC, etc.• Phase models, efficacy/effectiveness studies• Green/Glasgow external validity criteria• Experimental/quasi-experimental designs, cluster RCTs • Stetler et al, formative evaluation, process evaluation• Economic evaluation• CONSORT-like statements, templates• Diffusion, dissemination, implementation theories, models• Qualitative research methods• Publication guidelines (protocols, main and supporting)

Key challengesIncrease quality and quantity of implementation research

Quantity• Steve Woolf’s Breakeven Point: greater benefit from investing in enhanced use of

existing innovations vs. developing new innovations

• Negative relationship between QI success and evaluation rigor: temporal trends, QI paradox

Quality• Modest contribution of QI/implementation research (symptom)

• Inconsistencies in peer reviews, approaches/methods, legitimacy

• Lack of academic programs, standards, journals, conferences