Sifting Through theTranslational Toolbox

Ralph Gonzales, MD, MSPHProfessor of Medicine; Epidemiology & Biostatistics

13 May 2008

Where Do “Tools” Fit In T2?--Taxonomy

• Conceptual Framework– Understanding behaviors

• Theoretical Approach– Determining intervention targets

• Intervention Implementation Strategy– Determining intervention components (tools)

• Program Evaluation

• Analytical Design

Where Do “Tools” Fit In T2?--NIH T2 Grant

A. Specific AimsB. Background; Rationale; Significance

– Needs Assessment– Conceptual Framework

C. Preliminary Studies– Formative Research

D. Research Methods– Theoretical Approach– Implementation Strategy & Tools– Program Evaluation– Analytical Design

E. Human Subjects

The Translational Toolbox-individual behavior change targetsCommunity

• Health fairs• Mass media• Educational

outreach• Health

Coaches• Insurance

Category Key Knowledge Enablement Prof. Service Incentives

The Translational Toolbox-individual behavior change targetsCommunity

• Health fairs• Mass media• Educational

outreach• Health

Coaches• Insurance

Patient• Education

– Printed– Computer– Internet– Video/multi-media

• Decision Aids• Disease

management– Coaches– Action plans

• Motivational interviewing

• Copayments• P4P

KeyKnowledgeEnablementProf ServiceIncentives

The Translational Toolbox-individual behavior change targetsCommunity

• Health fairs• Mass media• Educational

outreach• Health

Coaches• Insurance

Patient• Education

– Printed– Computer– Internet– Video/multi-media

• Decision Aids• Disease

management– Coaches– Action plans

• Motivational interviewing

• Copayments• P4P

Physician• Education

– CME– Outreach– Detailing

• Guidelines• Decision

support– Reminders

• Registries• Performance

feedback• P4P• Prior Auth’n

KeyKnowledgeEnablementProf ServiceIncentives

Tools

Provider-Focused• Practice Guidelines• Clinical Decision Support Systems• Audit and Feedback

Patient-Focused• Patient Education• Patient Decision Aids• Reminders

Tool Specs

• What is it?– Cost– Feasibility– Complexity

• Summary of evidence

• Ideal uses– Target behaviors– Target barriers

Practice Guidelines

– The Beginning: AHCPR Guidelines– Currently: Produced by professional societies,

governmental agencies, expert panels– Evidence-based frameworks– Recommended behaviors implicit or explicit

– Conclusion: necessary, but not sufficient• Relate back to transtheoretical model, or cognitive theory

(knowledge/awareness must be present before action)

Practice Guideline Specs• What is it?

– Cost: person-hours– Feasibility: buy-in; participation– Complexity: varies

• Summary of evidence ineffective in isolation

• Ideal uses– Target behaviors single, simple actions– Target barriers knowledge/attitudes

• Conclusion: it’s all about ‘implementation’

• Assemble a multi-disciplinary Panel (1-2 mos)– IM, FP, EM, ID

• Use evidence-based principles to assess evidence (2-3 mos)– AHRQ; ACP-CEAS

• Obtain professional society input and/or endorsement (2-3 mos)– ACP; AAFP; ACEP; IDSA

• Write (and re-write) manuscript/documents (4 months)

5 for the price of 1?• Gonzales R, Bartlett JG, Besser RE, Cooper RJ, Hickner JH, Hoffman JR,

Sande MA. Principles of appropriate antibiotic use for treatment of acute respiratory tract infections in adults: Background, Specific Aims and Methods. Annals of Internal Medicine, 2001;134:479-486.

• Gonzales R, Bartlett JG, Besser RE, Cooper RJ, Hickner JH, Hoffman JR, Sande MA. Principles of appropriate antibiotic use for treatment of uncomplicated acute bronchitis: background. Clinical Practice Guideline, Part 2. Annals of Internal Medicine, 2001;134:521-529.

• Gonzales R, Bartlett JG, Besser RE, Hickner JH, Hoffman JR, Sande MA. Principles of appropriate antibiotic use for treatment of nonspecific upper respiratory tract infections in adults: background. Clinical Practice Guideline, Part 2. Annals of Internal Medicine, 2001;134:490-494.

• Cooper RJ, Hoffman JR, Bartlett JG, Besser RE, Gonzales R, Hickner JH, Sande MA. Principles of appropriate antibiotic use for acute pharyngitis in adults: background. Clinical Practice Guideline, Part 2. Annals of Internal Medicine, 2001;134:509-517.

• Hickner JH, Bartlett JG, Besser RE, Gonzales R, Hoffman JR, Sande MA. Principles of appropriate antibiotic use for acute rhinosinusitis in adults: background. Clinical Practice Guideline, Part 2. Annals of Internal Medicine, 2001;134:498-505.

SUMMARY OF PRINCIPLES

• Don’t prescribe antibiotics for colds & URIs

• Don’t prescribe antibiotics for acute bronchitis when comorbidity is absent

• Limit antibiotics to adults with sinusitis symptoms lasting at least 1 week

• Limit antibiotics to adults with sore throat who have a positive test or clinical screen for strep

Practice Guidelines seem to be most effective…

• for acute care conditions

• when quality of evidence is superior

• when compatible with existing values

• when decision making complexity is low

• when desired performance/behavior is clearly understood

• when new skills or organizational support is not necessary for behavior change

The influence of intervention strategy and organisational factors on practice guideline effectiveness.

Adapted from BMC Health Services Research 2006;6:53

INTERVENTION

Educational Meeting

Educational Material

Consensus Meeting

Reminders

Feedback

Patient-Mediated

Outreach

Opinion Leader

Revision of Prof Roles

Financial

Organisational

SETTING

Inpatient

Outpatient

ORGANISATIONAL EFFECT MODIFIERS

Leadership (Management Support)

Learning Environment (Academic)

Physician Type and Specialty

Local Consensus (Development)

OUTCOMES

-behavioral

-clinical

Effectiveness of Specific Intervention Components

BMC Health Services Research 2006;6:53

Effect Modifiers of CPG Implementation Strategies

• Readiness to change– time in practice; age

– perception of a gap between current and optimal practices

– motivation

• The “Messenger”– opinion leader; colleagues

• “Practice enabling” strategies– information systems

– team building/support staff

– standing orders

– computerized medical records

• Reinforcements– reminders; profiling

– financial incentives

– liability

SUMMARYCPG Interventions

• Development– identify clinician knowledge and behavior gaps– identify barriers to change– evidence-based “best practice”– quantify benefit of CPG compliance on system, practice and

patient– local input & endorsement

• Implementation– opinion leader; clinical champion– point-of-service reminders– feedback/profiling

Clinical Decision Support

Clinical Decision Support SpecsKawamoto K et al. BMJ 2005

• What is it? – “…any electronic or non-electronic system designed

to aid directly in clinical decision making, in which characteristics of individual patients are used to generate patient-specific assessments or recommendations that are presented to clinicians for consideration”.

– Manual or computer-assisted preventive care– CPOE

• Cost: low-medium if infrastructure in place• Feasibility: depends heavily on IT officer buy-in• Complexity: potential for high complexity

Implementation Options for Clinical Decision Support Systems

Implementation Options for Clinical Decision Support Systems

Results of Meta-Regression of 71 studies. Kawamoto et al. BMJ 2005.

Clinical Decision Support SpecsKawamoto K et al. BMJ 2005

• Summary of evidence:– Automatic provision of support in clinical work-

flow strongly predicts success– Real-time decision support; recommendations

(not just assessments); and use of computers also predict success

– Simple prompts better than advanced systems• Ideal uses

– Target behaviors: management > diagnosis, especially drug-dosing and prevention

– Target barriers: doctors too busy; low priority problem

• Conclusion: key features of CDSS need to make system easy for doctors to use

Audit and Feedback

Audit and Feedback Specs-Jamtvedt G et al. Qual Saf Health Care 2006;15:433-6.

• What is it? – “any summary of clinical performance of

healthcare over a specified period of time”– Profile at individual, group or regional

level• Cost: fairly low depending on data source• Feasibility: not feasible for complex tasks;

ideal for testing, prescribing, referrals, procedures

• Complexity: low; acknowledge limitations of administrative data and inclusion criteria

Colorado Medical Society Joint Data Project

Truman Medical Center

URI Bronchitis Pharyngitis

Pneumonia AECB Other

0

20

40

60

80

All ARIs

EMNet Average Year 1 Truman Year 1

Truman Year 2 EBM Target

0

20

40

60

80

100

URI Bronchitis Pharyngitis AECB

Ant

ibio

tic

Pre

scri

ptio

n R

ate

EMNet Average year 1 Truman year 1Truman year 2 EBM Target

Truman Medical Center

*

URI, Bronchitis, Pharyngitis: excludes COPD, and antibiotic-responsive secondary diagnosesAECB: as 1st diagnosis, or URI/bronchitis 1st diagnosis in patient with PMHx COPD* < 5 visits

*

Audit and Feedback Specs-Jamtvedt G et al. Qual Saf Health Care 2006;15:433-6.

• Summary of evidence:– Alone: mild-to-modest effect– In Combination: modest-to-strong effect

• Ideal uses– Target behaviors: test ordering; prescribing– Target barriers: doctors too busy; low

priority problem

• Conclusion: use in combination with education, outreach, reminders

Public and Patient Education

Consumer Education: Lots of Options!

• type of instructional media– verbal, written, audiotapes, audiovisual, computer-

assisted instruction

• type of learning activity– lecture, discussion, demonstration, practice,

interactive vs. non-interactive

• nature of follow-up– reminders, self-monitoring, support groups, feedback,

reinforcement, written action-plan

• degree of structure– planned instruction vs. unstructured information

• nature of content

Patient Education-Bottom Line

Search Strategy:

<insert disease here> and “patient education” and “randomized clinical trial”

Patient Decision Aids

Patient Decision Aid SpecsO’Connor AM et al. Cochrane Reviews 2003

• What is it? – An adjunct to counseling that

• explains options• clarifies personal values for the benefits vs. harms• guides patients in deliberation and communication

– Decision Quality• Decisions are informed (knowledge; risk perception)• Decisions based on personal values (congruence)’

– Most common conditions… most are web-based:• Breast, prostate and colon cancer screening & treatment• Menopause options• Cardiovascular disease management• Prenatal testing

Effect of a Decision Aid on Knowledge and Treatment Decision Making for Breast Cancer Surgery

Whelan et al. JAMA 2004

Results

t0 +6 mo +12m

Rx C Rx C Rx C

• Knowledge 67 59

• Conflict 1.4 1.6 1.4 1.5 1.5 1.5

• Satisfaction 4.5 4.3 4.5 4.3 4.4 4.4

• Anxiety no diff

• Depression no diff

• BCS 94% 76%

• “offered clear choice” 87% 69%

Patient Decision Aid SpecsO’Connor AM et al. Cochrane Review 2003

• Cost: development… low-medium—person-hours• Feasibility: very feasible• Complexity: potential for high complexity • Summary of evidence:

– Most RCTs measured process/intermediate outcomese (knowledge; realistic expectations; decisional conflict)

• Main effects are on knowledge and realistic expectations, with OR about 1.4-1.6.

• Reductions in decisional conflict appear modest• 5/9 studies showed improvement in satisfaction with decision

• Ideal uses– Target behaviors: health care decisions that depend on

patient preferences for harms/benefits of different options– Target barriers: poor patient knowledge; doctors too busy;

low priority problem• Conclusion:

CASE STUDY 1:Colorado Joint Data Project on

Careful Use of Antibiotics

Clinical Practice Guidelines (local)

+

Performance Feedback (individual)

+/-

Patient Education

CMS Joint Data Project-Community Partners & Collaborators

Key Organizations

Colorado Medical Society

Colorado Clinical Guidelines Collaborative

Colorado Dept of Public Health and Envt

University of Colorado Health Sciences Center

MCOs

Cigna Healthcare of CO

Community Health Plan of the Rockies

HMO Colorado (BCBS)

One Health Plan

PacifiCare CO

Sloans Lake Health Plan

UnitedHealthcare of CO

Intervention Design: Year 1•7 Health Plans representing 1 million covered

lives

•Target Conditions: pharyngitis & bronchitis

•All CMS Physicians (n=2500)

• practice guidelines for acute respiratory illnesses (Colorado Clinical Guidelines Collaborative)

• patient education sheet

•Physicians > 10 visits in MCO data (n=750)

• Individual physician profiles based on aggregated MCO data

Intervention Design: Year 2•All Physicians > 5 visits Winter 1999 (n=750)

• pre/post physician profiles on bronchitis and pharyngitis

• practice guidelines for acute respiratory illnesses (Colorado Clinical Guidelines Collaborative)

Colorado Medical Society Joint Data Project

Are Administrative Data Valid?-Maselli et al, J Clin Epidemiol, 2001.

• Random medical record review of CMS Data Project office visits for acute bronchitis (medical record=“gold standard”)

• Verification of diagnosis (Age 18-64 years; n=497): 79%• Verification of antibiotic prescription for acute bronchitis

Administrative DataMedical Record antibiotic prescription

+ -+ 357

96 - 9 48

sensitivity (95% CI) 79% (75-83%)specificity (95% CI) 84% (81-87%)concordance (95% CI) 79% (75-83%)positive predictive value (95% CI) 98% (97-99%)negative predictive value (95% CI) 33% (29-37%)

Sub-Intervention Design: Year 2•Randomly selected profiled physicians (n=18)

• MCO member households received educational materials (n=14,400) (distributed across participating MCO plans)

• materials production and delivery sponsored by GlaxoSKB and Abbott

Adult Office Visits for Acute Uncomplicated BronchitisCMS Joint Data Project

0

20

40

60

80

100

'98 '99 '00 '98 '99 '00 '98 '99 '00

Adj

uste

d A

ntib

ioti

c R

x R

ate

P=0.4259 P=0.0009

P=0.0037

No Profile Profile Profile + Education

Physician Group**Each year represents a 4 month winter period beginning Nov of that year. 98 is the baseline winter, 99 is the first winter in which profiles were mailed, and 00 is the second year in which profiles were mailed, as well as household patient educational materials to a subset of profiled physicians.

CASE STUDY 1:Colorado Joint Data Project on

Careful Use of AntibioticsClinical Practice Guidelines (local)

+

Performance Feedback (individual)

+/-

Patient Education

CONCLUSIONS

• Guidelines & Feedback do not appear effective without patient education

CASE STUDY 2:IMPAACT Trial

Clinical Practice Guidelines (national)

+

Performance Feedback (group)

+

Patient Education

The IMPAACT TrialR Gonzales – co-PI – AHRQJ Metlay – co-PI – VAMCC Camargo – Co-I – EMNetT MacKenzie (UCHSC)C McCulloch (UCSF)

IMPAACT Intervention Sites

UNM Health Sciences CenterAlbuquerque VAMC

Medical College of GeorgiaAugusta VAMC

Northwestern Memorial Hospital Chicago VAMC

Lincoln Medical CenterBronx VAMC

IMPAACT Multi-Dimensional Intervention Strategy

• Four regions randomized to receive:1. Provider education (practice guidelines)

delivered by local opinion leaders

2. Group audit and feedback

3. Patient education

• Sites provided individualized adaptation of components

0

20

40

60

80

100

URI Bronchitis Pharyngitis AECB

Ant

ibio

tic

Pre

scri

ptio

n R

ate

EMNet Average year 1 Truman year 1Truman year 2 EBM Target

Truman Medical Center

*

URI, Bronchitis, Pharyngitis: excludes COPD, and antibiotic-responsive secondary diagnosesAECB: as 1st diagnosis, or URI/bronchitis 1st diagnosis in patient with PMHx COPD* < 5 visits

*

Patient Education

• Waiting Room Patient Education– Pamphlets/Cards– Informational Kiosk

• Examination Room Materials– Bronchitis Posters

Exam Room Poster

KIOSK

• Waiting room signs directed patients to kiosk

• Patients were encouraged to use kiosk by ED staff

• Rotating messages on screen suggested content

• All text on screen could be heard through speakers

• Bilingual educational printout at end of program

Kiosk Care Plan printout(Spanish and English)

Adjusted Abx Rx Rates for all ARIs

-15

-10

-5

0

5

10

15

Control Sites Intervention Sites

p= .17

% V

isit

s P

res

cri

be

d A

nti

bio

tic

s:

Inte

rve

nti

on

- B

as

eli

ne

Pe

rio

ds

Adjusted Abx Rx Rates for URI/AB

-15

-10

-5

0

5

10

15

Control Sites Intervention Sites

p = .04

% V

isit

s P

resc

rib

ed A

nti

bio

tics

:In

terv

enti

on

- B

asel

ine

Per

iod

s

CASE STUDY 2:IMPAACT Trial

Clinical Practice Guidelines (national)

+

Performance Feedback (group)

+

Patient Education

CONCLUSIONS

• Multidimensional Intervention IS effective at reducing overuse of antibiotics in EDs.

ABx Treatment of URIs/Bronchitis Decreased at Intervention Sites

Metlay et al, Ann Emerg Med, 2007.

SUMMARY

• Uncommon to have any single tool prove >10% effect… thus, use multifaceted implementation strategies

• Guidelines/Knowledge necessary starting point, but rarely sufficient– Nicely augmented by performance feedback, opinion

leaders, and reminders– Consider adding patient education when appropriate

• Decision aids can be very useful, particularly when at point of service/decision making

Effects of Organisational Features on Guideline Impact

BMC Health Services Research 2006;6:53