Changing physician behavior:
an exercise in futility?
Jeremy GrimshawCanada Research Chair in Health
Knowledge Transfer and UptakeDirector, Clinical Epidemiology Program
Ottawa Health Research Institute
Personal perspective
Focus has been on: professional and organizational behavior change. improving technical aspects of care ie how do we
ensure patients get the right (evidence based) treatments at the right time.
populations of physicians (and health care organizations).
Background
Research is consistently producing new findings that may contribute to effective and efficient patient care
Watt observed that 50% reduction in mortality in ischemic heart disease over past 50 years has been due to advances in clinical care and suggested we were entering a post McKeown era where evidence based health services can improve population health
The findings of such research will not change population outcomes unless health services and health care professionals adopt them in practice.
Grimshaw, Ward, Eccles (2001). Oxford Handbook of Public Health.Watt (2002) Lancet
Background Consistent evidence of failure to translate research
findings into clinical practice 30-40% patients do not get treatments of proven
effectiveness 20–25% patients get care that is not needed or
potentially harmful IOM round table identified failure to translate
research findings as one of the major blocks to improved health outcomes
‘Evidence based medicine should be complemented by evidence based implementation’
Schuster, McGlynn, Brook (1998). Milbank Memorial QuarterlyGrol (1997) BMJ
Sung et al (2003). JAMA
Grol R (2001). Med Care
Background
Local health care organisations have relatively few resources for implementation activities and decision makers need to consider how best to use these to maximise benefits
Dissemination and implementation strategies are not without costs.
In some circumstances, the costs of development and introduction are likely to outweigh their potential benefits. In other circumstances, it may be more efficient to adopt less costly but less effective dissemination and implementation strategies.
Background
If decision makers are to make decisions aboutimplementation strategies informed by research evidence, they need information on: likely effectiveness of interventions (direction of
effect, predicted effect size of intervention and relative effectiveness of different interventions);
effect modifiers (practice environment, potential adopter, characteristics of behaviour);
and, resources need to deliver interventions.
Rigorous evaluations (mainly randomised controlled trials) provide the best evidence of effectiveness of different interventions because: Effects of interventions are modest Limited understanding of likely confounders Substantial opportunity costs if ineffective or
inefficient dissemination and implementation strategies used
Eccles (2003) Quality and Safety in Health Care
Background
Background
Systematic reviews of rigorous evaluations of implementation interventions should inform decisions because they: identify and summarise evidence on the
effectiveness of interventions allow the generalisability and consistency of
research findings to be assessed allow exploration of data inconsistencies to be
explored. Mulrow (1994) British Medical Journal
Grimshaw (2003) Quality and Safety in Health Care
Overview of previous systematic reviews
Generally ineffective
Mixed effects Generally effective
Dissemination of printed educational materials
Audit and feedback Reminders
Didactic educational session
Local opinion leaders Educational outreach
Multifaceted interventions
Bero et al (1998). BMJ
Grimshaw et al (2002). Medical Care
Problems with previous systematic reviews
Use of vote counting techniques: Problems handling studies where statistical significance
of comparison is uncertain (eg studies with unit of analysis errors)
Don’t provide any estimate of effect size Inadequate consideration of quality of primary
studies Conflation of multifaceted interventions into single
intervention category eg educational outreach
Systematic review of guideline dissemination and implementation strategies
Objective Systematic review of the effectiveness and
efficiency of guideline dissemination and implementation strategies to promote improved professional practice.
Systematic review of guideline dissemination and implementation strategies
Inclusion criteria Study designs – RCTs, CCTs, CBAs, ITS Participants – medically qualified health care
professionals Interventions – guideline dissemination and
implementation strategies Outcomes – objective measures of provider behaviour
and/or patient outcome
Search strategy Cochrane Effective Practice and Organisation of Care
group’s search strategy
Results – included studies
Search strategy identified approx 150,000 hits
5,000 hits identified as potentially relevant Full text 863 reports retrieved Included 285 reports of 235 studies, yielding
309 separate comparisons
Results – methodological quality
Overall methodological quality poor (eg unit of analysis errors common)
Poor description of interventions Only 27% of studies used theories and/or
psychological constructs 29.4% comparisons reported any economic
data
Systematic review of guideline dissemination and implementation strategies
Intervention Number of CRCTs
Median effect size
Range
Educational materials
5 +8.1% +3.6%, +17.0%
Audit and feedback
5 +7.0% +1.3%, +16.0%
Reminders 14 +14.1% –1.0%, +34.0%
Results – single interventions
Systematic review of guideline dissemination and implementation strategies
16 28 46 63 56 N =
Absolute effect size
Number of interventions in treatment group
>4 4 3 2 1
80%
60%
40%
20%
0%
-20%
-40%
-60%
-80%
Systematic review of guideline dissemination and implementation strategies
Multifaceted interventions including educational outreach13 RCT
Median effect +6.0% (range –4% to +17.4%)
Systematic review of guideline dissemination and implementation strategies
Conclusions Imperfect evidence base for decision makers Many current rigorous evaluations have
methodological weaknesses (eg unit of analysis errors)
Poor reporting of study settings, barriers to change, content and rationale of intervention
Generalisability of study findings is frequently uncertain
Systematic review of guideline dissemination and implementation strategies
Conclusions Improvements in direction of effect in 86% of
comparisons Reminders most consistently observed to be effective Educational outreach only led to modest effects Dissemination of educational materials may lead to
modest but potentially important effects (similar effects to more intensive interventions)
Multifaceted interventions not necessarily more effective than single interventions
UK MRC Framework for Evaluating Complex Interventions
Theory Modelling Exploratory trial Definitive RCT Long termimplementation
Pre-clinical
Phase I
Phase II
Phase III
Phase IV
Continuum of increasing evidence
Innovation
Potential Adopters
Practice Environment
Interventions
Adoption
Outcomes
Ottawa Model Of Research Use
Assess + Monitor + Evaluatebarriers & supports interventions outcomes & degree of use
Copyright, Logan, J. & Graham, ID 2003
Towards evidence based practice
Ferlie and Shortell suggested four levels at which interventions to improve the quality of health care might operate:
the individual health professional; health care groups or teams; organisations providing health care (e.g., NHS
trusts); the larger health care system or environment in
which individual organizations are embedded. Ferlie, Shortell (2001). Milbank Quarterly
Towards evidence based practice
Levels of engagement Policy
Macro (national/provincial) Meso (organisational)
Managerial Provider Consumer
Towards evidence based practice
Most clinical care occurs within the context of a provider – patient dyad.
Interventions at other levels are mediated through the provider – patient dyad.
Provider behaviour is most proximal determinants of evidence based practice.
Changing provider behaviour is (one of) the right objective(s) to promote evidence based behaviour.
Implications for KU research
Further research is required to develop a better theoretical understanding of professional behavior change by exploring determinants of provider and organisational behaviour to better identify modifiable and non modifiable factors Incremental predictive or action orientated models Emphasis should be on developing standard
methods for operationalising theories and then testing their utility
Implications for KU research
Further research is required to estimate the effectiveness and efficiency of dissemination and implementation strategies in the presence of different barriers and effect modifiers. Rigorous designs (pragmatic RCTs) Maximise informativeness by exploring causal
mechanisms (based on theoretical constructs) and economic evaluations
Implications for KU research
Further research is required to develop practical methods of identifying barriers and facilitators to change outside an academic context.
Implications for KU research
Barriers to progress Lack of consensus around overarching
frameworks and candidate theories Measurement issues Lack of interdisciplinary approach Lack of programmatic funding and
approaches Lack of cohesion in KU research community
Summary
KU research is about saving lives, improving health outcomes and enhancing the quality of health services.
The good news – changing physician behavior is possible though current efforts only achieve modest effects.
I am currently more optimistic than ever that with concerted research effort within increasingly cohesive research community, it should be possible to improve incrementally KU.