Some Principles of Evidence-based Medicine
William J. Powers, M.D.
Financial Disclosures for William J. Powers, M.D.
Salary and Research Support: National Institutes of HealthWashington University School of
Medicine Barnes-Jewish Hospital
No personal financial relationships with any pharmaceutical or medical device companies including consulting, lecture fees, honoraria, gifts, licensing revenues, equity interests or other compensation
Users' Guides to the Medical Literature: XXV. Evidence-Based Medicine: Principles for Applying
the Users' Guides to Patient Care.
Guyatt, et al: JAMA 2000 284:1290-1296Guyatt, et al: JAMA 2000 284:1290-1296
Hierarchy of Clinical Evidence
In the N of 1 RCT, Patients undertake pairs of treatment periods in which they receive a target treatment in 1 period and a placebo or alternative in the other. Patients and clinicians are blind to allocation, The order of the target and control are randomizedPatients make quantitative ratings of their symptoms during each period N of 1 RCTs can provide definitive evidence of treatment effectiveness in individual patients, and may lead to long-term differences in treatment administration.
N of 1 RCTs are unsuitable for short-term problems; therapies that cure therapies that act over long periods of time or prevent rare or unique events (such as stroke, myocardial infarction, or death)
N of 1 RCTN of 1 RCT
Guyatt, et al: JAMA 2000 284:1290-1296Guyatt, et al: JAMA 2000 284:1290-1296
When considering any source of evidence about treatment other than N of 1 RCTs, clinicians are generalizing from results in other people to their patients, inevitably weakening inferences about treatment impact and introducing complex issues of how trial results apply to individuals.
Inferences may nevertheless be strong if results come from a systematic review of methodologically strong RCTs with consistent results and are generally somewhat weaker if we are dealing with only a single RCT unless it is large and has enrolled a diverse patient population.
Users' Guides to the Medical Literature: XXV. Evidence-Based Medicine: Principles for Applying
the Users' Guides to Patient Care.
Guyatt, et al: JAMA 2000 284:1290-1296Guyatt, et al: JAMA 2000 284:1290-1296
Meta-analysis
• Identification of Trials• Definition of Outcomes• 2 x 2 outcome tables summed over
all trials• Calculation of treatment effects• Statistical significance
An
nu
alize
d S
troke R
ate
900-1300 mg/day ASA 150-300 mg/day DPM
p=NS
Secondary Prevention of Stroke with High Dose Aspirin and Dipyridamole
Meta-analysis of 1574 patients from Three Trials*
Secondary Prevention of Stroke with High Dose Aspirin and Dipyridamole
Meta-analysis of 1574 patients from Three Trials*
*Rev Neurol 1082; 138:367-385 Stroke 1983; 14:5-14 Stroke 1985; 16:406-415
European Stroke Prevention Study 2Low Dose Aspirin plus Extended Release Dipyridamole
(J Neurol Sci 1996; 143:1-13)
01
2
3
4
5
67
8
9
10 Placebo
ASA
DPM
ASA+DPM
An
nu
alized
Str
oke R
ate
* ***
* Better than placebo
** Better than other 3 groups
ASA 50 mg/dayDPM 400ER mg/day
Secondary Prevention of Stroke with Aspirin and Dipyridamole
Meta-analysis of 4873 patients from Four Trials with Any Dose of Aspirin
Diener HC: Neurology1998; 51(Suppl 3):S17-S19
Secondary Prevention of Vascular Events in High Risk Patients
Aspirin vs. Dipyridamole and Aspirin
0
5
10
15
20
25
Aspirin
ASA+DPM
Perc
en
t of
Vascu
lar
Even
ts
p=NS
Meta-analysis of 25 Trials
Users' Guides to the Medical Literature: II. How to Use an Article About Therapy or Prevention:
A. Are the Results of the Study Valid?
Guyatt, et al: JAMA 1993; 270:2598-2601
Why is the intent-to-treat principle so important?
• It mimics the real life decisionAt diagnosis, you have to decide what treatment to try
• It accounts for sicker patients going off protocolBenefits can be inflated by non-responders dropping out
• It accounts for side effectsTreatment A for cancer yields a 90% 5-year survival but
only 10% survive treatmentTreatment B yields a 75% survival and all survive treatment
Measuring the Effects of Therapy
Guyatt, et al: JAMA.1994; 271:59-63
Comparison of Aspirin with other Anti-platelet Agents for the Secondary Prevention of Stroke:
Absolute Risks & Risk ReductionA
nn
ualiz
ed
Str
oke
Rate
(%
)
25
20
15
10
5
0ESPS2 TASS
25
20
15
10
5
0ESPS2 TASS
Aspirin Agent Relative Risk Reduction
Absolute Risk Reduction
Comparison of Aspirin with other Anti-platelet Agents for the Secondary Prevention of
Stroke: Prognosis
0
10
2030
40
50
6070
80
90
100
ESPS2 TASS
AspirinAgent
% W
ith
ou
t S
troke
At
On
e Y
ear
The solid line represents the confidence interval around the first example in which there were 100 patients per group and the number of events in the active and control groups were two and four, respectively. The broken line represents the confidence interval around the second example in which there were 1000 patients per group and the number of events in the active and control groups were 20 and 40, respectively.
From: Guyatt: JAMA 1994: 271:59-63
Confidence Intervals for Risk Reduction
Warfarin-Aspirin Recurrent Stroke Study(WARSS)
(Mohr et al: NEJM 2001; 345:1444-1451)
PRIMARY RESULTSAlthough this trial demonstrated no
significant difference between warfarin and aspirin, it did not establish
equivalence Hazard ratio 1.13 95% CI 0.92-1.38
This means that there is less than a 5% chance
that warfarin is more than 8% better than aspirin
HAESTLancet 2000; 355:1205-1210
HAESTLancet 2000; 355:1205-1210
Recurrent ischemic stroke within 14 days in 449 patients with acute (< 30 hrs) ischemic stroke who had atrial fibrillation within 2 years (89%
present on admission)
Dalteparin 19/224 (8.5%)Aspirin 17/225 (7.5%)
p=.73
OR(95% CI) = 1.13 (.57 to 2.24)
Number Needed to Treat
The number needed to treat is 1 divided by the absolute
risk reduction
Number Needed to Treat to Prevent 1 Stroke at 2 Years After Carotid Endarterectomy
Gorelick PB: Stroke. 1999; 30:1745-50.
Study
% Stenosi
sNNT
Symptomatic NASCET 70–99 8ECST 70–99 8NASCET 50–69 20NASCET <50 67
Asymptomatic
Veterans Affairs 48ACAS 83
Economic Analysis
• Cost-effectiveness analysis - health outcomes are not valued, but reported in physical units such as life years gained or cases successfully treated.
• Cost-utility analysis - outcomes of different types are weighted to produce a composite index, such as the quality-adjusted life year (QALY) or healthy years equivalent.
• Cost-benefit analyses - the health consequences are valued by asking health care consumers what they would be willing to pay for health services that achieve combinations of outcomes of particular types.
Drummond, et al:. JAMA 1997; 277:1552-1557Drummond, et al:. JAMA 1997; 277:1552-1557
Cost Utility of PET Selection for EC/IC Bypass Surgery for Symptomatic Carotid Occlusion
Derdeyn et al, J Nucl Med 2000; 41:800-807
If the Carotid Occlusion Surgery Study hypothesis is proven true, EC/IC bypass surgery will improve outcome and reduce cost:
49 QALY/10 yrs/ 100 patients cost savings of $11,000 per patient
The Important Issue of the Primary Endpoint
The Primary Endpoint should be an event that is clinically important to
patients such as death, stroke or myocardial infarction
The Important Issue of the Primary Endpoint
• The choice of the Primary Endpoint will affect the size and duration of the study• Sample size is determined by the number of
events in the control group and the projected treatment effect
• The choice of the Primary Endpoint will affect the outcome of the study • All components of the primary endpoint are
weighted equally• Serious Adverse Events NOT included in Primary
Endpoint are not included in the primary analysis
Comparison of Stenting with Minimally Invasive Bypass Surgery for Stenosis of the Left Anterior Descending
Coronary Artery NEJM 2002; 347:561-566
Stenting Group
(N=108)
Surgical Group (N=106) P
Cardiac Death 0 2 .99
Acute MI 3 5 .68
Revascularization 31 9 .003
Primary Endpoint (All)
34 16 .02
Secondary Prevention of Stroke with Aspirin and Dipyridamole ESPS2
0
5
10
15
20
2 y
ear
Str
oke R
ate
(%
)
Bleeding
Tissue Plasminogen Activator for Acute Ischemic Stroke (NINDS)
NEJM 1995; 333:1581-1587; Stroke 1997; 28:2109-2118
Outcome at 3 Months in 624 Patients
Tissue Plasminogen Activator for Acute Ischemic Stroke (NINDS)
NEJM 1995; 333:1581-1587; Stroke 1997; 28:2109-2118
Outcome at 3 Months in 624 Patients
0102030405060708090100
rt-PA Placebo rt-PA Placebo rt-PA Placebo
p < .001
GoodRankin 0-1
PoorRankin 2- 5
Dead
SymptomaticHemorrhage
Observational studies may provide compelling evidence if untreated outcomes are consistent and treatment effects are sufficiently large and consistent. For instance , observational studies have allowed extremely strong inferences about the efficacy of insulin in diabetic ketoacidosis or penicillin for pneumococcal meningitis, both of which are almost uniformly fatal if untreated.
However, when outcome is variable and, especially, when the poor outcome occurs in only a small proportion of patients, evidence derived only from non-randomized observational studies should be regarded with extreme caution.
Observational Studies
Divergent Data on Post-Menopausal Hormone TherapyNEJM 2003; 348:645-650
Users' Guides to the Medical Literature: XXV. Evidence-Based Medicine: Principles for Applying
the Users' Guides to Patient Care.
Users' Guides to the Medical Literature: XXV. Evidence-Based Medicine: Principles for Applying
the Users' Guides to Patient Care.
Guyatt, et al: JAMA 2000; 284:1290-1296
Physiologic studies and unsystematic clinical observations provide the weakest inferences about treatment effects.
Problems in the "Evidence" of
"Evidence-Based Medicine" Feinstein AR & Horwitz RI: Am J Med 1997;103:529-535
• The data do not include many types of treatments or patients seen in clinical practice.
• The results show comparative efficacy of treatment for an "average" randomized patient, not for pertinent subgroups formed by such cogent clinical features as severity of symptoms, illness, co-morbidity, and other clinical nuances.
The laudable goal of making clinical decisions based on evidence can be impaired by the restricted quality and scope of the "best available evidence." The authoritative aura, however, may lead to major abuses that produce inappropriate guidelines or doctrinaire dogmas for clinical practice.
0
5
10
15
Primary Prevention Secondary Prevention
Aspirin
OAC
% P
rim
ary
End
poin
tp
er
year
Extrapolating from Primary Prevention to Secondary Prevention
Oral Anticoagulation vs. Aspirin in Atrial Fibrillation
AFASAK-1 Lancet 1989; 1:175-178; European Atrial Fibrillation Trial Lancet 1993; 342: 1255-1262
Extrapolating from Primary Prevention to Secondary PreventionAspirin for Primary Stroke Prevention
Non-fatal Stroke per 10,000 subject-years
Aspirin Control Odds Ratio
British1 35.6 31.7 1.12
US2 20.1 16.8 1.20
1 BMJ 1988; 296:313-3162 NEJM 1989; 321:129-135
Extrapolating from Primary Prevention to Secondary Prevention
Antiplatelet Therapy for Secondary Stroke Prevention1
Non-fatal Stroke
Antiplatelet Control Odds Ratio
8.3% 10.8% .77*
* p< .0001
1 BMJ 2002; 324:71-86
The Problem of Subgroup AnalysisTOAST Primary Analysis
JAMA 1998;279:1265
• 1281 patients within 24 hrs of ischemic stroke
• Dose-adjusted danaparoid IV vs. placebo x 7d
• Primary endpoint: favorable functional outcome (GOS I or II and BI 12-20) at 3 months
Favorable outcome
Danaparoid 75.2%Placebo 73.7%
p = .49OR 1.09 (0.85 – 1.41)
TOASTTOAST
Stroke Subtype Analysis
5 subtypes analyzed for two different endpoints in addition to 4 analyses for total group = 14 analyses
p < .05/14 = p< .0036
Large artery atherosclerosisFavorable outcome p=.04Very favorable outcome p=.02
NO STATISTICALLY VALID SIGNIFICANT DIFFERENCE
Lumping vs. Splitting WARSS
(Mohr et al: NEJM 2001; 345:1444-1451)
Design:Prospective, randomized, double-blind, multi-center clinical trialEligibility:Non-cardioembolic stroke < 30 d, no CEA plannedTreatment:Warfarin (INR 1.4-2.8, mean 2.1) vs aspirin 325 mg/day
Primary endpoint:Recurrent ischemic stroke or death within two years
WARSS (Mohr et al: NEJM 2001; 345:1444-1451)
PRIMARY RESULTS2206 patients followed for 2 years
Ischemic Stroke/ Death Major Hemorrhage /100 pt-yrs
Warfarin 17.8% 2.22Aspirin 16.0% 1.49
p=.25
Hazard ratio 1.13 95% CI 0.92-1.38
Lumping vs. SplittingWASID
(Chimowitz MI, et al: Neurology 1995; 45: 1488-1493)
Design:Retrospective, non-randomized, unblinded, multi-center concurrent cohort study based on angiogram review Eligibility:Symptomatic 50-99% stenosis of major intracranial artery by arteriographyTreatment:MD choice: Warfarin (PT 1.2 –1.6 x control) vs aspirin (usually 325 mg/d)
Primary endpoint:Stroke, MI or sudden death
WASID(Chimowitz MI, et al: Neurology 1995; 45: 1488-1493)
RESULTS OAC ASA
Number of Patients 88 63Median follow-up (mo) 14.7
19.3 Primary Endpoint 14 26
p < 0.01
Hazard ratio 0.46 95% CI 0.23-0.86
In addition to the strength of the evidence, the following considerations will bear on the strength of the treatment recommendation:
-the magnitude and precision of the treatment effect -patients’ risk of the target event being prevented
-the nature of the benefit-the magnitude of the risk associated with
treatment -variability in patient preferences
-variability in regional resource availability -cost considerations
Translating Evidence into Recommendations(Guyatt, et al: Chest 2001; 119:3S-7S)
Issues Examples
Less serious outcomePreventing postphlebitic syndrome with thrombolytic therapy in DVT rather than death from PE.
Smaller treatment effect
Clopidogrel vs aspirin leads to a smaller stroke reduction in TIA (8.7% RRR) than anticoagulation vs placebo in AF (68% RRR).
Imprecise estimate of treatment effect
Aspirin vs placebo in AF has a wider CI than aspirin for stroke prevention in patients with TIA.
Lower risk of target event
Some surgical patients are at very low risk of postoperative DVT and PE, while others surgical patients have considerably higher rates of DVT and PE.
Higher risk of therapy Warfarin has a much higher risk of serious hemorrhage than aspirin.
Higher costs tPA has much higher cost than streptokinase in acute MI.
Varying values
Most young, healthy people will put a high value on prolonging their lives (and thus incur suffering to do so); the elderly and infirm are likely to vary in the value they place on prolonging their lives (and may vary in the suffering they are ready to experience to do so).
Translating Evidence into Recommendations Factors That May Weaken a Recommendation to Treat
(Guyatt, et al: Chest 2001; 119:3S-7S)
Further Reading on Evidence Based Medicine
Users' Guides to the Medical Literature: Essentials of Evidence-Based Clinical Practice edited by Gordon Guyatt and Drummond Rennie, includes CD-ROM, 442 pp, soft cover, $34.95, ISBN 1-57947-191-9, Chicago, Ill, AMA Press, 2002.
Interpreting the Medical Literatureby Stephen H. Gehlbach, 4th ed, 296 pp, paper, $34.95, ISBN 0-07-138762-5, New York, NY, McGraw-Hill, 2002.
Evidence-Based Medicine: How to Practice and Teach EBMby David L. Sackett, Sharon E. Straus, W. Scott Richardson, William Rosenberg, and R. Brian Haynes, 2nd ed, with CD-ROM, 261 pp, softcover, $43.95, ISBN 0-443-06240-4, Philadelphia, Pa, Churchill Livingstone, 2000.