Date post: | 28-Mar-2015 |
Category: |
Documents |
Upload: | mia-parker |
View: | 220 times |
Download: | 0 times |
Systematic Reviews-intro level
Phil WiffenDirector of Training UK Cochrane CentreEditor, Cochrane Collaborative Review Groupin Pain, Palliative & Supportive Care
The rationale
•Evidence based medicine
•What are systematic reviews ?
•How do we find, understand and evaluate systematic reviews?
•Tools to present data
561882563770
4401941305
300946
127279816716506196322831167442649
257169183
308636100
762222898726186
2 4 6 8 10 12 14 16 18
Number needed to treat with 95% Confidence Intervals
Ibuprofen 800 mgIbuprofen 600 mgIbuprofen 400 mgIbuprofen 200 mgIbuprofen 100 mg
Diclofenac 100 mgDiclofenac 50 mgDiclofenac 25 mgNaproxen 440 mgNaproxen 550 mg
Naproxen 220/250 mgParacetamol 1000 mg/Codeine 60 mg
Aspirin 1200 mgParacetamol 600 or 650 mg/Codeine 60 mg
Aspirin 1000 mgAspirin 600/650 mg
Dextropropoxyphene HCl 65mg/Paracetamol 650 mgParacetamol 1000 mg
Paracetamol 600/650 mgParacetamol 300 mg/Codeine 30 mg
Paracetamol 500 mgPethidine 100 mg (intramuscular)
Morphine 10 mg (intramuscular)Dextropropoxyphene HCl 65mg
Dihydrocodeine 30 mgCodeine 60 mg
Tramadol 150 mgTramadol 100 mg
Tramadol 75 mgTramadol 50 mg
Total Numberin Comparison
League table of NNTs to produce at least 50% pain reliefover 4-6 hours compared to placebo in pain of
moderate or severe intensity
What evidence-based medicine is:
Evidence-based medicine is the conscientious, explicit and judicious use of current best evidence in making decisions about the care of individual patients.
Sackett (BMJ 1996; 312: 71-2)
What evidence-based medicine is:
The practice of EBM requires the integration of
• individual clinical expertise with the
• best available external clinical evidence from systematic research.
Another definition of EBM
Evidence based medicine is an approach to health care that promotes the collection,
interpretation and integration of valid, important and applicable patient reported, clinician observed and research derived evidence. The best available evidence,
moderated by patient circumstances and preferences, is applied to improve the
quality of clinical judgements.
McKibbon KA et al ‘The medical literature as a resource for Evidence Based Care’http://hiru.mcmaster.ca/hiru/medline/mdl-ebc.htm
“There are perhaps 30000 biomedical journals in the world, and they have grown steadily by 7% a year since the seventeenth century.
Yet about 15% of medical interventions are supported by solid scientific evidence...
...only 1% of the articles in medical journals are scientifically sound”
R. Smith quoting Prof. D. Eddy, BMJ 1991; 303: 798-99
More than 20 000 RCTs have been
published in pain relief research since
1950
“...approximately 17000 new biomedical books are published annually.”
Lowe and Barnett, JAMA 1994; 271: 1103-8
The size of the task
How many biomedical papers are there ?
•Medline 17 million records, 5000 journals, 80 countries
•Embase 14 million records, 7000 journals, 70 countries
•CINAHL 1 million records 2900 journals. 13 languages
•Others: e.g. LILACS ????
April 2009
enthusiastUNBIASED‘good’ RCT
numbers / inclusion / exclusionblindingpower
open ‘biased’ study
local use‘expert’ clinical practice
+systematic reviews
Tools not Rules
I Strong evidence from at least 1 systematic review of multiple well-designed randomised controlled trials
II Strong evidence from at least 1 properly designed randomised controlled trial of appropriate size
III Evidence from well designed trials without randomisation, single group pre-post, cohort, time series or matched case-controlled studies
IV Evidence from well-designed non experimental studies from more than 1 centre or research group
V Opinions of respected authorities, based on clinical evidence, descriptive studies or reports of expert committees
Type & Strength of Evidence
What is a systematic review ?
•Filing Cabinets
•Friends
•Foreigners ?
•The world literature on a subject
Systematic Reviews
“Clinical review articles should be as scientific as the articles they review”
Haynes, BMJ 1992; 304: 330-1
“Clinical review articles should be as scientific as the articles they review”
Haynes, BMJ 1992; 304: 330-1
“ The fundamental difference between a review and a primary study is the unit of analysis, not the scientific principles that apply”
Oxman & Guyatt, CMAJ 1988; 138: 697-703
“ The fundamental difference between a review and a primary study is the unit of analysis, not the scientific principles that apply”
Oxman & Guyatt, CMAJ 1988; 138: 697-703
0. Frame question
1. Search for trials
2. Score for quality
3. Validity of trial results
4. Vote-counting 4. Extract data
qualitative quantitative
5. Meta-analysis
narrative review (overview)
systematic review ± meta-analysis
Finding systematic reviews
• Medline has a filter
Finding systematic reviews
• Cochrane Library
• DARE database on Cochrane Library
Understanding systematic reviews
• Meta-analysis (forest plot)
• NNTs
• L’Abbe plots
• Assessment of bias in the included studies
Ibuprofen 400 mg vs. paracetamol 1000 mg for acute postoperative pain
0
1
2
3
4
5
6
7
-20 -10 0 10 20 30
Cooper et al, 1984
Cooper, 1984
Cooper et al, 1989Schachtel al, 1989
Mehlisch et al, 1990
Overall weighted difference
mean differences and 95% CI( % of the maximum possible TOTPAR value)
favours ibuprofenfavours paracetamol
no difference
difference between the mean effects within the trial and 95% CI
A closer look at a forest plot (the meta-analysis)
Cochrane Database of Systematic ReviewsPublished by John Wiley & Sons, Ltd
There is a label to tellyou what the comparisonis and what the outcomeof interest is
At the bottom there’sa horizontal line. This is the scale measuringthe treatment effect.Here the outcome is ……..
Take care to read whatthe labels say – things tothe left do not always mean the treatment is better than the control.
The vertical line in themiddle is where thetreatment and control have the same effect – there is no differencebetween the two
For each study there is an ID
The data foreach trial are here, divided into the treatment and control groups
This is the % weightgiven to this study in the pooled analysis
Each study is given a blob, placed where the data measure the effect.
The size of the blob is proportional to the % weight The horizontal line is called a confidence interval and is a measure of how we think the result of this study might vary with the play of chance.
The wider the horizontal line is, the less confident we are of the observed effect.
The label above the graph tells you what statistic has been used
The data shown in the graph are also given numerically
The pooled analysis is given a diamond shapewhere the widest bit in the middle is located at the calculated best guess (point estimate), and the horizontal width is the confidence interval
** Note on interpretation **
If the confidence interval crosses the line of no effect, this is equivalent to saying that we have found no statistically significant difference in the effects of the two interventions
Cochrane Database of Systematic ReviewsPublished by John Wiley & Sons, Ltd
Numbers needed to treat (NNTs)
The Number of people who have to be treatedfor ONE to benefit
Number-needed-to-treat (NNT)
Number of patients
Improved = Clinical end point
Actives
Nact
Impact
Controls
Ncon
Impcon
1
Impcon
Ncon
Impact
Nact
-NNT =
Number-needed-to-treat (NNT)
1
0
100
100
100 -NNT =
NNT is treatment specific -takes into account the event rate in controls:• may be a placebo effect• may be the effect of another treatment
active control
improved 80 20
N 100 100
Relative Risk (RR) = (Impact/Nact) / (Impcon/Ncon)
Relative Risk Reduction (RRR) = (1-RR) / 100
Absolute Risk (AR) = (Impact/Nact) - (Impcon/Ncon)
Number Needed to Treat (NNT) = 1/AR
RR = 4; AR = 0.6; NNT = 1.7 (best 1.25)
0
25
50
75
100
0 25 50 75 100
L'Abbé plot for treatment
equal
ity
Treatment betterthan
control
Control betterthan treatment
Proportion improved
with treatment
Proportion improved with control
0
10
20
30
40
50
60
70
80
90
100
0 10 20 30 40 50 60 70 80 90 100
0
100
200
600/650
500
1000
At least 50% pain relief with placebo
At least 50% pain relief with
paracetamol
World literature on paracetamol
Moore et al Pain 1997;70:193
‘Risk of bias’ assessment in Cochrane reviews
Risk of bias summary
• Here ‘Blinding’ and ‘Incomplete outcomes data’ have been assessed for two sets of outcomes
Ad
eq
ua
te s
eq
ue
nce
ge
ne
ratio
n
Barry 1988 +
Baylis 1989 +
Cooper 1987 +
Dodd 1985 +
Goodwin 1986 +
Sanders 1983 +
Allo
ca
tio
n c
on
ce
alm
en
t
-
+
?
?
+
+
Blin
din
g (
Pa
tie
nt-
rep
ort
ed
ou
tco
me
s)
+
+
-
+
+
-
Blin
din
g (
Mo
rta
lity
)
+
+
?
+
+
?
Inco
mp
lete
ou
tco
me
da
ta a
dd
resse
d (
Sh
ort
-te
rm o
utc
om
es (
2-6
wks))
-
?
-
+
+
-
Inco
mp
lete
ou
tco
me
da
ta a
dd
resse
d (
Lo
ng
er-
term
ou
tco
me
s (
> 6
ks))
-
?
-
-
+
-
Fre
e o
f se
lective
re
po
rtin
g
-
+
+
?
+
-
Fre
e o
f o
the
r b
ias
-
?
?
?
+
-
Evaluating systematic reviews
• Critical appraisal skill programme (CASP)
• 10 questions to make sense of a review
Ten questions to make sense of a review Adapted from Oxman AD et al Users Guide to the Medical Literature VI How to use an overview. JAMA 1994; 272 (17): 1367-71 For each question answer : YES, NO or DON’T KNOW A. Are the results of the review valid ? 1. Did the review address a clearly focused issue ?
e.g. the population , intervention and or outcomes 2. Did the authors look for the appropriate sort of papers ?
Did they deal with the issues and have appropriate study design ? Is it worth continuing ?? 3. Do you think the important relevant studies were included ? Look for search methods, reference list use, unpublished studies and non English language 4. Did the authors do enough to assess the quality of included studies ? 5. If the results of studies have been combined, was it reasonable to do so ?
B. What are the results ? 6. What is the overall result of the review ?
Is there a clear numerical expression ? 7. How precise are the results ?
Confidence intervals ? C. Will the results help my local situation ? 8. Can the results be applied locally ? 9. Were all important outcomes considered ? 10. Are the benefits worth the harms and costs ? Phil Wiffen Sep2011 ([email protected])
Conclusions
• Evidence based medicine has emphasised the importance of systematic reviews as evidence for care
• Need to know how find systematic reviews
• Need to know how to read a meta-analysis