30
Meta-analysis Quantitative systematic review Combines data from previously
conducted clinical trials (and epidemiologic research) and performs statistical analyses on pooled results
NOTE: different from a review article
Meta-analysis
Useful when:1. definitive clinical trials are impossible,
unethical or impractical2. randomized trials have been performed but
results are conflicting3. results from definitive trials are being
awaited4. new questions not posed at the beginning of
the trial need to answered5. sample sizes are too small
Meta-analysis
Purposes include:1. to increase statistical power for primary
endpoint and or subgroups2. to resolve uncertainty when reports disagree3. to improve estimates of size of effect4. to answer new questions not posed at the
start of the trials5. to bring about improvements in quality of
primary research
Meta-analysis: SSRIs Whittington CJ, et al. Selective
serotonin reuptake inhibitors in childhood depression: systematic review of published versus unpublished data. Lancet 2004;363:1341-1345.
Meta-analysis: Cox-2
Association Between Valdecoxib and Cardiovascular Events*
Number
Cardiovascular Events
Study
Valdecoxib
Placebo
Valdecoxib
Placebo
RR
95% CI
Ott et al3 311 151 14 2 3.40 0.82–13.98
2nd CABG4 1088 548 17 3 2.85 0.81–10.02
Meta-analytic RR5 ... ... ... ... 3.08 1.20–7.87
"Valdecoxib" indicates valdecoxib alone or in combination with parecoxib. Meta-analytic P=0.019; P heterogeneity=0.86.
*Cardiovascular events include coronary and cerebrovascular events.
Furberg CD, et al. Parecoxib, valdecoxib, and cardiovascular risk. Circulation 2005;111:249.
Meta-analysis: the process
1. Problem formulation2. Data collection3. Evaluation of the collected data4. Analysis and interpretation5. Presentation of Results
Problem formulation Clearly define the clinical question
specify variables evaluate relationship between
variables (cause and effect)
Data collection Describe details of literature search
databases published vs. unpublished additional sources (i.e.., reference lists,
meetings) Describe inclusion/exclusion criteria
study design participants treatment outcome measures
Evaluation of dataEven before the “data” Author Funding Relevant information
Important part of data evaluation.Different ways to incorporate into meta-
analysis: exclusion, weighting, stratifying
Evaluation of data Raw data, individual patient data
preferred difficult to obtain
Summary data more commonly utilized
Evaluation of data Homogeneity vs. heterogeneity
L’abbe plot Cochran-Q
Publication Bias Funnel plot
Funnel Plot
Meta-analysis continued
Remember:
Meta-analysis is an observational study of evidence. It is retrospective.
Evaluation of data Scrutinize validity of trials
randomization techniques sample size compliance blinding intention to treat vs. per protocol
Primary studies may be weighted to reflect quality of research design. Weighting of data is controversial.
Investigators should be blinded to: authors, institutions, journals, funding, acknowledgements.
Analysis and interpretation Appropriate statistical analyses
standardized outcome measure Continuous (i.e., blood pressure): differences,
standard deviations Binary (i.e., dead or alive): odds ratio, relative risk
overall effect; combining data fixed effects model--assumes same effect across
studies random effects model--assumes different
underlying effect for all studies and others…
Analysis and interpretation Odds Ratio
Sample Group Disease No Disease
Treatment or exposure a b
Control or no exposure c d
Total a+c b+d
controlsin exposure of Odds
casesin exposure of OddsOR
bc
ad
b/d
a/cOR
d
b
d)d/(b
d)b/(bcontrolsin exposure of Odds
c
a
c)c/(a
c)a/(acasesin exposure of Odds
Cigarette smoking and lung cancer (Doll and Hill BMJ 1950 ii 739-748). Results for men.
Lung cancer cases Controls
Smokers 647 622
Non-smokers 2 27
OR=?
Analysis and Interpretation: Odds Ratio
Cigarette smoking and lung cancer (Doll and Hill BMJ 1950 ii 739-748). Results for men.
Lung cancer cases Controls
Smokers 647 622
Non-smokers 2 27
Odds ratio = (647x27) / (2x622) = 14.04Lung cancer cases 14 x more likely to be smokers.
Analysis and Interpretation: Odds Ratio
Analysis and interpretation Relative Risk
d)c/(c
b)a/(a
group controlin event ofy Probabilit
groupnt in treatmeevent ofy ProbabilitRR
Sample Group Disease No Disease
Total
Treatment or exposure
a b a+b
Control or no exposure
c D c+d
Total a+c b+d
Analysis and interpretation Sensitivity analysis
Overall effect calculated by different methods (fixed vs. random)
Reanalysis with exclusion of poor-quality studies
Reanalysis with exclusion of small studies
Reanalysis of exclusion of studies with short duration of follow-up
Presentation of results Often graphically displayed with
confidence intervals Type I and II error should be
discussed Robustness of findings/sensitivity
should be discussed
Strengths Can summarize from available
studies the effects of interventions across many patients
Can reveal research designs as moderators of study results
Can reduce false negative results Can clarify heterogeneity between
study results
Strengths Can assist in accurate calculation
of sample size needed in future studies
Can suggest promising research questions for future study
Can allow more objective assessment of evidence and thereby reduce disagreement
Weaknesses Can pass along inflated estimates
of size effects based previously reported results
Cannot overcome subjectivity in choice of outcomes and their weighting in analysis
Can be compromised by publication bias
Weaknesses Arithmetic nature of meta-analysis
can produce false impression of certainty in an inherently uncertain process with many subjective elements
Cochrane Collaboration The Cochrane Collaboration is an
international not-for-profit organization, providing information about the effects of health care
Source of qualitative and quantitative systematic reviews with good methodological rigor
www.cochrane.org
Conclusions Interpret with caution
remembering that conclusions depend on the quality of the studies included
Findings of subsequent randomized controlled trials may differ
References Malone PM et al. Drug information: a
guide for pharmacists. McGraw-Hill. New York. 2nd edition. 2001.
Noble Jr JH. Meta-analysis: methods, strengths, weaknesses, and political uses. J Lab Clin Med 2006;147:7-20.
