Date post: | 01-Jan-2016 |
Category: |
Documents |
Upload: | valentine-shields |
View: | 212 times |
Download: | 0 times |
1
Systemic Review and Meta-Analysis in Cancer Epidemiology
Chun Rebecca Chao, Ph.D.
Kaiser Permanente Southern California
Department of Research and Evaluation
2
Overview
Introduction of systematic review and meta-analysis
Methods of systemic review and meta-analysis
3
What is Systemic Review?
A review that has been prepared using a systemic approach to minimize biases and random errors (which is documented in a materials and methods section)*. Review in which there is a comprehensive search for
ALL relevant studies on a specific topic, and those identified are then appraised and synthesized according to a predetermined and explicit method.
Systemic review vs. Narrative review Narrative Review: traditional expert review, subjective,
no formal rules in selecting studies, no standard statistical methods for combining studies.
*Chalmers and Altman. Systemic Reviews, BMJ Publishing Group, 1995.
4
What is Meta-Analysis?
A statistical analysis of the results from independent studies, which generally aims to produce a single estimate of a treatment effect.
A systemic review may or may not include a meta-analysis.
It is always appropriate and desirable to systemically review a body of data, but it is sometimes inappropriate to statistically pool results from different studies.
5
The Need for Systemic Review
Health Care Professional: number of biomedical publication has been increasing rapidly.
Guideline and Policy Makers: Evidence based medicine is the trend for patient care, clinical guideline development and policy making.
Researchers: future research direction can be guided by systemic reviews.
Consumers
6
Strength of Evidence Concerning Efficacy of Treatment Case report Case series without controls Series with literature controls Series with historical controls Case control studies Cohort studies Randomized controlled trials (RCTs) SR/Meta-analysis of RCTs Prospective meta-analysis of RCTs with individual
data
7
Strengths of Systemic Review
To provide a more objective appraisal of the evidence
Contribute to resolve uncertainty when original research, reviews and editorials disagree.
Meta-analysis of the BCG vaccine trials
Graph adopted from Egger, Smith and Altman. Systemic Reviews in Health Care. BMJ
8
Strengths of Systemic Review (Cont.)
To provide a more objective appraisal of the evidence
Contribute to resolve uncertainty when original research, reviews and editorials disagree.
To reduce the probability of false negative results To explore treatment effects in subgroups of patients To explore and explain heterogeneity between study results To guide the direction of future studies
To understand current gap and limitation in the literature To generate new research questions to be addressed.
9
Advantages of Use of Meta-Analysis to Combine Studies When individual trials or studies are too small
to give reliable answers When large trials or studies are impractical or
impossible Potentially lead to more timely introduction of
effective treatment When there have been many trials or studies
showing small effects may be important
10
Number of Publications about Meta-Analysis, 1985-2005
Results from MEDLINE search using MeSH and text word “meta-analysis”
0
500
1000
1500
2000
2500
11
Potential Limitation of Conducting Systemic Review Bias can be introduced in reviews in several
ways. Problems associated with design or reporting
of original studies Limitations of using published data Publication bias
12
Publication Bias – A real threat for systemic review Studies with significant results are
more likely to be published More likely to be published without delay (lag time) More likely to be published in English More likely to be cited More likely to be published more than once
Outcome reporting bias Significant outcomes are more likely to be reported than
non-significant outcomes. Should unpublished data be included in systemic
review? Pre-specified inclusion (quality) criteria are
recommended.
13
A Demonstration of Publication Bias
Graph adopted from Egger, Smith and Altman. Systemic Reviews in Health Care. BMJ
14
Predictors of Publication
Graph adopted from Egger, Smith and Altman. Systemic Reviews in Health Care. BMJ
15
Language Bias
Graph adopted from Egger, Smith and Altman. Systemic Reviews in Health Care. BMJ
16
Pan Z et al. PLoS Med. 2005 Dec;2(12):e334
Publication/Reporting Bias?
17
Meta-Analysis of Observational Studies In observational studies, bigger is not necessarily
better. This is a danger that meta-analysis of observational
data produces precise but spurious results. A careful systemic review and examination of source
of heterogeneity are more important than combining results. Statistical combination of data should not be the main
component of systemic review of observational data. It is often desirable to have individual level data for
this purpose. Confounding and effect modifiers.
18
Example 1. Fat Intake and Risk of Breast Cancer
Graph adopted from Egger, Smith and Altman. Systemic Reviews in Health Care. BMJ
Recall bias might play a role in the findings from case-control studies.
19
Example 2: Intermittent Sunlight Exposure and Risk of Melanoma
Graph adopted from Egger, Smith and Altman. Systemic Reviews in Health Care. BMJ
Lack of blinding of the hypothesis to the subjects may introduce recall bias.
20
Example 3. Formaldehyde Exposure and Lung Cancer
Graph adopted from Egger, Smith and Altman. Systemic Reviews in Health Care. BMJ
Dose-response or selection bias?
21
Carotenoids and the Risk of Developing Lung Cancer: A Systemic Review*. Objective: Systemic review of the association
between carotenoids and lung cancer. Included both RCT and prospective observational
studies (smoking adjusted). Examined total carotenoids, β-carotene, α-carotene,
β-cyrptoxanthin, lycopene, and lutein-zeaxanthin. Examined carotenoid supplements, dietary intake of
carotenoids, and serum carotenoid concentrations.
*Gallicchio L et al, Am J Clin Nutr 2008; 88: 372-83.
22
β-carotene Supplement Use in RCT
Gallicchio et al, Am J Clin Nutr 2008; 88: 371-83
23
Dietary Total Carotenoid Intake: Prospective Cohort Studies
Gallicchio et al, Am J Clin Nutr 2008; 88: 371-83
24
Dietary β-carotene Intake: Prospective Cohort Studies
Gallicchio et al, Am J Clin Nutr 2008; 88: 371-83
25
Serum Total Carotenoid Concentrations: Prospective Cohort Studies.
Gallicchio et al, Am J Clin Nutr 2008; 88: 371-83
26
Dose Response
Gallicchio et al, Am J Clin Nutr 2008; 88: 371-83
27
Subgroup Analysis
Gallicchio et al, Am J Clin Nutr 2008; 88: 371-83
28
Method for Systemic Review
1. Define research questions 2. Define inclusion/exclusion criteria 3. Search the literature 4. Select articles 5. Evaluate the internal/external validity of the studies 6. Extract/Abstract Data 7. Calculate effect size and standard error 8. Examine heterogeneity 9. Assess publication bias 10. Combining study results if appropriate 11. Influence analysis, sensitivity analysis 12. Interpretation of results 13. Reporting
29
Stages of a Systemic Review
Stage 1Planning
Identification of the need
for a systemic review
Development of a proposal
Stage 2Conducting
Stage 3Reporting
Identification of the research
Selection of studies
Study quality assessment
Data extraction
Data synthesis
The report and recommendations
30
1. Clearly Formulated Question The research question is extremely important!
Get feedback about your specific research question from many people (content expert, savvy clinician in the field, methodologist)
The question should be clearly specified What is the study objective
To validate results in a large population To guide new studies
Pose questions in both biologic and health care terms specifying with operational definitions Population Intervention/exposure Outcomes (both beneficial and harmful)
31
Study Protocol
Develop a protocol of what will be done
Background: why is this important? Specify the research question
Research question defines the following criteria Provide an overview of the methods, including search
strategy, inclusion/exclusion criteria, quality assessment, how data will be abstracted, and an analysis plan
Specify any subgroup analyses or sensitivity analyses (best if these are a priori)
Initially, inclusion criteria should be overly broad E.g., search all alcohol when specifically interested in wine. E.g., search all dietary, anthropometric related papers when
interested in carotenoids.
32
2. Develop Inclusion Criteria
Use clinical/scientific judgment to enhance
validity and homogeneity
Validity (Study quality)
Homogeneity Similar study design
Similar patient populations
Similar interventions/exposures
Similar outcomes
33
Typical Inclusion Criteria
Study Design (e.g., RCT, prospective cohort) Population (risk group) Interventions (dosage, regimen)/exposure of
interest Outcomes (definitions, follow-up period)
34
Practical Considerations in Defining Eligibility for a Systemic Review Study designs to be included Years of publication or study conduct Languages Choice among multiple publications Restrictions due to sample size or follow-up
duration Similarity of treatment and/or exposure Completeness of information
35
Copyright restrictions may apply.
Chao, C. et al. Am. J. Epidemiol. 2008 168:471-480; doi:10.1093/aje/kwn160
List of excluded studies
36
3. Comprehensive Literature Search
Need a well formulated and coordinated effort Seek guidance from a librarian Specify language constraints Requirements for comprehensiveness of the
search depends on the field and question to be addressed
37
Searching the Literature
Database searches: MEDLINE, EMBASE, ISI Web of Science, PsychLit, CancerLit, Cochrane, Dissertations online
Reference lists of retrieved articles Manual searching of related journals, conference proceedings,
textbooks Experts in the field Granting agencies Study/trial registries Industry (device manufacturers, pharmaceutical companies)
38
How to Develop a Search Strategy (Example)
Study Question What is the relationship between consumption of different
types of alcoholic beverage (beer, wine and liquor) and risk of lung cancer?
Break down the question into facets (not all may be needed for searching) Exposure: Alcoholic beverage consumption Outcome: Lung cancer Study design: Observational studies with internal
comparison group (case-control or cohort)
39
How to Develop a Search Strategy (Cont.) Identify synonyms, spelling variants, and
subject headings associated with each facet Text terms (title/abstract) MeSH
alcohol alcohol drinking
ethanol ethanol
alcoholic
alcoholics
beer
wine
liquor
40
How to Develop a Search Strategy (Cont.)
Text terms (title/abstract) MeSH
lung lung neoplasm
in combination with
cancer
neoplasm
carcinoma
tumor
incidence
mortality
41
4. Study Selection
Unbiased Study Selection Two independent reviewers select studies Based on a priori specification of the population,
intervention, outcomes and study design Differences are resolved by consensus Specify reasons for rejecting studies
Keep a record of what is excluded and reason (journals may require this and the search must be reproducible)
42
Flow Diagram of Study Selection ProcessPotentially relevant references identified after liberal screening of the electronic search (n=#)
Excluded by Title/Abstract (n=#) List the reasons
Articles retrieved for more detailed evaluation (n=#)
Articles excluded after evaluation of full text (n=#) List the reasons
Relevant studies included in the meta-analysis (n=#)
Figure 1.
43
Carotenoids and Lung Cancer Systemic Review
Gallicchio et al, Am J Clin Nutr 2008; 88: 371-83
44
Study Quality Assessment
Quality of study conduct vs. quality of reporting Controversial since this process is subjective and can introduce bias No standard method to assess quality of a study Scales and checklists exist but can be challenging to incorporate for
observational studies Pre-specify algorithm for quality assessment Assess quality of each study in uniform, systematic and complete
manner Once quality is assessed then what to do with that information
Consider weighting each study result by quality score Exclude studies with ‘poor’ quality Stratifying by methodological quality component (component
approach)
45
5. Data Extraction
Develop an easy-to-fill-out form for collecting data from each article Study methods (quality) Study description (population) Results (outcomes, side effects)
Two independent reviewers extract data Should be explicit, unbiased, and reproducible Include all relevant measures of benefit and harm of the
intervention Contact investigators of the studies for clarification in
published methods, data Differences in data extraction are resolved by consensus
46
Data Extraction: Study Characteristics Types of publication (journal article, abstract or
unpublished data) Publication year and country of origin Study participants (sample size, age, gender, race, health
status) Design details (case-control, cohort, parallel or cross-over,
randomization, blinding) Nature of treatment and control Study duration Measurement of compliance Definition and measurement of outcome Other confounders
47
Continuous data Outcomes summarized as means (blood pressure, weight)
Dichotomous or binary data Each individual must be in one of two states (dead/alive,
smoking/not smoking) These data are summarized using odds ratios, risk ratios, or risk
differences Survival or time to event data
Outcome of interest is the time to the occurrence of an event Usually summarized using hazard ratios
Other data Some outcomes may be
Short ordinal scales (pain scales: individuals’ rate their pain as none, mild, moderate, severe) for which it is not sensible to calculate a mean
Event counts (number of asthma attacks per month)
Data Extraction:Study Outcome
48
Gallicchio et al, Am J Clin Nutr 2008; 88: 371-83
49
6. Calculating Effect Size and Standard Error
Perform a narrative, qualitative summary when data are too sparse, or too low quality or too heterogeneous to proceed with a meta-analysis
The results from each study are converted into an Odds Ratio (OR) or Effect Size (ES)
95% confidence intervals (CI) are calculated for each study-specific OR or ES
For a meta-analysis, If only confidence intervals are given, computation will be required to obtain estimates of the standard error
50
7. Examining Heterogeneity
Statistical test for heterogeneity Visual inspection/Graphical approach
Forest plot, Galbraith plot Meta-regression
Unit of regression: study Dependent variable: study-specific effect
estimate Independent variables: study-specific
characteristics (e.g., study design, geographic location, length of follow-up)
51
Examining Forest Plot for Heterogeneity
Graph adopted from Egger, Smith and Altman. Systemic Reviews in Health Care. BMJ
52
Meta-regression STATA Output
Type 2 diabetes and risk of NHL: systemic review and meta-analysis
---------------------------------------------------------------------------------------- | Coef. Std. Err. z P>|z| [95% Conf. Interval] ---------------------------------------------------------------------------------------- design | .4256531 .203504 2.09 0.036 .0267926 .8245136 _cons | .167641 .083123 2.02 0.044 .0047228 .3305591 ---------------------------------------------------------------------------------------- Reference group: case-control design ---------------------------------------------------------------------------------------- | Coef. Std. Err. z P>|z| [95% Conf. Interval] ---------------------------------------------------------------------------------------- europe | .1077322 .1625632 0.66 0.508 -.2108859 .4263503 asia | .4499953 .1966564 2.29 0.022 .064556 .8354347 _cons | .107974 .1095071 0.99 0.324 -.106656 .3226041 ---------------------------------------------------------------------------------------- Reference group: US studies
53
8. Assessing Publication Bias
Results because negative studies are less likely to be submitted or published
Can bias the results of a meta-analysis toward a positive finding
Can evaluate publication bias graphically (funnel plot) or through statistical analysis Egger’s test, Begg’s test
54
Begg’s Funnel Plot
Graph adopted from Egger, Smith and Altman. Systemic Reviews in Health Care. BMJ
55
Copyright restrictions may apply.
Chao, C. et al. Am. J. Epidemiol. 2008 168:471-480; doi:10.1093/aje/kwn160
Begg's funnel plot with pseudo 95% confidence limits for assessment of publication bias
56
9. Meta-Analysis for Calculating a Summary Effect Estimate Several methods are available for combining
study results Inverse variance method M-H methods Peto’s odds ratio method
Fixed effect vs. random effect
57
Subgroup Analyses
Pre-specify hypothesis-testing subgroup analyses and keep few in number
Label all posteriori subgroup analyses When subgroup differences are detected, interpret in
light of whether they were: Established a priori Few in number Supported by plausible mechanisms Important (qualitative vs. quantitative) Consistent across studies Statistically significant (adjusted for multiple testing)
58
10. Influence Analysis
Examine how each study influence the summary statistic by removing one study at a time and re-calculate the combined estimate.
A graphically display can be used for visual inspection of influential studies.
59
Influence Analysis STATA Output
Wine consumption and risk of lung cancer: systemic review and meta-analysis ------------------------------------------------------------------------------ Study ommited | e^coef. [95% Conf. Interval] ------------------------------------------------------------------------------ Bendera 1992 | .831761 .67413002 1.0262506 De Stefani 1993 | .78909296 .65476894 .95097321 Carpenter 1998 | .81794792 .66839695 1.0009602 De Stefani 2002 | .82806259 .67865324 1.0103652 Hu 2002 | .82721388 .67337066 1.0162053 Freudenheim 2003 | .82455081 .67135686 1.0127014 Benedetti I 2006 | .82831144 .67354399 1.0186415 Benedetti II 2006 | .81791788 .66001374 1.0135996 Benedetti II 2006 | .82721388 .67337066 1.0162053 Pollack 1984 | .78993553 .67183363 .92879862 Prescott 1999 | .84944367 .70591372 1.0221567 Prescott 1999 | .83032089 .69030547 .99873579 Freudenheim 2005 | .81104547 .65682006 1.001484 Freudenheim 2005 | .78433573 .64188826 .9583950 ------------------------------------------------------------------------------- Combined | .81770466 .67530395 .99013327 -------------------------------------------------------------------------------
60
Influence Analysis STATA Output
Graph adopted from Egger, Smith and Altman. Systemic Reviews in Health Care. BMJ
61
10. Sensitivity Analysis
Test robustness of results relative to key features of the studies and key assumptions and decisions
Include tests of bias due to retrospective nature (e.g., with/without studies of lower methodological quality)
62
Component Approach
Graph adopted from Egger, Smith and Altman. Systemic Reviews in Health Care. BMJ
63
11. Interpretation of Findings
Interpret results in context of clinical practice State methodological limitations of the individual studies
included and in the meta-analysis Consider size of effect in studies and meta-analysis,
consistency of effect sizes and any dose-response relationship
Interpret results in light of other valuable evidence Make recommendations clear and practical Propose future research agenda (clinical and methodological
requirements)
64
12. Describe Studies and Findings
Guideline for reporting systemic review The QUOROM
statement (Quality Of Reporting Of Meta-analysis)
Moher D, et al. Improving the quality of reports of meta-analyses of randomised controlled trials: the QUOROM statement. Quality of Reporting of Meta-analyses. Lancet 1999, 354(9193):1896-900.
65
QUOROM Check List
Moher D, et al. Lancet 1999, 354(9193):1896-900.
66
Summary
A well conducted systemic review allows for a more objective appraisal of the evidence than traditional narrative reviews
Systemic review may contribute to resolve uncertainties and disagreements in original research
Meta-analysis may enhance the precision of estimates of treatment effects
Exploratory analyses (i.e., subgroups who are likely to respond well to a treatment) may guide cost effective treatment decisions
Systemic review may demonstrate areas where the evidence is inadequate and thus identity areas where further research is needed