Date post: | 21-Dec-2015 |
Category: |
Documents |
View: | 218 times |
Download: | 0 times |
1
An Introduction to Systematic Reviews
2
Information explosion
3
Daily
•46 RCTs
•1000 Medline New articles
•6,000 biomedical articles
4
3 million articles30,000 journals
Annually
5
Increasing Knowledge
0
2000
4000
6000
8000
1966 1976 1986 1996
Number of articles on Hypertension cited in Medline by Year
Articles
6
Most research published in medical journals is
too poorly done
or
insufficiently relevant
to be clinically useful
7
Too much information, too little time
• There is simply too much information around for people to keep up to date.
• On top of this, high quality information is often not easy to find.
8
Review articles
9
• A ‘review’ is the generic term for any attempt to synthesis the results and conclusions of two or more publications on a given topic.
10
Some reviews are usually based on narrative or commentary and are produced by a
‘content expert’
11
What’s the problem with
“Expert Opinion”?
12
Professor Paul Knipschild has described how Nobel prize winning biochemist Linus Pauling
used selective quotes from the medical literature to "prove" his theory that
vitamin C helps you live longer and feel better.
13
When Knipschild and his colleagues searched the literature systematically for evidence
They found that
14
• One or two trials did strongly suggest that vitamin C could prevent the onset of the common cold.
• There were far more studies which did not show any beneficial effect.
15
Unfortunately, expert reviewers often :
• Make conflicting recommendations
• Their advice frequently lags behind
• Inconsistent with the best available evidence.
16
The use of unsystematic approaches to collecting and summarizing the evidence.
17
In one study, self-rated expertise was inversely related to the methodologic rigor of the review
Oxman AD, Guyatt GH. The science of reviewing research. Ann N Y Acad Sci. 1993;703:125-133; discussion 133-134.
18
The problems are that …
• Too much information, too little time
• Many too poorly done or insufficiently relevant to be clinically useful
• Many have conflicting results
• On top of these, high quality information is often not easy to find
19
Timely response
• A classic example: streptokinase in myocardial infarction – First trial published in 1959
– 15 trials published up to 1977
– Introduced as normal practice in late 80s
– Meta-analysis of treatments in 1992: fall in mortality by 1977, after inclusion of 15 trials
20
21
5
101
1 2
8
7
8
12
4
3
1
1
2
8
7
2
1
1
1
2
8
1
5
15
6
No
t M
en
tio
ne
d
Ro
uti
ne
Ex
pe
rim
en
tal
Ra
re/N
ev
er
Sp
ec
ific
M
M
M
M
M
M
Textbook/ReviewRecommendations
Odds Ratio (Log Scale)
0.5 1.0 2.0
Favours Treatment Favours Control
RCTs Pts
1 23
2 65
3 149
4 316
7 1793
10 254411 265115 331117 392922 5452
P<.01
23 5767
27 612530 634633 657143 21 05954 22 051
67 47 53165 47 185
70 48 154
P<.001
P<.00001
CumulativeYear
1960
1965
1970
1980
1985
1990
Thrombolytic Therapy
21
Bias
Bias means something that will cause a consistent deviation from the truth.
This is different from the play of chance.
22
There are three possible sources of bias in reviews:
• bias arising from the studies included in the review
• bias arising from the studies not included in the review
• Bias arising from the way the review is done.
23
Minimizing bias
We need to do as much as possible to minimize the effects of anything that will cause the results to deviate from the truth.
24
What is a Systematic Review?
25
Systematic review Or Overview
Comprehensively
• locates
• evaluates
• synthesizes
all the available literature on a given topic
using a strict scientific design which
must itself be reported in the review
26
A ‘systematic review’, therefore, aims to be:
• Systematic (e.g. in its identification of literature)
• Explicit (e.g. in its statement of objectives, materials and methods)
• Reproducible (e.g. in its methodology and conclusions)
27
The ‘systematic’ part of systematic reviews is all about
minimizing bias in the way
the review is carried out
28
Systematic reviews are the same as ordinary reviews, only bigger!
• not simply "comprehensive"
• but to answer a specific question
• to reduce bias in the selection and inclusion of studies (language, database, publication, reporting, citation, multiple publication)
• to appraise the quality of the included studies
– Internal validity: minimised systematic error (bias)
External validity: generalisability of findings
• to summarise them objectively
29
They are different!!
30
The UK Medical Research Council now requires a systematic review has been prepared before it will commission a new trial.
• The question has not already been answered
• The results of previous research are used in designing the new trial.
31
The Cochrane Collaboration
www.cochrane.org
32
“It is surely a great criticism of our profession that we have not organised a critical summary, by specialty or subspecialty,
adapted periodically, of all relevant randomised controlled trials.”
Archie Cochrane
33
Section 2
34
Conducting a Systematic Review
35
Stages of a systematic review
• Planning the review – i.e. identifying the need for a review, and documenting the methodology
• Conducting the review – i.e. finding, selecting, appraising, extracting and synthesising primary research studies
• Reporting and dissemination – i.e. writing up and disseminating the results of the review
36
Development of a review protocol
37
Formulating review questions
Searching & selecting studies
Study quality assessment
Data synthesis
Extracting data from studies
38
Step 0
انجام اين مطالعات به پول، زمان و نيروی •متخصص نياز دارند.
بندي مداخالت و به منظور اولويت• الزم بود شيوع HIVهاي مربوط به پژوهش
ي و قدرت اثرهريك ازعوامل تعيين كنندهاين بيماري محاسبه گردد.
39
Formulating review questions
• The first and most important decision in preparing a review is to determine its focus
• This is best done by asking clearly framed questions.
40
Formulating review questions
• define a four part clinical question
breaking the question down into its component parts
41
Question components : PICO
• What types of Participants?
• What types of Interventions?
• What types of Comparison?
• What types of Outcomes?
42
A PICO questionA PICO question
What is the best strategy to prevent smoking in young people?
Time-consuming question:
43
An answerable questionAn answerable question
Q. Are mass media (or school-based or community-based) interventions effective in preventing smoking in young people?
44
How are these questions different?
• Does aspirin improve survival after acute myocardial infarction?
• In patients with acute myocardial infarction, does daily, low-dose, oral aspirin lead to higher survival rates as compared to placebo?
45
Formulation of a therapy question
Is Zinc effective in treating cold?
In children with common cold, is oral Zinc effective in reducing the duration of symptoms, as compared to placebo?
Intervention Outcome
InterventionPatient/problem
Outcome Comparison+ RCTs
46
سواالت مرور سيستماتيك
شيوع عدم آگاهي در مورد راه هاي انتقال •HIV رفتار پر خطر جنسي و اعتياد تزريقي ،
14در گروه هاي مختلف مردم ايراني باالي سال اخير چه ميزان مي باشد؟10سال در
بزرگي اثر هر يك از عوامل فوق در آلودگي • در گروه هاي مختلف مردم ايراني HIVبا
سال چگونه است؟14باالي
47
Formulating review questions
Searching & selecting studies
Study quality assessment
Data synthesis
Extracting data from studies
48
Selecting studies
• performing a comprehensive, objective, and reproducible search of the literature
• selecting studies which meet the original inclusion and exclusion criteria
can be the most time-consuming and challenging task in preparing a systematic review
49
Data sources for a systematic review• Electronic databases
– MEDLINE and EMBASE
– Specialized or local databases (CINAHL, CENTRAL, AMED,…)
• Hand searching• “Grey literature” ( thesis, Internal reports, pharmaceutical industry files)
• Checking reference lists• Unpublished sources known to experts in the specialty (seek by
personal communication)
• Raw data from published trials
50
Developing a search strategy
• It is always necessary to strike a balance between comprehensiveness and precision when developing a search strategy.
51
Selecting studies
• assessing titles and abstracts
• full text
52
Searching the literature
• Published and unpublished literature
• unbiased search (not just Medline which contains < half of trials)
• conference papers, company reports, et.
• personal communication with experts
53
شرايط انتخاب نوع مطالعه• توصيفي
توصيفي-تحليلي تحليلي
سال ايراني14مردان يا زنان باالي تحت مطالعه جمعيت• _ جهان مديترانه شرقي_ايران مكان مطالعه•
مطالعه • زماناخير 10 سال
HIVآگاهي و نگرش نسبت به انتقال يا پيشگيري از محتوي مطالعه• اثر آموزش بر آگاهي يا نگرش نسبت به راه هاي انتقال
HIV اعتياد تزريقي
رفتار پر خطر جنسي كيفيت قابل قبول مطالعات•
54
جستجوي مقاالت و اسناد
•NGOهانظر مصاحبه با افراد صاحب•
وكليدي(UN-WHOسازمان هاي جهاني)•
منابع جستجو:•IRAN MEDEX•IRAN DOCچكيده مقاالت كنگره ها•پايان نامه ها•مجالت داخلي و خارجي•مراكز دولتي )وزارت بهداشت، •
مطبوعات، بهزيستي، وزارت ارشاد، مركزمديريت بيماريها،
، مراكز تحقيقاتي(زندان•Cochrane Library•MEDLINE•EMBASEبررسي مراجع مقاالت يافت شده•
55
لغات كليدي مورد استفاده
آموزش• نگرش• آگاهي• ايدز•سندرم نقص ايمني اكتسابي•شيوع•/ عوامل تعيين كنندهعوامل خطر /• ايران•اعتياد تزريقي، سوء مصرف •
تزريقي مواد، اعتيادرفتار پر خطر جنسي، روسپي •
گري، بيماري هاي منتقله از راه جنسي، زنان ويژه
قدرت اثر، نسبت شانس، خطر •نسبي، نسبت خطر، بزرگي اثر
• AIDS• HIV
• Knowledge• Attitude• Knowledge/ Attitude/practice• Risk factors• Intra venous drug use (IDU)• High risk sex• Sex workers, heterosexual,
homosexual, MSM• STI• IRAN• Prevalence• relative risk,odds ratio, effect
56
مرور خالصه مقاالت
مرور خالصه مقاالت توسط دو نفر به •صورت مستقل و انتخاب با توجه به
صورت گرفت. معيارهاي انتخاب 93 خالصه مقاله 1000در نهايت از بيش از •
مطالعه انتخاب شدند. مطالعه از نظر 93سپس متن كامل اين •
كيفيت و معيار هاي انتخاب مورد بررسي HIV assessment form.docقرار گرفت
57
Experts in a particular area frequently have pre-formed opinions that can bias their assessments of both the relevance and validity of articles
58
Formulating review questions
Searching & selecting studies
Study quality assessment
Data synthesis
Extracting data from studies
59
Assessment of study quality
• Assess each study for:
– eligibility for inclusion
– study quality
– reported findings
• Ideally will involve two independent reviewers
60
Assessment of study quality
• Validity: the degree to which the trial design, conduct, analysis, and presentation have minimized or avoided systematic biases.
61
'Quality' scales and checklists
62
های در این مطالعه صرف نظر از گزارش • مطالعه 93نظران، از شفاهی صاحب
مطالعه دارای معیارهای انتخاب 53موجود، بودند
63
Formulating review questions
Searching & selecting studies
Study quality assessment
Data synthesis
Extracting data from studies
64
Collecting data
• data collection forms• Methods
• Participants
• Interventions
• Outcome measures and results
65
داتنجدول مست
گروه هدف •جمعيت تحت مطالعه )سن، جنس، شغل(• عامل يا عوامل خطر مورد بررسي درمقاله•مكان مطالعه)كشور / شهر(•زمان مطالعه• (population based، مطالعه )توصيفي / تحليلينوع• و نحوه ي كنترل آن ها مخدوش كننده هاي مورد بررسي•
مطالعهدرگيري متغيرهاي مورد مطالعه )پرسشنامه / مدارك ابزار اندازه•
موجود(روايي و پايايي ابزار مورد استفاده•
و قدرت اثر هر يك از عوامل تعيين كننده مذكورميزان شيوع •Extraction Form Final.xls
66
Formulating review questions
Searching & selecting studies
Study quality assessment
Data synthesis
Extracting data from studies
67
Graphical Display
• The graphical display of results from individual studies on a common scale is a “Forest plot”.
In the forest plot each study is represented by a black square and a horizontal line (CI:95%).The area of the black square reflects the weight of the study in the meta-analysis.
Forest plot is an important step, which allows a visual examination of heterogeneity between studies.
68
Odds RatioOdds Ratio
more than 1less than 1 1
Line of no significance
69
Odds Ratio – with Odds Ratio – with pooled effect sizepooled effect size
more than 1less than 1 1
Best/point estimate
Confidence Interval
70
Forest Plot
var1.012601 .071399
Combined
71
Forest Plot
var1-.004408 .60356
Combined
72
Meta-Analysis
• when an overview incorporates a specific statistical strategy for assembling the results of several studies into a single estimate
73
• Systematic reviews do not have to have a meta-analysis
• There are times when it is not appropriate or possible.
75
تجزيه و تحليل نهايي
امكان انجام متاآناليز با توجه به وجود •هتروژنيتي ) نمودار و آماري( بررسي گرديد.
شيوع عدم آگاهي در زير گروه هاي مختلف • محاسبه گرديد.6/32 تا 4/24ما بين
76
77
A Range of Function of Systematic Review
• 1- Identify heterogeneity in effects among multiple studies and, where appropriate, provide summary measure.
• 2- Increase statistical power and precision to detect an effect
• 3- Develop, refine and test hypothesis• 4- Reduce the subjectivity of study
comparisons by using systematic and explicit comparison procedure
78
Remain:
• 5- Identify data gap in the knowledge base and suggest direction for future research
• 6- Calculate sample size for future studies
79
Section 3
80
Steps of a systematic review
• Step 1: Framing question for a review
• Step 2: Identifying relevant literature
• Step 3: Assessing the quality of the literature
• Step 4: Summarizing the evidence
• Step 5: Interpreting the finding
81
• The principal findings should be related to the main question formulated in step1.
• Other finding should be considered secondary.
82
Validity of the main finding
• Are the searches adequate?
• Is there a risk of publication and related biases?
• Is the quality of the included studies high enough?
83
Limitations of Systematic Review
• Reporting bias and the inadequate quality of primary research are potentially serious problems for systematic reviews.
• The dissemination of research findings is not a random process; rather it is strongly influenced by the nature and direction of results.
84
Type of reporting bias
• Publication bias • The publication or non-publication of research findings, depending on the nature and direction of the results
85
Publication bias
• 1-Arising from the researchers deciding whether or not to submit result
• 2- Arising from the tendency of journals to reject negative studies
• 3-sponsorship
• ….
86
Methods of preventing publication bias
• 1-Registeries
• 2-Editorial policy
87
• Time lag bias • The rapid or delayed publication of research finding, depending on the nature and direction of the results
88
• Multiple (duplicate) publication bias
• The multiple or singular publication of research finding, depending on the nature and direction of the results
89
• Citation bias • The citation or non-citation of research
finding, depending on the nature and direction of the results
90
• Language bias • The publication of research finding in a particular language, depending on the nature and direction of the results
91
• Outcome reporting bias
• The selective reporting outcomes but not of others , depending on the nature and direction of the results
92
Funnel Plot• Plots of the trials’ effect estimates against sample
size, may be useful to assess the validity of meta-analyses
• A symmetrical shape is expected, since greater scatter in estimate is expected for smaller study.
• The cardinal sign of publication bias is a hole in the middle or one side of the plot, that is an area where we would expect to see study result but where there are apparently none.
93
94
Thanks