Date post: | 02-Jan-2016 |
Category: |
Documents |
Upload: | ashley-stone |
View: | 213 times |
Download: | 0 times |
EBC course10 April 2003
Critical Appraisal of the Clinical Literature:
The Big Picture
Cynthia R. Long, PhDAssociate Professor
Palmer Center for Chiropractic Research
Outline
• Overall purpose of critical appraisal• Consider “levels of evidence”• Some tips on interpreting
“significant” findings• Potentially useful references
Overall Purpose
• To critically appraise a research article in order to assess the validity of the authors’ conclusions
Sections of an Article
• Introduction: Background and explanation of rationale for study.
• Methods: How was study done? Should allow study to be replicated.
• Results: Report results (data).• Discussion: Interpret results. Draw
conclusions from results.• Abstract: Article summary.
Recommendation
• Read the abstract last when familiarizing yourself with critically appraising the literature.
• Assess evidence from reading the article, not the abstract.
• Read the abstract in deciding whether or not you are interested in the topic of the article.
Critical Appraisal
• similar across types of studies for Introduction and Discussion sections
• information in Methods sections may differ
• information in Results sections may differ
Overall Questions to Ask
• Is the study design appropriate to address the research question?
• In the Discussion Section: Are the findings... – ...consistent with the research question of
the study?
Terminology:Clinical Intervention Studies
• Evaluates which treatment interventions are most useful and effective for a given clinical condition
Clinical Intervention Studies
• One-group Pre/post Study • Pilot RCT• Randomized Clinical Trial (RCT)• Systematic Review of RCTs
Levels of Evidence
designed studies
clinical observations
RCT
One-group pre/post
SystematicReviews
case series, N-of-1, time series designs
case reports
Pilot RCT
Levels of Evidence
• Is the study design appropriate to address the research question?
• In the Discussion Section: Are the findings... – ...consistent with the research question of
the study?
One-group pre/post design
• Addresses: do patients improve after treatment?
• Can’t address: is treatment effective for patients?– No comparison group– Patients may improve: natural course of
condition, in study, change in lifestyle, treatment
One-group pre/post design: Example
Hawk C, Long CR, Azad A. Chiropractic Care for Women with Chronic Pelvic Pain: A Prospective Single-Group Intervention Study. JMPT 1997;20:73-79. – Results: women improved!– Conclusion: needs further study
Pilot RCT
• Addresses: is the RCT feasible?• Can’t address: is treatment
effective for patients?– Not powered (i.e. sample size not large
enough)
• Note: often only published if RCT is determined to be unfeasible
Pilot RCT: Example 1
Hawk C, Long CR, et al. Issues in planning a placebo-controlled trial of manual methods: Results of a pilot study. J of Alt Comp Med 2002;8:21-32. – Results: recruitment not feasible;
standardization of treatment protocol difficult among multiple sites
– Conclusion: put on hold
Pilot RCT: Example 2
Bronfort G, et al. Nonoperative treatments for sciatica: A pilot study for a randomized clinical trial. JMPT 2000;23:536-544. – Results: recruitment not feasible!– Conclusion: put on hold
RCT
• Addresses: is the treatment effective for patients?
• Caveats:– Must be a powered study
• sample size must be formally justified in the Methods section (based on: effect size—minimally important clinical difference; variability of outcome measure; statistical test)
RCT: Example
Hurwitz E, et al. A randomized trial of chiropractic manipulation and mobilization for patients with neck pain: Clinical outcomes from the UCLA neck-pain study. Am J Public Health 2002;92:1634-1641. – Results: no statistically or clinically significant
differences among groups– Conclusion: cervical spine mobilization is as
effective as manipulation in reducing neck pain and related disability among chiropractic patients
Overall Questions to Ask
• In the Discussion Section: Are the findings... – ...consistent with the research question of
the study?– …consistent with the results presented? – …given in the context of current evidence?
Systematic Review of RCTs
• Addresses: is the treatment effective for patients?
• Looks at all RCTs of the treatment for patients and combines based on quality of original RCTs
RCT: Example
Bronfort G, et al. Efficacy of Spinal Manipulation for Chronic Headache: A Systematic Review. JMPT 2001;24:457-466. – Conclusions: can’t make firm conclusions
(few trials of adequate methodological quality)
RCT: Example
– Conclusions:• Cervicogenic: SMT better effect than massage• Tension-type and migraine: SMT effect
comparable with commonly used first-line prophylactic prescription meds
Terminology: Descriptive Statistics
(Results)• Patient characteristics at baseline
– Examples: mean, standard deviation, median, range, percentage
– Assess group comparability on baseline characteristics
• Assess generalizability of results to target population
Terminology: Analytical Statistics (Results)• Assess statistical significance with
confidence intervals and p-values– Within and between group differences– Make inference about target population
• Must be appropriately interpreted in the context of the research question and the study design
Terminology
• P-values and confidence intervals:– Reflects measure of effect relative to
variation and sample size
Statistical Significance
Interpreting p-values: p<0.01 statistically significant difference!
0.01p0.05 statistically significant difference
0.05<p0.10 borderline statistically significant difference
p>0.10 no statistically significant difference
Clinical Significance
• i.e. clinical importance• is defined before study is
conducted• assessed with descriptive statistics
(e.g. mean improvement in outcome measure)
Possible Scenarios
• statistically and clinically significant findings
• clinically significant, but not statistically significant
• statistically significant, but not clinically significant