Physiologic StudiesPhysiologic StudiesIntuition or BeliefsIntuition or Beliefs

Systematic Reviews Systematic Reviews MetaanalysisMetaanalysisDouble-blindDouble-blind

Randomized Controlled TrialsRandomized Controlled TrialsCohort StudiesCohort Studies

Case-Control Case-Control StStudiesudiesCases Report/SeriesCases Report/Series

Hierarchy of Clinical Evidence Hierarchy of Clinical Evidence Obs

ervati

onal

Design of a cohort studyTIME

direction of inquiry

people exposed diseasepopulation without the no disease disease not exposed disease no disease“at risk”

Design of a case-control study

TIMEdirection of inquiry Start with:

Exposed cases (people with disease)Not exposed

PopulationExposed controls (people without disease)Not exposed

Questions to ask when an association is reported in the literature

(eg estrogen and CHD)Explanation Finding

Association

Bias in selection Yes Noor measurement

Chance Likely Unlikely

Confounding Yes No

Cause Cause

Association and causeExplanation Finding

Association

Bias in selection Yes Noor measurement

Chance Likely Unlikely

Confounding Yes No

Cause Cause

Case report?

Case series?

Association and causeExplanation Finding

Association

Bias in selection Yes Noor measurement

Chance Likely Unlikely

Confounding Yes No

Cause Cause

P value

Association and causeExplanation Finding

Association

Bias in selection Yes Noor measurement

Chance Likely Unlikely

Confounding Yes No

Cause Cause

Criteria for causation

1. Is there evidence from true experimentation in humans?

2. Is the association strong?3. Is the association consistent from study to study?4. Is the temporal association correct?5. Is there a dose-response gradient?6. Does the association make biological sense?7. Is the association specific?(Adapted from Bradford Hill)

Randomization

1. Guarantees equal probability of receiving control/experimental treatment to all participants (removes investigator bias)

2. Protects against imbalances in known and unknown confounders

3. Provides basis for statistical analysis

Hierarchy of study methods for clinical decision-making

Sys

tem

atic

revi

ews

Is observational evidence equivalent to

experimental evidence?

Benson NEJM 2000;342:1877Concato NEJM 2000;342:1887

In some cases -YES

In others- NO !

Clinical trials are selective!-Select group gets in - chance decides who gets treatment; treatment effect decides who does better

Observational evidence is also selective: self-selection of exposure: this may decide who does better

How much of Medicine is How much of Medicine is Evidence-Based ?Evidence-Based ?

Matzen P. Ugeskr laeger 2003;165:1431-5• General Internal Medicine - 50%• Psychiatry- 65%• Others (surgery, general practice,

dermatology) - lessLai Br. J Ophthal . 2003;4:385-90: • 42.9% of patient interventions were based on

evidence from RCT, meta-analysis or systematic reviews (23% on no evidence)

Major Disadvantage of RCTs:Selectivity

Clinical trials

Should not be performed unless there is a realistic chance of providing a valid/reliable answer to a well-defined medical question

But…SHEP study Of 447,921 (100%) identified

31,960 (11.6%) met initial criteria

4,736 (1.03%) randomized

01.4

10.1Comorbidity: General Pop

vs. Subgroup

0

5

10

15

20

25

12345

AP MI CHF CVA DM

%

of P

op

Men, gen’l pop

Wom., gen’l popSHEP pop

% of Pop

DM CVD CHF MI Angina

Coke tastes better…….!

Than what?

Coke tastes better…….!

CAVEATS in using RCT evidence to guide patient management

1. Tendency to extend application of new treatments to patient groups other that those for whom data exist

2. Extrapolation of data to agents of the same class but untested for specific indication

Don’t drown in the evidence

Use predigested sources

ACP J Club, Bandolier, POEMS, clinical evidence, Cochrane Reviews, Clinical guidelines clearinghouse

Good luck!