The ABC of Evidence-Base Medicine

Dr Max Mongelli 2016

http://drmaxmongelli.weebly.com

What is EBM?

““The conscientious, explicit and The conscientious, explicit and judicious use of judicious use of current best current best evidenceevidence in making decisions about in making decisions about the care of individual patients”the care of individual patients”

Prof. David L. Sackett, 1997 Prof. David L. Sackett, 1997


Why EBM?



Primum non nocerePrimum non nocere““First do no harm”First do no harm”

Hippocrates, Hippocrates, EpidemicsEpidemics


Sources of Evidence in Medicine

Traditional TeachingTraditional Teaching TextbooksTextbooks Basic sciencesBasic sciences Observational studiesObservational studies Computer simulationComputer simulation Decision AnalysisDecision Analysis Case-Control StudiesCase-Control Studies Randomised Controlled Trials (RCT)Randomised Controlled Trials (RCT)• Meta-analysesMeta-analyses


RCOG Classification of Evidence LevelsRCOG Classification of Evidence Levels

1++ High quality meta-analyses of 1++ High quality meta-analyses of RCT'sRCT's

1+ Meta-a. Or RCT's at low risk of bias1+ Meta-a. Or RCT's at low risk of bias 1- Meta-a. Or RCT's at high risk of bias1- Meta-a. Or RCT's at high risk of bias 2++ High quality meta-analyses of 2++ High quality meta-analyses of

CC'sCC's 2+ Well-conducted cc or cohort 2+ Well-conducted cc or cohort

studiesstudies 2- Case-control or cohort studies 2- Case-control or cohort studies

with ? biaswith ? bias 3 Case reports3 Case reports 4 Expert opinion4 Expert opinion


““Effectiveness and Effectiveness and Efficiency: Random Efficiency: Random Reflections of Reflections of Health ServicesHealth Services “, “, 19711971

Archie Cochrane (1909-88)Archie Cochrane (1909-88)


http://www.google.com.au/imgres?imgurl=http://www.kkmk.hu/onszolg/eletrajz/kepek/nagy/cochrane.jpg&imgrefurl=http://www.kkmk.hu/onszolg/eletrajz/%3Flap%3DCo&usg=__mUMOS1lJqQSNrR6p8rIZ2nxDO-I=&h=400&w=300&sz=19&hl=en&start=1&itbs=1&tbnid=qR_e14VI5K4XLM:&tbnh=124&tbnw=93&prev=/images%3Fq%3DArchie%2BCochrane%26hl%3Den%26sa%3DG%26gbv%3D2%26tbs%3Disch:1

Randomized Controlled TrialsRandomized Controlled Trials

““Gold standard” in evaluating Gold standard” in evaluating new therapies or surgical new therapies or surgical techniquestechniques

May also be applied to new May also be applied to new diagnostic testsdiagnostic tests


Objectives of RCT:Objectives of RCT: Minimize bias by randomisationMinimize bias by randomisation Achieve statistical power through Achieve statistical power through

adequate sample sizeadequate sample size ““Blinding “ - single or doubleBlinding “ - single or double Analysis by intention to treatAnalysis by intention to treat




RandomisationRandomisation Several techniques availableSeveral techniques available Computer software linked to central Computer software linked to central

monitoring stationmonitoring station ““Block “ randomisationBlock “ randomisation Sealed envelope methodSealed envelope method



http://upload.wikimedia.org/wikipedia/commons/0/06/Flowchart_of_Phases_of_Parallel_Randomized_Trial_-_Modified_from_CONSORT_2010.jpg

What about observational studies?What about observational studies?


RCT’s and Observational StudiesRCT’s and Observational Studies

• Two studies published in the NEJM in 2000 suggested that RCTs and observational studies overall produced similar results

• JAMA 2001: “discrepancies beyond chance do occur and differences in estimated magnitude of treatment effect are very common”

• RCTs may be unnecessary for treatments that have dramatic and rapid effects relative to the expected


RCT’s and Industry FundingRCT’s and Industry Funding

•RCT’s funded by industry are significantly more likely to report positive results

•Possibly due to publication bias

•RCTs may be unnecessary for treatments that have dramatic and rapid effects relative to the expected


RCT’s and Statistical ErrorRCT’s and Statistical Error

• Type I error – “false positive”

• Type II error – “false negative”

• Sample size calculations often inaccurate


Diagnostic TestsDiagnostic Tests


2 X 2 Table2 X 2 Table

Disease present

Disease absent

Test Positive

a b

Test Negative

c d



Sensitivity = TP =Sensitivity = TP =

Disease present

Disease absent

Test Positive

a b

Test Negative

c d



Sensitivity = TP = a/(a+c)Sensitivity = TP = a/(a+c)

Disease present

Disease absent

Test Positive

a b

Test Negative

c d



False Positive Rate = FP = False Positive Rate = FP =

Disease present

Disease absent

Test Positive

a b

Test Negative

c d



FP = b/(d+b)FP = b/(d+b)

Disease present

Disease absent

Test Positive

a b

Test Negative

c d



Specificity = 1 - FP = Specificity = 1 - FP =

Disease present

Disease absent

Test Positive

a b

Test Negative

c d



Specificity = 1 - FP = d/(d+b)Specificity = 1 - FP = d/(d+b)

Disease present

Disease absent

Test Positive

a b

Test Negative

c d



Positive predictive value (PPV) =Positive predictive value (PPV) =

Disease present

Disease absent

Test Positive

a b

Test Negative

c d



PPV = a/(a+b)PPV = a/(a+b)

Disease present

Disease absent

Test Positive

a b

Test Negative

c d



Negative predictive value (NPV) = Negative predictive value (NPV) =

Disease present

Disease absent

Test Positive

a b

Test Negative

c d



NPV = d/(c+d)NPV = d/(c+d)

Disease present

Disease absent

Test Positive

a b

Test Negative

c d


PPV and PrevalencePPV and Prevalence

Steep drop in positive predictive value as Steep drop in positive predictive value as disease prevalence decreasesdisease prevalence decreases


PPV and PrevalencePPV and Prevalence

PPV = PPV = (sens x prev)(sens x prev) (sens x prev +(1 - spec)x(1 - prev))(sens x prev +(1 - spec)x(1 - prev))


The Likelihood RatioThe Likelihood Ratio

Single value to indicate Single value to indicate the clinical utility of a testthe clinical utility of a test

Independent of Independent of prevalenceprevalence

LR = Sensitivity/(1- Spec.)LR = Sensitivity/(1- Spec.) LR >8 : tests usually LR >8 : tests usually

clinically usefulclinically usefulDr Max Mongelli 2016

The Likelihood RatioThe Likelihood Ratio

LR is an odds modifier:LR is an odds modifier:

Posterior odds =Posterior odds =prior odds x LRprior odds x LR


Odds and ProbabilityOdds and Probability

Inter-convertible:Inter-convertible:

Odds = p/(1-p)Odds = p/(1-p)


Can tests be combined ?Can tests be combined ?

Rare conditions: high rates Rare conditions: high rates of false positivesof false positives

Lead to excessive Lead to excessive unnecessary interventionunnecessary intervention

Can be reduced by Can be reduced by combining tests e.g. combining tests e.g. intrapartum monitoringintrapartum monitoring


Impact of new diagnostic Impact of new diagnostic test on clinical outcomes:test on clinical outcomes:

RCTRCT Cohort studyCohort study Case-control studyCase-control study Before and after studyBefore and after study


SYSTEMATIC REVIEWSSYSTEMATIC REVIEWS


"It is surely a great criticism of our "It is surely a great criticism of our profession that we have not organised profession that we have not organised a critical summary, by specialty or a critical summary, by specialty or subspecialty, adapted periodically, of subspecialty, adapted periodically, of all relevant randomized controlled all relevant randomized controlled trials." trials."

Archie Cochrane, 1972Archie Cochrane, 1972


Role of systematic reviewsRole of systematic reviews

Before commencing a new project: to determine Before commencing a new project: to determine whether further studies are reallywhether further studies are really indicated: ‘state-indicated: ‘state-of-the-art’ literature review.of-the-art’ literature review.

Gain in statistical power for average estimates.Gain in statistical power for average estimates. 'Cumulative' meta-analysis can determine when 'Cumulative' meta-analysis can determine when

further studies are nofurther studies are no longer indicated.longer indicated. Design of subsequent studies.Design of subsequent studies. Setting policy for treatment and health care – Setting policy for treatment and health care –

making the best use of themaking the best use of the data available.data available.


Can Studies be Combined?Can Studies be Combined?

Identification of optimal inclusion criteria can be Identification of optimal inclusion criteria can be difficult.difficult.

The most critical step is choosing the appropriate The most critical step is choosing the appropriate research question.research question.

A fairly general question is more preferable to a very A fairly general question is more preferable to a very specific one. specific one.

Tukey : "...far better an approximate answer to the Tukey : "...far better an approximate answer to the right question, than an exact answer to the wrong right question, than an exact answer to the wrong question.."question.."


Publication BiasPublication Bias

Entire research studies may fail to reach publication Entire research studies may fail to reach publication because of the nature of the results.because of the nature of the results.

Identification of unpublished trials can be very Identification of unpublished trials can be very difficult - in one study it accounted for 22% of the difficult - in one study it accounted for 22% of the papers included in the meta-analysis. papers included in the meta-analysis.

Failure to publish rests with the investigators rather Failure to publish rests with the investigators rather than editors. than editors.


Comparison of the meta-analyses of smaller studies Comparison of the meta-analyses of smaller studies with the corresponding result of the largest study.with the corresponding result of the largest study.

30 meta-analyses including a total of 185 randomised 30 meta-analyses including a total of 185 randomised controlled studies (RCT) obtained from the Cochrane controlled studies (RCT) obtained from the Cochrane pregnancy and childbirth database. The meta-pregnancy and childbirth database. The meta-analyses were only included if they had at least one analyses were only included if they had at least one trial with a total sample size of over 1000.trial with a total sample size of over 1000.

Calculations differ from the Cochrane database in Calculations differ from the Cochrane database in that the largest trial was excluded, this being used as that the largest trial was excluded, this being used as the 'gold standard' for outcomethe 'gold standard' for outcome

PREDICTIVE ABILITY OF META-ANALYSESPREDICTIVE ABILITY OF META-ANALYSES Villar et al, Lancet 1995 Villar et al, Lancet 1995


There was total agreement between the meta-analysis and the largest study in 18/30 (60% [C.I. 42-78]) of comparisons.

There was partial agreement between the meta-analysis and the largest study in 6/30 (20% ) of comparisons.

There was disagreement in 6/30 (20% [C.I. 6-34] ) of comparisons.

PREDICTIVE ABILITY OF META-ANALYSESPREDICTIVE ABILITY OF META-ANALYSES Villar et al, Lancet 1995 Villar et al, Lancet 1995



The Cochrane Collaboration The Cochrane Collaboration

• Established in 1993 by Sir Iain Chalmers• International: 100 countries• Independent• Not-for-profit• Over 27000 contributors


RANZCOG and EBMRANZCOG and EBM

“RANZCOG endorses the principles of Evidence-based Medicine and recognizes the NHMRC levels of evidence and grades of recommendations”

College Statement C-Gen 15, Nov. 2009


…a scientific idea can never be proven true, because no matter how many observations seem to agree with it, it may still be wrong. On the other hand, a single contrary experiment can prove a theory forever false…

Sir Karl Popper


Date post:	15-Apr-2017
Category:	Health & Medicine
Upload:	dr-max-mongelli
View:	188 times
Download:	6 times

The ABC of Evidence-Base Medicine

Health & Medicine