Assessing DiagnosticAssessing DiagnosticArticles: Likelihood RatiosArticles: Likelihood Ratios

Evidence Based Medicine:Evidence Based Medicine:Pedi 101, Part 1Pedi 101, Part 1

Section of Emergency MedicineSection of Emergency MedicineBaylor College of MedicineBaylor College of Medicine

Houston, TexasHouston, Texas

The patientencounter

Formulating theclinical question

Searching theevidence

Appraising theevidence

Uncertainty indiagnosis ortherapy,controversies

PatientInterventionComparisonOutcome

Hierarchy ofevidencePre-appraisedresources

Drawingconclusionsthat impacton practice

SAEM. Evidence Based Medicine Online Course 2005

Review the PICO questionReview the PICO question

Patient InterventionComparison(if relevant)

Outcome

In children withfever andpetechiae

- does well-appearing vs

not well-appearing

predictbacteremia?

PICO questions can bePICO questions can becategorized:categorized:

EtiologyEtiology

DiagnosisDiagnosis

TherapyTherapy

PrognosisPrognosis

Clinical queryClinical query

Where do we look for answers?Where do we look for answers?

Possible resources:Possible resources:

www.www.cochranecochrane.org.org

http://ebm.bmjjournals.comhttp://ebm.bmjjournals.com

www.library.tmc.eduwww.library.tmc.edu

Up to DateUp to Date

Hierarchy of EvidenceHierarchy of Evidence

MetaMeta--analysis ofanalysis of RCTsRCTssystematic review ofsystematic review of RCTsRCTs

Individual RCTIndividual RCT

Observational studiesObservational studiespatientpatient--important outcomesimportant outcomes

Basic researchBasic researchtest tube, animal, human physiologytest tube, animal, human physiology

Clinical experienceClinical experienceJAMA. User’s Guide to the Medical Literature 2002.

Critical Appraisal: DiagnosisCritical Appraisal: Diagnosis

ValidityValidity Will the study reflect the truth?Will the study reflect the truth?

Rapid review: Do they give a search strategy andRapid review: Do they give a search strategy andassess quality?assess quality?

ImportanceImportance Are the effects big enough to matter?Are the effects big enough to matter?

Rapid review: How precise is the answer?Rapid review: How precise is the answer?

ApplicabilityApplicability Is it useful here and now?Is it useful here and now?

Rapid review: Are the costs similar? Is the benefitRapid review: Are the costs similar? Is the benefitworth the cost?worth the cost?

Defining riskDefining risk

PrePre--test and posttest and post--test probabilitiestest probabilities

Likelihood ratiosLikelihood ratios

Sensitivity and specificity of a testSensitivity and specificity of a test

PrePre--test probabilitytest probability

Defined as the prevalenceDefined as the prevalence

The probability of a disease beforeThe probability of a disease beforeany testing resultsany testing results

The assumptions we function withThe assumptions we function with

CaseCase

7 year old with7 year old withsore throat,sore throat,tonsillartonsillarhypertrophy,hypertrophy,cervicalcervicallymphadenopathylymphadenopathy,,and absence ofand absence ofcoryzacoryza..

Google Images, 2005. “pharyngitis”

Given minimal but appreciable risksGiven minimal but appreciable risksfrom therapy with Penicillin,from therapy with Penicillin,

would you treat ifwould you treat if……

the probability ofthe probability ofdisease were 70%?disease were 70%?

the probability ofthe probability ofdisease were 98%?disease were 98%?

A diagnostic test (rapidA diagnostic test (rapidstrep), can change yourstrep), can change yourprobability of disease.probability of disease.

This is a postThis is a post--test probability.test probability.

PostPost--test probabilitytest probability

Probability of having a condition after theProbability of having a condition after thetest result is availabletest result is available

The reason a test is obtained is toThe reason a test is obtained is toincrease or decrease the probability ofincrease or decrease the probability ofdisease that existed prior to the testdisease that existed prior to the test

CaseCase

In the example, theIn the example, theprepre--test probabilitytest probabilitywas 70%, letwas 70%, let’’ssassume the postassume the post--test probability withtest probability witha positive rapida positive rapidstrep is now 98%.strep is now 98%.Now would youNow would youtreat?treat?

Google Images, 2005. “pharyngitis”

In essence, we are interestedIn essence, we are interestedin how a diagnostic test altersin how a diagnostic test alters

the likelihood of a disease.the likelihood of a disease.

Likelihood ratios.Likelihood ratios.

Understanding likelihoodUnderstanding likelihoodratiosratios……

what does it mean to the clinician?what does it mean to the clinician?

DiseasePresent

DiseaseAbsent

TestPositive

TruePositive

FalsePositive

TestNegative

FalseNegative

TrueNegative

Sensitivity:Sensitivity:

The proportion ofThe proportion ofpatients with thepatients with thedisorder under studydisorder under studywho have a positivewho have a positiveresult.result.

The probability thatThe probability thatgiven the disease isgiven the disease ispresent, an abnormalpresent, an abnormaltest result indicatestest result indicatespresence of disease.presence of disease.

SensitivitySensitivity

=TP/(TP+FN)=TP/(TP+FN)

DiseasePresent

DiseaseAbsent

TestPositive

TruePositive

FalsePositive

TestNegative

FalseNegative

TrueNegative

Specificity:Specificity:

The proportion ofThe proportion ofpatients who do notpatients who do nothave the target disorderhave the target disorderand whose test resultsand whose test resultsare negative.are negative.

The conditionalThe conditionalprobability that given theprobability that given thedisease is absent, adisease is absent, anormal test resultnormal test resultexcludes the disease.excludes the disease.

SpecificitySpecificity

=TN/(FP+TN)=TN/(FP+TN)

Determining postDetermining post--test probabilitytest probability

CalculateCalculate

Or do it theOr do it thesimple waysimple waywith awith anomogramnomogram!!

Pre-TestProbability

Multiply by LR

Post-TestProbability

Convert to Post- Test Odds

Convert to Pretest Odds

Adapted from Comprehensive Case Studies: ACEP, EBM Listserve 2003

Relationship between odds andRelationship between odds andprobabilityprobability

To use LR: convert between odds and probabilityTo use LR: convert between odds and probability

yProbabilit1

yProbabilitOdds

1Odds

OddsyProbabilit

What if we donWhat if we don’’t know thet know thelikelihood ratio?likelihood ratio?

Likelihood ratio positiveLikelihood ratio positive

The LR for a positive test result is theratio of true positive results to falsepositive results.

A helpful test will have a large LR positive.

yspecificit1ysensitivite)LR(positiv

Likelihood ratio negativeLikelihood ratio negative

The LR for a negative test result is theratio of false negative results to truenegative results.

A helpful test will have a small LRnegative.

yspecificitysensitivit-1e)LR(negativ

TheThe nomogramnomogram

From: Fagan,From: Fagan, NomogramNomogram forfor BayesBayes’’ Theorem.Theorem.NEJM, 1975: 293: 257NEJM, 1975: 293: 257

Adapted from Comprehensive Case Studies: ACEP,EBM Listserve 2003

TheThe nomogramnomogram!!

For our child withFor our child withpharyngitispharyngitis: assume: assumethe rapid strep used inthe rapid strep used inthe unit carries a LR+the unit carries a LR+of 25.of 25.

For our rapid strep (QFor our rapid strep (Q--test)test)

Given a sensitivity of 94.2% and a specificityGiven a sensitivity of 94.2% and a specificityof 96.3%:of 96.3%:

yspecificit1ysensitivite)LR(positiv

25.9631

.942e)LR(positiv

TheThe nomogramnomogram!!

For our child withFor our child withpharyngitispharyngitis: the rapid: the rapidstrep used in the unitstrep used in the unitreally does carry areally does carry aLR+ of 25.LR+ of 25.

SummarySummary

Related EBM to practiceRelated EBM to practice

PICO questionsPICO questions

QueriesQueries

Diagnosis studiesDiagnosis studies

Pre and post test probabilitiesPre and post test probabilities

Likelihood ratiosLikelihood ratios

Optional case exerciseOptional case exercise

2 1/2 year old girl is brought in by mother2 1/2 year old girl is brought in by mother

Fever to 103Fever to 103°°FF

You remember a SBI lecture and wonderYou remember a SBI lecture and wonderabout the etiology of fever as it relates toabout the etiology of fever as it relates toUTIUTI

PICO: In children with fever without aPICO: In children with fever without afocus, what is the prevalence of UTI?focus, what is the prevalence of UTI?

Query:Query: www.library.tmc.eduwww.library.tmc.edu

DiagnosisDiagnosis

9% of children 09% of children 0--36 months with fever in36 months with fever inthe ED have a UTIthe ED have a UTI

Urine dipstick used to measure leukocyteUrine dipstick used to measure leukocyteesterase and bacterial nitrates.esterase and bacterial nitrates.

Sensitivity and specificity are:Sensitivity and specificity are:Sensitivity = 0.40 & Specificity = 0.98Sensitivity = 0.40 & Specificity = 0.98

Is a urine dipstick going to change yourIs a urine dipstick going to change yourdecisiondecision--making?making?

Adapted from Comprehensive Case Studies: ACEP, EBM Listserve 2003

Calculating LR:Calculating LR:

PrePre--testtestprobability isprobability is9%9%

Pre-TestProbability

Multiply by LR

Post-TestProbability

Convert to Post- Test Odds

Convert to Pretest Odds

Adapted from Comprehensive Case Studies: ACEP, EBM Listserve 2003

DiagnosisDiagnosis

Sensitivity = 0.40Sensitivity = 0.40 Specificity= 0.98Specificity= 0.98

LR+ =LR+ = SensSens / (1/ (1 -- Spec) = 0.4 / 0.02 = 20Spec) = 0.4 / 0.02 = 20

LRLR-- = (1= (1 -- Sens) / Spec = 0.6 / 0.98 = 0.61Sens) / Spec = 0.6 / 0.98 = 0.61

Pretest odds of disease = 0.09 / 0.91 = 0.1Pretest odds of disease = 0.09 / 0.91 = 0.1

BayesBayes' theorem: 0.1 x 20 = 2 = Post test' theorem: 0.1 x 20 = 2 = Post testodds of a UTI.odds of a UTI.

Post test probability = 2 / 3 = 0.67Post test probability = 2 / 3 = 0.67Adapted from Comprehensive Case Studies: ACEP, EBM Listserve 2003

Using the FaganUsing the Fagannomogramnomogram

LR+

Pretest probability

Posttest probability

Adapted from Comprehensive Case Studies: ACEP, EBM Listserve 2003

DiagnosisDiagnosis

This means that the patient is very likelyThis means that the patient is very likely(67%) to have a UTI.(67%) to have a UTI.

Similarly for a negative test:Similarly for a negative test:

LRLR-- = (1= (1 -- SensSens) / Spec = 0.6 / 0.98 = 0.61) / Spec = 0.6 / 0.98 = 0.61

BayesBayes' theorem: 0.1 x 0.61 = 0.06,' theorem: 0.1 x 0.61 = 0.06,

Post test odds = 0.06Post test odds = 0.06

If test is negative, still 6% chance of UTI.If test is negative, still 6% chance of UTI.

Adapted from Comprehensive Case Studies: ACEP, EBM Listserve 2003

SummarySummary

Related EBM to practiceRelated EBM to practice

PICO questionsPICO questions

QueriesQueries

Diagnosis studiesDiagnosis studies Pre and post test probabilitiesPre and post test probabilities

Likelihood ratio: how many times more likelyLikelihood ratio: how many times more likelyis the test to be positive when the disease isis the test to be positive when the disease isthere than when the test is positive but thethere than when the test is positive but thedisease is absentdisease is absent

Questions?Questions?

CreditsCredits

Society for Academic Emergency Medicine:Society for Academic Emergency Medicine:Evidence Based Medicine Course 2005Evidence Based Medicine Course 2005

JAMA. UserJAMA. User’’s Guides to the Medical Literatures Guides to the Medical Literature20022002

BarrattBarratt A, et al. Risk Reduction. 2005. EBMA, et al. Risk Reduction. 2005. EBMTeaching Tips Working GroupTeaching Tips Working Group

Moyer V et al. Evidence Based Pediatrics andMoyer V et al. Evidence Based Pediatrics andChild Health 2000 BMJ BooksChild Health 2000 BMJ Books

Katz D. Clinical epidemiology and evidenceKatz D. Clinical epidemiology and evidencebased medicine 2001 Sage Publicationsbased medicine 2001 Sage Publications