Evidence Based MedicineWorkshop

DiagnosisMarch 18, 2010

Objectives - Diagnosis

• At today’s end you’ll be able to…– consider rationale for diagnosis– define and calculate test characteristics– state the ideal research design for studying

diagnostic tests– critically appraise articles about diagnostic

testing

Seemingly Dumb Question…

• Why make a diagnosis?

Why Diagnose?Heart Failure Therapy / Prognosis

• What are advantages and disadvantages of diagnostic testing?

Diagnostic Testing

• Advantages– can assess parameters beyond the 5 senses– can be more ‘objective’ than clinical data

• Disadvantages– test results can be incorrect– test results may lead you in the wrong direction– tests cost money– tests may confer risk– some diseases have no diagnostic test– tests may add little to what is already known

Issues in Diagnostic Testing

• Invasiveness– urine sample versus brain biopsy versus autopsy

• Cost– glucoscan strip ~ $1.00 versus MRI $900.00

• Availability– hemogram versus Positive Emission Tomogram

• Patient Acceptability– urine sample versus 3 day fecal fat collection

Let’s Back Up…

• What type of ‘testing’ is the cheapest, lowest risk, available anywhere, and needs no requisitions?

Clinical Scenario

• A 70 year old man – presents to the ED– 1 hr x chest pain & shortness during 10 hr car trip

• PMH– prostate cancer

• Exam– distressed with splinting respiration (pleuritic cp)– HR 130 / min, RR 32 / min.

• What’s your working diagnosis?

Test for Pulmonary Embolism

• Gold Standard: pulmonary angiogram– invasive– costly– not readily available– risky

• Other tests:– D-dimer, V/Q scans, Spiral CT scan– ? may be helpful in right setting with right results

- complex

PE - diagnosis

Pulmonary angiogram

- gold standard

PE - diagnosis (spiral CT scan)

PE - diagnosis (V/Q scan)

• high probability V/Q scan (2 defects)

Pulmonary Thromboembolism

How well does the test perform?

• Welcome to the world of

TEST CHARACTERISTICS

Take a deep breath...

Test Characteristics

• Sensitivity

• Specificity

• Positive predictive value

• Negative predictive value

• Accuracy

• Likelihood ratio

Individual A

Cross-sectional survey: measure disease status & test status at same time point.

Inidividual D

Inidividual C

Individual B

D+ & T+D+ & T-D- & T+D- & T-

DISEASE

Present Absent

Positive TRUE POSITIVE

FALSE POSITIVE

TEST

Negative FALSE NEGATIVE

TRUE NEGATIVE

DISEASE (PE)

Present Absent

Positive TRUE

POSITIVE a = 80

FALSE POSITIVE

b = 20 a + b = 100

TEST (V/Q

scan) Negative

FALSE NEGATIVE

c = 10

TRUE NEGATIVE

d = 90 c + d = 100

a + c = 90 b + d = 110 a+b+c+d = 200

Hypothetical Test Results

DISEASE (PE)

Present Absent

Positive TRUE

POSITIVE a = 80

FALSE POSITIVE

b = 20 a + b = 100

TEST (V/Q

scan) Negative

FALSE NEGATIVE

c = 10

TRUE NEGATIVE

d = 90 c + d = 100

a + c = 90 b + d = 110 a+b+c+d = 200

SensitivityProbability that test is positive given that disease is present.

80 / (80 + 10) = 88.9%

DISEASE (PE)

Present Absent

Positive TRUE

POSITIVE a = 80

FALSE POSITIVE

b = 20 a + b = 100

TEST (V/Q

scan) Negative

FALSE NEGATIVE

c = 10

TRUE NEGATIVE

d = 90 c + d = 100

a + c = 90 b + d = 110 a+b+c+d = 200

Specificity Probability that test is negative given that disease is absent.

90 / (90 + 20) = 81.8%

Sensitivity / Specificity Trade-off

Sensitivity DecreasesSpecificity Increases

Test Characteristic Issues

• Highly Sensitive Tests:– tend to be less invasive, less risky, less costly– best for screening programs– best for ruling out disease: “SNOUT”

• Highly Specific Tests:– tend to be more invasive, more risky, more

costly– best for confirming (ruling in) disease: “SPIN”

DISEASE (PE)

Present Absent

Positive TRUE

POSITIVE a = 80

FALSE POSITIVE

b = 20 a + b = 100

TEST (V/Q

scan) Negative

FALSE NEGATIVE

c = 10

TRUE NEGATIVE

d = 90 c + d = 100

a + c = 90 b + d = 110 a+b+c+d = 200

Positive Predictive Value

Probability that disease is present given that the test was positive.

80 / (80 + 20) = 80.0%

DISEASE (PE)

Present Absent

Positive TRUE

POSITIVE a = 80

FALSE POSITIVE

b = 20 a + b = 100

TEST (V/Q

scan) Negative

FALSE NEGATIVE

c = 10

TRUE NEGATIVE

d = 90 c + d = 100

a + c = 90 b + d = 110 a+b+c+d = 200

Negative Predictive ValueProbability that disease is absent given that the test was negative.

90 / (90 + 10) = 90.0%

Issue

Sensitivity / Specificity

versus

Positive / Negative Predictive Values

Change Disease Prevalence from 90 to 110 per 200

DISEASE (PE)

Present Absent

Positive

TRUE POSITIVE

a = 80 97.7

FALSE POSITIVE

b = 20 16.4

a + b = 114.1 TEST

(V/Q scan)

Negative

FALSE NEGATIVE

c = 10 12.2

TRUE NEGATIVE

d = 90 73.6

c + d = 85.8

a + c = 90

110 b + d = 110 90

a+b+c+d = 200

prevalence = 110 / 200 = 0.55 = 55% (was 45%)

sensitivity = 97.7 / 110 = 88.8% (unchanged)specificity = 73.6 / 90 = 81.7% (unchanged)

positive predictive value = 86.5% (was 80%)negative predictive value = 85.8% (was 90%)

DISEASE (PE)

Present Absent

Positive TRUE

POSITIVE a = 80

FALSE POSITIVE

b = 20 a + b = 100

TEST (V/Q

scan) Negative

FALSE NEGATIVE

c = 10

TRUE NEGATIVE

d = 90 c + d = 100

a + c = 90 b + d = 110 a+b+c+d = 200

Accuracy

(80+90) / (80+ 20 + 10 + 90) = 85.0%

Positive (test) Likelihood Ratio

• Ratio of:

probability of positive test when disease is present

--------------------------------------------------------------------

probability of positive test when disease is absent

DISEASE (PE)

Present Absent

Positive TRUE

POSITIVE a = 80

FALSE POSITIVE

b = 20 a + b = 100

TEST (V/Q

scan) Negative

FALSE NEGATIVE

c = 10

TRUE NEGATIVE

d = 90 c + d = 100

a + c = 90 b + d = 110 a+b+c+d = 200

Positive Likelihood Ratio

(80 / 90) / (20 / 110) = 4.89

Utility of LRPretest odds x Likelihood Ratio = Posttest odds

Palpable 5.6Screen 2.2

Critical Appraisal of an Article about Diagnosis

Validity

Results

Applicability

Validity• Primary Guides

– Was there an independent, blind comparison with a reference standard?

– Did the patient sample include an appropriate spectrum of patients to whom the diagnostic test will be applied in clinical practice?

• Secondary Guides– Did the results of the test being evaluated influence the

decision to perform the reference standard?– Were the methods for performing the test described in

sufficient detail to permit replication?

Results

• Are likelihood ratios for the test results presented or data necessary for their calculation provided?

 

LRs >10 generate large changes from pre- to post-test probability;

LRs 5-10 generate moderate changes in pre- to post-test probability;

LRs 2-5 generate small (but sometimes important) changes in probability;

LRs 1-2 alter probability to a small (and rarely important) degree.

Applicability

• Will the reproducibility of the test result and its interpretation be satisfactory in my setting?

• Are the results applicable to my patient?

• Will the results change my management?

• Will patients be better off as a result of the test?

Sensitivity: 670 / 744 = 0.90 or 90%Specificity: 640 / 842 = 0.76 or 76%PPV: 670 / 872 = 0.77 or 77%NPV: 640 / 714 = 0.90 or 90%Accuracy: (670 + 640) / 1586 = 0.83 or 83%

Positive Likelihood Ratio (670 / 744) / (202 / 842) = 3.75

Let’s Design a Study…

End of the Line…