Studying Diagnosis

7/30/2019 Studying Diagnosis

1/24

UniversitaireMasterstudieEvidenceBasedPracticeAMC-UvA

1

Module 2

Epidemiology and

Evidence Based Practice:

Designs

Lecture 6

Probabilistic research II:

Studying diagnosis

UniversitaireMast

erstudieEvidenceBasedPracticeAMC-UvA

22

Learning objectives Lecture 6

Student can describe and explain theanalysis of research studyingdiagnosis, i.e. sensitivity/specificity,predictive values, likelihood ratios,ROC-curves

Student can name methodologicalissues concerning internal and externalvalidity in research studying diagnosis

UniversitaireMasterstudieEvidenceBasedPracticeAM

C-UvA

3

Agenda

1. Intro diagnostics

2. Studying Diagnosis: analysis(sensitivity/specificity, predictive values,likelihood ratios, ROC-curves)

3. Studying Diagnosis: methodology(cross-sectional design)


2/24


4

1.Intro diagnostics


55

Probabilistic research

Predictive relation between one ormore determinants and one(diagnosis) or more (prognosis)outcomes

Descriptive

No interest in confounding Data collection reflects practice

Prognosis, Diagnosis


C-UvA

66

Diagnosis

Predicting the presence, type, severityof disease based on patients profile

- Individual prediction based on clinical profile

- Preferably multiple determinants (i.e. tests)

- Often one outcome

- Transversal


3/24


7

Diagnostic reasoning

Descriptive- pattern recognition

- hypothesis testing

Probabilistic

- rational and quantitative

- from pre-test to post-test probabilities


8

Thomas Bayes (1702-1761)

Estimating probabilitiesfrom new data usingpre-existing knowledge

Bayes theorem:

Post-test probability = Pre-test probability * X


C-UvA

9

Clinical decision-makingWill the results of the test move a decision

across the test (1) or treatment (2) threshold?

1 2

Do not test

Do not treat

Do not test,

but treat now!

Test and, depending on

the results, treat (or not)

pre-test probability


4/24

History &

Examination

Multistage, multidimensional diagnostic process

Presenting

symptoms

Further

Testing 1

TherapyPatient

Outcomes

Further

Testing 2

No therapyPatient

Outcomes

Cave: focus on single disease

Test A or B?

risk reduction as observed in trial

+ risk side effects

Test A

True positives [p*seA]

True negatives [(1-p)*spA]

Test B

False positives [(1-p)*(1-specA)]

False negatives [p*(1-seA)]

True positives [p*seB]

False positives [(1-p)*(1-specB)]

False negatives [p*(1-seB)]

True negatives [(1-p)*spB]

Treat

Treat

No Treat

Treat

Treat

No Treat

No Treat

No Treat

risk side effects, no treatment effect

risk untreated

none

risk reduction as observed in trial

+ risk side effects

risk untreated

none

Sutton et al. Integration of meta-analysis and economic decision modelling for evaluating tests. MDM 2008

risk side effects, no treatment effect

p = prevalence

se = sensitivity

sp = specificity


C-UvA

1. Classical diagnostic thinking

Uni- or multivariable approach:

probability of disease being present givenclinical profile

one or multiple pieces of information

estimate independent contribution (weight)

uni- or multivariable analysis

clinical decision rules

Goal: better prediction/discrimination


5/24


Tests can be

history

physical examination

blood tests

imaging procedures

questionnaires

etcetera


14

2.

Studying Diagnosis:

analysis


C-UvA

Diagnostic accuracy

The extent to which the test results reflectthe true state

The ability of a test to discriminate amongpatients with and without the suspecteddisease


6/24

How accurate is the test?

R e f e r e n c e t e s t

R e f e r e n c e t e s t R e f e r e n c e t e s t

R e f e r e n c e t e s t

I

n

d

e

x

t

e

s

t

I

n

d

e

x

t

e

s

t

I

n

d

e

x

t

e

s

t

I

n

d

e

x

t

e

s

t

P

a

t

i

e

n

t

N

o

r

m

a

l

P a t i e n t

N o r m a l

Threshold

TP

True Positive

FP

False Positive

FN

False Negative

TN

True Negative


17

Basic design

Patients with a suspected disease

Index test

Reference test (gold standard)

Compare test results


C-UvA

18

Clinical problem

Patient with chest pain suggestive foracute myocardial infarction (AMI)

Does this patient have an AMI?


7/24


19

Clinical problem

Diagnostic value of creatine kinase(CK) measurement

Does CK measurement distinguishbetween those with and withoutmyocardial infarction?


20

Anatomy of the study

Target population: patients with chest pain

Index test: CK measurement

Target condition: acute myocardial infarction

Final diagnosis based on WHO criteria(reference standard):

clinical outcome ECG-changes

enzym values

autopsy


C-UvA

21

Our example

Patients with chest pain

CK measurement

WHO criteria for AMI

Cross-classification


8/24


22

Results of CK study

231

a bc d

129

360130230

CK

AMI

high

(>80)

low

Present Absent

16215

15 114


23

Measures of accuracy

360130230

16215

15 114

231

129

CK

AMI

high

(>80)

low

Present Absent

sensitivity 215 / 230 = 93% < Pr(T+|D+) >

specificity 114 / 130 = 88% < Pr(T-|D-) >

Cut-off value


9/24


25

Accuracy: Sens/Spec

dichotomous test outcomes depend on cut-off value

(trade-off; FP/FN)

independent of disease prevalence

Se/Sp of a single test may vary widelyacross studies

Se/Sp: from disease status to test result(testing the test)


26

Measures of accuracy

360130230

16215

15 114

231

129

CK

AMI

high

(>80)

low

Present Absent

PPV 215 / 231 = 93% < Pr(D+|T+) >

NPV 114 / 129 = 88% < Pr(D-|T-) >


C-UvA

PrevalencePrevalence: percentageof patients with the target

disorder at a certain point in time

formula: (TP + FN) / N

reference / standard

positive negative

index test positive TP FP TP+FP

negative FN TN FN+TN

TP+FN FP+TN N


10/24


Effect of prevalence on measures

of outcome

D-dimer test in GP practice

(low prevalence DVT)

D-dimer test in academic hospital

(high prevalence DVT)


High prevalence

DVT no DVT

D-dimer positive 215 16 231

negative 15 114 129

230 130 360

prevalence = 230 / 360 = 64%

Sens = 215 / 230 = 0.93

Spec = 114 / 130 = 0.88

PPV = 215 / 231 = 0.93

NPV = 114 / 129 = 0.88


C-UvA

Low prevalence

DVT no DVT

D-dimer positive 215 248 463

negative 15 1822 1837

230 2070 2300

prevalence = 230 / 2300 = 10%

Sens = 215 / 230 = 0.93

Spec = 1822 / 2070 = 0.88

PPV = 215 / 463 = 0.46

NPV = 1822 / 1837 = 0.99


11/24


31

Accuracy: PVs

post-test probabilities

dichotomous test outcomes

depend on Se/Sp

dependent upon prevalence

PVs: from test result to disease status(testing the patient)


32

Estimating post-test probabilities

Unknown prevalence:

- SpPin/SnNout

Known prevalence:

- directly use PVs (similar prevalence)

- calculate PVs using Se/Sp (prevalence not similar)

- use likelihood ratios (LR +/-)


C-UvA Unknown prevalence:

ruling out the target disorder

If a test has a sufficiently high Sensitivity,

a Negative result rules outthe target disorder

the rule of SnNout

low FN rate

TNFN

FPTPe.g. breast cancer screening


12/24

UniversitaireMasterstudieEvidenceBasedPracticeAMC-UvA Unknown prevalence:

ruling inthe target disorder

If a test has a sufficiently high Specificity,

a Positive result rules inthe target disorder

the rule of SpPin

low FP rate

TNFN

FPTP


35

Likelihood ratios

Direct link from pre-test probabilities topost-test probabilities

Applicable in situations with more than

two test outcomes


C-UvA

36

Likelihood ratios Summarize predictive power of a test

result in a single measure

Likelihood ratio of a positive andnegative test result

How more often a positive test resultoccurs in persons with compared tothose without the target condition


13/24


37

Positive likelihood ratio

Likelihood ratio of a positive test result

How more likely a positive test result isin persons with the target conditioncompared to those without the targetcondition


38

Positive likelihood ratio

360130230

16215

15 114

231

129

CK

AMI

high(>80)

low

Present Absent

6.7130/16

230/215

)|Pr(

)|Pr(==

+

++=+

DT

DTLR


C-UvA

39

Negative likelihood ratio

Likelihood ratio of a negative test result

How less likely a negative test result isin persons with the target conditioncompared to those without the targetcondition


14/24


40

Negative likelihood ratio

07.0130/114

230/15

)|Pr(

)|Pr(==

+=

DT

DTLR

360130230

16215

15 114 129

CK

AMI

high

(>80)

low

Present Absent

231


41

64%

93%

LR+ = 7.6


C-UvA

42

Advantages of likelihood ratios

Relatively stable (function of Se/Sp)

Still useful when there are more thantwo test outcomes


15/24


43

CK is a continuous measurement

Dichotomisation of CK (high vs low)means loss of information

Higher values of CK are moreindicative of myocardial infarction


44

Results of CK study

CK Present Absent Total

280 97 1 98

80-279 118 15 133

40-79 13 26 39

1 - 39 2 88 90

Total 230 130 360

MI


C-UvA

45

Likelihood ratios

Stratum-specific likelihood ratios incase of more than two test results

)|Pr(

)|Pr()(

=

+===

DxT

DxTxTLR


16/24


46

Stratum-specific likelihood ratios

8.54130/1230/97

)|280Pr()|280Pr()280( ==

+=

DCKDCKCKLR

CK Present Absent

280 97 1

80-279 118 15

40-79 13 26

1 - 39 2 88

Total 230 130

MI


47

Stratum-specific likelihood ratios

CK Present Absent LR

280 97 1 54.83

80-279 118 15 4.45

40-79 13 26 0.28

1 - 39 2 88 0.01

Total 230 130 360

MI


C-UvA

48

Bayes rule

Post-test odds for disease

=

Pre-test odds for disease * Likelihood ratio


17/24


49

Bayes rule

odds = probability / (1 probability)

probability = odds / (1 + odds)

)Pr(1

)Pr()(Odds

+

+=+

D

DD

)(Odds1

)(Odds)(Pr

++

+=+

D

DD


50

Bayes rule:patient with CK between 80-279

Pre-test probability = 0.5

Pre-test odds = 0.5 / (1-0.5) = 1

LR(CK 80-279) = 4.45

Post-test odds = 1 * 4.45 = 4.45 Post-test probability = 4.45 / (1+4.45)

= 0.82


C-UvA

51

Bayes rule:

patient with CK lower than 40

Pre-test probability = 0.5

Pre-test odds = 0.5 / (1-0.5) = 1

LR(CK < 40) = 0.013

Post-test odds = 1 * 0.013 = 0.013

Post-test probability = 0.013 / (1+0.013)

= 0.013


18/24


52

Probability of MI after CK

CK LR

280 54.83

80 - 279 4.45

40-79 0.28

1-39 0.013

Pre-test prob.

Post test prob.

50%

98%

82%

22%

1%


53

Probability of MI after CK

CK LR 5% 50%

280 54.83 74% 98%

80 - 279 4.45 19% 82%

40-79 0.28 1% 22%

1-39 0.013 0% 1%

Pre-test prob.

Post test prob.


C-UvA

Classification of LR values

>10 and


19/24


55

ROC-curve

ROC stands for Receiver OperatingCharacteristic

ROC-curve shows the pairs ofsensitivity and specificity thatcorrespond to various cut-off points forthe continuous test result


56

Cut-off value

MI-patientsMI absent

TN

FP

CK measurement


C-UvA

57

Cut-off value

MI-patientsMI absent

FNTP

CK measurement


20/24


58

Change in cut-off value

MI absent MI present

CK measurement

SpecFP


59


MI absent MI present

CK measurement

FN Sens


C-UvA

60


and effect on Se/Sp

Cut-off Sensitivity Specificity

9999 0.0% 100.0%

280 42.2% 99.2%

80 93.5% 87.7%

40 99.1% 67.7%

1 100.0% 0.0%


21/24


61

0%

20%

40%

60%

80%

100%

0% 20% 40% 60% 80% 100%

1-specificity

Sensitivity

ROC-curve CK

Cut-off: 280

Cut-off: 80


62

ROC-curve

AUC 0.91


C-UvA

63

ROC-curve

Shows the effect of different cut-offvalues on sensitivity and specificity

Better tests have curves that lie closer tothe upper left corner

Area Under the ROC-Curve (AUC) is asingle measure of test performance(0-1, higher is better)


22/24


64

So far,

Accuracy: how close to the true state?

- measures of prediction

(Se/Sp, PVs, LRs)

Accuracy: distinguishing between patients

- measures of discrimination

(DOR, ROC with AUC)


65

3.

Studying Diagnosis:

methodology


C-UvA

66

Basic design

Patients with a suspected disease

Index test

Reference test (gold standard)

Compare test results


23/24


67

WhitingP et al. Ann Intern Med 2004;140:189-202


2. More modern diagnostic thinking

Determine the most likely role and position of anew test within the test-treatment pathwayrelative to current practice- replacement, add-on, triage

Compare differences in:

- test safety etc.- Se/Sp

- treated population

- management following positive (TP/FP) and negative (TN/FN)

test results

- treatment effects

- patient outcomes


C-UvA

69

Copyright2006 BMJ Publishing Group Ltd.

Bossuyt et al. BMJ 2006;332:1089-92

Roles of tests and positions in existing diagnostic pathways


24/24


70

Van den Bruelet al. J Clin Epidemiol2007;60:1116-22

Test evaluation research


Further reading

Knottnerus JA, Buntinx F (Ed.). Theevidence base of clinical diagnosis. Theoryand methods of diagnostic research, 2e

edition. Blackwell Publishing, 2009

Newman TB, Kohn MA. Evidence-baseddiagnosis. Cambridge University Press, 2009

Date post:	14-Apr-2018
Category:	Documents
Upload:	camilo-parra-cuadros
View:	217 times
Download:	0 times

Studying Diagnosis

Documents