1

Credible Intervals, Bayes Theorem +Diagnostic Tests

2

Outline

• Credibile Intervals – Posterior Distribution and Bayes Theorem

• Sensitivity

• Specificity

• Positive Predictive Value

• ROC curve

See Pagano- Chapter 6- section 1-4

3

Credibile Intervals• For Prob(Smoking)=p in a Population:

– p could be 0.05, 0.10, … 0.90, 0.95,1– Prob of p: (prior probability)– Data: x=4 out of n=10 people smoke

• Get Posterior Distribution using Bayes Theorem

p4

ˆ 0.410

p

• Credible Interval: 95% Credible Interval: 2.5th and 97.5th percentile of posterior distribution

• Example: Suppose the prior probability is the same for all p (uniform prior)

p p

,|

ˆ|

ˆ|

p p

p pall all

P Data pP p Data

P Data

P Data p p

P Data p p

p p p

ˆ ˆ| | ||

ˆ ˆ| | |all all all

cP Data p P p p L p pP p Data

cP Data p P p p L p p

p c

|P p Data

p

2.5%2.5%

Credible Interval

Posterior Distribution

4

Diagnostic Tests

• Diagnostic tests are routinely used to detect disease

• Events related to individual’s health status:– Individual has disease (D)– Individual is disease free (Dc)

• Outcomes of a diagnostic test:– Positive test result (T+)– Negative test result (T-)

5

Diagnostic tests

D = “have disease”

Dc =“do not have disease”

T+=“positive screening result”

Find P(D |T+)

Diagnostic Tests

Find the probability that an individual who tests positive actually has disease

6

Diagnostic Tests

• Positive predictive value = P(D | T+)

• Sensitivity = P(T+ | D)

• Specificity = P(T- | Dc )

• Prevalence = P(D)

7

Example: X-ray screening for tuberculosis

X-ray

Tuberculosis

Yes

Positive

x=22

Negative

8

Total n=30

22Sensitivity .7333

3

0 = =

ˆx

pn

8

Example: X-ray screening for tuberculosis

X-ray

Tuberculosis

Yes No

Positive

51

Negative

x=1739

Totaln=179

01739Specificity .9715

1790= =

ˆx

pn

9

Example: X-ray screening for tuberculosis

X-ray

Tuberculosis

TotalYes No

Positive

22 51 73

Negative

8 1739 1747

Total 30 1790 1820

22Positive Predictive Value .3014

73 = =

10

Example: Using Bayes Theorem for PPV

X-ray

Tuberculosis

TotalYes(D

1)No(D2)

Positive(T+)

22 51 73

Negative(T-)

8 1739 1747

Total 30 1790 1820( ) ( )( ) ( ) ( ) ( )

|Pos.Pred.Value

|

|

P T D P D

P T D P D P T D P D

+

+ +=

+

1 1

1 1 2 2

11

Example: Cotinine levels and smoking

• Outcome of interest – Smoking status– Problem: People may not report

honestly– Cotinine level may provide ‘objective’

asessment of smoking– Cotinine levels don’t work perfectly

• Diagnostic test – Concentration of cotinine– i.e. If Cotinine level > c Smoker– If Cotinine level <= c NonSmoker

12

Cotinine Level

(ng/ml)Smokers

Non-smokers

0--13 78 3300

14--49 133 72

50--99 142 23

100--149 206 15

150--199 197 7

200--249 220 8

250--299 151 9

300+ 412 11

Total 1539 3445

13

Cotinine Level

(ng/ml)Smokers

Non-smokers

Sensit-ivity

0--13 78 3300 1461/1539

14--49 133 72

50--99 142 23

100--149 206 15

150--199 197 7

200--249 220 8

250--299 151 9

300+ 412 11

Total 1539 3445

14

Cotinine Level

(ng/ml)Smokers

Non-smokers

Sensit-ivity

0--13 78 3300 1461/1539

14--49 133 72 1328/1539

50--99 142 23

100--149 206 15

150--199 197 7

200--249 220 8

250--299 151 9

300+ 412 11

Total 1539 3445

15

Cotinine Level

(ng/ml)Smokers

Non-smokers

Sensit-ivity

0--13 78 3300 1461/1539

14--49 133 72 1328/1539

50--99 142 23 1186/1539

100--149 206 15 980/1539

150--199 197 7 783/1539

200--249 220 8 563/1539

250--299 151 9 412/1539

300+ 412 11

Total 1539 3445

16

Cotinine Level

(ng/ml)Smokers

Non-smokers

Sensit-ivity

0--13 78 3300 0.95

14--49 133 72 1328/1539

50--99 142 23 1186/1539

100--149 206 15 980/1539

150--199 197 7 783/1539

200--249 220 8 563/1539

250--299 151 9 412/1539

300+ 412 11

Total 1539 3445

17

Cotinine Level

(ng/ml)Smokers

Non-smokers

Sensit-ivity

0--13 78 3300 0.95

14--49 133 72 0.86

50--99 142 23 1186/1539

100--149 206 15 980/1539

150--199 197 7 783/1539

200--249 220 8 563/1539

250--299 151 9 412/1539

300+ 412 11

Total 1539 3445

18

Cotinine Level

(ng/ml)Smokers

Non-smokers

Sensit-ivity

0--13 78 3300 0.95

14--49 133 72 0.86

50--99 142 23 0.77

100--149 206 15 980/1539

150--199 197 7 783/1539

200--249 220 8 563/1539

250--299 151 9 412/1539

300+ 412 11

Total 1539 3445

19

Cotinine Level

(ng/ml)Smokers

Non-smokers

Sensit-ivity

0--13 78 3300 0.95

14--49 133 72 0.86

50--99 142 23 0.77

100--149 206 15 0.64

150--199 197 7 783/1539

200--249 220 8 563/1539

250--299 151 9 412/1539

300+ 412 11

Total 1539 3445

20

Cotinine Level

(ng/ml)Smokers

Non-smokers

Sensit-ivity

0--13 78 3300 0.95

14--49 133 72 0.86

50--99 142 23 0.77

100--149 206 15 0.64

150--199 197 7 0.51

200--249 220 8 563/1539

250--299 151 9 412/1539

300+ 412 11

Total 1539 3445

21

Cotinine Level

(ng/ml)Smokers

Non-smokers

Sensit-ivity

0--13 78 3300 0.95

14--49 133 72 0.86

50--99 142 23 0.77

100--149 206 15 0.64

150--199 197 7 0.51

200--249 220 8 0.37

250--299 151 9 412/1539

300+ 412 11

Total 1539 3445

22

Cotinine Level

(ng/ml)Smokers

Non-smokers

Sensit-ivity

0--13 78 3300 0.95

14--49 133 72 0.86

50--99 142 23 0.77

100--149 206 15 0.64

150--199 197 7 0.51

200--249 220 8 0.37

250--299 151 9 0.27

300+ 412 11

Total 1539 3445

23

Cotinine Level

(ng/ml)Smokers

Non-smokers

Sensit-ivity

1—Specif-

icity

0--13 78 3300 0.95 145/3445

14--49 133 72 0.86

50--99 142 23 0.77

100--149 206 15 0.64

150--199 197 7 0.51

200--249 220 8 0.37

250--299 151 9 0.27

300+ 412 11

Total 1539 3445

24

Cotinine Level

(ng/ml)Smokers

Non-smokers

Sensit-ivity

1—Specif-

icity

0--13 78 3300 0.95 145/3445

14--49 133 72 0.86 73/3445

50--99 142 23 0.77

100--149 206 15 0.64

150--199 197 7 0.51

200--249 220 8 0.37

250--299 151 9 0.27

300+ 412 11

Total 1539 3445

25

Cotinine Level

(ng/ml)Smokers

Non-smokers

Sensit-ivity

1—Specif-

icity

0--13 78 3300 0.95 145/3445

14--49 133 72 0.86 73/3445

50--99 142 23 0.77 50/3445

100--149 206 15 0.64 35/3445

150--199 197 7 0.51 28/3445

200--249 220 8 0.37 20/3445

250--299 151 9 0.27 11/3445

300+ 412 11

Total 1539 3445

26

Cotinine Level

(ng/ml)Smokers

Non-smokers

Sensit-ivity

1—Specif-

icity

0--13 78 3300 0.95 0.04

14--49 133 72 0.86 73/3445

50--99 142 23 0.77 50/3445

100--149 206 15 0.64 35/3445

150--199 197 7 0.51 28/3445

200--249 220 8 0.37 20/3445

250--299 151 9 0.27 11/3445

300+ 412 11

Total 1539 3445

27

Cotinine Level

(ng/ml)Smokers

Non-smokers

Sensit-ivity

1—Specif-

icity

0--13 78 3300 0.95 0.04

14--49 133 72 0.86 0.02

50--99 142 23 0.77 50/3445

100--149 206 15 0.64 35/3445

150--199 197 7 0.51 28/3445

200--249 220 8 0.37 20/3445

250--299 151 9 0.27 11/3445

300+ 412 11

Total 1539 3445

28

Cotinine Level

(ng/ml)Smokers

Non-smokers

Sensit-ivity

1—Specif-

icity

0--13 78 3300 0.95 0.04

14--49 133 72 0.86 0.02

50--99 142 23 0.77 0.01

100--149 206 15 0.64 35/3445

150--199 197 7 0.51 28/3445

200--249 220 8 0.37 20/3445

250--299 151 9 0.27 11/3445

300+ 412 11

Total 1539 3445

29

Cotinine Level

(ng/ml)Smokers

Non-smokers

Sensit-ivity

1—Specif-

icity

0--13 78 3300 0.95 0.04

14--49 133 72 0.86 0.02

50--99 142 23 0.77 0.01

100--149 206 15 0.64 0.01

150--199 197 7 0.51 28/3445

200--249 220 8 0.37 20/3445

250--299 151 9 0.27 11/3445

300+ 412 11

Total 1539 3445

30

Cotinine Level

(ng/ml)Smokers

Non-smokers

Sensit-ivity

1—Specif-

icity

0--13 78 3300 0.95 0.04

14--49 133 72 0.86 0.02

50--99 142 23 0.77 0.01

100--149 206 15 0.64 0.01

150--199 197 7 0.51 0.01

200--249 220 8 0.37 20/3445

250--299 151 9 0.27 11/3445

300+ 412 11

Total 1539 3445

31

Cotinine Level

(ng/ml)Smokers

Non-smokers

Sensit-ivity

1—Specif-

icity

0--13 78 3300 0.95 0.04

14--49 133 72 0.86 0.02

50--99 142 23 0.77 0.01

100--149 206 15 0.64 0.01

150--199 197 7 0.51 0.01

200--249 220 8 0.37 0.006

250--299 151 9 0.27 11/3445

300+ 412 11

Total 1539 3445

32

Cotinine Level

(ng/ml)Smokers

Non-smokers

Sensit-ivity

1—Specif-

icity

0--13 78 3300 0.95 0.04

14--49 133 72 0.86 0.02

50--99 142 23 0.77 0.01

100--149 206 15 0.64 0.01

150--199 197 7 0.51 0.01

200--249 220 8 0.37 0.006

250--299 151 9 0.27 0.003

300+ 412 11

Total 1539 3445

33

ROC Curve for Cotinine and Smoking

0

0.2

0.4

0.6

0.8

1

0 0.01 0.02 0.03 0.04 0.05

1-Specificity

Se

nsi

tivi

tyExample: Cotinine levels and

smoking

34

ROC Curve for Cotinine and Smoking

0

0.2

0.4

0.6

0.8

1

0 0.01 0.02 0.03 0.04 0.05

1-Specificity

Se

nsi

tivi

ty

Example: Cotinine levels and smoking

Perfect

Choose cutpoint using points closest to perfect.