Wittgenstein in Heidelberg

7/29/2019 Wittgenstein in Heidelberg

1/20

Martin Diller and Paul Artes, IMAGE 2013Friday, March 22, 2013


2/20

Data are complex.Analyses are complex.

Friday, March 22, 2013


3/20

Please write down yourjudgements!

(A to F, yes or no)



4/20



5/20

A: The likelihood that this testresult is abnormal is >99.5%.

B: The likelihood that this patienthas a normal visual field is


6/20

beyond 99.9% P.L.

between 95% and 99.9% P.L.

above 95% P.L.



7/20

E: The likelihood that this testresult is abnormal is >99.9%.

F: The likelihood that this patienthas a healthy optic disc is 99.9% likely.

H: The likelihood that this patient

has glaucoma is >99.9%.



8/20

A - H are allwrong.

(Some are dangerous.)



9/20

Why do we need L?Studies show ~50% undiagnosed disease.

Patients still go blind from glaucoma.

A lot of these patients present late.

Many diagnostic and therapeutic decisions are poor.All this despite a lot of technological advances.



10/20

positive rate in VFnegative group

positive

rate

in

VFpositive

group

0.0 0.2 0.4 0.6 0.8 1.0

0.0

0.2

0.4

0.6

0.8

1.0

A

B

C

D

AUC=0.82

87%

r=0.84

mean.Diff=0.38

criterion=0.67

A

!0

5

10

15

20

25

30

response

re

sponse

latency(s)

d h p h n s p d d d

0

5

10

15

20

25

30

29 40 10 11 10

positive rate in VFnegative group

positive

rate

in

VFpositive

group

0.0 0.2 0.4 0.6 0.8 1.0

0.0

0.2

0.4

0.6

0.8

1.0

A

B

C

D

AUC=0.79

79%

r=0.61

mean.Diff=0.27

criterion=1.3

A

0

5

10

15

20

25

30

response

re

sponse

latency(s

d h p h n s p d d d

0

5

10

15

20

25

30

62 18 9 8 3



11/20

Case Example

Assessment of Glaucomatous Optic Disc Damage

by Ophthalmology Residents at the University of So PauloJayme A. Vianna1, Alexandre S. Reis1,2, Lucas P. Vicente1, Marcelo Hatanaka1, Paul H. Artes2

Title

Results

Purpose

Conclusion

1Department of Ophthalmology, University of So Paulo, So Paulo, Brazil; 2Ophthalmology and Visual Sciences, Dalhousie University, Halifax, Canada

Jayme A Vianna, ARVO 2012, #644

[email protected]

To examine performance at diagnosingglaucomatous optic disc damage in residents at

different stages of training.

MethodsAt the end of the academic year, 40

ophthalmology residents (14, 14, 12 in the 1st,

2nd, and 3rd year or residency training) tested

themselves with the Discus software.1

The software randomly displayed 100 non-

stereoscopic optic disc photographs. Twenty(20%) were from patients with glaucoma and

confirmed visual field defect, and 80 (80%) from

patients with suspected glaucoma or ocular

hypertension with reproducibly normal visual

fields. Twenty-six (26%) of images were repeated

to evaluate consistency.

Each image was displayed for 10 seconds, and

observers had unlimited time to rate it (Figure 1).Fig 2. Graphic results of Discus, containing:

User ROC curve (colored)

Expert reference ROC curve (gray)

Area under ROC (AUC)

User AUC / Expert AUC (percentage)

Rank correlation of user and expert responsesMean difference of repeated images

Likelihood to diagnose damage as mean of responses (criterion)

Response latency for each category (inset boxplot)

Graphic representation of criterion (red line)

Expert reference of criterion (gray dashed line)

Median AuROC was smaller in the first year.

Median response latency was larger in the second

year. Decision criteria and correlation with experts

were similar among the 3 years (Table 1).

There was moderate correlation between the two

performance measures (AuROC and correlation

with experts, Spearmans = 0.61, P< 0.001), but

no relationship between either performance

measure and the decision criteria ( = -0.13 and

-0.01, P> 0.10).

Table1. Perfor

(numbers are

mance of resi

ean [standar

ents at differe

d deviation])

nt years of traiining

1st Year 2nd Year 3rd Year P*

AuROC

Correlationw/ Experts

DecisionCriteria

ResponseLatency (s)

0.69 (0.07) 0.74 (0.05) 0.74 (0.07) 0.61

0.66 (0.15) 0.65 (0.10) 0.65 (0.11) 0.86

1.92 (0.34) 1.85 (0.30) 1.81 (0.26) 0.52

5.6 (2.42) 7.6 (2.82) 5.4 (2.17) 0.04

*Kruskal-Walli

AuROC, Area

- test

, under the Rec

-

, eiver Operatin

-

, g Characterist

-

, ic

There were considerable differences in

performance, criterion, and speed, between

residents in each year of training.

Residents in the second and third year tended to

perform better than those at the first year of

training. These differences were not statistically

significant.

Discus provides a simple, rapid and objective

assessment of performance that should be useful

in many training programs. Our results will be

useful as a reference for comparing other trainees.

Fig 1. Screenshot of Discus software, showing an optic disc

photograph, and the rating scale from definitely healthy (+2) to

definitely damaged (-2). Images are displayed for up to 10 sec.

Reference

Denniss et al. Discus: investigating subjective

judgment of optic disc damage. Optometry and

Vision Science. 2011; 88(1):E93-101.

Diagnostic performance (AuROC), response

latency, decision criteria for individual

participants, and summary statistics by year, are

shown in Figure 3.

Try Discus yourself!www.discusproject.blogspot.com

Fig 3. Discus results stratified by year of residency training.

Each circle represents a single resident.

Sizes of the circles are proportional to the response latency,

colors are coded according to criterion,

bold circles and horizontal dashed lines are group medians,

and vertical dashed lines give the 25th and 75th percentiles.

The horizontal gray line and shaded area give the mean and

range from of a reference group of 10 experts (Denniss, 2011)

After completion of the test, each resident sresponses were automatically analyzed and

the results presented in a graphic (Figure 2).

unda, pril 9, 0Friday, March 22, 2013


12/20

Imaging really has a place

in primary and secondaryglaucoma care.

We need to provide better

guidance on use andinterpretation.



13/20Friday, March 22, 2013


14/20

Aim

jargon-free statements

simple, but technically accurateunderstandable to non-experts

useful rather than trivial



15/20

Guiding Principles

"Was sich berhaupt sagen lt, das kann manklar sagen; und wovon man nicht reden kann,

darber mu man schweigen."

"What can be said, can be said clearly,

and what cannot be said clearly, thereof weshould be silent."



16/20

Things should be madeas simple as possible.but no simpler.

Man muss die Dinge so einfachwie mglich machen.

Aber nicht einfacher.



17/20

Categories

Changetopographical analysisrate of rim area change (mm2/y * 10-3)

power (to detect a rate of -20.0 units)

Relevance of model / validity of assumptionsoutlierstests of nonlinearity & autocorrelation

Data qualityImage quality (MPHSD)Overlap of images across the series



18/20

Principles

Visualisationtransform each statistic via its reference distribution.show quantiles [0-10, .., 90-100] on a simple 10-segment bar chart.

Translation into verbal statementsThe data are of good quality. Typically, only 1 of 10 patients havebetter images. No change exceeding chance was statistically detectableover 5 years of follow-up (10 tests). If rapid deterioration had

occurred, it would almost certainly have been detected. Another testshould be obtained within the next 18 months.



19/20

Open Questions

Combining different indices to single metrice.g: series quality = median MPHSD + 85th percentile MPHSD + overlapweighted mean, linear & non-linear discriminant functions

Reference datapublished research studies vs large clinical databaseswhose data is it anyway?



20/20

We hide it effectively.

Date post:	03-Apr-2018
Category:	Documents
Upload:	paul-h-artes
View:	226 times
Download:	0 times

Wittgenstein in Heidelberg

Documents