1

From the analytical uncertainty to uncertainty in data interpretation

D. Concordet, J.P. Braund.concordet@envt.fr

jp.braun@envt.fr

2

Biological data vary according several sources of variations

Biological- species- status (healthy, sick)- breed- format, age...- inter-individual- intra-individual

Metrological- intra- laboratory variations (imprecision)- inter-laboratories variations (bias)

controlled

of interest

not controlled

parasitic

3

Influence of imprecision on the reference interval

Pl-creatinine (mol/L)

10 30 50 70 90 110 130 150

CV=0%

Population distribution of Pl-Creatinine of healthy dogs

4

Influence of imprecision on the reference interval

Pl-creatinine (mol/L)

10 30 50 70 90 110 130 150

CV=0%

CV=10%

Population distribution of Pl-Creatinine of healthy dogs

Y = X + with E (

5

Influence of imprecision on the reference interval

Pl-creatinine (mol/L)

10 30 50 70 90 110 130 150

CV=0%

CV=10%

CV=20%

Population distribution of Pl-Creatinine of healthy dogs

Y = X + with E (

6

Sampling variations of the reference interval

95%

CI

of th

e re

fere

nce

inte

rval

Pl-

crea

tinin

e (

mol

/L)

10

30

50

70

90

110

130

150

170

N=100 dogs

CV=0% CV=10% CV=20%

7

Use of reference interval

Pl-creatinine (mol/L)40 135

Effect of large imprecision

8

Use of reference interval

Pl-creatinine (mol/L)40 135

When the precision is better

9

Replicates decrease the influence of imprecision

Y1 = X +

Y2 = X + with E (i

SD(Yi)= SD(i

Yp = X + p

p

YSD

CV(Yi) = 20 %

p CV(Y)

2 14.1%

3 11.5%

4 10.0%

10

20 40 60 80 100 120 140 160 180

Ref.

Influence of inter laboratories variability on the reference interval

Pl-creatinine (mol/L)

11

20 40 60 80 100 120 140 160 180

Ref.

CV = 10% / Ref

Influence of inter laboratories variability on the reference interval

Pl-creatinine (mol/L)

12

20 40 60 80 100 120 140 160 180

Ref.

CV = 10% / Ref

CV = 20 % / Ref

Influence of inter laboratories variability on the reference interval

Pl-creatinine (mol/L)

13

Reference range of Pl-creatinine11 books of animal clinical biochemistry

0

25

50

75

100

125

150

175

200

225

250P

l-C

reat

inin

e (µ

mo

l/l)

[2] [6] [7] [8] [27] [28] [34] [45] [46] [55] [59]

Lefebvre HP et al. 1998.

35 µmol/l

245 µmol/l

Reference number

14

A “demographic” source of variation

GFR value

0

1

2

3

4

5

Dachshund Labrador Great Dane

GFR

(m

L/kg

/min

)M

ean

SD

8 8 6

GFR value depends on breed

Lefebvre HP unpublished results

15

Influence of the population’s structure

20 70 120 170 220 270

breed 1 = 30%

breed 2 = 30%

breed 3 = 20%

breed 4 = 10%

breed 5 = 5%

breed 6 = 5%

20 70 120 170 220 270

breed 1 = 30%

breed 2 = 30%

breed 3 = 20%

breed 4 = 10%

breed 5 = 5%

breed 6 = 5%

total

130Pl-creatinine (mol/L)

16

Influence of the population’s structure

20 70 120 170 220 270

breed 1 = 16%

breed 2 = 16%

breed 3 = 16%

breed 4 = 16%

breed 5 = 16%

breed 6 = 20%

20 70 120 170 220 270

breed 1 = 16%

breed 2 = 16%

breed 3 = 16%

breed 4 = 16%

breed 5 = 16%

breed 6 = 20%

total

210Pl-creatinine (mol/L)

17

Influence of multiplicity : case of independence

Pl-creatinine (mol/L)40 135

40

95% of healthy animals

95%

of

hea

lth

y an

imal

s

90% of healthy animals

18

Imprecision and multiplicity

Pl-creatinine (mol/L)40 135

40

19

Imprecision and multiplicity

Pl-creatinine (mol/L)40 135

40

CV= 15 %82 % of healthy animals

CV = 15%93% of healthy animals

20

Diagnostic tests : a better way to proceed

20 70 120 170 220 27020 70 120 170 220 270

Diseased animalsWithout the disease

Threshold

Considered as sick

Pl-creatinine (mol/L)

Considered as healthy

Sensitivity :SeSpecificity :Sp

21

Performances of the test

Sensitivity = Se% of diseased animals with a result > threshold

Specificity = Sp% of animals without the disease with a result < threshold

22

Several definitions of specificity

Specificity = Sp

• Sp1 :% of healthy animals with a result < threshold

• Sp2 :% of animals without the disease (healthy + with any other disease) with a result < threshold

23

20 70 120 170 220 270

Several definitions of specificity

Healthy animalsSp1

Threshold

Healthy animals+ with other diseasesSp2

Sp2<<Sp1

24

The operational indices : the predictive values

Spe

ePPV

1Pr1SPr

SPr

Spe

SpNPV

Pr1S1Pr

Pr1

Positive Predictive Value (PPV) : Probability that the animal has the disease when its result >threshold

Negative Predictive Value (NPV) : Probability that the animal has not the disease when the result < threshold

25

The clinician experience : Pr

Pr = Pre-test probabilityprobability that the animal has the disease

Pr = 1 : The clinician is sure that the animal has the disease

Pr = 0 : The clinician is sure that the animal has not the disease

PPV =1

NPV =0whatever Se and Sp

PPV =0

NPV =1whatever Se and Sp

Positive diagnostic gain : PPV-Pr

Negative diagnostic gain : NPV-(1-Pr)

Pr = 0.5 : The clinician does not know (coin tossing)

26

20 70 120 170 220 270

Sp = 80%

Influence of imprecision

Threshold

Pl-creatinine (mol/L)20 70 120 170 220 270

Se = 92%

healthy1

Sp = 73%

CV = 15 %

Se = 81%

CV = 15%

27

0

0.10.2

0.3

0.40.5

0.6

0.7

0.80.9

1

0 0.2 0.4 0.6 0.8 1

PPV

NPV

Consequences on interpretation

Pre-test probability

PP

V/N

PV

0

0.10.2

0.3

0.40.5

0.6

0.7

0.80.9

1

0 0.2 0.4 0.6 0.8 1

PPV

NPV

PPV CV 15%

NPV CV 15%

28

Sampling variations of sensitivity and specificity

0.5

0.55

0.6

0.65

0.7

0.75

0.8

0.85

0.9

0.95

1

0 50 100 150 200 250 300

Sample size : n95%

co

nfi

den

ce in

terv

al o

f S

e=0.

92, S

p=

0.80

N

SSS

N

SSS

12;

12

Se = 0.92

Sp = 0.80

95% confidence interval of Se/ Sp :

29

Consequences on interpretation9

5% c

on

fid

enc

e in

terv

al o

f P

PV

/NP

V

Pre-test probability

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

0 0.2 0.4 0.6 0.8 1

N=50

30

Consequences on interpretation9

5% c

on

fid

enc

e in

terv

al o

f P

PV

/NP

V

Pre-test probability

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

0 0.2 0.4 0.6 0.8 1

N=100

31

Consequences on interpretation9

5% c

on

fid

enc

e in

terv

al o

f P

PV

/NP

V

Pre-test probability

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

0 0.2 0.4 0.6 0.8 1

N=300

32

Effects of the biological sources of variation

Increase the overall dispersion of the results

Improvement possible if Se and Sp are determined for each level of the factors of variation (e.g. breed)

decrease Se and Sp

decrease PPV and NPV for a fixed pre-test proba

33

The future ?

Individualisation

Blood sample when the animal is young and healthy

Follow-up of the evolution of the appropriate marker

Critical difference

Independent of demographic factors (breed, sex…)

dependent only on intra-individual variability

analytical errors

34

The end