IPH

Accreditation & ValidationJoris Van Loco

Scientific Institute of Public HealthFood Section

Method Validation

Is method validation• analyzing 6 samples ?• Calculating the bias, repeatability,

reproducibility,… of a method ?• Knowing the detection limits of the method ?• knowing the uncertainty associated with a

method?• satisfying ISO 17025 assessors?

What is Method Validation?

Method validation is the process of proving that an analytical method is acceptable for its intended purpose

Why is Method Validation Necessary? To prove what we claim is true To increase the value of test results To justify customer’s trust To trace criminals

Examples• To value goods for trade purposes• To support health care• To check the quality of drinking water

When and How should Methods be Validated New method

development Revision of established

methods When established

methods are implemented in new laboratories

Interlaboratory Comparison

Single lab validation• Full Validation• Implementation Validation

Method performance parameters are determined using• equipment that is:

• Within specification• Working correctly• Adequately calibrated

• Competent operators

Validation and Quality Control

In house validation• (Bias), recovery• Repeatability• Within lab

reproducibility

Internal QC• Control charts

Starting data

Proficiency Testing• Bias (trueness)

Collaborative trial• Reproducibility• Bias (trueness)

Long term within lab reproducibility

Method Validation

Accuracy• Trueness (CRM)• Recovery (spikes)

Precision• Repeatability• (Within) reproducibility

Selectivity (& Specificity) Detection capability

• LOD, LOQ, CC, CC Linearity – calibration range Robustness

• Applicability – stability

Method ValidationPerformance Characteristics 2002/657/CE

Detection Capability

CCß

Decision Limit CC

Trueness/ Recovery

Precision Selectivity/ Specificity

Applicability/ Ruggedness/

Stability

S + - - - + + Qualitative methods C + + - - + +

S + - - + + + Quantitative methods C + + + + + +

S: Screening methods

C: Confirmatory methods

Linearity

Purpose• to evaluate the linear response of your instrument

How• Evaluating your calibration model

• Mandels fitting test• Lack-of-Fit• Residuals

Conclusion• Linear model • <> other (i.e. quadratic) regression model

Simple regression: confidence and prediction interval

-0,02

0

0,02

0,04

0,06

0,08

0,1

0,12

0,14

0 0,5 1 1,5 2 2,5 3 3,5 4 4,5

concentration (ng/ml)

AU

Linearity

Residual plots (ei)

• with

Statistical tests• Lack-of-fit• Mandel’s fitting test

Linear relationship

-12

-8

-4

0

4

8

12

0 20 40 60 80 100

X

Re

sid

ua

ls

iii YYe ˆ

ii bXaY ˆ

Curvilinear relationship

-20

-10

0

10

20

0 20 40 60 80 100

X

Re

sid

ua

ls

Curvilinear relationship

-20

-10

0

10

20

0 20 40 60 80 100

X

Re

sid

ua

ls

Coefficient of correlation (r)

Is NOT a suitable measure for linearity

0

10

20

30

40

50

60

70

80

0,9975 0,998 0,9985 0,999 0,9995 1

correlation coefficient (r)

F m

and

el's

fit

tin

g t

est

Pb

Cd

Matrix Effect

Purpose• To evaluate whether you have

a concentration dependent systematic error due the matrix

• i.e. ion suppression How

• comparison of standard curve with matrix matched standard curve

Conclusion• Standard solutions, spiked

extracts or spiked samples for the calibration line.

4

)2()2(

11)(

22

2

,

2

,

as

aasa

aaissi

as

nn

SnSnSp

XXXXSp

bbbt

-0,05

0

0,05

0,1

0,15

0,2

0,25

0,3

0,35

0,4

0 50 100 150 200 250

addition

standard

Detection limit

DIN 32645 from blanks

from calibration data

Funk dynamic model

IUPAC

Coleman recursive formula

explicit formula

2

D x 0 f;αx

1 1 xx s t

m n Q

LD f;α

s 1 1x t

b m n

2

c y n2

ii 1

0 x1y b s t 1

nx x

2

y cD n

22i

i 1

s t y y1x 2 1

a na x x

c 1 α,v 0S t sαβ,v 0 1 α,v 0

D

δ σ 2t σK Kx

A I A I

B A1 α,v

0

σ σK 1 r(B,A) t

σ A

2

A1 α,v

σI 1 t

A

11

2 22 2D

D n 2,1 α n 2,1 βxx xx

x xs 1 x 1x t 1 t 1

a n S n S

1

2 2

D H V

J J 4HKx DL

2H

n 2,1 β

aA

s t

12 2

n 2,1 α

n 2,1 β xx

t 1 xB 1

t n S

2xx

1F B 1 S

n

xxG 2AB S 2 1xxH A S J G 2x 2K F x

Detection Limits

A) DIN 32645

Detection limit

by fast estimation:

Capability limit

Determination limit

by fast estimation

Factor for fast estimation

2k

D x0 f;ax

y - a 1 1 xx = = s t + +

b m n Q

n;α x0Dx = 1,2 Φ s

2

C NG x0 f;βx

1 1 xx = x + s t + +

m n Q

2DDT x0 f;α

x

x - x1 1x = k s t + +

m n Q

n;α x0DTx = 1,2 k Φ s

n;α f;α

1Φ = t 1+

n

Detection Limits

B) Funk

Detection limit dynamic model

Determination limit dynamic model

2

c y n2

ii 1

0 x1y b s t 1

nx x

2

y cD n

22i

i 1

s t y y1x 2 1

a na x x

2y

c n2

ii 1

s t 1 xx 1

a nx x

2

ch y n

2

ii 1

x x1y b 2 s t 1

nx x

2

y hhDT n

22i

i 1

s t y yy b 1x 1

a a na x x

Detection Limits

C) IUPAC

Detection limitc 1 α,v 0S t s

αβ,v 0 1 α,v 0D

δ σ 2t σK Kx

A I A I

B A1 α,v

0

σ σK 1 r(B,A) t

σ A

2

A1 α,v

σI 1 t

A

Detection Limits

Detection limits “How to”

Choose a definition and stick to it • Describe the equation used in the validation file• Problems

• statistics <> practical limitations• statistics <> ID-criteria

Practical LOD• Analyzing samples with decreasing concentration• Minimum concentration which fulfills the identification

criteria = practical limit of detection• Repeat the experiment

S/N• i.e. LOD=3xS/N

Quantitation Limit (LQ)

The quantification limit is the minimum signal (concentration or amount) the can be quantified. • the residual standard deviation (RSD) is

included in the definition.

• The IUPAC default value for RSDQ= 0.1 (or 10%). LQ=10sQ.

QQ

Q RSDL 1

- and -error

-error• risk of erroneously rejecting H0

• i.e. risk of the conviction of an innocent -error

• risk of erroneously accepting H0

• i.e. risk of the non conviction of a criminal

Detection CapabilityCase of a permitted limit (MRL)

MRL CC

Signal orConcentration

CC

=

+1.64MRL +1.64sample

Determination of CC and CC with ISO 11843Plot of Fitted Model

Conc (µg/kg)

un

corr

ect

ed

re

sult

0,5 0,7 0,9 1,1 1,3 1,50

0,3

0,6

0,9

1,2

1,5

CC CC

yc

MRL

CC 1,12

CC 1,25

Detection CapabilityCase of a permitted limit (MRL)

2

2

1, .

11

xx

xx

JISytbxay

i

MRLdfMRLc

b

ayCC c

2

2

1, .

11

xx

xCC

JISytbCCay

i

dfc

Detection CapabilityCase of no established permitted limit or banned substance

Xblank CC

Signal orConcentration

CC

≠

+2.33blank +1.64sample

Presence of Heteroscedasticity

Nitroimidazoles in plasma (MNZ-OH)

Residuals plot• “<“ - shape

Plot of S vs conc• Linear relationship

between S and concentration

Heteroscedasticity

Impact on CCa and CCb• CCa and CCb are

incorrectly calculated

• Sblank ↓ CCa ↓

• CCb ↓ or ↑

MNZ-OH y = 0,0831x + 0,0023

R2 = 0,9924

0

0,05

0,1

0,15

0,2

0,25

0,3

0,35

0,4

0,45

0,5

0 1 2 3 4 5 6

Conc

St

De

v

RESIDUALS PLOT OF THE LINEAR MODEL

-0,8

-0,6

-0,4

-0,2

0

0,2

0,4

0,6

0,8

0 1 2 3 4 5 6

Concentration

Res

idua

l

Other examples

RNZy = 0,0764x - 0,0172

R2 = 0,7899

-0,05

0

0,05

0,1

0,15

0,2

0,25

0,3

0,35

0,4

0 0,5 1 1,5 2 2,5 3 3,5 4 4,5 5

Conc

St

De

v

AMOZy = 0,0426x + 0,0105

R2 = 0,9768

-0,02

0

0,02

0,04

0,06

0,08

0,1

0,12

0,14

-0,5 0 0,5 1 1,5 2 2,5 3

Conc

St

Dev

MNZ y = 0,0389x + 0,0146

R2 = 0,7416

0

0,02

0,04

0,06

0,08

0,1

0,12

-0,5 0 0,5 1 1,5 2 2,5 3

Conc

St

De

v

•Nitroimidazoles in plasma •Nitrofurans in honey

•Corticosteroids in liverTriamcinolone Acetonide y = 0,1975x - 0,1607

R2 = 0,8451

-0,2

-0,1

0

0,1

0,2

0,3

0,4

0,5

0,6

0,7

0,8

0 0,5 1 1,5 2 2,5 3 3,5 4 4,5 5

Conc

St

Dev

Weighted regression equations for CC and CC

2

2

221,

..

1y

wii

w

iCC

df SxxwJ

x

wJS

b

tCC

2

2

22,,

..

1y

wii

w

iCC

df SxxwJ

x

wJS

bCC

•Solved by iteration

Conclusion detection capability

Many definitions of detection limits• detection limit (≈ CC_banned substances)• determination limit (≈ CC_banned substances)• Quantition limit

Complicated statistics KISS

• demonstrate with real (spiked) samples at low concentration level practical limit of detection

Selectivity/Specificity

Identity: Signal to be attributed to the analyte • GLC (change column/polarity), GC/MS, Infra-red

Selectivity: The ability of the method to determine accurately the analyte of interest in the presence of other components in a sample matrix under the stated conditions of the test.

Specificity is a state of perfect selectivity

Selectivity

The procedure to establish selectivity:• Analyze samples and reference materials• Assess the ability of the methods to confirm

identity and measure the analyte• Choose the more appropriate method.• Analyze samples • Examine the effect of interferences

Selectivity: Verification of the identification criteria (2002/657/EC) MS – criteria

• 3 or 4 identification points• 1 precursor and 2 transition ions

• Relative ion intensities LC – criteria

• Relative retention time (RRT): +/- 2.5 % (LC)

UV – criteria• Spectrum match• +/- 3 nm

CC is concentration at or above the calculated CC for which the ID criteria are fulfilled in 95% of the cases.

CC is concentration at or above the calculated CC for which the ID criteria are fulfilled in 50% of the cases.

Ruggedness and Robustness

Intra-laboratory study to check changes due to environmental and/or operating conditions • Usually it is part of method development• Deliberate changes in

• Temperature• Reagents ( e.g. different batches)• Extraction time• Composition in the sample• etc

Precision – ISO 5725 1-6 (1994)

Expresses the closeness of agreement (dispersion Expresses the closeness of agreement (dispersion level, relative standard deviation) between a series of level, relative standard deviation) between a series of measurements from multiple sampling of the same measurements from multiple sampling of the same homogeneous sample (independent assays) under homogeneous sample (independent assays) under prescribed conditions.prescribed conditions.

Irrespective of whether mean is a correct representation of the true value.

Gives information on random errorsrandom errors Evaluated at three levelsthree levels: repeatabilityrepeatability intermediate precision (within laboratory)intermediate precision (within laboratory) reproducibility (between laboratory)reproducibility (between laboratory)

Precision (cont.) – ISO 5725 1-6 (1994)

RepeatabilityRepeatability: precision under conditions where the results of independent assays are obtained by the same analytical procedure, on identical samples, in the same lab, by the same operator, using the same equipment and during short interval of time

Intermediate precisionIntermediate precision: ISO recognizes M-factor M-factor different intermediate precision conditionsdifferent intermediate precision conditions (M = 1, 2 or 3) M = 1M = 1: only 1 of 3 factors (operator, equipment, time) is

differentM= 2 or 3M= 2 or 3: 2 or all 3 factors differ between determinations

Precision (cont.) – ISO 5725 1-6 (1994)

ReproducibilityReproducibility: precision under conditions where results obtained: by same analytical procedureon identical sample in different laboratories, different operators, different

equipment Reproducibility established by interlaboratory study

(standardisationstandardisation of an analytical procedure)

Intermediate precisionIntermediate precision RepeatabilityRepeatability ReproducibilityReproducibility

Evaluation of Precision 10 samples for each conc.under r,R, within lab R

• Standard Deviation

Determination in pairs under r,R, within lab R• Std. Dev. between two single determinations

• a-b, the difference between the values, d, the number of pairs

1

)(1

2

n

xxn

i 100x

sRSD

d

bas ii

2

)( 2

sr

SR

SRw

Repeatability (r) and within-lab reproducibility (Rw) ANOVA table for a single factor balanced design with 3 replicate samples on the same day.

Source Sum of squares df Mean Squares Expected mean squares

day SSdays ndays - 1 MSdays =

SSdays / (ndays – 1)

σrepl² + 3σdays²

replicate SSrepl nT – ndays MSrepl =

SSrepl / (nT – ndays)

σrepl²

Total SST nT – 1 SST = MST / (nT – 1)

repeatability (Sr²) and within-lab reproducibility variances (SRw²)

Sr² = Srepl²

SRw² = Sr² + Sdays²

The Srepl²and Sdays² can be obtained from mean squares as (nrepl = 3):

Srepl² = MSrepl

Sdays² = (MSdays – MSrepl) / 3

Repeatability and reproducibility

The value of 2.8?• Variance of difference between 2 replicate

measurements is 2s² • Confidence interval at 95% level on the difference is

0 ± 1.96 √2 s ± 1.96 x 1.41 sr = ± 2.8 sr

95% probability that difference between duplicate determinations will not exceed 2.8 sr

r = limit of the repeatability r = 2.8 sr

R = limit of the reproducibility R = 2.8 SR

Precision criteria 2002/657/CE

Horwitz: RSDR(%) = 2(1-0.5logC)

-140

-100

-60

-20

20

60

100

1,E

+00

1,E

-01

1,E

-02

1,E

-03

1,E

-04

1,E

-05

1,E

-06

1,E

-07

1,E

-08

1,E

-09

1,E

-10

1,E

-11

1,E

-12

1,E

-13

1,E

-14

C

%

Drugs

Aflatoxines

Dioxins

narcotics in food pesticide residu's

CRRSD log5,012%

Determination of Trueness

Using Certified Reference Materials Using RM or In-house materials Using Reference methods

• Single sample• Many samples

Via Interlaboratory study

Trueness, extraction yield (recovery) and apparent recovery Trueness means the closeness of agreement

between the average value obtained from a large series of test results and an accepted reference value. Trueness is usually expressed as bias

Recovery (extraction yield): yield of a preconcentration or extraction stage of an analytical process for an analyte divided by amount of analyte in the original sample.

Apparent recovery: observed value derived from an analytical procedure by means of a calibration graph divided by reference value.

Trueness criteria 2002/657/CE

When no such CRMs are available, it is acceptable that trueness of measurements is assessed through recovery of additions of known amounts of the analyte(s) to a blank matrix. Data corrected with the mean recovery are only acceptable when they fall within the ranges