Measurements for 8 Common Analytes in Native Sera Identify Inadequate Standardization among 6 Routine Laboratory Assays

H.C.M. Stepman, U. Tiikkainen, D. Stöckl,

H.W. Vesper, S.H. Edwards, H. Laitinen, J. Pelanti,

L.M. Thienpont, and Participating Laboratories

June 2014

www.clinchem.org/content/60/6/855.full

© Copyright 2014 by the American Association for Clinical Chemistry

© Copyright 2009 by the American Association for Clinical Chemistry

IntroductionIntroduction

Proficiency Testing (PT), also known as External Quality Assessment (EQA), has earned a well-deserved position as an element of laboratory quality management

PT with commutable samples and high-quality targets is essential for assessing the accuracy of diagnostic assays and the interchangeability of generated results

The “intrinsic” quality of a manufacturer’s assay may be influenced by the laboratory using it; therefore, assessment under routine conditions is essential

See Editorial by Horowitz GL. Assessing Accuracy on the Front Lines: A Pragmatic Approach for Single-Donor Proficiency Testing. Clin Chem 2014

© Copyright 2009 by the American Association for Clinical Chemistry

QuestionsQuestions

What were the special aspects of the experimental design of the PT survey described?

How was the data assessment done? Quality indicators? Targets? Limits?

© Copyright 2009 by the American Association for Clinical Chemistry

Materials and Methods – Study DesignMaterials and Methods – Study Design

Use of 20 fresh-frozen single donation serum samples prepared according to CLSI C37-A

Selection of laboratories using “homogeneous tests” Reagent, calibrator, platform from the same manufacturer

Inclusion of assays installed on random access platforms

Abbott Architect, Beckman Coulter AU, Ortho Vitros, Roche Cobas, Siemens Advia, and Thermo Scientific Konelab

Measurement of 8 analytes Creatinine, glucose, phosphate, uric acid, total cholesterol, HDL cholesterol, LDL cholesterol, and triglycerides

© Copyright 2009 by the American Association for Clinical Chemistry

Materials and Methods – Data AssessmentMaterials and Methods – Data Assessment

Quality indicators Intra-assay and combined imprecision (including sample

matrix interference) Bias and total error

Targets Peer group “All-method trimmed mean” (AMTM) Reference (REF) method values (cholesterol, creatinine,

uric acid)

Limits Reflecting state-of-the-art performance Related to biological variation

© Copyright 2009 by the American Association for Clinical Chemistry

QuestionsQuestions

What were the main findings of the study?

What were the limitations?

© Copyright 2009 by the American Association for Clinical Chemistry

Results – SummaryResults – Summary Excellent peer performance for most assays (except

HDL and LDL cholesterol)

Intrinsic quality sufficiently robust for satisfactory performance in a daily laboratory context

Considerable bias for some assays (Table 1/Figures 1 & 2)

Siemens Advia creatinine (-4.2%), Ortho Vitros phosphate (8.9%), Beckman Coulter AU triglycerides (5.4%) and LDL (15%), Thermo Scientific uric acid (6.4%)

More biases at low & high concentration

Inter-laboratory differences >30% (Table 2)

© Copyright 2009 by the American Association for Clinical Chemistry© Copyright 2009 by the American Association for Clinical Chemistry

ResultsResults

Table 1. AMTM/REF bias estimates (at low, mid, and high concentration) for each assay (named by manufacturer). The blue, underlined values indicate violation of the limits. CHOL, cholesterol; CREA, creatinine; GLU, glucose; HDL, HDL cholesterol; LDL, LDL cholesterol; PHOS, phosphate; TRIGL, triglycerides; UA, uric acid; NA, not applicable.

  CHOL CREA GLU HDL LDL PHOS TRIGL UABias Limit (%) 4 4 4.5 4.5 4.5 4.5 4.5 4Abbott 3.0 5.1 -0.4 5.5 2.7 0.1 2.5 -4.9  2.5 3.7 -0.2 -1.0 1.5 0.0 0.1 -1.2  2.3 2.8 0.2 -5.4 0.9 0.0 -1.0 1.9Beckman 2.7 -0.6 2.1 0.3 NA 0.8 5.8 -2.6  3.8 0.5 1.9 -3.2 NA 0.5 5.4 -1.3  4.4 1.1 1.6 -5.6 NA 0.4 5.3 -0.3Ortho -0.6 1.6 -3.0 0.0 NA 14.6 -0.9 -2.2  0.0 1.2 -2.8 -1.1 NA 8.9 -0.3 -2.6  0.4 0.9 -2.4 -1.9 NA 6.7 0.0 -2.8Roche 3.5 2.6 -0.7 -0.3 5.2 -1.1 0.0 -4.6  2.5 2.7 -0.6 3.9 2.0 -0.7 -1.6 -3.4  1.9 2.7 -0.6 6.7 0.4 -0.5 -2.3 -2.4Siemens 0.7 -5.5 1.2 2.8 0.0 1.3 0.3 0.4  0.0 -4.2 1.0 2.2 -0.3 1.1 0.7 0.8  -0.4 -3.4 0.5 1.7 -0.4 1.1 1.0 1.2Thermo Scientific 2.2 -1.5 0.8 -8.3 2.1 NA -1.9 5.2  2.6 -0.3 0.7 -0.7 -0.1 NA 1.0 6.4  2.8 0.5 0.6 4.4 -1.3 NA 2.3 7.3

© Copyright 2009 by the American Association for Clinical Chemistry© Copyright 2009 by the American Association for Clinical Chemistry

Figure 1. Assay bias (% difference = (mean of peer group result - target value/target value)*100) and total error vs AMTM (glucose, HDL-cholesterol) or REF target values (cholesterol, creatinine). Abbott (red diamond), Beckman (blue square), Ortho (black triangle), Roche (yellow circle), Siemens (red square), and Thermo Scientific (blue diamond). The red-broken bias limits are those listed in Table 1; the blue-broken limits are optimal bias limits from biological variation (online Supplemental Table 6).

ResultsResults

For conversion of the traditional units to SI units used in the online Supplemental Figs., multiply by 0.02586 for cholesterol (mmol/L), 88.40 for creatinine (µmol/L), 0.05551 for glucose (mmol/L),and 0.02586 for HDL cholesterol (mmol/L).

-6

-4

-2

0

2

4

6

8

100 120 140 160 180 200 220 240

Ass

ay D

iffer

ence

(%)

Cholesterol REF (mg/dL)-10

-8

-6

-4

-2

0

2

4

6

8

0.5 0.7 0.9 1.1 1.3 1.5

Ass

ay D

iffer

ence

(%)

Creatinine REF (mg/dL)

-5

-4

-3

-2

-1

0

1

2

3

4

5

70 90 110 130 150 170 190

Ass

ay D

iffer

ence

(%)

Glucose AMTM (mg/dL)-15

-10

-5

0

5

10

15

20 30 40 50 60 70 80 90 100

Ass

ay D

iffer

ence

(%)

HDL-Cholesterol AMTM (mg/dL)

© Copyright 2009 by the American Association for Clinical Chemistry© Copyright 2009 by the American Association for Clinical Chemistry

Figure 2. Assay bias (% difference) and total error vs AMTM (LDL cholesterol, phosphate, triglycerides) or REF target values (uric acid). Abbott (red diamond), Beckman (blue square), Ortho (black triangle), Roche (yellow circle), Siemens (red square), and Thermo Scientific (blue diamond). The red and blue broken limits are the same as described for Fig. 1.

ResultsResults

For conversion of the traditional units to SI units used in online Supplemental Figs., multiply by 0.02586 for LDL cholesterol, 0.3229 for phosphate (mmol/L), 0.01129 for triglycerides (mmol/L), and 59.48 for uric acid (µmol/L).

-20

-10

0

10

20

30

40

50 60 70 80 90 100 110 120 130 140 150 160

Ass

ay D

iffer

ence

(%)

LDL-Cholesterol AMTM (mg/dL)-5

-3

-1

1

3

5

7

9

11

13

15

2.3 2.6 2.9 3.2 3.5 3.8 4.1 4.4Ass

ay D

iffer

ence

(%)

Phosphate AMTM (mg/dL)

-15

-10

-5

0

5

10

15

20 70 120 170 220 270 320 370

Ass

ay D

iffer

ence

(%)

Triglyceride AMTM (mg/dL)-8

-6

-4

-2

0

2

4

6

8

10

3 4 5 6 7 8 9

Ass

ay D

iffer

ence

(%)

Uric Acid REF (mg/dL)

© Copyright 2009 by the American Association for Clinical Chemistry© Copyright 2009 by the American Association for Clinical Chemistry

ResultsResults  CHOL CREA GLU HDL LDL PHOS TRIGL UABias (%)Min -16 -9 -8 -20 -17 -6 -15 -7Max 6 17 6 8 25 13 13 11Diff 1 22 26 14 27 43 19 28 18Diff 2 11 22 10 16 25 14 21 16Diff 3 9 17 9 13 21 14 15 13Bias Low (%)Min -18 -10 -10 -16 -22 -8 -35 -7Max 6 28 6 14 42 21 29 14Diff 1 24 38 16 30 64 29 64 21Diff 2 12 36 12 22 52 22 38 18Diff 3 9 35 10 19 39 21 23 14Bias High (%)Min -15 -9 -6 -22 -15 -6 -14 -7Max 6 10 6 10 17 10 9 10Diff 1 21 19 12 32 32 16 23 17Diff 2 11 17 11 18 16 12 13 14Diff 3 9 16 8 18 14 11 11 13

Table 2. Observed AMTM bias in the participating laboratories. The blue, underlined values refer to maximum absolute laboratory biases >15% and differences between laboratories >30%. CHOL, cholesterol; CREA, creatinine; GLU, glucose; HDL, HDL cholesterol; LDL, LDL cholesterol; PHOS, phosphate; TRIGL, triglycerides; UA, uric acid; NA, not applicable. Diff 1, the difference between the most deviating laboratories, diff 2 and 3 the 2nd and 3rd most deviating laboratories. Note: “bias low and high” stand for the bias at the limits of the concentration range covered by the panel.

© Copyright 2009 by the American Association for Clinical Chemistry

Limitations of the studyLimitations of the study

The CLSI C37-A protocol does not in itself prove that the sampels were commutable

The number of laboratories included was, of necessity, relatively small

Concentration ranges covered by the samples were small; no reflection of performance at pathological concentrations

Targets traceable to reference methods not available for all analytes

Selected state-of the-art limits can be debated

© Copyright 2009 by the American Association for Clinical Chemistry

ConclusionsConclusions

The results indicated room for improved quality of performance, standardization and interchangeability of results

Notable was the observation that this applies for the simple analytes (and at concentrations) assessed here

The study demonstrated that dedicated PT surveys are a powerful tool to uncover and hopefully solve certain problems

Dedicated PT surveys do not substitute conventional PT but are complementary

© Copyright 2009 by the American Association for Clinical Chemistry

Final Comment in EditorialFinal Comment in Editorial

“For now, we should celebrate the insights that Stepman et al. have provided. As good as conventional PT may be, we can do better.”

“We are indebted to these authors for shedding light on a problem we may have assumed we did not have and, more important, for providing a powerful tool to help us make things better.”

Horowitz GL. Assessing Accuracy on the Front Lines: A Pragmatic Approach for Single-Donor Proficiency Testing. Clin Chem 2014.

© Copyright 2009 by the American Association for Clinical Chemistry

Thank you for participating in this month’sClinical Chemistry Journal Club.

Additional Journal Clubs are available atwww.clinchem.org

Download the free Clinical Chemistry app on iTunes for additional content!

Follow us