Statistics & graphics for the laboratory 35 Analytical Quality Specifications ("Goals") Introduction...

Statistics & graphics for the laboratory 1

Analytical Quality Specifications ("Goals")

Introduction

Concepts• Clinical concepts• Questionnaires to clinicians• Goals from biology• Goals from experts• State-of-the-art

Comparison of goals

Comparison "state-of-the-art" with goals

Analytical goals – Translation into practice

Goals – future vision

Outlook

References

Content


Introduction

The analytical quality triangleThe establishment of analytical performance goals (quality specifications) is an important part of the analytical quality triangle, which should guarantee medically relevant quality of laboratory tests.

In the words of ISO 15196 – Analytical goals (discontinued) this is expressed as: "Well-defined analytical performance goals for clinical laboratories have practical, regulatory, and commercial implications. The goals help to determine the day-to-day practices of clinical laboratories. In turn, these goals should shape, if not drive the plans of IVD manufacturers to produce devices that can meet the analytical needs. In a proper synergy, regulatory agencies should base their expectations of laboratory quality on well-conceived analytical goals".

Unfortunately, the clinical community has not been able to establish a consensus on goals for analytical quality. A major problem is the involvement of many groups with different interests: "analytical performance goals shall be reviewed by the laboratory director to verify that they a. meet local medical needs; b. meet applicable regulatory requirements; c. are economically feasible; d. are technically achievable" (ISO 15196).

Nevertheless, what has been achieved is a consensus on a hierarchy of models to be used for the establishment of analytical goals.

Introduction

ImprovementAssurance

Control

Quality creation

Quality specification

Planning

Quality management

InstrumentChemistry

State-of-artExpert

BiologyClinical


Introduction

Hierarchy of models for the establishment of analytical goals• Clinical concepts• Concepts based on biological variation• Expert opinion• Regulations• "State-of-the-art" Consensus Statement (Stockholm 1999).Scand J Clin Lab Invest 1999;59:585.

In the following, these concepts will be reviewed with the emphasis on their underlying statistical basis. For easier understanding, a list of abbreviations of biological and analytical components of variance is given below.

Abbreviations for biological variation• SDW-S/CVW-S (or CVw) = Within-subject

• SDB-S/CVB-S (or CVb) = Between-subject

• SDG/CVG (or CVg) = "Group"

= SQRT(CV2W-S + CV2

B-S)

= ¼ of the reference interval• SDBiol/CVBiol : if the component (within, etc.) is not specified

CAVE: SDB-S often is understood as SDG

Abbreviations for analytical variation/error• SDA/CVA (or CVa) = Analytical

• TEA = Total allowable analytical error

Introduction


Unifactorial health/disease models (1 analyte/1 disease)

For example:• HbA1c for long-term control of diabetes.

2 Situations• Separate distributions (bimodal) of the analyte in sick and healthy • One distribution (unimodal) of the analyte in sick and healthy

Unimodal situation Bimodal situation

The bimodal situation

Prerequisites The “gold-standard”The true status of each population has to be established by other means than the test being subject to evaluation, namely, a so-called “gold standard”.

Defining a decision point (“cut-off” value)A decision point (sick/healthy) must be defined. Note that this point must not lie at the crossing of the two distributions. Dependent on the importance of false negatives or false positives, it can be moved towards increased sensitivity or specificity.

Note: For monitoring, a medically significant change has to be defined.

Classification of resultsWith respect to the gold standard, test outcome is classified as “true positive”, “false positive”, “true negative”, or “false negative”.

Clinical concepts

Clinical concepts


Analytical error and test outcome• At a given clinical situation, the influence of analytical quality on test

sensitivity and specificity is investigated by simulation studies.

Simulation of systematic error• The introduction of a systematic error in the direction of the diseased

population increases the false positive results. The introduction of a systematic error in the direction of the healthy population would increase the false negative results.

Simulation of random error• The introduction of random analytical error, generally, deteriorates test

accuracy.

Simulation of systematic error Simulation of random error

Bias diseased > FP, FNGeneral deteriorationBias healthy > FP, FN of test accuracy

See also: Hyltoft Petersen P. Scand J Clin Lab Invest 1999;59:517-22

Clinical concepts – ResultsFrom considerations about maximum tolerable deterioration of the intrinsic clinical quality of a test, maximum values for analytical inaccuracy and imprecision are derived by simulation studies. The derived specifications• are restricted to the specific clinical situation addressed• depend on the opinion of the clinicians involved (e.g., definition of the “cut-off” value) • depend on the parameter that was used for optimization (sensitivity, specificity, cost/benefit considerations, etc).

Clinical concepts – Bimodal situation (ctd.)

Clinical concepts


Clinical concepts

Unimodal – Decision limits

Serum-CholesterolUni-modal distributionThe whole populationClassificationEffect of analytical quality

Serum cholesterol in men – Effect of biasKlee et al. Scand J Clin Lab Invest 1999;59:509-12 (wrong numbers: ±1, 3, 10%).(200 mg/dl = 5.17 mmol/l; 240 mg/dl = 6.2 mmol/l)On the basis of the hospital population at Mayo, the % misclassifications due to bias were investigated.

A bias of ±1% caused a -6.5% to +5.8% change in the classification of patients for cardiac disease. A bias of of ±3% caused a -18.4% to +16.7% change in the classification. From these data, a maximum bias of 1% was proposed for cholesterol.

Clinical concepts

250

200

150

100

50

050 100 150 200 250 300 350

Nu

mb

er o

f p

atie

nts — 200 mg/dl

– – – ± 2%

• • • • • ± 6%

• – • – ± 20%

Cholesterol, mg/dl

250

200

150

100

50

050 100 150 200 250 300 350

Nu

mb

er o

f p

atie

nts — 200 mg/dl

– – – ± 2%

• • • • • ± 6%

• – • – ± 20%

250

200

150

100

50

050 100 150 200 250 300 350

Nu

mb

er o

f p

atie

nts — 200 mg/dl

– – – ± 2%

• • • • • ± 6%

• – • – ± 20%

Cholesterol, mg/dl


Clinical concepts

Unimodal – Decision limits

Serum cholesterol in men – Effect of biasSimilar investigations have been made by Hyltoft Petersen et al. (Clin Chim Acta 1997;260:189-206). However, they made a Gaussian model of the distribution and read the effect of bias from the cumulated distribution. Moreover, they mirrored the distribution and centered it at the decision point, for an easier visual assessment.

A bias of +3%, for example, incresed the positives from 48% to 62% (= +14%). However, no limit was recommended in that publication.

Effect of interferencesAssume a "low prevalence situation" where 0.2% of patients are found above a certain cutoff (= 2 of 1000). If the test shows 0.2% interferences that cause otherwise normal to be above the cutoff, 4 are tested positive (100% increase). For a prevalence of 2% (= 20 of 1000), 22 will test positive with a test with 0.2% interference (10% increase). In a “high prevalence” situation, interference may be less a problem.

Clinical concepts

Bias (mmol/l)

% with CHOL >6.2 as function of bias

6.2

± 3 %

+1 mmol/l

-1 m

mol

/l

Bias (mmol/l)

% with CHOL >6.2 as function of bias

6.2

± 3 %

+1 mmol/l

-1 m

mol

/l

Incr

ease

Cumulateddistribution

% at or above given CHOL

S-CHOL (mmol/l)

Incr

ease

Cumulateddistribution

% at or above given CHOL

S-CHOL (mmol/l)


Clinical concepts

Examples from literature• Haemoglobin A1c for long term diabetic control [1]

• Theophylline in serum for therapeutic drug monitoring [2]• Creatine kinase isoenzyme in the diagnosis of acute myocardial infarction [3]• Blood thyroid-stimulating hormone in screening for congenital hypothyroidism [3] • Cholesterol as a measure for risk of coronary heart disease [3]• AFP, prenatal screening for neural tube defects [5]• Reviews: Hyltoft Petersen & Hørder [3, 4] and a special issue of the Upsala Journal of Medical Sciences [5].

Summary clinical concepts• Their establishment is very complicated• They offer several values, dependent on the model used to derive them (e.g., optimisation of specificity, sensitivity, cost/benefit, etc.)• Only a few analytes have been tackled• They are extremely valuable

Clinical concepts


Clinical concepts

References1 Larsen ML, Fraser CG, Petersen PH. A comparison of analytical goals for haemoglobin A1c assays derived using different strategies. Ann Clin Biochem 1991;28:272-8.2 Jenny RW. Analytical goals for determination of theophylline concentration in serum. Clin Chem 1991;37:154-8.3 Petersen PH, Hørder M. Ways of assessing quality goals for diagnostic tests in clinical situations. Arch Pathol Lab Med 1988;112:435-43.4 Hyltoft Petersen P, Hørder M. Influence of analytical quality on test results. Scand J Clin Lab Invest 1992;52 Suppl 208:65-87.5 de Verdier C-H, Groth T, Hyltoft Petersen P, editors. Medical need for quality specifications in clinical laboratories. Upsala J Med Sci 1993;98:189-491.

References specific to TSH1 Klee GG. Clinical interpretation of reference intervals and reference limits. A plea for assay harmonization. Clin Chem Lab Med 2004;42:752-7.2 Fatourechi V, Klee GG, Grebe SK, Bahn RS, Brennan MD, Hay ID, McIver B, Morris JC 3rd. Effects of reducing the upper limit of normal TSH values. JAMA 2003;290:3195-6.3 Fatourechi V, et al. Factors influencing clinical decisions to initiate thyroxine therapy for patients with mildly increased serum thyrotropin (5.1-10.0 mIU/L). Mayo Clin Proc 2003;78:554-60.4 Klee GG, Schryver PG, Kisabeth RM. Analytic bias specifications based on the analysis of effects on performance of medical guidelines. Scand J Clin Lab Invest 1999;59:509-12.5 Hay ID, Klee GG. Linking medical needs and performance goals: clinical and laboratory perspectives on thyroid disease. Clin Chem 1993;39:1519-24.6 Ward G, McKinnon L, Badrick T, Hickman PE. Heterophilic antibodies remain a problem for the immunoassay laboratory. Am J Clin Pathol 1997;108:417-21.7 Despre´s N, Grant AM. Antibody interference in thyroid assays: a potential for clinical misinformation. Clin Chem 1998;44:440–54.8 Ismail AAA et al. Wrong Biochemistry Results: Two Case Reports and Observational Study in 5310 Patients on Potentially Misleading Thyroid-stimulating Hormone and Gonadotropin Immunoassay Results. Clin Chem 2002;48:2023–29.

Clinical concepts


Questionnaires to clinicians

MethodThe clinicians or patients (self-monitoring) are presented for case histories or situations in the form of paper vignettes.The physician / patient is often asked about a critical difference (CD, or med)

necessary to initiate/change an action.

Case story/situation-Relevant problem-Frequently encountered-Laboratory test of especially importance -Setting ”easily” described-Conversational writing style; ”quoting” patients.

Example

Dependent on the question, the CD can comprise• Pre-analytical variation• Imprecision under defined conditions• Within-subject variation (CVW-S)

• Group biological variation (CVG)

• Bias

Clinical concepts

Suppose you find in a patient under treatment the first laboratory result given in the table. You follow the course of this patientand/or the effect of treatment. Please circle the value that would represent a significant change, in these cases improvement.

50

240

2.5

120

2

56545248Protein (g/l)

210220230250Glucose (mg/dl)

2.82.72.62.4Potassium (mmol/l)

123122121119Sodium (mmol/l)

5431

Result

50

240

2.5

120

2

56545248Protein (g/l)

210220230250Glucose (mg/dl)

2.82.72.62.4Potassium (mmol/l)

123122121119Sodium (mmol/l)

5431

Result

Elion Gerritzen WE. Analytical precision in clinical chemistry and medical decisions. Am J Clin Pathol 1980;73:183-95.


Probability considerations ("z")

Probability/certainty that patients (physicians) attribute to the answer of the question depends on:-Semantic of the question and - of course - the clinical situation-Willingnes to accept false positive/negative actions-”One sided” or ”two sided” situations

Establishment – Summary• Production of adequate questionnaires• Extraction of a medical action limit: med

• Transformation of med into an analytical specification for CV, bias or TE

CalculationsMonitoring: med = Bias+z•2•SQRT[CVA

2+CVW-S2]

CVA = SQRT[{(med-Bias)/z•2}2-CVW-S2]

Diagnosis: med = Bias+1.65•SQRT[CVA2+CVG

2]

CVA = SQRT[{(med-Bias)/1.65}2-CVG2]

Note: z = 1.65 (1.96) for "better/worse" ("change")

Calculation exampleImagine that you have measured your blood glucose level to be 8.0 mmol/lA. To what value do you think your blood glucose must increase to before you would be sure that it represents a true increase? ___B. To what value do you think your blood glucose must decrease to before you would be sure that it represents a true decrease? ___Answers


50th percentile

25th percentile

38%30%

25%25%

Decrease from 8.0 mmol/l

Increase from 8.0 mmol/l

50th percentile

25th percentile

38%30%

25%25%

Decrease from 8.0 mmol/l

Increase from 8.0 mmol/l

Calculations

CVA = SQRT[{(med-Bias)/z•2}2-CVW-S2]

med = 25% increase from 8 mmol/l

Bias = 0

z = 1.65

CVW-S = 5% CVA ~ 10%


Medical decision limits investigated

Medical decision limits (med) related to biological variation (CVW-S) and analytical

state-of-the-art (CVA) (the data of Linnet and Skendzel were used for the

calculation) When CVA = 0, is med/CVW-S = 2.77

When CVA/CVW-S = 0.5, is med/CVW-S = 3.10

When CVA/CVW-S = 1.5, is med/CVW-S = 5.0

We observe incongruent med values:

med/CVW-S should be ~3.5 for glucose

med/CVW-S should be ~2.8 for triglycerides (biological variation underestimated?)

Summary – Goals from Questionnaires to Clinicians• Their establishment is complicated• Usually, they reflect the analytical quality the clinicians got offered• Sometimes they are based on incongruent med values

• Often, biological variation is not adequately considered, with the consequence that the derived values are mostly too generous• They are very useful when they were established in the correct manner


2610.2Triglycerides

231.90.5Bilirubin

4.4

4.4

1.8

4.4

0.8

CVW-S

2.70.5Potassium

5.10.8Glucose

5.41.3Calcium

5.91.4Creatinine

4.51.5Sodium

med/CVW-SCVA/CVW-SAnalyte

2610.2Triglycerides

231.90.5Bilirubin

4.4

4.4

1.8

4.4

0.8

CVW-S

2.70.5Potassium

5.10.8Glucose

5.41.3Calcium

5.91.4Creatinine

4.51.5Sodium

med/CVW-SCVA/CVW-SAnalyte


References

• Barnett RN. Medical significance of laboratory results. Am J Clin Pathol 1968;50:671-6.• Barrett AE, Cameron SJ, Fraser CG, Penberthy LA, Shand KL. A clinical view of analytical goals in clinical biochemistry. J Clin Pathol 1979;32:893–6. • Elion Gerritzen WE. Analytical precision in clinical chemistry and medical decisions. Am J Clin Pathol 1980;73:183-95.• Skendzel LP, Barnett RN, Platt R. Medical useful criteria for analyte performance of laboratory tests. Am J Clin Pathol 1985;83:200-5.• Thue G, Sandberg S, Fugelli P. Clinical assessment of haemoglobin values by general practitioners related to analytical and biological variation. Scand J Clin Lab Invest 1991;51:453-9.

See also• Gilbert RK. Progress and analytical goals in clinical chemistry. Am J Clin Pathol 1975;63:960-73.• Koch DD, Oryall JJ, Quam EF, et al. Selection of medically useful quality-control procedures for individual tests done in a multitest analytical system. Clin Chem 1990;36:230-3.



Goals for "multipurpose" analytes

Potassium test in a general laboratory

Urgency situations• Digitalis intoxication• Heart arrhytmias• Acute renal failure• Diarrhea

Therapy decisions, treatment with• Angiotensin-converting enzyme blocker • Digitalis • Diuretics

Monitoring illness• Diabetes mellitus• Chronic renal failure• Gastrointestinal diseases

Conclusions from the potassium exampleIt is NOT useful to set different goals for all the different situations.Usually, 1 test is used for all applications in the laboratory.

Take the most stringent goal, then all situations are covered• The proposed goals are independent of the magnitude of potassium present in a specimen• Always gaussian distribution and 5% test-level is considered

Weakness of the approach• "One size NEVER fits all"• If there are better goals, use them

Proposal: Establish goals from biological variation

Goals for "multipurpose" analytes


General considerations for analytical goals

Monitoring:• Stable operation and low imprecision compared to the within-subject

biological variation.Diagnostic Testing:

• Sufficient accuracy to allow the use of common reference intervals.

Goals from biology – The most important concepts

Tonks (1967)Empirical conceptThe total allowable analytical error (TEA) should not exceed one quarter of the

reference interval (RI).

TEA 1/4[RI]/[mean of RI] x 100%.

Statistical backgroundReference interval = 4 • CVG TEA CVG

Cotlove, Harris & Williams (1970)Relation of the analytical error (SDA) to a biological standard deviation (SDBiol)

SDA 0.5 SDBiol

Statistical background (in CV-terms)CVA = 0.5 • CVBiol,

CVA adds only 12% to the

total test variability (CVT)

CVT = [(0.5 CVBiol)2 + CVBiol2]1/2

CVT = 1.12 CVBiol

Relationship between conceptsTonks & Cotlove, Harris & Williams are quite similarIf bias is neglected, TEA defined by Tonks equals 2 SDA

The reference interval represents the mean ± 2 SDG = 4 SDG

Tonks formula can then be written as: 2 SDA 0.25 x 4 x SDG, equals SDA 0.5 SDG

0

50

100

150

0.0 0.5 1.0 1.5 2.0CVA/CVBiol

Incr

eas

e o

f v

ari

ati

on

(%

)

12%

Goals from biology

Goals from biology


Harris (1988)

Bias (B) and imprecision were both taken into account.

The original equation: • SDA 0.5 SDW-S (SDG)

Was thus changed into: • [SDA

2 + B2]1/2 0.5 SDW-S (SDG).

Gowans et al. (1988)Statistical backgroundConfidence interval (CI) of IFCC recommendations for establishment of a reference-interval: n = 120... translated into increase of SDG by SDA

Goals from biology

Sample size

10,000

1,000

100

1090 % CI 90 % CI

N = 120


Gowans et al. (1988)

Statistical background > ImprecisionConfidence interval n = 120 translated into increase of SDG by SDA:

SDT = SQRT[SDG2 + SDA

2]: =translated into imprecision

Statistical background > BiasConfidence interval n = 120 translated into bias

Statistical background: IFCC reference-interval, n = 120

Consequence: Confidence interval for sample size• n = 120, maximum 4.6% outside each limit• n = , maximum 2.5% outside each limit

Model• Allocate the difference n = 120 versus n =

to analytical errorOutcome of the concept

• B 0.25 SDG (without imprecision)

• [SDA 0.6 SDG (without bias)] (original publication, meanwhile adapted)

• SDA 0.52 SDG (without bias; from "diagnostic curve": see later!)

Goals from biology

-5 -4 -3 -2 -1 0 1 2 3 4 5

SDG alone, n = 2.5% outside

SDT, n = 4.6% outside

90% CI, n = 120

-5 -4 -3 -2 -1 0 1 2 3 4 5

No bias, n = 2.5% outside

Bias, n = 4.6% outside

90% CI, n = 120

CVG

0.25 CVG


Statistical background: Influence of imprecision and bias of a test on its clinical specificityDefinition of an analytical "error budget": SQRT[SDA

2+Bias2]

Limit: 50% increase in the false-positive rate for classifying healthy subjects (unimodal distribution,± 2s decision limit)

Outcome of the concept:Error budget: 0.45 SDG

Allocated as• 0.36 SDG for bias, &

• 0.18 SDG for SDA

Monitoring goals from biologyThe reference change value (RCV) (Least "medical significant difference") (Harris & Yasaka Clin Chem 1983;29:25-30)

CalculationsSmallest medically significant difference (med) that analytically can be detected for

two consecutive measurements with P = 0.05 and SDA = 0:

• med = 1.96 x 2 x SDW-S = 2.77 x SDW-S

When SE > 0 and SDA > 0, the equation can be rewritten (Larsen et al. 1991)

• med = 2.77 x (SDA2 + SDW-S

2)1/2 + SE Eq. 1

When SE = 0 and SDA = 0.5 SDW-S follows:

• med = 2.77 (0.25 SDW-S2 + SDW-S

2)1/2

• med = 3.10 SDW-S

When SDA = 0 and med = 3.10 SDW-S, Equation 1 can be rearranged:

• 2.77 x SDW-S + SE = 3.10 SDW-S

Resulting in:• SE 0.33 SDW-S

Summary SDA 0.5 SDW-S

med = 2.77 SDW-S if SDA = 0

med = 3.10 SDW-S if SDA = 0.5 SDW-S

SE 0.33 SDW-S if SDA = 0

-5 -4 -3 -2 -1 0 1 2 3 4 5

3.41% versus2.28% outside

Klee (1993)

Goals from biology


Reference change (med) concept (left figure):

relationship between allowable drift and CVA in fractions of CVW-S (SE/ CVW-S and CVA/CVW-S ).The figure shows the case med = 3.10 CVW-S

Common reference interval concept (right figure):relationship between allowable bias and CVA in fractions of CVG (SE/CVG and CVA/CVG). Case where 4.6% are outside each reference limit.

Goals – Monitoring & Diagnosis: Which is most stringent?Monitoring and diagnosis use different biological CV• Assume CVb (between) = 1.75 • CVw (within) CVg ~2 CVw

• Transform the CVg scale in diagnosis to CVw

Monitoring gives more stringent goals

Monitoring

0

0,1

0,2

0,3

0,4

0,5

0,6

0 0,25 0,5 0,75 1

CVa/CVw

SE

/CV

w

Diagnosis transformed to monitoring (CVb = 1.75 * CVw)

0

0,1

0,2

0,3

0,4

0,5

0,6

0 0,25 0,5 0,75 1CVa/CVw

SE

/CV

w

When SE and RE are present (complex calculations!)

Goals from biology

Monitoring

0

0.1

0.2

0.3

0.4

0.5

0 0.1 0.2 0.3 0.4 0.5 0.6

CVA/CVW-S

SE

/CV

W-S

Diagnosis

0

0.1

0.2

0.3

0.4

0.5

0 0.1 0.2 0.3 0.4 0.5 0.6

CVA/CVG

SE

/CV

G


Missing data for SDW-S and SDB-S

Assumption• SDB-S = 1.75 • SDW-S, then SDG = SQRT(1.752 + 12) = 2 • SDW-S

With• RI = 4 • SDG (RI: reference-interval)

Follows• RI = 4 • 2 • SDW-S = 8 • SDW-S > SDW-S = RI/8

Assumption• SDA ~ 0,5 • SDW-S

Follows• SDA ~ 1/15 RI,

Conclusion for SDA

• When data for SDW-S and SDB-S are missing, take RI/15 to comply with the

Harris concept

Goals from biology


Summary

Goals from biology – Summary

Conclusion: most values for CVA are ~0.5 CVBiol; most values for bias are ~0.25

or 0.33 CVBiol. Values for total error vary, another concept will be shown later (see

Ricos concept).

Goals from biology – Most often used

For monitoring as:• SDA 0.5 SDW-S (in the absence of unidirectional systematic changes)

• SE 0.33 SDW-S (when imprecision is negligible)

For diagnostic testing as:• B 0.25 SDG (when the imprecision is negligible), or

• SDA 0.52 SDG (when bias is negligible; from "diagnostic curve")

Goals from biology

$Dependent on the concept, SDBio is within-, between-, or group biological variation.

0.45 SDBio0.36 SDBio0.18 SDBioKlee

"Graphics""Graphics"

0.33 SDBio

0.25 SDBio

0.5 SDBio

0.52 SDBioGowans et al.

[SDA2+SE2]1/2

0.5 SDBio——Harris

——0.5 SDBio$Cotlove et al.

0.5 SDBio——Tonks

Total Error(TE)

Bias, Shift/Drift(SE, or SE)

Imprecision(SDA)

$Dependent on the concept, SDBio is within-, between-, or group biological variation.

0.45 SDBio0.36 SDBio0.18 SDBioKlee

"Graphics""Graphics"

0.33 SDBio

0.25 SDBio

0.5 SDBio

0.52 SDBioGowans et al.

[SDA2+SE2]1/2

0.5 SDBio——Harris

——0.5 SDBio$Cotlove et al.

0.5 SDBio——Tonks

Total Error(TE)

Bias, Shift/Drift(SE, or SE)

Imprecision(SDA)


The Ricos et al. concept$ (TE = SE + z • RE)$Ricós C, Baadenhuijsen H, Libeer J-C, Hyltoft Petersen P, Stöckl D, Thienpont LM, Fraser CG. Eur J Clin Chem Clin Biochem 1996;34:159-65.

TE = 0.25 • CVg + 1.65 • [0.5 • CVw]

CVg = group biological variation (can be calculated from CVw and CVb;

alternatively = ¼ of the reference interval)CVw = within-subject biological variation

CVb = between-subject biological variation

Note: in www.westgard.com CVg = CVb

Remark: The Ricos et al. concept is a simplified, practical concept and a hybrid between monitoring and diagnosis.The Ricos et al. concept combines concepts for monitoring & diagnosis by use of the respective RE/SE goals with SE = 0 and RE = 0.

RemarkTEa is not constant in the biological concepts; it depends on the SE/RE-ratio:Select a realistic SE/RE-ratio. Choose, in the monitoring case the point SE = RE TE = 0.66 CVw

TERicos = 0.25•CVg (=2•CVw) +1.65•[0.5•CVw] = 1.3 • CVw

The Ricos concept gives too big TE-values.

TE monitoring (k: 1.65 to 2, SE/RE<1)

0

0,2

0,4

0,6

0,8

1

1,2

0 0,1 0,2 0,3 0,4 0,5CVa/CVw

SE

/CV

w

SE/RE-l ine

TE-l ineRealistic

TE

TE from biology

Goals from biology

Monitoring

0

0,1

0,2

0,3

0,4

0,5

0 0,1 0,2 0,3 0,4 0,5 0,6

CVa/CVw

SE

/CV

w

Diagnosis

0

0,1

0,2

0,3

0,4

0,5

0 0,1 0,2 0,3 0,4 0,5 0,6

CVa/CVg

SE

/CV

g


TE from biology

BUT: Some are too stringent for current technologyBiology gives extreme low values for some analytes: TEa sodium: Ricos: 0.9% (http://www.westgard.com/biodatabase1.htm; but note: CVg on that site = CVb)

Apply a bottom-line##Stöckl D. Desirable Performance criteria … based on biological analyte variation - hindrances to reaching some and reasons to surpass some. Clin Chem 1993;39:913-4.

Bottom-line (for stable process!)CV: 1%Bias: 1,5%TE: 3,2%

Consider to surpass someThink about using more stringent goals than derived from biology the goals are a compromise; account for IQC; some goals are surpassed, by far, by current technique.

• Stöckl D. Desirable Performance criteria … based on biological analyte variation - hindrances to reaching some and reasons to surpass some. Clin Chem 1993;39:913-4.• Tonks: 10% upper limit for TE

Goals from biology

Mon DiagSE=0 RE=0 Ricos"0,5" "0,25" "1,65"

Analyte CVw CVb CVg REmax SEmax TEmaxSodium 0.7 1.0 1.2 0.4 0.3 0.9Chloride 1.2 1.5 1.9 0.6 0.5 1.5Calcium 1.9 2.8 3.4 1.0 0.8 2.4Protein 2.7 4.0 4.8 1.4 1.2 3.4Albumin 3.1 4.2 5.2 1.6 1.3 3.9

Mon DiagSE=0 RE=0 Ricos"0,5" "0,25" "1,65"

Analyte CVw CVb CVg REmax SEmax TEmaxUrea 12.3 18.3 22.0 6.2 5.5 15.7Triglycerides 20.9 37.2 42.7 10.5 10.7 27.9CK 22.8 40.0 46.0 11.4 11.5 30.3ALT 24.3 41.6 48.2 12.2 12.0 32.1Bilirubin 25.6 30.5 39.8 12.8 10.0 31.1Iron, total 26.5 23.2 35.2 13.3 8.8 30.7

http://www.westgard.com/biodatabase1.htm


TE from biology

NOTEWhen data for CVw and CVb are not available,

"Ricos":take 1/6th of the reference interval! (assumption CVb = 1.75 • CVw)

Monitoring ("Most stringent"):take 1/10th of the reference interval!

Conclusions

Desirable analytical quality should be based on within- and between-subject biological variation because:• the model is simple to understand and apply • there are many data on biological variation • the within- and between-subject biological variations are nearly constant over geography• when these criteria are fulfilled, the analytical quality will satisfy most clinical needs• the model has a great practical and educative value.

ButSometimes they are too narrow (e.g., electrolytes), sometimes they are too broad (e.g., enzymes).They do not consider that different requirements might be necessary for different concentrations.Current approaches give too generous values.

Goals from biology


Goals from "Experts"

Statistical background• Usually, empiric

Establishment• EQA-performance• "State-of-the-art" performance

Goals from experts


State-of-the-art versus State-of-the-art

Comparison of imprecision data (“mean claims”) of the instrument generation 2001 from 4/5 manufacturers.

Comparison of goals

Ortho

Roche

Abbott

Bayer

Beckman

Manufacturer

Vitros ECiVitros 700

ElecsysModular

ArchitectAeroset

Advia Centaur---

AccessSynchron LX20

ImmunoassaysClinical chemistry

Ortho

Roche

Abbott

Bayer

Beckman

Manufacturer

Vitros ECiVitros 700

ElecsysModular

ArchitectAeroset

Advia Centaur---

AccessSynchron LX20

ImmunoassaysClinical chemistry

Analyte Level total-CV (%) Analyte Level total-CV (%)Best Worst Best Worst

Na 120 0.5 1.1 Mg 0.6 3 4mmol/l# 145 0.7 1.2 mmol/l 0.9 2 4.4

160 0.8 1.2 2 1 3.5

Cl 85 0.8 1.4 Crea 0.9 1.1 5mmol/l 100 0.8 1.5 mg/dl 1.5 1 3.6

120 0.6 1.4 6 1.1 3

Ca 2 1.3 1.8 K 3 1.1 2mmol/l 2.4 1.2 2 mmol/l 4.5 1 2

3.4 1.2 2.3 7 0.6 1.3

Prot 4 1.1 2 Gluc 90 1.2 2.2g/dl 5.5 1 2 mg/dl 150 1.5 2.2

7 1.1 2.5 300 1.2 2.3

Alb 2.5 0.9 2.3 Chol 100 1.5 1.8g/dl 3 0.8 2 mg/dl 150 1.6 1.8

4.5 0.6 1.7 250 0.8 2

#Note: SI Units, except ALP 100 3 4.5when all used conventional U/l 250 1.1 3

600 2.5 3


LDH 150 1.3 5.7 Urea 20 1.8 6.1U/l 500 1.7 3.2 mg/dl 50 1.8 2.1

1000 0.9 2.7 80 1.6 1.9

Phos 2.5 1.1 2.7 Trigly 80 1.1 2.2mg/dl 3.5 1.1 2.1 mg/dl 150 0.8 1.6

8 0.5 2.4 250 1.6 1.8

UA 3 1.9 2.3 CK 100 2 5mg/dl 5 1.4 1.6 U/l 300 1.3 4.6

8 0.8 1.9 600 0.9 4

ACP 5 3.9 4.9 ALT 30 2.2 6U/l 10 3 5.3 U/l 150 1.3 2.9

30 1.1 2.7 400 0.7 2.3

Amyl 60 1 3.5 Bili 0.8 3.4 5.6U/l 350 1 1.6 mg/dl 4 1.4 4.3

500 0.8 1.6 20 1.2 3.8

AST 30 1.6 4.5 Fe 80 1 3.5U/l 150 0.8 2.4 µg/dl 120 1.2 2

250 0.7 2.4 250 0.7 2.1


State-of-the-art versus State-of-the-art

• There is a big difference between best and worst state-of-the-art

• Note: “Best/worst” do not refer to the complete system. E2 may be best on 1 system, but the same system may have a poor fT4 test

• For clinical chemistry, total CV’s of the best systems are generally <2%, with a minimum of 0.5%

• For immunochemistry, total CV’s of the best systems are generally <5%, with a minimum of 2%

Comparison of goals


TSH 0.1 2 7 Prog 3 6 13µIU/ml# 5 2 4 nmol/l 40 3 8

30 3 5 85 3 8

TT4 50 5 7 E2 150 6 20nmol/l 100 2 6 pmol/l 500 4 8

200 3 8 2000 3 6

fT4 6 4 10 Testo 2 7 8pmol/l 15 3 7 nmol/l 15 3 6

40 4 6 40 3 6

TT3 1 3 8 LH 5 3 9nmol/l 2 2 5 mIU/ml 50 2 4

5 2 5 100 2 5

fT3 3 4 15 FSH 5 4 7pmol/l 10 3 9 mIU/ml 30 2 5

20 3 8 80 3 6

#Note: SI Units, except PRL 60 4 8when all used conventional mIU/l 400 3 5

2000 4 6


AFP 10 3 6 B12 200 8 12IU/ml 100 2 5 pg/ml 500 4 8

500 3 6 1000 4 10

CEA 5 3 6 Folate 2 9 10ng/ml 30 3 5 ng/ml 7 6 8

80 3 3 20 5 7

ßhCG 10 5 7 Ferritin 10 4 5mIU/ml 200 3 6 ng/ml 150 3 6

1000 3 5 450 3 6

PSA (t) 0.5 2 6ng/ml 4 3 5

35 3 4

CA 19-9 30 4 9IU/ml 100 3 5

250 4 4

CA 125 30 4 5IU/ml 100 3 4

400 3 5


Questionnaire & Biology

The table compares analytical goals from questionnaires [1], with analytical goals from biology [2]. It shows that goals from questionnaires may be very generous (for example, Calcium: CV = 4.8%; Cholesterol: CV = 12.3%). A weakness in that publication was that clinicians may have neglected the biological variation of the analytes (see also p. 46).

[1] Skendzel LP, Barnett RN, Platt R. Medical useful criteria for analyte performance of laboratory tests. Am J Clin Pathol 1985;83:200-5 (Questionnaire).[2] Fraser CG, Hyltoft Petersen P, Ricos C, Haeckel R. Proposed quality specifications for the imprecision and inaccuracy of analytical systems for clinical chemistry. Eur J Clin Chem Clin Biochem 1992;30:311-7 (Biology).

Comparison of goals

6.312.2Urea

2.44.8Potassium

2.211.2Glucose

2.712.3Cholesterol

0.94.8Calcium

CVA[2]CVA[1]Analyte

6.312.2Urea

2.44.8Potassium

2.211.2Glucose

2.712.3Cholesterol

0.94.8Calcium

CVA[2]CVA[1]Analyte


European EQA (% limits from 13 countries) – "Expert goals"

Analytes arranged by increasing biological variation. From: Ricos et al. Eur J Clin Chem Clin Biochem 1996;34:159-65.

The table shows an extreme variation in EQA-goals.

Proposed goals – Is there a consensus?TEa of serum sodium measurement

TEa of serum glucose measurement

Unfortunately, there is no consensus in goal-setting!

Comparison of goals

0

10

20

30

40

50

60

70

5German EQA

2.9American EQA (CLIA)(in % mean of RI)

2.2¼ of the reference-interval

0.9Biological variation(Ricos data, “Westgard page”)

TEa (%)Source

5German EQA

2.9American EQA (CLIA)(in % mean of RI)

2.2¼ of the reference-interval


TEa (%)Source

2.7Boyd & Bruns(CC, 2001;47:209)

20Glucometers ("Upper A-line")

15German EQA

10American EQA (CLIA)(in % mean of RI)

10¼ of the reference-interval


TEa (%)Source

2.7Boyd & Bruns(CC, 2001;47:209)

20Glucometers ("Upper A-line")

15German EQA

10American EQA (CLIA)(in % mean of RI)

10¼ of the reference-interval


TEa (%)Source


Overview

Goals for analytical quality –Résumé

Most obvious for daily practice• The “own” state-of-the-art• Legislation/EQA

Most relevant for the patient• ”Clinical”• ”Biological”

Most realistic as goal• The “better” state-of-the-art• ”Monitoring”/”diagnosis”

Relevance beyond establishing analytical goals

Clinical conceptImportant for test selection (“a-priori quality”)

Questionnaires to cliniciansImportant for the use of a test in practice

Biology• Determines the treshold for action (med monitoring)

• Determines the “gray” zone in diagnosis

Comparison of goals

Actually available quality

Most useful for staying in business

Useful as goals in method selection

For special centers (endocrine labs)

Remark

Some diversity

Big diversityFairFairAgreement

GeneralGeneral

General (note: "too wide/too strict")

Restricted to specific situations

Applica-bility

GeneralGeneralGeneralVery fewAvail-ability

State-of-the-artRegulationBiologicalClinical

Actually available quality

Most useful for staying in business

Useful as goals in method selection

For special centers (endocrine labs)

Remark

Some diversity

Big diversityFairFairAgreement

GeneralGeneral

General (note: "too wide/too strict")

Restricted to specific situations

Applica-bility

GeneralGeneralGeneralVery fewAvail-ability

State-of-the-artRegulationBiologicalClinical


State-of-the-art/"Harris" (CVa 0.5 CVW-S)

Some remarks at the beginning

Currently insufficiently known:• Bias from comparison with a reference method• SE from calibration tolerance/IQC limits• The global effect of unspecificity/interference

Usually, SE goals cannot be investigated and TE goals, only, can be compared with 2 • CV Comparisons are restricted to RE

TE goals from biology: "1 number for all" Comparisons are restricted to CV's in the reference interval


BestGoalLevelAnalyte

1.13.2250ALP, U/l

1.63150Chol, mg/dl

1.52.5150Glu, mg/dl

12.44.5K, mmol/l

12.21.5Crea, mg/dl

21.80.9Mg, mmol/l

0.81.63Alb, g/dl

11.45.5Prot, g/dl

1.212.4Ca, mmol/l

0.80.6100Cl, mmol/l

0.70.4145Na, mmol/l

CVa (tot)


1.13.2250ALP, U/l

1.63150Chol, mg/dl

1.52.5150Glu, mg/dl

12.44.5K, mmol/l

12.21.5Crea, mg/dl

21.80.9Mg, mmol/l

0.81.63Alb, g/dl

11.45.5Prot, g/dl

1.212.4Ca, mmol/l

0.80.6100Cl, mmol/l

0.70.4145Na, mmol/l

CVa (tot)

1.213.3120Fe, µg/dl


1.412.84Bili, mg/dl

1.312.2150ALT, U/l

1.311.4300CK, U/l

0.810.5150Trigl, mg/dl

1.86.250Urea, mg/dl

0.86150AST, U/l

14.8350Amyl, U/l

34.510ACP, U/l

1.44.35UA, mg/dl

1.14.33.5Phos, mg/dl

1.73.3500LDH, U/l

CVa (tot)

1.213.3120Fe, µg/dl


1.412.84Bili, mg/dl

1.312.2150ALT, U/l

1.311.4300CK, U/l

0.810.5150Trigl, mg/dl

1.86.250Urea, mg/dl

0.86150AST, U/l

14.8350Amyl, U/l

34.510ACP, U/l

1.44.35UA, mg/dl

1.14.33.5Phos, mg/dl

1.73.3500LDH, U/l

CVa (tot)



Other way of presentationWe can look at the contribution of the analytical variance to the total variance. For CVa 0.5 CVW-S this translates to CVA

2 20% of CVtot2;

CVtot2 = [CVA

2 + CVW-S2] = [0.25 CVW-S

2 + CVW-S2] = 1.25 CVW-S

2.



27.350LH, mIU/ml

374PSA (t), ng/ml

36.8100CA 125, IU/ml

25.130FSH, mIU/ml

34.730CEA, ng/ml

34.415Testo, nmol/l

24.42TT3, nmol/l

3410fT3, pmol/l

33.815fT4, pmol/l

33.5400PRL, mIU/l

23100TT4, nmol/l

CVa (tot)


27.350LH, mIU/ml

374PSA (t), ng/ml

36.8100CA 125, IU/ml

25.130FSH, mIU/ml

34.730CEA, ng/ml

34.415Testo, nmol/l

24.42TT3, nmol/l

3410fT3, pmol/l

33.815fT4, pmol/l

33.5400PRL, mIU/l

23100TT4, nmol/l

CVa (tot)


6---7Folate, ng/ml

4---500B12, pg/ml

3---200ßhCG, mIU/ml

2---100AFP, IU/ml

3---40Prog, nmol/l

312.3100CA 19-9, IU/ml

411.3500E2, pmol/l

39.95TSH, µIU/ml#

37.5150Ferritin, ng/ml

CVa (tot)


6---7Folate, ng/ml

4---500B12, pg/ml

3---200ßhCG, mIU/ml

2---100AFP, IU/ml

3---40Prog, nmol/l

312.3100CA 19-9, IU/ml

411.3500E2, pmol/l

39.95TSH, µIU/ml#

37.5150Ferritin, ng/ml

CVa (tot)


Contribution of analytical variance to total

CVA2 in % of [CVA

2 + CVW-S2]; Basis: Best CVA and CVW-S

CVA2 in % of [CVA

2 + CVG2]; Basis: Best CVA and CVG


1st summary and remarks• Bias (SE), and consequently TE, cannot be assessed reliably

The optimistic point-of-viewFor imprecision: the best state-of-the-art seems to be close to the currently used biological goals: Exceptions: Na, Cl, Ca & Mg

BUT: GOALS are for the laboratory! Consequences for the manufacturer?

What if bias, interferences, etc. are considered?


0%10%

20%30%

40%50%60%

70%80%

90%100%

So

diu

mC

hlo

rid

eC

alciu

mM

ag

ne

siu

mP

RL

fT4

fT3

Pro

tein

Te

sto

ste

ron

eA

cid

Ph

os

ph

.T

T4

CEA

Glu

co

se

Ch

ole

ste

rol

Alb

um

inL

DH

Cre

ati

nin

eT

T3

CA

125

PS

A (

t)P

ota

ss

ium

Fe

rrit

inFS

HEs

trad

iol

Alk

. P

ho

sp

h.

Uri

c a

cid

TS

HU

rea

LH

Ph

os

ph

ate

CA

19-9

a-A

myl

as

eA

ST

CK

Bilir

ub

inA

LT

Iro

n, to

tal

Tri

gly

ce

rid

es

"Go

al"

CVA = 0.5 CVW

0%10%

20%30%

40%50%60%

70%80%

90%100%

So

diu

mC

hlo

rid

eC

alciu

mM

ag

ne

siu

mP

RL

fT4

fT3

Pro

tein

Te

sto

ste

ron

eA

cid

Ph

os

ph

.T

T4

CEA

Glu

co

se

Ch

ole

ste

rol

Alb

um

inL

DH

Cre

ati

nin

eT

T3

CA

125

PS

A (

t)P

ota

ss

ium

Fe

rrit

inFS

HEs

trad

iol

Alk

. P

ho

sp

h.

Uri

c a

cid

TS

HU

rea

LH

Ph

os

ph

ate

CA

19-9

a-A

myl

as

eA

ST

CK

Bilir

ub

inA

LT

Iro

n, to

tal

Tri

gly

ce

rid

es

"Go

al"

CVA = 0.5 CVW

0%10%

20%30%

40%50%

60%70%

80%90%

100%

So

diu

mC

hlo

rid

eC

alci

um

Mag

ne

siu

mA

cid

Ph

os

ph

.fT

4P

rote

inG

luco

se

Alb

um

inFe

rrit

inT

T4

Po

tas

siu

mT

es

tos

tero

ne

Estr

adio

lT

T3

LD

HC

ho

les

tero

lT

SH

Ph

os

ph

ate

Ure

aC

reat

inin

eU

ric

acid LH

CA

125

FSH

CEA

PR

LA

lk. P

ho

sp

h.

PS

A (

t)A

ST

Bili

rub

inIr

on

, to

tal

a-A

myl

ase

CA

19-

9C

KA

LT

Tri

gly

ceri

de

s"G

oal

"

CVA = 0.52 CVG

0%10%

20%30%

40%50%

60%70%

80%90%

100%

So

diu

mC

hlo

rid

eC

alci

um

Mag

ne

siu

mA

cid

Ph

os

ph

.fT

4P

rote

inG

luco

se

Alb

um

inFe

rrit

inT

T4

Po

tas

siu

mT

es

tos

tero

ne

Estr

adio

lT

T3

LD

HC

ho

les

tero

lT

SH

Ph

os

ph

ate

Ure

aC

reat

inin

eU

ric

acid LH

CA

125

FSH

CEA

PR

LA

lk. P

ho

sp

h.

PS

A (

t)A

ST

Bili

rub

inIr

on

, to

tal

a-A

myl

ase

CA

19-

9C

KA

LT

Tri

gly

ceri

de

s"G

oal

"

CVA = 0.52 CVG



GOALS are for the laboratory: TE = SE + 1.65 x RE!

Translation requirements for the manufacturer• Give room for the laboratory• Leave room for realistic IQC procedures As manufacturer,consume only a part of the goal! : 1/2 to 1/4? What about goals expressed as total error?How to fractionate (allocate) TE in SE/RE components?

To answer these questions, we will address-Finding the "manufacturer line"-TE-calculation with 1- and 2-sided random error-Distributions at different SE/RE ratios (position of an "operating point")-"Fractionation" of TE at various ratios of SE/REFor this purpose, we will make use of the Westgard total error charts.

Construction of TE-charts (Example TE = 10%)

TE charts are constructed by use of the formula TE = SE + k x RE. The SE component is plotted on the y-axis, the RE component on the x-axis.First, SE is calculated for RE = 0; SE = 10%Second, RE is calculated with SE = 0; RE = 10%/k; for k = 2 RE = 5%.Then, the 2 points are connected. All points on the line follow the respective equation 10% = SE + k x RE.

Laboratory line: k = 1.65!


0

2

4

6

8

10

0 2 4 6

CV (%)

Bia

s (%

)

k = 1.65

k = 2

k = 3

k = 4

k = 6


How big should “k” be?Give room for IQC

We look into the Westgard TE-charts used for IQC: "OPSpecs-charts"

From IQC OPSpecs: “k” should (at least) be 4(Note: nearly all IQC-lines are at the left of k = 4)

Finding the "manufacturer line"

OPSpecs Chart TEa 10% with 90% AQA (SE)

0

2

4

6

8

10

12

0 1 2 3 4 5 6 7

Allowable CV (%)

Allo

wa

ble

bia

s (

%) k = 1.65

IQC multirule

IQC 1,2.5s(n=4)& k = 4

IQC 1,2.5s(n=2)

The laboratory line

The manufacturer line



TE-lines: the 1-sided/2-sided problem

Populations at different operating points (TE = 10%)

Move operating point along the 1.65-line (1-sided 95% TE limit)Populations with high SE: 1-sided, 5% out of +10% TE-limitPopulations with low SE: 2-sided, 10% out of ±10% TE-limitThe 1-sided case (high SE) changes gradually to the 2-sided caseThe 2-sided case becomes relevant at SE RE

TE curves at fixed “out-of-limit-percentage”, principally, should respect 1- & 2-sided probabilities.

At SE RE (SE/RE 1) the probabilities become practically 1-sided

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

-5 0 5 10 15

RE = 0,5%

RE = 2%

RE = 6,1%

TE

1,65s

0

2

4

6

8

10

0 2 4 6 8

RE (%)

SE

(%

)

1,65 s

2 sSE = RE



Distributions at different SE/RE ratios

Chart with the TE-lines: TE = SE + k • RE (k = 1.65/2/3/4)Move along different SE/RE ratios and look at the populations at the intersection points of the SE/RE-lines with the TE-lines at different k-values.

0

2

4

6

8

10

12

0 2 4 6 8

1,65 s

2 s

3 s

4 s

SE/RE = 16

SE/RE = 2

SE/RE = 1

SE/RE = 0,5

SE/RE = 1/16

Moving along SE/RE = 1/16

0.000

0.050

0.100

0.150

0.200

0.250

-5 0 5 10 15

1,65 s

2 s

3 s

4 s

TE

SE/RE = 1/16 Robust situationAn excellent method(k = 4) allows additional SE

Moving along SE/RE = 16

0.000

0.200

0.400

0.600

0.800

-5 0 5 10 15

1,65 s

2 s

3 s

4 s

TE

SE/RE = 16 Labile situationAn excellent method (k = 4) does not allow much more SE



Observations from the Westgard TE-chart

Position of an operating point

Operating points at big SE/RE ratios describe “labile” analytical procedures-The difference between excellent and poor method (Westgard

terminology) is marginalOperating points at small SE/RE ratios describe “robust” analytical procedures.

Method decision chartThe distinction between excellent to poor methods is meaningless at high SE/RE ratios.

"Fractionation" of TE at various ratios of SE/RECalculation of TE along SE/RE = 16 & 1/16 at different k-valuesTE = SE + 1.65 • RE (SE/RE = 16)TE = SE + 2 • RE (SE/RE = 1/16)

Further observations from the Westgard TE-grid (TE = 10%)Along:The SE/RE = 16 line The SE/RE = 1/16 linek TE k TE1.65 10 2 103 9.3 3 6.74 8.8 4 5.1 Fractionation of TE occurs only at “medium to low” SE/RE ratios

0

1

2

3

4

5

6

7

8

0 1 2 3 4

CV (%)

Bia

s (%

)

Oper.point

Excellent

Poor

Marginal

Good



TE goal 10% Direction [0;0] to [SE=RE]; move from k = 1.65 to k = 4

• k = 1.65: SE = 3.8%; RE = 3.8% at k = 4: SE = 2%; RE = 2% TE (k = 4) = 5.3%

Direction [0;0] to [SE=0;RE]; move from k = 2 to k = 4• k = 2: SE = 0%; RE = 5%

at k = 4: SE = 0%; RE = 2.5% TE (k = 4) = 5%

Movement in the Method Decision Chart at a SE/RE = 1 ratio from the 1.65s-line to the 4s-line

TE drops to 53% of originalMovement in the Method Decision Chart at SE = 0 from the 2s-line to the 4s-line

TE drops to 50% of original

The fraction of the manufacturerStart from the lab TE for monitoring (#) and use

TE = SE + 1.65 • RE

Move in the Method Decision Chart at a SE/RE = 1 ratio from the 1.65s-line to the 4s-line [for SE = 0 from the 2s-line to the 4s-line]

TE manufacturer 50% of TE lab

Take as bottom-line (Stöckl, Clin Chem):CV ~ 0.5%, SE ~ 1%,TE ~ 1.8% (k = 1.65)

Laboratory bottom-line (TE~2 • manufacturer):CV ~ 1%, SE ~ 1.5%, TE ~ 3.2%

(#) When the biological variation is unknown, useTE = 1/10 RI ( TE monitoring at RE = SE).

“Allocation and fractionation” of TE

Moving along SE/RE = 1

-5 0 5 10 15SE (%)

TE-goal1.65 s

4 s

2%

3.8%

Method Decision Chart

CV (%)

Bia

s (

%)

- k = 1.65 - k = 2 - k = 3 - k = 4

Direction of error fractio-

nation



Laboratory goals from biology – future vision

• Always consider SE & RE

• Choose a TE-value in relation to a certain SE/RE-ratio

• Use “bottom-line” goalsRE = 1%, SE = 1.5%, TE = 3.2%

• [Use maximum values]

RECALL“Sometimes too small, sometimes too broad”

Be practical:Tonks/Stöckl: Use upper-/lower limits

Tonks: 10% upper limit for TEStöckl: 1% (0.5%) lower limit for bias (imprecision)

Tonks DB. A study of accuracy and precision of clinical chemistry determinations in 170 canadian laboratories. Clin Chem 1963;9:217-33.

Stöckl D. Desirable Performance criteria for quantitative measurements in medical laboratories based on biological analyte variation - hindrances to reaching some and reasons to surpass some. Clin Chem 1993;39:913-4.

Current and future goals compared

LaboratoryCurrent: CVa 0.5 CVW-S ("Harris"; without bias)

Future: CVa 0.25 CVW-S ("Advanced laboratory goal"; with bias)

ManufacturerCurrent: CVa 0.5 CVW-S ("Harris"; without bias)

Future: 50% of "Advanced laboratory goal"CVa 0.125 CVW-S ("Advanced manufacturer goal")

Future = ¼ of current!



CVA – Manufacturer "advanced"/"State-of-the-art"

Contribution of analytical uncertainty to total (compare with p. 67!)UA

2 in % of [UA2 + CVW-S

2]

Laboratory UA (incl. Bias, etc) = 4 x best CVA and CVW-S

UA2 in % of [UA

2 + CVG2]

Laboratory UA (incl. Bias, etc) = 4 x best CVA and CVG

2nd summary and remarks• Assumption: bias, interferences, etc• Assumption: the uncertainty of the laboratory is 4 x the best manufacturer performance

The pessimistic point-of-viewThere are very many problem analytes according to the biological concept, in particular, in the monitoring situation!


0%10%

20%30%

40%50%60%

70%80%

90%100%

So

diu

mC

hlo

rid

eC

alci

um

Mag

ne

siu

mP

RL

fT4

fT3

Pro

tein

Te

sto

ste

ron

eA

cid

Ph

os

ph

.T

T4

CEA

Glu

cos

eC

ho

les

tero

lA

lbu

min

LD

HC

reati

nin

eT

T3

CA

125

PS

A (

t)P

ota

ss

ium

Ferr

itin

FSH

Estr

adio

lA

lk. P

ho

sp

h.

Uri

c ac

idT

SH

Ure

aL

HP

ho

sp

hate

CA

19-

9a-

Am

ylas

eA

ST

CK

Bili

rub

inA

LT

Iro

n, t

ota

lT

rig

lyce

rid

es

"Go

al"

CVA = 0.5 CVW

0%10%

20%30%

40%50%

60%70%

80%90%

100%

So

diu

mC

hlo

rid

eC

alci

um

Mag

ne

siu

mA

cid

Ph

os

ph

.fT

4P

rote

inG

luco

se

Alb

um

inFe

rrit

inT

T4

Po

tas

siu

mT

es

tos

tero

ne

Estr

adio

lT

T3

LD

HC

ho

les

tero

lT

SH

Ph

os

ph

ate

Ure

aC

reat

inin

eU

ric

acid LH

CA

125

FSH

CEA

PR

LA

lk. P

ho

sp

h.

PS

A (

t)A

ST

Bili

rub

inIr

on

, to

tal

a-A

myl

ase

CA

19-

9C

KA

LT

Tri

gly

ceri

de

s"G

oal

"

CVA = 0.52 CVG

0%10%

20%30%

40%50%

60%70%

80%90%

100%

So

diu

mC

hlo

rid

eC

alci

um

Mag

ne

siu

mA

cid

Ph

os

ph

.fT

4P

rote

inG

luco

se

Alb

um

inFe

rrit

inT

T4

Po

tas

siu

mT

es

tos

tero

ne

Estr

adio

lT

T3

LD

HC

ho

les

tero

lT

SH

Ph

os

ph

ate

Ure

aC

reat

inin

eU

ric

acid LH

CA

125

FSH

CEA

PR

LA

lk. P

ho

sp

h.

PS

A (

t)A

ST

Bili

rub

inIr

on

, to

tal

a-A

myl

ase

CA

19-

9C

KA

LT

Tri

gly

ceri

de

s"G

oal

"

CVA = 0.52 CVG


Analytical quality – still an issue?

We believe YESQuality compared to goalsStöckl D. Modern quality management misunderstood? Clin Chem 1998;44:1066-7.

Others believe NOQuality compared to own process specificationsQuality evaluated against “harm” but not against usefulness-Blumenthal D. The errors of our ways. Clin Chem 1997;43:1305.-Plebani M, Carraro P. Mistakes in a stat laboratory. Clin Chem 1997;43:1348-51.-Witte DL et al. Errors, mistakes, blunders, and outliers, or unacceptable results: how many? Clin Chem 1997;43:1352-6.

Analytical quality – The future?• Market transparency – independent of goals

-Why should I pay the same for different quality(compare with reference methods)

• Quality assessed against usefulness (“does not harm” is not sufficient anymore)-Evidence-based

• New analytes• Reconsider old/develop new diagnostic strategies (#)

(#) Hammond HC. Applying the value-of-information paradigm to laboratory management. Clin Lab Manage Rev 1996;10:98-106. (measuring, or not, glucose for the early identification of diabetes)

Priority list for improvement

Imprecision-medium priority (between day)

Bias/traceability-high priority for some analytes (be careful during “restandardisation”)

[Diagnostic]accuracy on individual sample-highest priority (see hCG example, Lancet 2000;355:712)

Outlook


Analytical quality – the future?

Remember, the influence on TE of:• Bias (traceability)/SE• Sample related effects (overall matrix)• Specificity (cross-reactions)• Common interferences (lipemia, etc.)• Effects of drugs• Auto-/heterophilic antibodies• Genetic variantsInsufficiently known

Important for the future:quality for the individual sample (patient!)

Remarks• Know the different concepts for deriving analytical quality specifications (= goals), but be able to use them in the appropriate manner• Make a difference between “desirable” and required quality• Take critically part in establishing (inter)national specifications• Stay informed about “state-of-the-art” quality

Analytical goals – Which ones to use?

Document your own quality

Compare it with the “state-of-the-art”• Identify therefrom “problem analytes”

Compare it with international goals• Identify therefrom “problem analytes” or “problem goals”

Identify “problem analytes” by communication with the laboratories/clinicians

Identify “possible” problem analytes by comparison with “biological“ specifications: biological variation, ref.-intervals

Verify “possible” problem analytes by comparison with “clinical“ specifications and “bottom-line” goals

Check whether you want to improve the quality of some tests, independent of proposed goals

Outlook


Data on biological variation

• Ross JW. Evaluation of precision. In: Werner M, ed. CRC Handbook of Clinical Chemistry. Boca Raton, Fla: CRC Press: 1982;1:391-422.• Fraser CG. The application of theoretical goals based on biological variation data in proficiency testing. Arch Pathol Lab Med 1988;112:404-15.• Fraser CG. Biological variation in clinical chemistry. An update: collated data, 1988-1991. Arch Pathol Lab Med 1992;116:916-23.• Fraser CG. Databases for facilitating work on setting quality specifications: biological variation. Upsala J Med Sci 1993:98:415-6. • Sebastian-Gambaro MA, Liron-Hernandez FJ, Fuentes-Arderiu X. Intra- and inter-individual biological variability data bank. Eur J Clin Chem Clin Biochem 1997;35:845-52.

Selected referencesAnalytical goals• Stöckl D, Baadenhuijsen H, Fraser CG, Libeer J-C, Hyltoft Petersen P, Ricós C. Desirable routine analytical goals for quantities assayed in serum (Discussion paper from the members of the EQA Working Group A on analytical goals in laboratory medicine). Eur J Clin Chem Clin Biochem 1995;33:157-69.• Fraser CG, Hyltoft Petersen P, Ricos C, Haeckel R. Proposed quality specifications for the imprecision and inaccuracy of analytical systems for clinical chemistry. Eur J Clin Chem Clin Biochem 1992;30:311-7.• Fraser CG, Hyltoft Petersen P, Ricos C, Haeckel R. Quality specifications. In: Haeckel R, editor. Evaluation methods in laboratory medicine. Weinheim: VCH Verlagsgesellschaft, 1993:87-99.• Ross JW. Evaluation of precision. In: Werner M, ed. CRC Handbook of Clinical Chemistry. Boca Raton, Fla: CRC Press: 1982;1:391-422.• Fraser CG. The application of theoretical goals based on biological variation data in proficiency testing. Arch Pathol Lab Med 1988;112:404-15.• Fraser CG. Biological variation in clinical chemistry. An update: collated data, 1988-1991. Arch Pathol Lab Med 1992;116:916-23.• Fraser CG. Databases for facilitating work on setting quality specifi-cations: biological variation. Upsala J Med Sci 1993:98:415-6. • Sebastian-Gambaro MA, Liron-Hernandez FJ, Fuentes-Arderiu X. Intra- and inter-individual biological variability data bank. Eur J Clin Chem Clin Biochem 1997;35:845-52.• Tonks DB. A study of accuracy and precision of clinical chemistry determinations in 170 canadian laboratories. Clin Chem 1963;9:217-33.• Cotlove E, Harris EK, Williams GZ. Components of variation in long term studies of serum constituents in normal subjects. III. Physiological and medical implications. Clin Chem 1970;16:1028-32.• Harris EK. Statistical principles underlying analytical goal-setting in clinical chemistry. Am J Clin Pathol 1979;72:374-82.• Elevitch FR, editor: Analytical goals in clinical chemistry. Proceedings of the CAP Aspen Conference; 1976; Skokie (IL). Skokie (IL): College of American Pathologists, 1977.• Proceedings of the subcommittee on analytical goals in clinical chemistry, World Association of Societies of Pathology, London, Analytical goals in clinical chemistry: Their relationship to medical care. Am J Clin Pathol 1979;71:624-30.• Harris EK. Proposed goals for analytical precision and accuracy in single point testing. Arch Path Lab Med 1988;112:416-20.

References


Selected references

• Ross JW. A theoretical basis for clinically relevant proficiency testing evaluation limits: Sensitivity analysis of the effect of inherent test variability on acceptable method error. Arch Path Lab Med 1988;112:421-34.• Ross JW, Fraser MD. Analytical goals developed from inherent error of medical tests. Clin Chem 1993;39:1481-94.• Gowans EMS, Hyltoft Petersen P, Blaabjerg O, Horder M. Analytical goals for the acceptance of common reference intervals for laboratories throughout a geographical area. Scand J Clin Lab Invest 1988;48:757-64.• Klee GG. Tolerance limits for short-term analytical bias and analytical imprecision derived from clinical assay specificity. Clin Chem 1993;39:1514-8.• Harris EK, Yasaka T. On the calculation of a "reference change" for comparing two consecutive measurements. Clin Chem 1983;29:25-30.• Hyltoft Petersen P, Fraser CG, Westgard JO, Lytken Larsen M. Analytical goal-setting for monitoring patients when two analytical methods are used. Clin Chem 1992;38:2256-60.• Queralto JM, Boyd JC, Harris EK. On the calculation of reference change values, with examples from a long-term study. Clin Chem 1993;39:1398-1403.

Goals from "experts"• Richtlinien der Bundesärztekammer zur Qualitätssicherung in medizinischen Laboratorien. Dtsch Ärztebl 1988;85:A699-A712.• Department of Health and Human Services. Medicare, Medicaid and CLIA programs; regulations implementing the Clinical Laboratory Improvement Amendments of 1988 (CLIA). Fed Regist, Feb 28, 1992;57:7002-186.• Belgisch Staatsblad - Moniteur Belge. 01.04.1996. N. 96-716.• Ricós C, Baadenhuijsen H, Libeer J-C, Hyltoft Petersen P, Stöckl D, Thienpont LM, Fraser CG. External quality assessment: currently used criteria for evaluating performance in European countries, and criteria for future harmonization. Eur J Clin Chem Clin Biochem 1996;34:159-65.

Reference intervals• Harris EK, Boyd JC. Statistical bases of reference values in laboratory medicine. Washington: AACC Press, 1995:384pp.• Burtis CA, Ashwood ER, eds. Tietz Textbook of Clinical Chemistry. 2nd ed. Philadelphia (PA): W.B. Saunders Co, 1994.• Heil W, Schuckließ F, Zawta B. Reference ranges for adults and children. Klinikum Wuppertal-Barmen, Universität Witten-Herdecke und Boehringer Mannheim GmbH: 1995.• Soldin SJ, Hicks JM, Gunter KC, Brugnara C (editors). Pediatric reference ranges. 2nd edition. Washington: AACC Press, 1997:181pp. • Perkins SL, Livesey JF, Belcher J. Reference Intervals for 21 clinical chemistry analytes in arterial and venous umbilical cord blood. Clin Chem 1993;39:1041-4.• Tietz NW, Shuey DF, Wekstein DR. Laboratory values in fit aging individuals - sexagenerians through centenarians (special report). Clin Chem 1992;38:1167-85.• Caroli S, Alimonti A, Coni E, Petrucci F, Senofonte O, Violante N. The assessment of reference values for elements in human biological tissues and fluids: a systematic review. Crit Rev Anal Chem 1994;24:363-98.

References


Notes

Notes


Notes

Notes

Date post:	15-Dec-2015
Category:	Documents
Upload:	daryl-bowser
View:	213 times
Download:	0 times

Statistics & graphics for the laboratory 35 Analytical Quality Specifications ("Goals") Introduction...

Documents