Biomarker Assay Validation ? Bringing Context of Use into practice · 2019-10-09 · Development of...

Designed by Scientists, Run by Scientists

European Bioanalysis Forum – Autumn Focus Workshop 18-19 September 2019

Biomarker Assay Validation ? Bringing Context of Use into practice

What can we learn from clinical labs?

John L Allinson FIBMSVice President – Biomarker Services

Immunologix Laboratories

IMMUNOLOGIX LABORATORIES

FOCUSED. EXPERIENCED. READY.FOCUSED. EXPERIENCED. READY.

• Scope

• Brief History of Biomarkers – discovery and technology

• Development of consensus for assay validation

• Assay performance & validation vs Changing COU (for same biomarker)

• Assay performance and validation vs Same COU (for different biomarkers)

• Application of true analytical Quality Control to BM BioA

What can we learn

from clinical labs?



• ~460 – 375BC “Biomarker” – idea arguably dates back to Hippocrates(noted relationships between various visible biological manifestations of disease)

• 1950s – term "biological marker" introduced

• 1980 - “Biomarker“ - widespread use

• 1998 - ?advent of biomarker-guided drug development:

- FDA approve trastuzumab (HER2 +ve metastatic breast cancer)

A Brief History of Biomarker



DATE DISEASE “SCIENTIST” TECHNIQUE

4000 BC Earliest recordingEarly pregnancy test

Sumerians / Eqyptians “Urinalysis”

1550 BC Diabetes Hesy-Ra -Physician

Urine – Ant attraction & frequency

2nd century AD Haemophilia ‘A’ Unknown - Talmud BM ID & Link to genetics

Middle Ages Various Unknown Uroscopy wheel



DATE DISEASE “SCIENTIST” TECHNIQUE

1845 Multiple Myeloma Henry Bence-Jones Urinalysis –

1st “Tumour Marker”1848 Multiple

Myeloma Henry Bence-Jones Quasi-quantitative

method published

1879 Lord Rayleigh (back to bare principles)Basis of Flow Cytometry

1906 SyphilisWassermann,

Julius Citron, &Albert Neisser

Complement Fixation Test

1934 Moldavan Cell counting in Flow

55°C 100°C



DATE DISEASE “SCIENTIST” TECHNIQUE / BM1940 &

1948Rheumatoid

Factor Dr Erik Waaler &

Dr H.M. Rose Haemagglutination

assays (Rose-Waaler)

1950’s Thyroid Disease Radio-isotopic Assays(not RIA) – “PBI”

1950’s Pernicious Anaemia

Microbiological Assays Vitamin B12 & Folate



DATE “SCIENTIST” TECHNIQUE / BM

1855 First Counting Chamber

1904 Folin* Dev. of quantitative methods in blood & urine1919 Dempster Mass Spec (first electron impact source)

1921 van Slyke Van Slyke manometer (TCO2 in blood)

1930-50’s e.g. Coulter Early colorimeters & Hb & Cell counters1957 Skeggs Fully Automated Analysis1979 Abbott First Automated Immunoassay



FOLIN* (1904 to 1922….)

developed quantitative analytical methods for several urine analytes:

• urea, ammonia, creatinine, uric acid, total nitrogen, phosphorus, chloride, total sulfate, acidity.

• also attempted to measure blood ammonia

• introduced Jaffe's alkaline picrate method for creatinine.

• showed the effect of uricosuric drugs on blood and uric acid levels in gout

• introduced the colorimetric method for measuring epinephrine

• is also responsible for establishing the relationship of uric acid, NPN, and blood urea nitrogen to renal

function.

• The Folin Cicalteau reagent among others developed by Folin, is still used today for protein

determinations.

Otto Folin1867 - 1934



• Scope






What can we learn

from clinical labs?



Development of “Clinical Laboratory” test methods for over a CENTURY

• 1912-3 - UK Professional body started

• 1933-6 - American Society of Clinical Laboratory Science ASCLS commenced

International Consensus to quantitative method validation criteria

• 1967 - National Committee for Clinical Laboratory Standards (NCCLS)

• 1977 - accredited by the American National Standards Institute (ANSI) as

a voluntary consensus standards organization

• 2005 – changed name to Clinical Laboratory Standards Institute (CLSI)



CLSI : Evaluation of Quantitative Measurement Procedures,Laboratory Error Sources and CLSI Evaluation Protocol Documents

Overview EP19-R

Clinical Utility

DiagnosticAccuracyGP10

Risk Management

Pre- and Post-Analytical ErrorEP18

MeasurementAccuracy

Total ErrorEP21

QualitativeMeasurementEP12

Precision Bias

Detection LimitsEP17

Precision and ComponentsEP5EP9EP15EP10

Prop. and Constant BiasEP9EP15EP10

Drift and CarryoverEP10

LinearityEP6EP10

InterferencesEP7EP14

© CLSI – Used with permission, CLSI, EP19-R A framework for CLSI Evaluation Protocols, www.clsi.org

http://www.clsi.org/



Diagnostics - CLSI – 9 selected guidelines • Document EP5A2: Evaluation of Precision Performance of Clinical Chemistry Devices : Approved Guidelines 2nd

edition. 2011

• Document EP6A: Evaluation of Linearity of Quantitative Measurement Procedures: A Statistical Approach: Approved Guideline 2011

• Document EP07A2: Interference Testing in Clinical Chemistry: Approved Guideline:2nd Edition.

• Document EP9A2IR: Method Comparison and Bias Estimation Using Patient Samples: Approved Guideline 2nd

edition: Interim Revision 2011

• Document EP10A3 : Preliminary Evaluation of Quantitative Clinical Laboratory Measurement Procedures : Approved Guidelines: 3rd edition 2011

• Document EP24 : Evaluation of Matrix Effects: Approved Guideline: 3rd edition 2014

• Document EP15A2: User Demonstration of Performance for Precision and Trueness: Approved Guideline 2nd

edition 2011

• Document EP17A: Protocols for Determination of Limits of Detection and Limits of Quantification : Approved Guideline 2011

• Document C28-A3c: Defining, Establishing and Verifying Reference Intervals in the Clinical Laboratory : Approved Guidelines: 3rd edition 2011

Continual review……

Guidance & white paper documents:

• FDA - 2001 …………………….25 pages• FDA – 2018…………………….35 pages• EMEA – 2011…………….……22 pages

• Lee et al – 2006 ………………7 pages• GCC – 2012………………........8 pages• EBF – 2012……………………..12 pages

• C-Path 2019……………………79 pages• CLSI (formerly NCCLS)…..613 pages



• Scope






What can we learn

from clinical labs?



Context of Use

• PK assays, COU is always the same : fixed validation and acceptance criteria makes sense

• Biomarker assays, COU often varies : variable criteria required

…….EVEN if it’s the SAME biomarker being measured

An example from the “Clinical Laboratory”

The Evolution of Thyroid Stimulating Hormone Assays:…..from a diagnostic to identify hypothyroidism to

Truly Personalised Healthcare/Therapy (achieved late 1980’s)





DATE DISEASE TECHNIQUE / BM

1950’s Thyroid Disease Radio-isotopic Assays (not RIA) – “PBI”

PBI (Protein-Bound Iodine) was really a “proxy” for Total Thyroid Hormones –Thyroxine (Tetra-iodothyronine (T4)) + Tri-iodothyronine (T3)

Assays for T4 (mainly) & T3 were used next and TSH came along later….(Negative Feedback)

High T4 = Hyperthyoid….but could not differentiate sufficiently between NORMALS and HYPOthyroidism

Due to NEG feedback on pituitary thought investigating TSH may be more sensitive due to physiological response (TSH)



DATE TECHNIQUE / BM COU Results (simplified) Critical/Additional Validation

1950’s Radio-isotopic Assays –“PBI” (T4 + T3)

?Hypo / Hyperthyroidism

High = HyperLow = Hypo -

1960’sTSH – 1st Generation CPB pAb & Radio-Label

sens = 1 – 2 mIU/L

Replace T4/T3 as first-line test for Hypo (develop data for hyper)

NR = 0.4 – 5.0 mIU/LHypo = >5.0 – give T4

Add. tests if >4.0

IACV@ 1 & 2 (≤20%)Add. QC @ 5 & “HIGH”

PARA & LOD, STAB

1970’sTSH – 2nd GenerationSandwich RIA & EIASens = 0.1-0.2 mIU/L

Differentiate normal & 1˚ hyperthyroid

(continue to dev. data)

<0.1 = 1˚ hyperthyroid>5.0 = hypo T4 doseAdd. tests if 0.1-0.2 or

>4.0

IACV@ 0.1 & 0.2 (≤20%)Add. QC @ 5 & “HIGH”

PARA & LOD, STAB

1980’s -90’s

TSH – 3rd GenerationmAbs EIA, FIA & CIA

Sens = 0.01-0.02 mIU/L

Monitor T4 repl. RxGuide ¯ dose

Monitor for 2˚ Hypopituitarism

<0.1 = 1˚ hyperthyroid>5.0 = hypo T4 dose

~0.02 – 0.1 = ¯ T4 dose~or <0.01 = ¯¯ T4 dose

IACV@ 0.01 & 0.02 (≤20%)Add QC @ 0.1, 5 & “HIGH”

PARA & LOD, STAB

1960 – 1990 = SAME BIOMARKER (TSH), EVOLVING COU’S



From 2˚ line test (after T4)

® first line test (only test for many) - (TSH 1st Gen)

®Dose escalation indices - (TSH 2nd Gen)

®Dose reduction indices then finally titre-ing dose to prevent hypopituitarism - (TSH 3rd Gen)

In this example the validation focus was maintaining precision at reducing concentrations (10x & 100x less)

& at critical decision concentrations

Continuing to gather datasets from multiple subjects until COU proven

Robust Analytical Quality Control at key concentrations



• Scope






What can we learn

from clinical labs?



Assay

performance and

validation vs

Same COU (for different biomarkers)

FACTORS IMPACTING UPON ASSAY PERFORMANCE REQUIREMENTS & ACCEPTANCE:

• PHYSIOLOGICAL VARIABILITY

COU = IS THE RESULT “NORMAL”? (DIAGNOSTIC):

• “NORMAL” OR “REFERENCE RANGE”

COU = IS THERE SIGNIFICANT CHANGE DURING CLINICAL STUDY

• SMALLEST CLINICALLY SIGNIFICANT CHANGE



IMPACT OF PHYSIOLOGICAL VARIABILITY ON ASSAY PERFORMANCE REQUIREMENTS (SAME COU)

Analyte

CVI CVg I(%) TE(%)pH (pH units) 0.2 --- 0.1 --- 7.38 - 7.44Sodium 0.6 0.7 0.3 0.7 135 - 146 mmol/LCalcium 2.1 2.5 1.1 2.6 2.2 - 2.6 mmol/L 8.6 - 10.3 mg/dLProtein 2.8 4.7 1.4 3.6 60 - 78 g/L 6.0 - 7.8 g/dLAlbumin 3.2 4.8 1.6 4.1 36 - 51 g/L 3.6 - 5.1 g/dLHemoglobin A1 C 1.9 5.7 0.9 3.0 <5.7 % HbMagnesium 3.6 6.4 1.8 4.8 0.62 - 0.99 mmol/L 1.5 - 2.4 mg/dLGlucose 5.6 7.5 2.8 7.0 3.9 - 5.6 mmol/L 70 - 100 mg/dLCreatinine 6.0 14.7 3.0 8.9 62 - 115 umol/L 0.70 - 1.25 mg/dLAlkaline phosphatase 6.5 26.1 3.2 12.0 40 - 115 IU/LPhosphate 8.2 10.8 4.1 10.1 0.97 - 1.45 mmol/L 3.0 - 4.5 mg/dLUrea 12.1 18.7 6.1 15.6 2.9 - 7.1 mmol/L 7 - 25 mg/dLAspartate aminotransferase (AST) 12.3 23.1 6.2 16.7 0 - 35 IU/LBilirubin total 21.8 28.4 10.9 26.9 3.4 - 20.5 umol/L 0.2 - 1.2 mg/dLCreatine kinase (CK) 22.8 40.0 11.4 30.3 30 - 170 IU/L

Physiological Variability

Assay Performance

Requirements"Normal Range"

S.I. USA Mass Units

AnalyteLimits

based on TE%

CVI CVg I(%) TE(%) (+/-%) +/- units +/- unitspH (pH units) 0.2 --- 0.1 --- 7.38 - 7.44 0.3 0.02 0.02Sodium 0.6 0.7 0.3 0.7 135 - 146 mmol/L 0.9 2 1Calcium 2.1 2.5 1.1 2.6 2.2 - 2.6 mmol/L 8.6 - 10.3 mg/dL 3.2 0.1 0.1Protein 2.8 4.7 1.4 3.6 60 - 78 g/L 6.0 - 7.8 g/dL 4.1 3 3Albumin 3.2 4.8 1.6 4.1 36 - 51 g/L 3.6 - 5.1 g/dL 4.8 2 2Hemoglobin A1 C 1.9 5.7 0.9 3.0 <5.7 % Hb 2.7 0.2 0.2Magnesium 3.6 6.4 1.8 4.8 0.62 - 0.99 mmol/L 1.5 - 2.4 mg/dL 5.4 0.04 0.04Glucose 5.6 7.5 2.8 7.0 3.9 - 5.6 mmol/L 70 - 100 mg/dL 8.4 0.4 0.3Creatinine 6.0 14.7 3.0 8.9 62 - 115 umol/L 0.70 - 1.25 mg/dL 8.9 8 8Alkaline phosphatase 6.5 26.1 3.2 12.0 40 - 115 IU/L 9.7 8 9Phosphate 8.2 10.8 4.1 10.1 0.97 - 1.45 mmol/L 3.0 - 4.5 mg/dL 12.2 0.1 0.1Urea 12.1 18.7 6.1 15.6 2.9 - 7.1 mmol/L 7 - 25 mg/dL 18.2 0.9 0.8Aspartate aminotransferase (AST) 12.3 23.1 6.2 16.7 0 - 35 IU/L 18.5 3.2 2.9Bilirubin total 21.8 28.4 10.9 26.9 3.4 - 20.5 umol/L 0.2 - 1.2 mg/dL 32.7 4 3Creatine kinase (CK) 22.8 40.0 11.4 30.3 30 - 170 IU/L 34.2 34 30

Physiological Variability

Assay Performance

Requirements"Normal Range"

S.I. USA Mass Units

3 x CV% (3SD) Limits for QC based on precision only



IMPACT OF SMALLEST CLINICALLY SIGNIFICANT CHANGE vs IS THE BM RESULT NORMAL? (SAME COU – ie “SAFETY BM”)

2.1 2.2 2.3 2.4 2.5 2.6 2.7

<2% change = not signif.

Flagged “HIGH” by N.R.

13.3% change – V.Sign

Flagged “NORMAL” by N.R.

Serum Calcium example

Smallest Clinically Significant Change is a

better way of evaluating on a

longitudinal basis during a clinical trial

than just using NORMAL RANGE.



• Scope






What can we learn

from clinical labs?



Application of true analytical Quality Control to BM BioA

• It doesn’t need to be complicated

• QC is about understanding how the method is

performing

• “Target” Acceptance criteria is to give warning of

performance deterioration

• Unlike PK – acceptance of sample results may be

accepted differently due to COU

Note: exact 95 & 99% CL = +/- 1.96 & 2.576 SD respectively



What can we learn

from clinical labs?

Application of true analytical Quality Control to BM BioA

QC results will inform you when sample results are “less reliable”

Even so, they may be good enough to deliver sufficiently robust data

For example – QC’s may be 30% biased vs 20% target acceptance

May be OK if results meet COU which may be multiple fold change or much larger % change

Obvious need to understand COU & Physiology vs what the statistics of the QC is informing you about the analytical results



3 of the Westgard QC Multirules can inform greatly on Analysis in BioA

Use QC Charts – They are a great way to visualize problems

Calculating and plotting results in terms of RE in SD’s allows easy comparison of ALL QC’s



Designed by Scientists, Run by Scientists

Thank you for your attention!

Date post:	19-May-2020
Category:	Documents
Upload:	others
View:	3 times
Download:	0 times