ISSUES WITH ASSAYS:CUT POINTS, CONFIRMATIONS, ANDFALSE ASSAY SIGNALS

Amy LoercherGlaxoSmithKline

CONFIRMATION ASSAYS

Majority of assays use excess of drug to confirm

Selection of spiked drug concentration is critical Need to have some idea of assay LOD and LLOQ

Typically aim to have spike concentration just at or below LLOQ

Calculations of false negative and false positive should give idea of appropriateness

IF THE SPIKE IS TOO HIGH… May result in a large number of false negatives

Confirmation cut point will be elevated compared to clinical samples Typically concern if higher than 70% Can effect validation parameters, but may not be

identified in validation

IF THE SPIKE IS TOO LOW… May result in a large number of false positives

Confirmation cut point will be decreased compared to clinical samples Matrix used to determine cut point should also reflect

clinical samples Care should be taken when purchasing from

commercial vendors (Case Study #2)

APPROACHES TO CUT POINTS

Lots of publications with standard formula for calculating fixed confirmation cut points

May be cases where need to use alternate approach Data dependent May be determined after pre-dose clinical samples

have been collected

CONFIRMATION ASSAY CUT POINTCALCULATIONS

“Calculate a fixed specificity cut point: Applying the mean and SD from step 2, the specificity cut point is now defined as mean + 3.09 × SD when log transformation is not applied. If the data were log transformed, compute the mean and SD of the logarithm of the ratio of drug spiked versus unspiked samples as suggested in step 2; compute mean − 3.09 × SD of these log ratios, and then take the anti-log to obtain the specificity cut point via the formula: 100 × (1 − antilog value). (Note that the value 3.09 corresponds to the 99.9th percentile of the normal distribution. Since this is a confirmation assay, only the samples that are truly positive and specific to the study drug should be reported, limiting the false-positive error rate to around 0.1%. Instead, if 1% false-positive rate is preferred, the threshold of 2.33 should be used in place of 3.09 in the above formula.)”

Shankar et al. (2008)

CONFIRMATION CUT POINTCALCULATIONS How many samples should be used in the calculation? The FDA guidance (2009) suggested 5–10 samples for assay

development and 50–100 samples for screening assay validation. While the FDA did recommend a confirmation assay a particular number of samples was not specified

The EMEA guidance (2010) did not provide any information on sample size requirements for either assay.

Shankar et al. (2008) suggested for screening assays: “Typically in clinical studies, matrix samples from more than 50 human subjects are analyzed. Due to practical considerations in nonclinical studies, at least 15 samples might be sufficient”

Schlain et al (2010) recommended that 30 samples would be a typical sample size for prestudy validation.

A t-test based approach proposed by Neyer et al.(2006), which compared signals of the native and drug-spiked sample, based results on a very limited number of observations (12 subjects) but it did not take into account the biological (inter-subject) variability.

NON-PARAMETRIC APPROACH

In cases where normal distribution cannot be determined

REGULATORY COMMENTS AROUNDCONFIRMATION ASSAYS

Question about the number of samples used confirmation assay GSK assay used results from 20 subjects, FDA was

looking for 50-100 subjects GSK assay used simulated data

Due to high variability seen in inhibition response, GSK statisticians determined we would need >300 subjects

Resulting cut point gave a ~10% false negative rate in 1 of 4 datasets

SIMULATION APPROACH TOCONFIRMATION CUT POINT A confirmation cut point that was the mean of percent

inhibition of the spiked samples was used to determine antibody specificity. This produced a confirmation cut point which required upwards of 70% inhibition in order to consider a sample positive in the initial assay.

Difficult to get good separation between the unspiked and the positive control-spiked samples, which resulted in a confirmation cut point that included a significant number of unspiked samples and a high false positive rate. This effect was primarily attributed to high variability

between the spiked samples which resulted in poor precision around the confirmation cut point that could not be resolved without the use of simulated data.

Increasing the number of samples tested would have improved this, but based on required sample size estimates (>300), this was not feasible and simulated data was used as an alternative.

Lucked out since ONLY samples tested on this were first tested in previous assay with <1% false negative

‘Percent’ represents the proportion of samples in the simulated whole population, and the response is the percent inhibition. To limit the false positive error rate, an upper 99% confidence limit on the SD for spiked and unspiked samples was calculated from the computer simulation and produced an original cutoff value of 36.07%, represented by the red dotted line. The lower 1st percentile (the final cutoff) of 30.34% is labeled “revised cutoff” and shown using a solid green line.

The lower 99% confidence limit of 36.07% inhibition is labeled “original cutoff” and shown using a dashed red line. The lower 1st percentile (the final cutoff) of 30.34% is labeled “revised cutoff” and shown using a solid green line

FALSE POSITIVE SIGNALS

May also be due to interferences in the samples Multivalent targets RF factor

Investigations should be initiated if positive signals are seen in pre-dose or placebo samples Helpful to define, if possible, what signal is If not, helpful to define the conditions under which it

occurs

CASE STUDY #1: SOLUBLE TARGET

Target is found as dimer in circulation Two binding sites Can bridge reagents like ADA Target accumulation after dosing

Two attempted approaches Removal of target from sample Blocking antibody against target

RESULTS FROM BLOCKING ANTIBODYAPPROACH

0ng/mL1 ng/mL

50ng/ml100 ng/mL

200 ng/mL500 ng/mL

0.000.050.100.150.200.250.30

Target Concentration

Ratio (RECL/[Ab])

Blocking Antibody

Mab Blocking of Target Detection

REGULATORY COMMENTS AROUND TARGETINTERFERENCE

Use of blocking antibody

How did we determine the concentration of blocking antibody? Wasn’t in original validation report Referenced R&D Runs

CASE STUDY #2: NC ISSUES

Assay was validated with serum purchased from commercial vendor and background was very high

Validation was performed on 100 serum samples 12% screened positive 8% confirmed positive 4% false positive rate

Clinical samples came in with much lower background (ECL signal)

Comparison of ECL Signals

Samples:  ECL1 ECL2 Mean ECL StDev %CVBRH645984 422 410 416 8.49 2.0

10% SeraBRH645986 481 475 478 4.24 0.9BRH645987 489 470 480 13.44 2.8BRH645995 416 433 425 12.02 2.8BRH646003 424 419 422 3.54 0.8

Samples:  ECL1 ECL2 Mean ECL StDev %CV RECLHQC 33462 34254 33858 560.03 1.7 601.92MQC 2960 2923 2942 26.16 0.9 52.29LQC 340 348 344 5.66 1.6 6.12NC 64 60 62 2.83 4.6 1.10S1 60 64 62 2.83 4.6 1.10S2 64 62 63 1.41 2.2 1.12S3 64 71 68 4.95 7.3 1.20S4 68 81 75 9.19 12.3 1.32S5 67 81 74 9.90 13.4 1.32S6 67 95 81 19.80 24.4 1.44S7 64 55 60 6.36 10.7 1.06S8 75 79 77 2.83 3.7 1.37

Clinical Samples

Validation Samples

FALSE NEGATIVE SIGNALS

Could also be due to interferences in the sample Need to look at assay format Possibilities for complexes in sample—soluble

receptors, binding proteins

More important to identify false negatives than false positives

SAMPLE COLLECTION TIMINGS

Will be important to put data in context Typically decrease IMGEN sample collections as

asset progresses

IgM Study #1 had a lot of Day 8 positives, probably IgM OLE studies had a lot of single week 4 positives, also

probably IgM Both examples showed an increased incidence of

immunogenicity compared to other clinical studies with same asset

TITERS

ISI: Reported clinical correlates for all subjects with titers above 320 FDA 2009 guidance states ADA at ~500ng/mL most

likely to have clinical correlates Comes out to about 300-400 titer for therapeutic

monoclonals

Request to provide clinical correlates for all subjects with titer greater than 32 New sensitivity requirements?

DATA PRESENTATION

Asset that was developed in 1997 Long history of clinical use

Stats team historically used post baseline positives Different than treatment-emergent antibodies Treatment of what defined a positive varied from

study to study Difficult to integrate all immunogenicity data

Encourage you to pick a standardized approach!

THANK YOU FOR YOUR ATTENTION

Questions? Comments?