A Bayesian Approach toParallelism Testing in BioassaySteven Novick, GlaxoSmithKlineHarry Yang, MedImmune LLC

Manuscript co-author

John Peterson, Director of statistics, GlaxoSmithKline

Warm up exercise

When are two lines parallel?Parallel: Being everywhere equidistant and not intersecting

Slope

Horizontal shift places one line on top of the other.

When are two curves parallel?Parallel: Being everywhere equidistant and not intersecting?

Can you tell by checking model parameters?

Horizontal shift places one curve on top of the other.

Where is parallelism important?Gottschalk and Dunn (2005)Determine if biological response(s) to two substances are similarDetermine if two different biological environments will give similar doseresponse curves to the same substance.

Compound screeningAssay development / optimizationBioassay standard curve

Screening for compound similar to gold-standardE.g., seeking new HIV compound with AZT-like efficacy, but different viral-mutation profile.

Desire for dose-response curves to be parallel.

Change to assay procedureE.g., change from fresh to frozen cells.

Want to provide same assay signal window.Desire for control curves to be parallel.

Assess validity of bioassayused for relative potency

Dilution must be parallel to original.Callahan and Sajjadi (2003)OriginalDilution

Replacing biological materials used in standard curveE.g., ELISA (enzyme-linked immunosorbent) assay to measure protein expression.Recombinant proteins used to make standard curve.Testing clinical sampleNew lot of recombinant proteins for standards. Check curves are parallelCalibrate new curve to match the old curve.

Potency often determined relative to a reference standard such as ratio of EC50Only meaningful if test sample behaves as a dilution or concentration of reference standard

Testing parallelism is required by revised USP Chapter and European Pharmacopeia

Linear: Two lots of Protein AEstimated ConcentrationsLot 2 is 1.4-fold higher than Lot 1=0.14Log10 Signal

If the lines are parallel

Shift Lot 2 line to the left by a calibration constant .

is log relative potency of Lot 2.Draft USP 2010

Testing for Parallelism in bioassay

Typical experimental designLog10 SignalSerial dilutions of each lot

Several replicates

Fit on single plate (no plate effect)

Tests for parallel curvesLinear model

Hauck et al, 2005; Gottschalk and Dunn, 2005

H0: b1 = b2H1: b1 b2May lack power with small sample sizeMight be too powerful for large sample sizeANOVA: T-test F goodness of fit test 2 goodness of fit test

A better ideaCallahan and Sajjadi 2003; Hauck et al. 2005Slopes are equivalent

H0: | b1 b2 | H1: | b1 b2 |<

Nonlinear: Two lots of protein BEstimated ConcentrationsLot 2 is 1.6-fold higher than Lot 1=0.21

Tests for parallel curves4-parameter logistic (FPL) model

Jonkman and Sidik 2009F-test goodness of fit statistic

H0: A1 = A2 and B1 = B2 and D1 = D2 H1: At least one parameter not equal

May lack power with small sample sizeMight be too powerful for large sample size

Calahan and Sajjadi 2003; Hauck et al. 2005; Jonkman and Sidik (2009)

Equivalence test for each parameter = intersection-union test

H0: |1 / 2| 1 or |1 / 2| 2H1: 1

Our proposalParallel Equivalence

Definition of ParallelTwo curves and are parallel if there exists a real number such that for all x.

Definition of Parallel Equivalence

Two curves and are parallel equivalent if there exists a real number such that for all x [xL, xU].

It follows that two curves are parallel equivalent if there exists a real number such that

It also follows that

Are these two lines parallel enough when xL < x < xU ?< ?

Linear-model solutionLinear model:

Just check the endpoints

Parallel equivalence = slope equivalenceSame as testing: | b1 b2 |< wlog

Parallel EquivalenceFPL model:

No closed-form solution.Simple two-dimensional minimax procedure.

< ?Are these two curves parallel enough when xL < x < xU ?

Testing for parallel equivalenceH0:

H1:

Proposed metric (Bayesian posterior probability):

Computing the Bayesian posterior probabilityFor each curve, assume

Data distribution:Prior distribution:i =1, 2 = reference or samplej = 1, 2, , N = observationsPosterior distribution proportional to:

Draw a random sample of the i of size K from the posterior distribution (e.g., using WinBugs).

The posterior probability

is estimated by the proportion (out of K) that the posterior distribution of:

= 0.14 (100.14=1.4-fold shift)= 0.07 90% probability to call parallel equivalent

= 0.33 (100.33=2.15-fold shift)= 0.52 90% probability to call parallel equivalent

Simulation: FPL ModelBased on protein B data

Simulation: FPL modelSimilar to Protein B protein-chip data.

Concentrations (9-point curve + 0):0, 102(=100), 102.5625, 103.125, , 106.5(=3,200,000)Three replicates

xL = 3.5=log10(3162) & xU = 5=log10(100,000) = 0.2. = 0.02, 0.04, 0.11, and 0.21 (%CV = 5, 10, 25, 50)

For each Monte Carlo run, I computed:

5,000 Monte Carlo Replicates

ScenarioOriginalNewMax DiffA1B1C1D1A2B2C2D2124.54-124.54-10224.54.7-1032.4454.9454.7-10.15424.54.7-1.3120.1552.615.114.7-1=0.2062.254.754.7-1.50.30

Example data (CV=10%)Diff: 0 0 0.15 0.15 0.20 0.30

Diff: 0 0 0.15 0.15 0.20 0.30

ScenarioFrequentist statistical powerMax Diff%CV=5%CV=10%CV=25%CV=50101.001.001.000.50201.001.001.000.5030.15 (shape 1)1.000.990.28< 0.0140.15 (shape 2)1.000.880.360.145 = 0.200.100.02< 0.01< 0.0160.300.000.00< 0.01< 0.01

SummaryStraight-forward and simple test method to assess parallelism.

Yields the log-relative potency factor.

Easily extended.

ExtensionsInstead of f(, x), could use

f(, x) / x = instantaneous slope

f-1(, y) = estimated concentrations

Whats next?Head-to-head comparison with existing methods

Choosing test level and , possibly based on ROC curve? Harry Yang paper

Guidance for prior distribution of 1 and 2.

ReferencesCallahan, J. D. and Sajjadi, N. C. (2003), Testing the Null Hypothesis for a Specified Difference - The Right Way to Test for Parallelism, Bioprocessing Journal Mar/Apr 1-6.Gottschalk P.J. and Dunn J.R. (2005), Measuring Parallelism, Linearity, and Relative Potency in Bioassay and Immunoassay Data, Journal of Biopharmaceutical Statistics, 15: 3, 437 - 463.Hauck W.W., Capen R.C., Callahan J.D., Muth J.E.D., Hsu H., Lansky D., Sajjadi N.C., Seaver S.S., Singer R.R. and Weisman D. (2005), Assessing parallelism prior to determining relative potency, Journal of pharmaceutical science and technology, 59, 127-137. Jonkman J and Sidik K (2009), Equivalence Testing for Parallelism in the Four- Parameter Logistic Model, Journal of Biopharmaceutical Statistics, 19: 5, 818 - 837.

Thank you!Questions?

Parallel indicates a change in potency only.*CD8 T cells, HIV ELISPOT assay**USP = United States PharmacopeiaUSP sets standards for the quality, purity, strength, and consistency of productscritical to the public health.*Depending on the response to back-calculate, the fold-change between lots 1 and 2 can range from as little as 1.5-fold to as much as 5.5-fold.If we examine the difference in estimated concentration for lot 2 from the x-range, we get a delta.x = 0.07 -> 1.17-fold maximum shift in estimated conc. Frequentist model point-estimate maximum shift = 1.07-fold. *If we examine the difference in estimated concentration for lot 2 from the x-range, we get a delta.x = 0.35 -> 2.15-fold maximum shift in estimated conc. Frequentist model point-estimate maximum shift = 1.67-fold. *