1

Optimal GNG decision rules for an adaptive seamless Ph2/3

oncology trial

Cong Chen

Linda Sun

BARDS, Merck & Co., Inc

22

Typical situation in oncology

Success rate in Ph3 is low, cost for Ph3 is high and competition is fierce

Data at Ph2 to Ph3 transition point– Plenty of data on early efficacy endpoints, of which the most

important is progression-free-survival (PFS)– Limited data on the clinical (late) endpoint, which is typically

overall survival (OS)– PFS is a reasonable surrogate biomarker of OS but can

hardly be considered a validated surrogate endpoint

33

Ph2 to Ph3 transition

Sequential– Ph2 followed with Ph3

Statistically seamless– Ph2 data are combined with Ph3 after proper

multiplicity adjustment due to dose selection

Operationally seamless– Ph3 starts immediately after Ph2 – Ph3 data won’t be combined with Ph2

4

GNG for seamless transition

Today’s focus– How to effectively

incorporate surrogate biomarker data into the decision matrix?

– How to derive objective GNG bars from a benefit-cost ratio perspective?

– How to fully realize the potential of a seamless design with proper risk mitigation?

Not today’s focus– Multiplicity adjustment

and dose selection– Validation of

assumptions made for setting GNG bars

– Cost-effective futility analysis of Ph3 to further mitigate the risk of a Go decision

– Technical details

5

Learning of the day

Relative effect size between the clinical endpoint and an early endpoint, and its application to GNG decisions

Benefit-cost ratio analysis for deriving objective GNG bars

6666

Relative effect size γ (Thing 1)

Ideally, estimation of distribution is based on appropriate meta-analysis of relevant historical data

Mean r and variance σ2

– Smaller r implies it takes greater early effect to achieve same benefit in later endpoint

– Smaller σ2 implies greater predictability of benefit in later endpoint from early endpoint data

7

Data utilization

Conventional Back-calculate effect size of

clinical interest for an early endpoint

– 40% hazard reduction in PFS ~ 25% hazard reduction in OS

Base upon early endpoint data for deriving GNG bar

– Uncertainty about relative effect size is not accounted for

– Role of late endpoint data is less clear (“review issue”)

– Lack of a contingency plan and frequent revisit of decisions

Proposed Explicitly incorporate (r, σ2)

into estimation of treatment effect on clinical endpoint

Directly base upon the estimate for deriving GNG bar

– Empirical relative effect size from the trial is compared to historical estimate before a GNG decision is made

8

Application to estimation of OS effect

weight*OS effect + (1-weight)*γ*PFS effect– Weight may be chosen to be inversely proportional to

variance after incorporation of σ2 into estimation When there is no OS data weight=0 but distribution of γ is still

incorporated into the decision matrix

Go if the joint estimate meets certain criterion and observed OS effect is consistent with or greater that predicted by the PFS effect

– Statistical criterion: type I/II errors and variants– Business criterion: max return on investment

9

Benefit-cost ratio analysis (Thing 2)

Ph2 POC trial is resourced for (α, β)=(0.1,0.2) for detecting an early endpoint effect of Δ

– Sample size is ~80 patients in oncology when Δ refers to a 50% hazard reduction on PFS

– Implicit GNG bar is ~0.6Δ associated with α=0.1 What is the optimal GNG bar?

– Senior manager: “The bar is too low!”– Team member 1: “How large is Ph3?”– Team member 2: “The drug is promising.”– Commercial: “The market is HUGE.”

10

A benefit-cost ratio analysis

Probability of Go if probability of drug truly active in the setting is POS

– Prob(Go) = (1-POS)*α+POS*(1-β)

Expected total sample size (SS)– Ph2 SS + Prob(Go)*Ph3 SS

Benefit-cost ratio (return on investment) when cost is measured by SS

– Power of carrying active drug (1-β) to Ph2 divided by expected total SS

11

Maximum benefit-cost ratio

POS Optimal (α, β) Optimal empirical

GNG bar

0.1 (6.7%, 26.7%) 0.71Δ

0.2 (7.2%, 25.3%) 0.69Δ

0.3 (8.0%, 23.7%) 0.66Δ

Ph3 SS is assumed to be 400 patients in above analyses

Bars should be set higher than 0.6 Δ to make it cost-effective

12

Application to a P2/3 oncology study

1313

Study design

A 3-arm operationally seamless Ph2/3 trial in platinum resistant ovarian cancer with an option of converting to sequential Ph2/3

Primary hypothesis– Test drug is superior to pegylated liposomal doxirubicin

(PLD) in OS– OR test drug is non-inferior to PLD in OS with a margin of

hazard ratio of 1.1 and superior to PLD in a set of pre-defined safety endpoints

14

Ph2 to Ph3 transition

A GNG bar to P3 is pre-set based on complete Ph2 data– ~210 patients in 1:1:1 randomization to two dose levels of test drug

(high and low), and PLD with 4-months of minimum follow-up Dose decision and preliminary GNG decision are made at an

interim analysis right after all patients are enrolled to trigger a seamless transition

– Go if conditional power of meeting the end of Ph2 bar is >80%– Hold otherwise until Ph2 completes

Go if bar for end of Ph2 is met and No Go otherwise Complete Ph2 data and OS data from extended follow-up are

used as a prior for helping set cost-effective futility bars in Ph3 interim analyses

1515

Flow chart

60 pts

380 pts

60 pts

60 pts

f/u

f/u

f/u

380 pts

1m

Ph2: ~135 PFS events

Ph3: ~508 OS events

Data cut-off for IA

GNG to Ph3

4m

1616

Joint estimate of OS effect

Historical estimate of γ– r=0.5 at log(hazard ratio) scale, and σ2= 0.22

50% hazard ratio on PFS implies ~70% hazard ratio on OS (95%CI: 54%, 93%)

0.15*OS effect + 0.85*γ*PFS effect – In this case study, 0.15 provides a robust estimate of weight

that approximately minimizes the variance when event numbers and other parameters are in a space of interest

17

Key input to benefit-cost ratio analysis

Test drug is equally likely to be superior, equivalent and inferior to PLD based on industry benchmark

Commercial values and approvability based on a poll from key stakeholders

– Relative value (superiority vs non-inferiority) is 5.6– Relative approvability (superiority vs non-inferiority) is 2.3

Relative cost before transition to Ph3 relative to the total cost of Ph2/3 program

– ~40% for seamless transition – ~25% for sequential transition

181818

GNG bars

GNG bar at end of Ph2 based on a benefit-cost ratio analysis

– Expected benefit: Average power of Ph3 over prior adjusted with outcome (superiority or non-inferiority) and its associated approvability and commercial value

– Expected cost: Cost before transition + Cost after transition*(average probability of Go to Ph3 over prior associated with the GNG bar)

GNG bar at interim of Ph2 for seamless transition– Conditional probability of meeting end of Ph2 bar is >80%– Assumption about r is double checked before GO

19

Key drivers behind benefit-cost ratio

In our case study, the optimal end of Ph2 bar is generally robust to the input parameters

– The bar corresponds to ~8% hazard reduction in terms of joint estimate of OS effect (implied OS improvement ~1.2 months)

Drivers with greatest impact on optimal GNG bar – Prior belief of drug activity: the stronger the belief the

lower the optimal bar– Cost structure: the higher the up front cost the lower

the optimal bar

20

Optimal GNG bars at end of Ph2

Assumption of drug activity

Relative Cost of Ph2 to whole

program

Implied OS improvement in optimized bar

Superior – 11% Equivalent – 22%

Inferior – 66%

25% - sequential ~ 1.7 months

40% - seamless ~ 1.3 months

Superior – 33%

Equivalent – 33%

Inferior – 33%

25% - sequential ~ 1.6 months

40% - seamless ~ 1.2 months

Superior – 50%

Equivalent – 33%

Inferior – 17%

25% - sequential ~ 1.5 months

40% - seamless ~ 1.1 months

21

GNG boundary at end of Ph2

0.6 0.8 1.0 1.2 1.4

0.0

0.5

1.0

1.5

Observed Hazard Ratio of PFS (MK/Control)

Ob

serv

ed

Ha

zard

Ra

tio o

f OS

(M

K/C

on

tro

l)

22

Check assumption about γ

0.6 0.8 1.0 1.2 1.4

0.0

0.5

1.0

1.5

Observed Hazard Ratio of PFS (MK/Control)

Ob

serv

ed

Ha

zard

Ra

tio o

f OS

(M

K/C

on

tro

l)

23

GNG boundary at interim analysis

0.6 0.8 1.0 1.2 1.4

0.0

0.5

1.0

1.5

Observed Hazard Ratio of PFS (MK/Control)

Ob

serv

ed

Ha

zard

Ra

tio o

f OS

(M

K/C

on

tro

l)

24

Check assumption about γ at IA

0.6 0.8 1.0 1.2 1.4

0.0

0.5

1.0

1.5

Observed Hazard Ratio of PFS (MK/Control)

Ob

serv

ed

Ha

zard

Ra

tio o

f OS

(M

K/C

on

tro

l)

Boundary for IABoundary for FAPredicted OS Effect95% CI of Predicted OS Effect at IA95% CI of Predicted OS Effect at FA

25

Discussion

Relative effect size γ (and its distribution) between OS and PFS or, more generally, between a late endpoint and an early endpoint holds key to statistical properties of GNG bars.

Assumptions about key parameters such as relative effect size, benefit, cost, and POS are implicit in all major clinical decisions, which are often heuristic and subjective in practice.

With max return on investment in mind, optimal GNG bars are derived after explicit incorporation of the assumptions into a utility function (e.g., benefit-cost ratio).

Statisticians can help formulate the problem, and help streamline the decision process.

26

References

1. Chen C, Sun L, Chih C. Evaluation of Early Efficacy Endpoints for Proof-of-concept Trials, Journal of Biopharmaceutical Statistics 2011, accepted.

2. Song Y, Chen C. Optimal Strategies for Developing a Late-stage Clinical Program with a Possible Subset Effect, Statistics in Biopharmaceutical Research 2011, accepted.

3. Robert A. Beckman, Jason Clark, and Cong Chen. Integrating Predictive Biomarkers and Classifiers into Oncology Clinical Development Programs: An Adaptive, Evidence-Based Approach. Nature Review Drug Discovery 2011, volume 10, 735-749.

4. Chen C, Sun, L. On quantification of PFS effect for accelerated approval of oncology drugs. Statistics in Biopharmaceutical Research 2011, DOI: 10.1198/sbr.2011.09046.

5. Chen C, Beckman RA. Hypothesis testing in a confirmatory Phase III trial with a possible subset effect. Statistics in Biopharmaceutical Research, 1, 431-440 (2009).

6. Chen C, Beckman, RA. Optimal cost-effective Go-No Go decisions in late stage oncology drug development. Statistics in Biopharmaceutical Research, 1, 159-169 (2009).

7. Chen C, Beckman RA. Optimal cost-effective Phase II proof of concept and associated Go-No Go decisions. J. Biopharmaceutical Statistics, 1, 431-440 (2009).

8. Song Y, Chen C. Optimal strategies for developing a late-stage clinical program with a possible subset effect. ASA Proceedings of the Joint Statistical Meetings 2009, 1408-1422.

9. Sun L, Chen C. Evaluation of early endpoints for go-no go decisions in late-stage drug development. ASA Proceedings of the Joint Statistical Meetings 2009, 2273-2283.

10. Chen C, Beckman RA. Optimal cost-effective designs of proof of concept trials and associated Go-No Go decisions. Proceedings of the American Statistical Association, Biometrics Section, (2007).

27

Technical details

Weighted estimate of treatment effect at information t– wΔt+(1-w)γδt where (Δt, δt) are treatment effect on (clinical endpoint, early endpoint)

and γ is estimated to be N(r, σ2) w=var(γδt)/(var(γδt)+var(Δt)) where var(γδt)= σ2(var(δt)+ δt

2)+(rδt)2

Corr(Δt, δt) may be estimated from a re-sampling based method or WLW method when both are time-to-event variables

Easily extended when there are more than one early endpoints of interest Estimated treatment effect on clinical endpoint at final analysis

– tΔt+(1-t)Δ1-t where Δ1-t is the treatment effect after information time t, independent of Δt

– Joint distribution of wΔt+(1-w)γδt and tΔt+(1-t)Δ1-t is obtained after the variance-covariance between the two are derived from the above

Conditional power, predicted power, and various other statistics of interest are easily obtained once the conditional distribution is obtained.

Exact distribution may also be obtained but is more complicated. Normal approximation is close enough for planning purpose.

Joint distribution of weighted estimates at two time points can be obtained similarly, which is used for calculation of conditional probability in case study