11

Optimal cost-Optimal cost-effective Go-No Go effective Go-No Go

decisionsdecisions

Cong Chen*, Ph.D.Cong Chen*, Ph.D.Robert A. Beckman, M.D.Robert A. Beckman, M.D.

*Director, Merck & Co., Inc.*Director, Merck & Co., Inc.EFSPI, Basel, June 2010EFSPI, Basel, June 2010

22

Sorry for not being able to attend in person…

3333

OutlineOutline

IntroductionIntroduction Benefit-cost ratio analysis of POC Benefit-cost ratio analysis of POC

design strategiesdesign strategies DiscussionDiscussion

– POC strategy and risk mitigationPOC strategy and risk mitigation– Phase III futility analysisPhase III futility analysis

4444

How to fish smartly?How to fish smartly?

Low success rate and

predictability

Constraint on

societal cost

Numerous POCpossibilities

Biology and tech revolution

5555

Proof-of-concept trialProof-of-concept trial

A A randomized double-blinded randomized double-blinded phase II trial with type I/II error rate phase II trial with type I/II error rate ((αα, , ββ) for detection of ) for detection of ΔΔ based on a based on a surrogate markersurrogate marker– Go to Phase III if p-value <Go to Phase III if p-value <αα

Choice of (Choice of (αα, , ββ, , ΔΔ) is based on a ) is based on a heuristic argument in practice and is heuristic argument in practice and is under-explored in statistical under-explored in statistical literatureliterature

6666

Issues to be addressedIssues to be addressed

What is a more cost-effective What is a more cost-effective sample size for a POC trial?sample size for a POC trial?

What is the optimal bar for a Go What is the optimal bar for a Go decision to Phase III?decision to Phase III?

How to re-allocate resource when How to re-allocate resource when there are more POC trials of there are more POC trials of similar interest? similar interest?

7777

Benefit-cost ratio Benefit-cost ratio analysisanalysis Probability of Go if Probability of Go if probability of probability of

drug truly active in the setting is drug truly active in the setting is POSPOS– (1-POS)*(1-POS)*αα+POS*(1-+POS*(1-ββ))

Expected total sample size (SS)Expected total sample size (SS)– Phase II SS + Prob(Go)*Phase III SSPhase II SS + Prob(Go)*Phase III SS

Benefit cost ratioBenefit cost ratio– Power of carrying active drug (1-Power of carrying active drug (1-ββ) to ) to

Phase III divided by expected total SSPhase III divided by expected total SS

8888

Two designsTwo designs

AssumptionsAssumptions– Same Same ΔΔ of interest, e.g., 50% improvement in of interest, e.g., 50% improvement in

median progression-free-survivalmedian progression-free-survival– Sample size for Phase III is fixed at 800 once a Sample size for Phase III is fixed at 800 once a

Go decision is made after POCGo decision is made after POC Two choices of (Two choices of (αα, , ββ))

– (10%, 20%) or a ~160 patient/~110 events (10%, 20%) or a ~160 patient/~110 events trial trial

– (10%, 40%) or a ~80 patient trial but higher (10%, 40%) or a ~80 patient trial but higher empirical bar (~0.8empirical bar (~0.8ΔΔ vs 0.6 vs 0.6ΔΔ) for a Go decision) for a Go decision

9999

Results for comparisonResults for comparison

POSPOS SizeSize Pr(Go)Pr(Go) PowePowerr

ExpecteExpected total d total

SS SS

Power/ Power/ total SStotal SS

10%10% 160160 17%17% 80%80% 300300 0.270.27

8080 15%15% 60%60% 200200 0.300.30

20%20% 160160 24%24% 80%80% 350350 0.230.23

8080 20%20% 60%60% 240240 0.250.25

30%30% 160160 31%31% 80%80% 400400 0.200.20

8080 25%25% 60%60% 280280 0.210.21

Smaller trial is more cost-effective. More gains Smaller trial is more cost-effective. More gains (15-30% improvement) can be realized after (15-30% improvement) can be realized after optimization.optimization.

1010

Optimal designs under fixed Phase II resource

POS ((αα, , ββ)) Empirical GNG bar

0.1 (6.7%, 26.7%) 0.71ΔΔ

0.2 (7.2%, 25.3%) 0.69ΔΔ

0.3 (8.0%, 23.7%) 0.66ΔΔAssumptions:1) Phase II is resourced for (αα, , ββ)=(0.1,0.2),

which has an implicit Go bar of 0.6ΔΔ 2) Relative sample size of Phase II to Phase III

is 20% (e.g., 160 pts vs 800 pts)

11111111

Resource optimization Resource optimization

Budgeted for conducting one 160 patient Budgeted for conducting one 160 patient POC trial, but has two POC trials of POC trial, but has two POC trials of similar interestsimilar interest– Consensus is that one has higher POS Consensus is that one has higher POS

(P1=30%) than the other (P2=20%)(P1=30%) than the other (P2=20%)– Phase III trial uses same design once GoPhase III trial uses same design once Go

Two scenarios for comparison under Two scenarios for comparison under varying ratio of POC budget (C2)/Phase III varying ratio of POC budget (C2)/Phase III cost (C3) assuming sample size is cost (C3) assuming sample size is proportional to cost proportional to cost – Two drugs have same valueTwo drugs have same value– The one with lower POS has 50% higher valueThe one with lower POS has 50% higher value

12121212

Optimal resource split Optimal resource split under same valueunder same value

13131313

Optimal resource split Optimal resource split and Go bar under same and Go bar under same valuevalue

14141414

Optimal resource split Optimal resource split and Go bar under and Go bar under different valuedifferent value

15151515

Conclusions Conclusions

Optimal (Optimal (αα, , ββ) can be easily optimized ) can be easily optimized from benefit-cost ratio analysisfrom benefit-cost ratio analysis

Number of POC trials and respective Go Number of POC trials and respective Go bars depend on Phase II resource, Phase bars depend on Phase II resource, Phase III cost, perceived POS and projected valueIII cost, perceived POS and projected value

Similar analysis reveals that a greater Similar analysis reveals that a greater ΔΔ has to be consideredhas to be considered when relationship when relationship between surrogate marker and OS is less between surrogate marker and OS is less certain certain – Uncertainty is highest in non-randomized Uncertainty is highest in non-randomized

trials!trials!

1616

POC strategy

More smaller trials, each with a More smaller trials, each with a higher Go bar, are generally higher Go bar, are generally preferred preferred – Adequately powered for larger Adequately powered for larger ΔΔ of true of true

interestinterest Similar analysis shows that Similar analysis shows that

simultaneous investigation is more simultaneous investigation is more cost-effective than sequential cost-effective than sequential investigationinvestigation

17171717

Avastin POC strategyAvastin POC strategy

IndicatioIndicationn

#pts/#pts/armarm

SourceSource

ColonColon 33-3633-36 JCO 2003; 21: 60-65JCO 2003; 21: 60-65

RCCRCC 37-4037-40 NEJM 2003; 349: 427-NEJM 2003; 349: 427-434434

NSCLCNSCLC 32-3432-34 JCO 2004; 22: 2184-91JCO 2004; 22: 2184-91

BreastBreast 10-1810-18 Semin Oncol 2003; Semin Oncol 2003; 5(suppl 16):1175(suppl 16):117

All trials have 3 arms (low/high dose and All trials have 3 arms (low/high dose and placebo) with 80% power for doubling of PFSplacebo) with 80% power for doubling of PFS

18181818

PFS effect of recently PFS effect of recently approved innovative approved innovative drugsdrugs

Trial Trial HR HR (central (central review)review)

HR (local HR (local review)review)

RCC/SorafenibRCC/Sorafenib 0.440.44 0.510.51

RCC/sunitinibRCC/sunitinib 0.420.42 0.420.42

CRC/CRC/panitumumabpanitumumab

0.540.54 0.390.39

BC/lapatinibBC/lapatinib 0.490.49 0.590.59

BC/Bev+pac vs BC/Bev+pac vs pacpac

0.420.42 0.480.48

19191919

Pros and consPros and cons

Smaller trials Smaller trials – Easier to accrue, faster to complete, and Easier to accrue, faster to complete, and

have better quality controlhave better quality control– Empirical findings of large treatment effect Empirical findings of large treatment effect

are more exciting, and help with Phase III are more exciting, and help with Phase III accrualaccrual

– More vulnerable to baseline imbalance More vulnerable to baseline imbalance More trialsMore trials

– Reduces missed opportunities (type III error) Reduces missed opportunities (type III error) and increases overall probability of success and increases overall probability of success

– May inflate program level type I error rateMay inflate program level type I error rate

20202020

Risk mitigationRisk mitigation

Apply minimization or other randomization Apply minimization or other randomization techniques for better baseline balancetechniques for better baseline balance

Follow-up patients for survival after primary Follow-up patients for survival after primary objective for Phase II is achievedobjective for Phase II is achieved– Initiation of Phase III may be delayed while waiting Initiation of Phase III may be delayed while waiting

for Phase II OS data to mature for Phase II OS data to mature – May revisit a Go or No-Go decision as necessary May revisit a Go or No-Go decision as necessary

after OS data become available after OS data become available – Strength of OS data may be used for setting futility Strength of OS data may be used for setting futility

bar of Phase III trial as appropriatebar of Phase III trial as appropriate Revisit those less promising ones from Phase Revisit those less promising ones from Phase

II after leading indications of same drug II after leading indications of same drug achieve major milestones in Phase III achieve major milestones in Phase III

2121

Futility bar at interim for an ongoing Phase III trial A hypothetical Phase III trial

– Designed to have 90% power for detection of Δ in OS before accounting for any futility analysis

– Trial stops for futility at interim if one-sided p-value > α based on survival info of fraction r after 50% of the cost is spent

Benefit = overall power adjusted for futility– May be further adjusted with value as needed

Expected cost = 0.5+0.5*Prob(Go)– where Prob(Go)=(1-POS)*α+POS*(1-β) and β satisfies Zα+Zβ=r1/2(Z0.025+Z0.1)

2222

Benefit-cost ratio analysis at 25% info for 30% POSα (cut-

off)Empirica

l barOvera

ll power

Expected cost

Power/cost

1 -∞ 90.0% 1.00 0.90

0.6 -0.16Δ 88.3% 0.86 1.03

0.5 0 86.8% 0.82 1.06

0.309* 0.31Δ 80.8% 0.74 1.09

0.2 0.53Δ 73.6% 0.69 1.07

2323

Optimal futility bars

POS Info (r) α (cut-off p)

Empirical bar

Overall power

30% 15% 45.0% 0.10Δ 80.2%

20% 36.8% 0.23Δ 80.3%

25% 30.9% 0.31Δ 80.8%

50% 15% 51.6% -0.03Δ 82.8%

20% 42.5% 0.13Δ 82.6%

25% 35.5% 0.23Δ 82.8%

Optimal bar decreases with POS and increases with information. Positive trend is generally required.

2424