1

Statistical Designs for Stratified Medicine

Cindy BillinghamProfessor of Biostatistics, School of Cancer Sciences

Director of Statistics, Cancer Research UK Clinical Trials Unit

Lung Cancer Research Stratified Medicine Educational EventBirmingham, June 22nd 2015

2

Agenda

Early phase clinical trials for stratified medicine Illustrated using National Lung Matrix Trial Bayesian adaptive design Example of an ‘umbrella trial’

Later phase clinical trials for stratified medicine Typical randomised controlled designs (RCTs) Umbrella and basket trials Do we always need an RCT to change clinical practice in

stratified medicine?

3

Actionable targets (biomarkers) and targeted drugs = 17 drug-biomarker cohorts

Prev A:AZD4547

B:AZD2014

C:Palbociclib

D:Crizotinib

E:Sel+Doc

F:AZD5363

G:AZD9291

A1: FGFR2/3 mutation-NSCLC 4.0%

B1: TSC1/2 mutation-NSCLC 2.7%

B2: LKB1 TIER1 mutation-NSCLC 6.4%

C1: Proficient Rb & p16 loss-SCC 29.0%

C2: Proficient Rb & CDK4 amp-NSCLC 7.0%

C3: Proficient Rb & CCND1 amp-NSCLC

7.3%

C4: Proficient Rb & KRAS mutation-ADC 25.8%

D1: Met amplified-NSCLC 2.3%

D2: ROS1 gene fusion-NSCLC 1.7%

E1: NF1 mutation-SCC 5.8%

E2: NF1 mutation-ADC 4.6%

E3: NRAS mutation-ADC 1.0%

F1: PIK3CA mutation-SCC 11.0%

F2: PIK3CA amp-SCC 15.0%

F3: PIK3/AKT deregulation-NSCLC 4.5%

F4: PTEN loss & mutation-SCC 20.0%

G1: EGFR mutation & T790M+-NSCLC 8.0%

4

National Lung Matrix Trial SchemaFinal biopsy result (diagnostic and/or repeat) mandated for National Lung Matrix Trial entry:

Biopsy failure

(diagnostic and repeat)or ineligible

No actionable genetic

aberration

Multiple actionable target with

open treatment arms

Single actionable target with

open treatment arm

Outcome Measures (common set for all arms with treatment-arm-specific primary): Best objective response rate (ORR), Change in total target lesion size (PCSD), Progression-free survival time (PFS), Time-to-

progression (TTP), Overall survival time (OS), Toxicity

Allocated to single

treatment arm relevant to actionable

target prioritised by CI if eligible

Allocated to single

treatment arm relevant to actionable

target if eligible

Standard treatment

OR recruitment to another relevant

trial

Allocated to no actionable genetic change (NA) cohort if

eligible

Standard Clinical

Outcome Measures

Actionable target but no

target therapy

arms open

5

Typical Single Arm Phase II Trial

Eligible Patients

NEW Treatment

Response rate

Historical data / clinical

experience of standard treatment

Benchmark response rate

0% 100%

?

Common Designs:A’Herns single stageSimon’s two-stage

6

Flexible design that embraces study complexity and facilitates decision-makingNeed efficient and flexible design that: has potential to stop early for lack of efficacy allows for differing prevalence rates of biomarkers allows continued recruitment to any sample size as appropriate has potential to incorporate relevant information from other biomarker

cohorts within each drug arm (‘borrowing information’) has potential to incorporate pre-existing evidence and emerging

external evidence enables cumulative learning

Question that we really want to answer: What is the probability that the TRUE signal of efficacy is above x% Make Go-NoGo decision for further research based on probability

Bayesian Adaptive DesignRef: Berry, Carlin, Lee, Muller; Bayesian Adaptive Methods for Clinical Trials, Chapman and Hall /CRC Biostatistics Series 2011

7

What is a Bayesian Approach to Statistical Analysis? Alternative method of statistical analysis to the

classical / frequentist approach ‘The explicit quantitative use of external evidence in the design,

monitoring, analysis, interpretation of a health-care evaluation’ Spiegelhalter et al 2004

Based on theorem devised by Reverend Thomas Bayes (1702-1761)

Basic maths:

Prior x Data → Posterior

)(

)()/()/(

Ap

BpBApABp

Bayes Theorem:

Posterior probability distribution

P(HR<1)=0.75

8

Start Recruitment

Interim Analysis 1 N=15

if P(<30%) ≥ 0.9STOP early @ Interim

Final AnalysisN=30

STOP @ Final

if P(>30%) < 0.5

if P(>30%) ≥ 0.5

GO

Arms A, B, D, F & GBayesian Adaptive Two-Stage Design

if P(<30%) < 0.9

Cohorts: A1, B1, B2, B3, D1, D2Primary outcome measure: objective response rate (ORR): true ORR

9

Illustrating Statistical Analysis Plan Example: GO-GO

Prior: Beta(1,1)

Interim Posterior:Beta(4,13)

Interim analysis: 3/15 = 20%Final analysis: 13/30 = 43%

P(<30%)=0.75

Final Posterior:

Beta(14,18)

P(>30%)=0.95

Beta-Binomial conjugate analysisPrior: ~ Beta(a0, b0)Posterior: |r,n ~ Beta(a0+r,b0+n-r)

10

Cumulative Learning Using Bayesian Analysis

n=1,r=1 n=2,r=1 n=3,r=1 n=4,r=2 n=5,r=2

n=6,r=2 n=7,r=2 n=8,r=2 n=9,r=2 n=10,r=2

n=11,r=2 n=12,r=2 n=13,r=2 n=14,r=2 n=15,r=3

n=16,r=4 n=17,r=5 n=18,r=5 n=19,r=6 n=20,r=7

n=21,r=7 n=22,r=7 n=23,r=8 n=24,r=8 n=25,r=8

n=26,r=9 n=27,r=10 n=28,r=11 n=29,r=12 n=30,r=13

11

How well does this design work? Are other designs better?Desirable operating characteristics

Sample size criteria: Need minimum of 10 and maximum of 15 at interim Need minimum of 20 and maximum of 40 at final

Interim analysis criteria: Need p(STOP early|=10%)>80% (true stopping rate) Need p(STOP early|=40%)<5% (false stopping rate)

Final analysis criteria: Need p(GO at final|=20%)<10% (false positive rate) Need p(GO at final|=40%)>80% (true positive rate)

Of all the designs that satisfy these criteria, the optimal design is that which maximises the true positive rate

12

Start Recruitment

Interim Analysis 1 N=15

if P(<30%) ≥ 0.9STOP early @ Interim

Final AnalysisN=30

STOP @ Final

if P(>30%) < 0.5

if P(>30%) ≥ 0.5

GO

Arms A, B, D, F & GDesign and Operating Characteristics

2/15=13% or less

8/30=27% or less

3/15=20% or more

9/30=30% or more

if P(<30%) < 0.9

=10% =20% =30% =40% =50%

STOP early 81.6% 39.8% 12.7% 2.7% 0.4%

STOP at final 18.2% 47.8% 31.8% 7.7% 0.7%

GO at final 0.2% 12.4% 55.5% 89.6% 99.0%

Operating characteristics based on exact binomial probabilities

: true ORR

13

Start Recruitment

Interim Analysis 1 N=15

if P(<40%) ≥ 0.9STOP early @ Interim

Final AnalysisN=30

STOP @ Final

if P(>40%) < 0.5

if P(>40%) ≥ 0.5

GO

Arm E (Selumetinib+Docetaxel)Design and Operating Characteristics

if P(<40%) < 0.9

=10% =20% =30% =40% =50%

STOP early 94.4% 64.8% 29.7% 9.1% 1.8%

STOP at final 5.6% 34.3% 54.7% 34.7% 8.7%

GO at final 0.0% 0.9% 15.6% 56.2% 89.5%

Operating characteristics based on exact binomial probabilities

Cohorts: E1, E2, E3 - objective response rate (ORR): true ORR

14

Start Recruitment

Interim Analysis 1 N=15

if P(<3mths) ≥ 0.8STOP early @ Interim

Final AnalysisN=30

STOP @ Final

if P(>3mths) < 0.5

if P(>3mths) ≥ 0.5

GO

Arm C (Palbociclib)Bayesian Adaptive Two-Stage Design

if P(<3mths) < 0.8

Cohorts: C1, C2, C3, C4Primary outcome measure: progression-free survival time (PFS): true median PFS (months)

15

Arm C (Palbociclib)Design and Operating Characteristics

True median = 1 month

True median = 2 months

True median = 3 months

True median = 4 months

True median = 5 months

True median = 6 months

STOP early 97.2% 49.9% 16.8% 5.4% 2.2% 1.0%

STOP at final

2.8% 49.0% 43.7% 11.4% 1.8% 0.3%

GO at final <0.1% 1.2% 39.4% 83.2% 96.0% 98.6%

Operating characteristics estimated through simulation

Cohort C1, recruiting at 93 patients per annum when all recruitment centres are open

16

Using Bayesian Hierarchical Modelling as Secondary Analysis

E1: NF1 mutant -

SCC

E2: NF1 mutant -

ADC

Arm E: Selumetinib+ Docetaxel

25/30 (83%)

2/15 (13%)

Ensures borderline decisions err on positive if drug has shown potential in other cohorts

Secondary analysis to aid decision-making,

particular when decisions are borderline

Can build in expected level of association

Never be used to negate a primary analysis that shows a potentially

positive result

External Evidence

17

National Lung Matrix Trial SchemaFinal biopsy result (diagnostic and/or repeat) mandated for National Lung Matrix Trial entry:

Biopsy failure

(diagnostic and repeat)or ineligible

No actionable genetic

aberration

Multiple actionable target with

open treatment arms

Single actionable target with

open treatment arm

Outcome Measures (common set for all arms with treatment-arm-specific primary): Best objective response rate (ORR), Change in total target lesion size (PCSD), Progression-free survival time (PFS), Time-to-

progression (TTP), Overall survival time (OS), Toxicity

Allocated to single

treatment arm relevant to actionable

target prioritised by CI if eligible

Allocated to single

treatment arm relevant to actionable

target if eligible

Standard treatment

OR recruitment to another relevant

trial

Allocated to no actionable genetic change (NA) cohort if

eligible

Standard Clinical

Outcome Measures

Actionable target but no

target therapy

arms open

18

No Actionable Genetic Change Cohorts (NA)

No actionable genetic

aberration

Drug NA1

Drug C

Drug NA2

Drug E

Drug B

Etc

Test in NEGATIVES once signal demonstrated in POSITIVES

ExamplePipeline of options that become available sequentially

Drug NA1: MEDI4736Statistical design: adaptive Bayesian design in line with actionable target cohortsTwo co-primary outcomes: ORR and PFS6 Initial sample size for interim analysis: N=20 (determined by drug supply)

No comparison will be made between treatments

19

Using NA Cohort for Decisions about Next Steps in Research Pathway

E.g. Drug D - Crizotinib

NACohort

15/30 (50%)

15/30 (50%)

10/20 (50%)

0/20 (0%)

All-comers design next

Enrichment design next

D1: Met amplified

- mixed

D2: ROS1 Gene fusions

- mixed

20

Measure Biomarkers

Biomarker+ Biomarker-

RANDOMISE

BM+Drug Control ControlBM+Drug

RANDOMISE

Stratified Trial Design

Marker-Based Strategy Design RANDOMISE

Marker-based treatment strategy Standard Care

Biomarker+ Biomarker-

Standard Care

Measure Biomarkers

BM+Drug

21

22

Basket Trial Umbrella Trial

23

When Might RCTs Not Be Needed / Ethical? Oxford / Sackett All or none criterion

E.g. Without intervention ALL patients die within 6 months VS With intervention NONE die within 6 months

Nick Black criteria Experimentation may be unnecessary when the effect is

so dramatic that unknown confounding factors could be ignored

Glasziou P, Chalmers I, Rawlins M, McCulloch P. When are randomised trials unnecessary? Picking signal from noise BMJ 2007;334:349-351 "10 x rule" – data from non-RCTs can be trusted if the

ratio of treatment effects between two alternative therapies > 10. In all other circumstances, the real treatment effects cannot be reliably separated from the effects of biases and random errors without employing RCT design.

http://personal.health.usf.edu/bdjulbeg/oncology/NON-RCT-practice-change.htm

24

Compelling Evidence From Single Case Studies / Clinical Experience

25

Compelling Evidence From A Single Arm Trial

26

What Treatment Can Create Such A Dramatic Effect? ALK Inhibitor

Patients whose tumour driven by ALK

27

Regulatory perspective on non-randomised evidence

99 trials supported approvals for 45 drugs for 68 rare cancer indications

28

Summary• More innovative statistical designs may be

needed as trials become complex• Umbrella and basket trials are an efficient

approach to stratified medicine research• Stratified medicine creates multiple rare cancers

that challenge conventional statistical designs • Developing the right drugs for the right patient to

target specific molecular drivers may create dramatic and biologically plausible treatment effects that do not require RCTs to change practice