Copyright © 2013 Quintiles

Predictive Modelling of Operational Characteristics at the Design and Monitoring Clinical Trial

PSI Annual Conference11-14 May 2014, London

Dr. Vladimir Anisimov

Sr Strategic Biostatistics Director

Predictive Analytics, InnovationVladimir.Anisimov@quintiles.com

2

Background

Stochastic behaviour and complexity of trial operation require

developing predictive analytic techniques accounting for • major uncertainties in input information• stochasticity of basic processes:

enrolment, trial start-up, various events (clinical & non-clinical)

Most of trial operational characteristics are driven by patient enrolment

which is stochastic by nature.

Inefficient enrolment forecast is one of the main reasons of trial failure: - more than 60-70% of trials fail to recruit in time

The majority of existing tools in pharma companies still use ad-hoc

simplified or deterministic models.

3

Outline

Discussion of predictive analytic techniques for

• Patient enrolment modelling

• Predicting trial/site enrolment performance and risk-based monitoring

• Forecasting trial operational characteristics - follow-up patients, visits, events, costs

• Event predictive modelling in event-driven trials

4

Patient enrolment

Analytic data-driven statistical methodology is developed *Stochastic modelling technique:

• Patients arrive at sites according to Poisson processes • Variation in enrolment rates is modelled using a gamma distribution• Delays in site initiation also be random (uniform, beta, gamma)

In site i, ni(t) = πλi (t – vi) χ(vi ≤t ≤ bi )

λi = γ(α,β) - rate, vi – delay in site initiation, bi - closure time.

Start-up predictions are created using expected or historical data. Interim re-forecasting uses real data and Bayesian re-estimation.

Closed-form expressions for mean and predictive bounds are derived (no need in Monte Carlo simulation).

Methodology is world-wide accepted, validated on tens of real trials.Basic version is implemented in some pharma companies.

* Anisimov & Fedorov, 2005, 2007; Anisimov, 2009-2011, presented at DIA, JSM (invited sessions), ISCB, PSI,...

5

The developed techniques and software tools (in C# and R)

allow:

• Compute mean and credibility bounds for the predictive number of patients recruited over time and for time to complete trial at any stage of the trial

• Realize a new paradigm: Plan with confidence e.g.

“70% chance that trial will complete enrolment in time”

• Evaluate the optimal number of sites needed to complete in time

• Advise on adaptive adjustment:o Evaluate Probability to recruit in time;o If study is likely to go late, calculate the number of additional sites

needed to complete in time with a given confidence

Enrolment forecaster tools

6

Enrolment adjustment

Scenario: 400 pts, 70 sites, time = 365 days. Sites initiated in 5-month period, half of sites will be closed in two months before the end of enrolment.

Initial design: to complete with 90% confidence.Predictive area: mean and confidence bounds.

Interim analysis after 150 days: 88 pts recruited.Enrolment is slower than predicted.

Interim adjustment: to complete with 90% confidence: 22 new sites to add. Adjusted prediction is going on time.

7

Site performance and risk-based monitoring

Technique/tools* for forecasting site/trial performance: Triggers for detecting unusual behavior

• Low and High-enrolling sites, • Late-start, inactive, high number of AE, etc.

Predictive triggers (interim time, data-driven): • Predict performance• Create dynamic forecasts in future periods

Opportunity for optimal decision-making:choose the optimal # of sites and allocation accounting for studycosts, completion time and risks per delays

Current triggers usually use assumptions of normality and detect unusual behaviour of sites within cohort using Mean and SD: X > Mean(cohort) + K*SD(cohort), K=1,2,3

However, many of variables describing operation of clinical trials are rather far from the normal distribution.

Thus, using triggers based on Mean and SD will lead to biased results.

* Predictive Analytics Team, Quintiles

8

Real data analysis

Histogram of the enrolment rate(# of patients)/(site enrolment duration) far from normal distribution, Red dot – 0.004: Mean + 3*SD,heavy tailed.Adequate model – Poisson mixed with gamma

Histogram - Time from Last Patient Enrolled till current time – red line – fitted Pareto distribution far from normal distribution, heavy tailed.Adequate model – Exponential mixed with gamma

9

Case study

Classification of Low-High enrolling sites using Probability Quantile trigger.

Different dots – different sites - 332. Each site is defined by (# of patients, exposure time)

Two-dimensional site’s characterization.Probability Quantiles: 95% - red line, 90% - magenta, 80% - brown.Adequate model - Poisson process with gamma rate.

Thresholds based on normal approximation:Black linear lines: Mean + SD; Mean + 2*SD; Mean + 3*SD – current trigger

10

Case study: predictive probabilities

Predictive probabilities to enrol: | zero | <= one | <= two | <= three | at least one patient |

ML estimation and Bayesian re-estimation.In particular, if at time t0 site i recruited zero patients during time interval vi , then

Prob (zero patients at time t) =

(α,β) – estimated by ML model parameters

Interim analysis, real study, 330 active sites.Sites: 1006 and 1017 – low enrollers Sites: 1901 and 1906 –good enrollers

11

Predictive enrolment bounds

Predictive 90%-upper bounds for the number of recruited patients in the future.Goal: to predict supply in advance, prepare staff in hospitals,..

Site 66 is highly enrolling; sites 57, 60, 68, 71 – low enrolling

SID 15 days 30 days 45 days57 2 2 358 7 12 1559 8 13 1760 2 2 361 9 14 1962 2 2 363 9 14 1964 9 14 1966 26 45 6367 8 13 1868 2 2 369 7 10 1470 7 10 1471 2 2 372 5 8 10

Real trial, interim look,544 sites active; 1790 patients screened

12

Dynamics of non-enrolling sites

Artificial case study: number of patients =450; number of clinical sites =175; time to complete enrolment =9 months;all centres initiated uniformly in 3-month period.

Blue curve: site’s start-upgreen curve: mean # of empty sitesred curves- 90%predictive bounds

At the end:mean 55, low bound = 45, upper bound = 67Proportion of empty sites at study completion ~ 30%

13

Dynamics of highly performing sites

Predicted number of sites recruited more than 8 pts each.The same case study: number of patients =450; number of clinical sites =175; time to complete enrolment =9 months; all centres initiated at start up.

Green curve: mean # of sitesrecruited > 8 ptsred curves- 90%predictive bounds

At the end of enrolment:mean = 8, low bound = 4, upper bound = 13

14

Modelling trial operation

Next stage: Predictive modelling operational processes.

A new technique using hierarchic evolving processes is developed.

Model: In centre i recruited at time tki patient k generates some follow-up

evolving process with random life time ξki(t- tki), tki ≤ t ≤ tki +τki, e.g.:

• follow-up visits, related events

(clinical or operational, e.g. - AE, CRF, monitoring visits,...)• associated costs (visits, maintenance, supply)

Zi(t) = Σk ξki(t – tki) - sum of evolving processes in centre i,

Statistical technique for forecasting evolving processes is developed.

Closed-form solutions are derived for many practical scenarios.

Tools in C#, R and RExcel are created *

* Predictive Analytics Team, Quintiles

15

Operation cost modelling

Input – tables: • Site’s initiation data • # of sites • Costs per sites/patient visits

Input: Operational costs:• Opening site = $10,000• Maintenance cost per site/day =

$100• Site closing cost = $1,000 • Monitoring cost per patient/day = $20

Monthly cost of trial operationInput: Costs per patient visits

Visit No Time# of visits Cost

1 1 600 1002 30 450 503 58 400 404 87 350 405 114 304 506 143 297 407 172 290 308 198 283 409 282 276 45

10 366 269 40

Mean and 90% credibility intervals.

16

Patients in trial/visits

Study design: Sites – 200, patient’s target – 800, enrolment duration – 1 year

Predictive number of follow-up patientsMean, Low and Upper 90% boundsRegion with 100 sites, μ – drop-out rate,Follow-up period L=180 days.

At some conditions – stable regime: for L < t <T,

n(t) ~ nstat – Poisson with rate λ(1 – e- μL) /μ

Predictive number of Visits No. 3Visits time-schedule:4 visits in total, each after 60 days

17

Event Forecaster tool

Event-driven trials. Patients are followed-up.

Analytic technique* is developed for predicting over time the number of events together with ongoing enrolment and time to stop trial

Assumptions:• Enrolment follows a Poisson-gamma model• Can be different types of events • For multiple events Markov evolving processes are used.

Predictive process:

and - transition probabilities of Markov chain,

- global posterior rate (using Bayesian re-estimation in sites).

Predictive characteristics are derived in a closed form(no need in Monte Carlo simulation)

Opportunity: to combine prediction of the # of events with predictive power.

* Anisimov, Pharma Stats, 2011

18

Scenario. Enrolment completion target time 21 months,Trial completion target time 70 monthsOther 150 sites (with 0.4 pts/month/site) are added after interim lookPA = 100% (enrolment complete in time)PB = 95% (events complete in time)Hazard ratio – 0.8, test on non-inferiority.Power depends on the # of events - randomPC = 97.5% - Prob( Power >0.9)

Target number of events

Trial completion target timeTrial completion target time

A

B C

Acknowledgment: Val Fedorov, Xiaoqiang Xue

Probability of operational success

19

Conclusions

The advanced statistical techniques for predictive analyticmodelling of trial operation are developed:

• Predict and adaptively adjust patient enrolment• Forecast trial operational characteristics

trial/site performance, visits, follow-up patients, operational costs...• Provide risk-based monitoring of the main characteristics• Forecast of different event’s

We are not aware about similar tools used by other pharma companies and CRO.

Supporting software tools are created and in use by Predictive Analytics Team, Innovation, Quintiles.

Tools open wide opportunities for improving the efficiency and quality of CT prediction, additional benefits and cost savings.

20

References

1. Anisimov V., Fedorov V., Modelling, prediction and adaptive adjustment of recruitment in multicentre trials, Statistics in Medicine, 26, 2007, 4958–4975.

2. Anisimov V., Predictive modelling of recruitment and drug supply in multicentre clinical trials, Proc. of JSM, Washington, August, 2009, 1248-1259.

3. Anisimov V., Predictive event modelling in multicentre clinical trials with waiting time to response, Pharmaceutical Statistics, 10, iss. 6, 2011, 517-522.

4. Anisimov V., Statistical modelling of clinical trials (recruitment and randomization), Communications in Statistics – Theory and Methods, 40: 19-20, 2011, 3684-3699.

5. Anisimov V., Predictive hierarchic modelling of operational characteristics in clinical trials, Communications in Statistics - Simulation and Computation (submitted)