When Is Stratification Detrimental to a Clinical Trial Design? Part II Katherine L. Monti, Ph.D....

When Is Stratification Detrimental to a Clinical Trial Design?

Part II

Katherine L. Monti, Ph.D.

Senior Statistical Scientist and Director of the Massachusetts Office, Rho, Inc.

Adjunct Associate Professor, BiostatisticsUniversity of North Carolina

Gretchen Marcucci, M.S.Biostatistician, Rho, Inc.

22003 Katherine L. Monti. All rights reserved. No part of this document may be copied without express written consent.

Introduction

• The method of randomizing clinical trial subjects to treatments in the presence of a (possibly important) prognostic factor requires a design decision: – To stratify on the factor or not to stratify?– Preconception: “Stratification can’t hurt.”

• This paper assesses that preconception for a particular design problem.


Outline

• Motivation for the assessment

• Description of the approach

• Results

• Conclusions


Motivation

• A drug company’s design:– 120 subjects

– 4 treatments (placebo, three drug doses)

– 30 sites

– 1 prognostic factor with 2 levels (hi and low levels, continuous covariate)

• Randomization:– At each site, NOT centralized– In blocks of 4 within factor level within site


Motivation

• Those designing the study thought that randomizing within level of the (prognostic ?) factor– would increase balance in the design – “couldn’t hurt”

• Others argued that randomizing within factor level would lead to – operational difficulties– greater imbalance in the design


Operational Difficulties

• Drug supply requirements increased. (~ 33%)

• Packaging/shipping costs increased.

• Additional training visits to sites were needed in order to explain the more complex randomization scheme.

• The project management burden increased.

• The misassignment of subjects to treatment was more likely.


Imbalance

• 120 subjects/ (30 sites) = 4 subjects per site

Perfect balance with 4 treatments

• 120 subjects/ (30 sites x 2 level) = 2 subjects per site for each levelBalance not assured with 4 treatments


Imbalance

• Although 120/30 = 4 is the “expected” number of subjects per site, 4 subjects enrolling at each site is not really expected! – We don’t know what the real enrollment pattern will be, but

we know that it is not likely to be exactly 4 subjects per site.

• Therefore we anticipate some overall treatment imbalance when randomizing within site.

• However, additionally restricting randomization to within level of the prognostic variable within site could only increase the imbalance.


Simulation

• I set out to compare the magnitude of the treatment imbalance if randomization were performed in permuted blocks of 4:– within site (WS) (30 strata)– within level of the factor within site (WLWS)

(60 strata)

• Used simulation.


Simulation

• Used SAS PROC PLAN to generate treatment assignments in permuted blocks of 4 – for 30 sites and – for 30 sites with 2 factor levels per site


Enrollment Patterns

• To compare the balance in treatment assignment when randomizing WS and WLWS, we need to assign subjects to treatments– at each site

and then reassign– in each level at each site

• To assign subjects to treatments, we need to know the the number of subjects ( Nij )– in site i (i=1-30) who have– factor level j (j=1,2)

The Nij are unknown….


Enrollment Patterns

…So we make assumptions

• However, instead of prescribing the exact Nij for each i and j in the simulation, I defined 9 different enrollment patterns for the 60 site/level combinations.


Enrollment Patterns

• Each enrollment pattern assumed a distribution of the number of subjects in the 60 site/levels, so that there were:– 30 sites, 2 levels per site– 120 subjects– 0-8 subjects in each of the 60 site/level strata

• Some enrollment patterns forced balance between the factor levels:

N.1 = N.2 = 120/2 = 60


An Enrollment Pattern

• EX: A simple pattern: – 20 site/levels strata w/ 3 subjects – 20 site/levels strata w/ 2 subjects – 20 site/levels strata w/ 1 subject

Total of 60 site/level strata w/ 60+40+20 = 120 subjects

Here, Nij = 1, 2 or 3

• Given that pattern– The 60 strata were randomly paired to construct 30 sites.

– Ni1 / Ni2 for site i could be any of the following: 3/3, 3/2, 3/1, 2/3, 2/2, 2/1, 1/3, 1/2, or 1/1


Evaluation Criteria

• For each enrollment pattern, we want to be able to compare the treatment balance when randomization is performed WS and WLWS using permuted blocks of 4 treatments.


Evaluation Criteria

• There are two types of treatment balance:– Overall balance

The extent to which the treatment assignments are balanced.

– Within-level balanceThe extent to which the treatment assignments are balanced within each of the 2 factor levels.


Evaluation Criteria

• 4 criteria used to assess the randomization results of each simulated run are reported here:– N[1] = smallest N of the 4 treatments

– N[1] + N[2] = smallest total sample size for any comparison of treatments

– % loss of power compared to a completely balanced design for the comparison based on N[1] + N[2] for a study designed for 90% power

– N[4] – N[1] = maximum difference in sample sizes


Evaluation Criteria

• Overall, treatment balance would beachieved if

– N1 = N2 = N3 = N4 = 120/4 = 30

– N[1] = 30

– N[1] + N[2] = 60 (No loss of power)

– N[4] – N[1] = 0


Simulation Scheme

• Do for enrollment pattern (EP) = 1 to 9 – Do 10,000 replications:

• Generate 60 strata of subjects (per EP).– Assign random numbers to each subject

(to determine order of enrollment at the site and the order of enrollment in the level at the site).

– Assign a random number to each stratum (to identify the levels, 1 vs 2).

• Randomly pair the 60 strata into 30 sites. (The stratum with the lower random number is level 1.)


Simulation Scheme

• Randomly assign treatments WS based on the order of enrollment in the site, then -determine the sample size of each treatment -compute the evaluation criteria

• Randomly assign treatments WLWS based on the order of enrollment in the site/level, then -determine the sample size of each treatment -compute the evaluation criteria

– Retain all the evaluation criteria

• Go to next enrollment pattern


Results Reported

• For each EP there are 10,000 values of the evaluation criterion

– when randomizing WS

– when randomizing WLWS


Overall Balance, Statistic = N[1]

Type 1 = Randomized Within Site, Overall Results Type 2 = Randomized Within Strata, Overall Results

1-1 1-2 1-3 1-4 1-5 1-6 1-7 1-8 1-9 2-1 2-2 2-3 2-4 2-5 2-6 2-7 2-8 2-9

10

15

20

25

30

Resu

lt

Randomization Type_Enrollment Plan


Overall Balance, Statistic = N[1] + N[2]


1-1 1-2 1-3 1-4 1-5 1-6 1-7 1-8 1-9 2-1 2-2 2-3 2-4 2-5 2-6 2-7 2-8 2-9

40

45

50

55

60

Resu

lt



Overall Balance, Statistic = % Change in Power


1-1 1-2 1-3 1-4 1-5 1-6 1-7 1-8 1-9 2-1 2-2 2-3 2-4 2-5 2-6 2-7 2-8 2-9

-20

-15

-10

-5

0

Resu

lt



Overall Balance, Statistic = Max Difference: N[4] - N[1]


1-1 1-2 1-3 1-4 1-5 1-6 1-7 1-8 1-9 2-1 2-2 2-3 2-4 2-5 2-6 2-7 2-8 2-9

0

5

10

15

20

25

30

Resu

lt



Within Strata Balance, Statistic = N[1] + N[2]

Type 3 = Randomized Within Site, Strata Results Type 4 = Randomized Within Strata, Strata Results

3-1 3-2 3-3 3-4 3-5 3-6 3-7 3-8 3-9 4-1 4-2 4-3 4-4 4-5 4-6 4-7 4-8 4-9

10

15

20

25

30

Resu

lt



Within Strata Balance, Statistic = Max Difference: N[4] - N[1]

Type 3 = Randomized Within Site, Strata Results Type 4 = Randomized Within Strata, Strata Results

3-1 3-2 3-3 3-4 3-5 3-6 3-7 3-8 3-9 4-1 4-2 4-3 4-4 4-5 4-6 4-7 4-8 4-9

0

5

10

15

20

Resu

lt



Conclusions

• With a relatively large number of treatments:– Randomizing within numerous small sites can lead

to some treatment imbalance and loss of power.– Randomization within levels of a prognostic factor

in those small sites will generally• Increase the treatment imbalance overall

• Increase the loss of power in overall pairwise comparisons

• Do little to reduce the treatment imbalance within the levels of the prognostic factor


Conclusions

• “When is stratification detrimental to a clinical trial design?”– I did not address this question directly.

• Is it the case that stratification by prognostic factor “can’t hurt”? – NO: In some cases, stratification can hurt

• Statistically (power)

• Operationally (money)


Contact Information

[email protected]

Slides: www.rhoworld.com

mailto:[email protected]

mailto:[email protected]

* Plans were constructed with and without forcing balance between strata.

No. Subjects

Enrollment Plan

1 2* 3* 4* 5 6

0 6 10 5 30

1 20 14 12 11

2 60 20 16 16 15

3 20 14 12 8

4 6 10 6 30

5 2

6 1

7 1

8 1

Entries are the number of strata having the indicated number of subjects.

Date post:	18-Dec-2015
Category:	Documents
View:	213 times
Download:	0 times

When Is Stratification Detrimental to a Clinical Trial Design? Part II Katherine L. Monti, Ph.D....

Documents