3 ACS001- Study Conduct Study Began in Study Began in First 10 patients – received CyIS (Cyclosporine Inhalation solution) as part of the open label pilot phase First 10 patients – received CyIS (Cyclosporine Inhalation solution) as part of the open label pilot phase Next 58 patients were randomized to either CyIS with propylene glycol as vehicle or propylene glycol alone. Next 58 patients were randomized to either CyIS with propylene glycol as vehicle or propylene glycol alone. Two of these 58 patients did not receive any dose of the study medication and were excluded from all analyses. Two of these 58 patients did not receive any dose of the study medication and were excluded from all analyses. The enrollment was stopped in August 2001 at 68 subjects. The enrollment was stopped in August 2001 at 68 subjects. Study was “Vehicle” Controlled. Study was “Vehicle” Controlled. All subjects received systemic immuno-suppresion All subjects received systemic immuno-suppresion
34
1 Pulminiq™ Pulminiq™ Cyclosporine Inhalation Cyclosporine Inhalation Solution Solution Pulmonary Drug Advisory Pulmonary Drug Advisory Committee Meeting Committee Meeting June 6, 2005 June 6, 2005 Statistical Evaluation Statistical Evaluation Jyoti Zalkikar, Ph.D. Jyoti Zalkikar, Ph.D. Office of Biostatistics Office of Biostatistics Center for Drug Evaluation and Research Center for Drug Evaluation and Research Food and Drug Administration Food and Drug Administration
Office of BiostatisticsOffice of BiostatisticsCenter for Drug Evaluation and ResearchCenter for Drug Evaluation and Research
Food and Drug AdministrationFood and Drug Administration
2
ACS001- Study ConductACS001- Study Conduct Small (56 patients in total), Phase II, Small (56 patients in total), Phase II,
Single Center (UPMC) StudySingle Center (UPMC) Study Not designed as a confirmatory study – Not designed as a confirmatory study –
planned sample size was 136 patients.planned sample size was 136 patients. Investigators did not use Case Report Investigators did not use Case Report
Forms during the conduct of the study. Forms during the conduct of the study. These were generated retrospectively by These were generated retrospectively by
the applicant.the applicant. No prospective Statistical Data No prospective Statistical Data
Analysis Plan or formal stopping rules. Analysis Plan or formal stopping rules.
3
ACS001- Study ConductACS001- Study Conduct Study Began in 1997.Study Began in 1997. First 10 patients – received CyIS (Cyclosporine First 10 patients – received CyIS (Cyclosporine
Inhalation solution) as part of the open label Inhalation solution) as part of the open label pilot phasepilot phase
Next 58 patients were randomized to either Next 58 patients were randomized to either CyIS with propylene glycol as vehicle or CyIS with propylene glycol as vehicle or propylene glycol alone.propylene glycol alone. Two of these 58 patients did not receive any dose of Two of these 58 patients did not receive any dose of
the study medication and were excluded from all the study medication and were excluded from all analyses. analyses.
The enrollment was stopped in August 2001 at The enrollment was stopped in August 2001 at 68 subjects. 68 subjects.
Study was “Vehicle” Controlled. Study was “Vehicle” Controlled. All subjects received systemic immuno-All subjects received systemic immuno-
suppresion suppresion
4
ACS001- Study DesignACS001- Study Design Patients were to be enrolled from 7-42 Patients were to be enrolled from 7-42
days, after transplant. days, after transplant. 3 violations – 47 days, 53 days and 77 days3 violations – 47 days, 53 days and 77 days
Randomization was stratified by Randomization was stratified by enrollment window (7-21 days and 22-42 enrollment window (7-21 days and 22-42 days) and CMV (D+/R-) mismatch status.days) and CMV (D+/R-) mismatch status. 2 violations - incorrect strata2 violations - incorrect strata
Patients were to be on treatment daily Patients were to be on treatment daily for the first 10 days and 3 times a week for the first 10 days and 3 times a week thereafter for a total of 2 years. thereafter for a total of 2 years. 30 patients discontinued treatment early30 patients discontinued treatment early
All Patients were followed for at least 33 All Patients were followed for at least 33 months. months.
5
ACS001 - Study DesignACS001 - Study Design Evaluation of Survival and Chronic Evaluation of Survival and Chronic
Rejection were NOT the primary Rejection were NOT the primary objectives. objectives.
Pre-specified Primary Objective – Pre-specified Primary Objective – Superiority over Superiority over propylene glycolpropylene glycol in in terms of Rate of Acute Rejection (AR).terms of Rate of Acute Rejection (AR).
Study Study failedfailed to achieve the primary to achieve the primary objective. objective. Mean number of AR higher in the CyIS arm Mean number of AR higher in the CyIS arm
(1.35) compared to the (1.35) compared to the propylene glycolpropylene glycol (1.20) arm.(1.20) arm.
Survival Probability Survival Probability MonthMonth 1212 2424 3636 4848 6060CyISCyIS 96%96% 88%88% 88%88% 84%84% 77%77%PlaceboPlacebo83%83% 77%77% 60%60% 56%56% 45%45%
Log-rank p-value=0.0165
7
Challenge for the Challenge for the Review TeamReview Team
Given that the study had failed on the Given that the study had failed on the primary endpoint, determine if the primary endpoint, determine if the
observed survival difference observed survival difference attributable to Cyclosporine Inhalation attributable to Cyclosporine Inhalation
Solution.Solution.
Review found Several Major Review found Several Major WeaknessesWeaknesses
8
Baseline ImbalancesBaseline Imbalances Though randomized, due to small Though randomized, due to small
sample size, the study failed to benefit sample size, the study failed to benefit from randomization – several baseline from randomization – several baseline characteristics that clinicians consider characteristics that clinicians consider to influence patient survival are not to influence patient survival are not balanced between the two groups. balanced between the two groups.
Therefore, groups are not comparable Therefore, groups are not comparable for evaluating survival or chronic for evaluating survival or chronic rejection. rejection.
9
Observed Important Baseline Observed Important Baseline Imbalances Imbalances
Statistical Adjustments “one factor at a Statistical Adjustments “one factor at a time” (Applicant’s briefing document and time” (Applicant’s briefing document and presentation) OR analyses with some (not presentation) OR analyses with some (not all) of the factors included are NOT all) of the factors included are NOT appropriate as the two groups are still not appropriate as the two groups are still not comparable due to imbalance with respect comparable due to imbalance with respect to the other factors. to the other factors.
The sample size of the study is too small The sample size of the study is too small for simultaneous adjustment for all the for simultaneous adjustment for all the factors of importance that show factors of importance that show imbalance. imbalance.
12
The questionable The questionable validity of any further validity of any further inferential statistical inferential statistical
analyses performed on analyses performed on the data from this the data from this
study lends to caution study lends to caution while interpreting the while interpreting the
results.results.
13
Blinding ?Blinding ? ABCD code as part of the patient ABCD code as part of the patient
number may have revealed the number may have revealed the treatment assignment to the treatment assignment to the investigators.investigators.
This together with the retrospective This together with the retrospective nature of the study makes it nature of the study makes it vulnerable to the introduction of bias, vulnerable to the introduction of bias, as inadvertent as it may be. as inadvertent as it may be.
14
Detection of the Detection of the presence and presence and
magnitude of bias is magnitude of bias is difficult, but its difficult, but its
possibility lends to possibility lends to caution when caution when
interpreting the interpreting the study results.study results.
15
An Illustration An Illustration of the effect of of the effect of
small small perturbations in perturbations in the assignment the assignment
Number of DosesNumber of DosesNumber of Doses Number of Doses CyIS CyIS
(n=26)(n=26)PG PG (n=30)(n=30)
1 to <10 doses1 to <10 doses 3 (12%)3 (12%) 3 (10%)3 (10%)10 to < 25 doses10 to < 25 doses 3 (12%)3 (12%) 2 (7%)2 (7%)25 to < 42 doses25 to < 42 doses -- 1 (3%)1 (3%)42 to < 78 doses42 to < 78 doses 3 (12%)3 (12%) 2 (7%)2 (7%)78 to <150 doses78 to <150 doses -- 3 (10%)3 (10%)150 to < 222 doses150 to < 222 doses 3 (12%) 3 (12%) 4 (13%)4 (13%)222 to < 294 doses222 to < 294 doses 7 (27%)7 (27%) 6 (20%)6 (20%)≥ ≥ 294 doses294 doses 7 (27%)7 (27%) 9 (30%)9 (30%)
21
Sensitivity analysesSensitivity analyses All 11 patients excludedAll 11 patients excluded All 6 CyIS patients treated as PG All 6 CyIS patients treated as PG
patientspatients Define Treatment Failure as Death Define Treatment Failure as Death
or Discontinuation within 25 doses or Discontinuation within 25 doses due to AE or Withdrawal of Consentdue to AE or Withdrawal of Consent
22
0.0
0.2
0.4
0.6
0.8
1.0
0 6 12 18 24 30 36 42 48 54 60 66Month
Surv
ival
Pro
babi
lity CyIS
PG
Survival Distributions Survival Distributions Patients with less than 25 doses Patients with less than 25 doses
excludedexcluded
Log-rank p-value=0.202
Hazard Ratio = .5195% CI = (.16, 1.4)
23
Sensitivity analysesSensitivity analyses All 11 patients excludedAll 11 patients excluded All 6 CyIS patients treated as PG All 6 CyIS patients treated as PG
patientspatients Define Treatment Failure as Death Define Treatment Failure as Death
or Discontinuation within 25 doses or Discontinuation within 25 doses due to AE or Withdrawal of Consentdue to AE or Withdrawal of Consent
24
0.0
0.2
0.4
0.6
0.8
1.0
0 6 12 18 24 30 36 42 48 54 60 66Month
Surv
ival
Pro
babi
lity CyIS
PG
Survival Distributions Survival Distributions Patients with less than 25 doses Patients with less than 25 doses
are treated as PG patientsare treated as PG patientsLog-rank p-value=0.194
Hazard Ratio = .5295% CI = (.17, 1.34)
25
Sensitivity analysesSensitivity analyses All 11 patients excludedAll 11 patients excluded All 6 CyIS patients treated as PG All 6 CyIS patients treated as PG
patientspatients Define Treatment Failure as Define Treatment Failure as
Death or Discontinuation within Death or Discontinuation within 25 doses due to AE or 25 doses due to AE or Withdrawal of ConsentWithdrawal of Consent
Distributions of Time to Distributions of Time to Treatment Failure*Treatment Failure*
Survival Probability Survival Probability MonthMonth 1212 2424 3636 4848 6060CyISCyIS 73%73% 65%65% 65%65% 61%61% 55%55%PGPG 77%77% 70%70% 53%53% 53%53% 43%43%
Log-rank p-value=0.556
* Treatment Failure = Death or Discontinuation Within 25 Doses due to Adverse Event or Withdrawal of Consent
27
The data from patients who The data from patients who discontinued early and are discontinued early and are
not expected to have any not expected to have any influence of the treatment influence of the treatment they are assigned to, have they are assigned to, have a big impact on the study a big impact on the study results and that lends to results and that lends to
caution when interpreting caution when interpreting the study results.the study results.
28
Lung Function – FEV1Lung Function – FEV1Number in bracket is sample sizeNumber in bracket is sample size
1.371 (17)1.371 (17)Mean(FEV1_3m)Mean(FEV1_3m) 33 2.175 (26)2.175 (26) 1.787 (26)1.787 (26)Mean(FEV1_6m)Mean(FEV1_6m) 66 2.326 (25)2.326 (25) 1.912 (27)1.912 (27)Mean(FEV1_12m)Mean(FEV1_12m) 1212 2.268 (24)2.268 (24) 1.867 (24)1.867 (24)Mean(FEV1_18m)Mean(FEV1_18m) 1818 2.135 (22)2.135 (22) 1.789 (22)1.789 (22)Mean(FEV1_24m)Mean(FEV1_24m) 2424 2.123 (20)2.123 (20) 1.643 (19)1.643 (19)______________________________________________________________________________________________Information on Pre-enrollment FEV1 is grossly Information on Pre-enrollment FEV1 is grossly
incomplete – 43% missing – Difference in pre-incomplete – 43% missing – Difference in pre-enrollment FEV1 between CyIS and Propylene enrollment FEV1 between CyIS and Propylene Glycol is statistically significant. Glycol is statistically significant.
29
Lung Function – FEV1Lung Function – FEV1
1.0
1.5
2.0
2.5
0 6 12 18 24Month
Mea
n FE
V1
CyIS(n=26)
PG(n=30)
17
26
27 2
4 22 1
9
15
26
25 24 22 2
0
30
0.0
0.2
0.4
0.6
0.8
1.0
0 6 12 18 24 30 36 42 48 54Month
Surv
ival
Pro
babi
lity
CyIS
PG
Time to BOS*Time to BOS*Log-rank p-value=0.214
No at RiskCyIS 26 26 24 20 16 12 11 9 5 0PG 30 27 21 18 14 8 6 4 3 1* Patients who died without diagnosis of BOS, as defined by the Applicant, were censored at the time of last follow-up for BOS.
31
Additional DataAdditional Data 10 patients in Study ACS001 in CyIS open 10 patients in Study ACS001 in CyIS open
label phase – Limited informationlabel phase – Limited information
Baseline CharacteristicsBaseline Characteristics CyIS- OLCyIS- OL CyIS-R CyIS-R PG PG
EmphysemaEmphysema 70%70% 35% 35% 63%63%Time in ICU > 10 daysTime in ICU > 10 days 0% 4% 23%0% 4% 23%
32
0.0
0.2
0.4
0.6
0.8
1.0
0 6 12 18 24 30 36 42 48 54 60 66 72 78 84
Month
Surv
ival
Pro
babi
lity
CyIS- Random
PGPGCyIS-OL
Survival DistributionsSurvival Distributions
33
SummarySummary Primary Endpoint of Acute Rejection - Primary Endpoint of Acute Rejection - No No
Treatment EffectTreatment Effect Risk Factor Imbalance - Risk Factor Imbalance - CautionCaution Blinding Concerns - Blinding Concerns - CautionCaution Sensitivity to small perturbations in assignment - Sensitivity to small perturbations in assignment -
HighHigh Discontinuations - Discontinuations - CautionCaution Lung Function - FEV1 and BOS - Lung Function - FEV1 and BOS - No Treatment No Treatment
EffectEffect Additional Data - Additional Data - Not supportiveNot supportive
34
The Question Still The Question Still Remains…Remains…
Is the Observed Survival Difference, between Is the Observed Survival Difference, between the two arms in the randomized portion of the two arms in the randomized portion of
the trial, the trial, due to Cyclosporine Inhalation Solution due to Cyclosporine Inhalation Solution
