11
Updates on Regulatory Updates on Regulatory Requirements for Missing Requirements for Missing
DataData
Ferran Torres, MD, PhDFerran Torres, MD, PhD
Hospital Clinic BarcelonaHospital Clinic Barcelona
Universitat Autònoma de Universitat Autònoma de BarcelonaBarcelona
22
DocumentationDocumentation
Power Point presentationPower Point presentation Direct links to guidelinesDirect links to guidelines List of selected relevant referencesList of selected relevant references
http://http://ferran.torres.name/edu/diaferran.torres.name/edu/dia
33
DisclaimerDisclaimer
The views expressed here are those The views expressed here are those of of the author and may not necessary the author and may not necessary reflect those of any of the following reflect those of any of the following institutions he is related to:institutions he is related to:
– Spanish Medical Agency - AEMPSSpanish Medical Agency - AEMPS– EMEA (SAWP; EWP)EMEA (SAWP; EWP)– Hospital Clinic BarcelonaHospital Clinic Barcelona– Autonomous University of BarcelonaAutonomous University of Barcelona
44
Regulatory guidance concerning Regulatory guidance concerning MDMD
1998: ICHE9. Statistical Principles for Clinical Trials
2001: PtC on Missing Data
Dec-2007-2008: Recommendation for the Revision of the PtC on MD
2009: Release for consultation
55
ICH-E9 ICH-E9 (3,6)(3,6)
Key pointsKey points::– Potential source of biasPotential source of bias– Common in Clinical TrialsCommon in Clinical Trials– Avoiding MDAvoiding MD– Importance of the methods Importance of the methods – Pre-specificationPre-specification– Lack of universally accepted method for Lack of universally accepted method for
handlinghandling– Sensitivity analysisSensitivity analysis– Identification and description of missingnessIdentification and description of missingness
66
Status in early 2000sStatus in early 2000s
In general, MD was not seen as a In general, MD was not seen as a source of bias:source of bias:– considered mostly as a considered mostly as a loss of power issueloss of power issue– little efforts in avoiding MDlittle efforts in avoiding MD
Importance of the Importance of the methodsmethods for dealing for dealing with:with:– Available Data OnlyAvailable Data Only– Handling of missingness: Mostly Handling of missingness: Mostly LOCFLOCF, ,
Worst CaseWorst Case
77
Status in early 2000sStatus in early 2000s
Very few information on the handling Very few information on the handling of MD in protocols and SAP (of MD in protocols and SAP (little pre-little pre-specificationspecification))
Lack of Sensitivity analysisLack of Sensitivity analysis, or only , or only one, and no justificationone, and no justification
Lack (little) identification and Lack (little) identification and description of missingnessdescription of missingness in reports in reports
88
99
PtC on MDPtC on MD
StructureStructure
• IntroductionIntroduction
• The effect of MD on data analysisThe effect of MD on data analysis
• Handling of MDHandling of MD
• General recommendationsGeneral recommendations
1010
Main PointsMain Points AvoidanceAvoidance of MD of MD
BiasBias: specially when MD was related to : specially when MD was related to the outcomethe outcome
Methods:Methods:– Warning on the Warning on the LOCFLOCF– Open the door to other methodsOpen the door to other methods::
Multiple imputation, Mixed Models…Multiple imputation, Mixed Models…
Sensitivity analysisSensitivity analysis
1111
Current status in 2008-9Current status in 2008-9Missing data remains a problem in
protocols and final reports:
Little or no critical discussion on pattern of MD data and withdrawals
None / only one sensitivity analysis
Methods:– Inappropriate methods for the handling of MD– LOCF: Still used as a general approach for too
many situations– Methods with very little use in early 2000 are
now common (Mixed Models)
1212
1313
New Draft PtCNew Draft PtC1.1. Executive SummaryExecutive Summary
2.2. IntroductionIntroduction
3.3. The Effect of MD on the Analysis & the InterpretationThe Effect of MD on the Analysis & the Interpretation
4. General Recommendations4. General Recommendations4.1 Avoidance of Missing Data4.1 Avoidance of Missing Data4.2 Design of the Study. Relevance Of Predefinition4.2 Design of the Study. Relevance Of Predefinition4.34.3 Final Report Final Report
5.5. Handling of Missing Data Handling of Missing Data 5.15.1 Theoretical FrameworkTheoretical Framework5.25.2 Complete Case Analysis Complete Case Analysis5.3 5.3 Methods for Handling Missing DataMethods for Handling Missing Data
6.6. Sensitivity AnalysesSensitivity Analyses
1414
Statistical frameworkStatistical framework
applicability of methods based on a applicability of methods based on a classification according to missingness classification according to missingness generation mechanisms:generation mechanisms:
– missing completely at random (missing completely at random (MCARMCAR) ) – missing at random (missing at random (MARMAR) ) – missing not at random (missing not at random (MNARMNAR) )
1515
36
32
28
24
20
16
12
8
4
0 2 4 6 8 10 12 14 16 18 Time (months)
> Worse
< Better
Options after withdrawalOptions after withdrawal
1616
Options after withdrawalOptions after withdrawal Ignore that information completely: Ignore that information completely: Available Data OnlyAvailable Data Only
approachapproach
To “force” To “force” data retrieval?data retrieval?::– ““Pure” estimates valid only when no treatment alternatives are Pure” estimates valid only when no treatment alternatives are
availableavailable– Otherwise the effect will be contaminated by the effect of Otherwise the effect will be contaminated by the effect of
other treatmentsother treatments
Single Imputation methodsSingle Imputation methods
MARMAR methods: methods:– Mixed-effect models for repeated measures (Mixed-effect models for repeated measures (MMRMMMRM))
MNARMNAR methods methods
1717
Single imputation methodsSingle imputation methods LOCF, BOCF and othersLOCF, BOCF and others
Many problems described in the previous PtCMany problems described in the previous PtC
Their potential for Their potential for bias bias depends on many factorsdepends on many factors– including true evolutions after dropoutincluding true evolutions after dropout– Time, reason for withdrawal and proportion of Time, reason for withdrawal and proportion of
missingness in the treatment arm missingness in the treatment arm – they do not necessarily yield a conservative estimation they do not necessarily yield a conservative estimation
of the treatment effectof the treatment effect
The imputation may distort the The imputation may distort the variance and the variance and the correlationscorrelations between variables between variables
1818
MMRM (and others MAR)MMRM (and others MAR) MAR MAR assumptionassumption
– MD depends on the observed dataMD depends on the observed data
– the behaviour of the post drop-out the behaviour of the post drop-out observations can observations can be predicted with the observed databe predicted with the observed data
– It seems It seems reasonablereasonable and it is and it is not a strong assumptionnot a strong assumption, , at least at least a prioria priori
– In RCT, the reasons for withdrawal are knownIn RCT, the reasons for withdrawal are known
– Other assumptions seem stronger and more arbitraryOther assumptions seem stronger and more arbitrary
1919
However…However… It is reasonable to consider that the treatment It is reasonable to consider that the treatment
effecteffect will somehow will somehow cease/attenuatecease/attenuate after after withdrawalwithdrawal
If there is a If there is a good responsegood response, MAR , MAR will not will not “predict” a bad response“predict” a bad response
=>MAR assumption =>MAR assumption not suitablenot suitable for early drop- for early drop-outs because of safety issuesouts because of safety issues
In this context MAR seems likely to be In this context MAR seems likely to be anti-anti-conservativeconservative
2020
The main analysis: The main analysis: What should reflect ? What should reflect ?
A) The “A) The “purepure” treatment effect:” treatment effect:– Estimation using the “on treatment” effect after Estimation using the “on treatment” effect after
withdrawal withdrawal – Ignore effects (changes) after treatment Ignore effects (changes) after treatment
discontinuationdiscontinuation– Does not mix up efficacy and safetyDoes not mix up efficacy and safety
B) The expected treatment effect in B) The expected treatment effect in “usual “usual clinical practice” clinical practice” conditionsconditions
2121
MARMAR
MMRM aims to estimate the treatment effect MMRM aims to estimate the treatment effect that would be seen if all patients had continued that would be seen if all patients had continued on the study as planned. on the study as planned.
In that sense MMRM results could be seen as In that sense MMRM results could be seen as not not fully compliant with the ITT principle fully compliant with the ITT principle
Regulatory assessment Regulatory assessment is focused onis focused on what could what could be expected "be expected "on averageon average" in a population, where " in a population, where not all patients have complied with the assigned not all patients have complied with the assigned treatmenttreatment for the full duration of the trial for the full duration of the trial
2222
Description of MDDescription of MD
Detailed description (numerical and graphical):Detailed description (numerical and graphical):
Pattern of MDPattern of MD
Rate and time of withdrawalRate and time of withdrawal– By reason, time/visit and treatmentBy reason, time/visit and treatment– Some withdrawals will occur between visits: use Some withdrawals will occur between visits: use
survival methodssurvival methods
OutcomeOutcome– By reason of withdrawal and also for completersBy reason of withdrawal and also for completers
2323
General recommendationsGeneral recommendations Sensitivity analysis (there is a new separate section)Sensitivity analysis (there is a new separate section)
Avoidance of MDAvoidance of MD
DesignDesign– Relevance of predefinition (Relevance of predefinition (avoid data-driven methods )avoid data-driven methods )– detailed detailed descriptiondescription– and and justificationjustification of absence of bias in favour of experimental of absence of bias in favour of experimental
treatmenttreatment
Final ReportFinal Report– Detailed description of the planned and amendments of the Detailed description of the planned and amendments of the
predefined methodspredefined methods
2424
Sensitivity Analyses Sensitivity Analyses One specific sectionOne specific section
a a set of analysset of analyseess showing the influence of different showing the influence of different methods of handling missing data on the study resultsmethods of handling missing data on the study results
Pre-defined and designed to Pre-defined and designed to assess the repercussion assess the repercussion on the resultson the results of the particular assumptions made in of the particular assumptions made in the handling of missingnessthe handling of missingness
Responder analysisResponder analysis
Sensitivity analysSensitivity analysees may give s may give robustnessrobustness to the to the conclusionsconclusions
2525
Concluding RemarksConcluding Remarks
Avoid and foresee MDAvoid and foresee MD
Sensitivity analysSensitivity analyseess
Methods for handling:Methods for handling:– No gold standard for every situationNo gold standard for every situation– In principle, “almost any method may be In principle, “almost any method may be
valid”:valid”:– =>But their appropriateness has to be justified=>But their appropriateness has to be justified
2626
2727
2828
2929
3030
http://http://ferran.torres.name/edu/diaferran.torres.name/edu/dia
