2008/2/27 1

FDA/INDUSTRY STATISTICS WORKSHOP: FDA/INDUSTRY STATISTICS WORKSHOP: Washington, D.C. Sept. 29, 2006Washington, D.C. Sept. 29, 2006

Statistical Issues in Medical Device Statistical Issues in Medical Device TrialsTrials

George Koustenis, FDA-CDRHDavid Breiter, Boston ScientificRoseann White, Abbott VascularGeorge Woodworth, Univ. of Iowa

2008/2/27 2

The Use of Objective Performance Criteria The Use of Objective Performance Criteria (OPC) in Medical Device Evaluation(OPC) in Medical Device Evaluation

FDA/Industry Statistics WorkshopSeptember 29, 2006Washington, D.C.

George KoustenisDivision of Biostatistics

Office of Surveillance and BiometricsCenter for Devices and Radiological Health

U.S. Food and Drug Administration

2008/2/27 3

OBJECTIVEOBJECTIVE

The purpose of this presentation is to provide a brief introduction to the possible use of Objective Performance Criteria (OPC) in the evaluation of medical devices during the regulatory approval process.

2008/2/27 4

DEFINITION OF OPCDEFINITION OF OPC

Fixed Target(s)Objective and Meaningful StandardProvides Comparison in Evaluating Safety and EffectivenessUsually a RateSurrogate for Control GroupBenchmark for Minimally Acceptable ValuesNot a Control Group

2008/2/27 5

CONTROLS CONTROLS

Standard Comparison for an Experimental TreatmentGroup of Patients with Same Condition, Demographics, & Prognostic ValuesBy Controlling for as Many Variables as Possible – Any Differences are Presumed Due to New Intervention

2008/2/27 6

TYPES OF CONTROLSTYPES OF CONTROLS

Concurrent Randomized Controlled Trial (RCT)The Gold StandardEffectively Minimizes BiasEffectively Balances Demographics & UnknownsSupports Basic Assumptions of Standard Statistical Methodology

2008/2/27 7

Historical ControlsHistorical Controls

Compares Current Therapy & Patients Against Others Studied in Previous InvestigationsNot Randomized to Current ConditionsMay Be Too Far Removed in TimeVery Difficult to Validate the Data from Historical TrialsTend to Produce More Positive Results

2008/2/27 8

ESTIMATES OF OPCESTIMATES OF OPC’’SS

Necessarily Driven by Historical DataRequires Appropriate Pooling of Different InvestigationsOPC’s Inherit All of Problems Seen with Historical ControlsEven Sophisticated Pooling/Analytical Techniques Cannot Eliminate Basic Problems

2008/2/27 9

USE OF OPCUSE OF OPC’’s In CDRHs In CDRH

Some Current Use of OPC’sCardiac Ablation CathetersReplacement Heart ValvesOphthalmicsHip Replacement Systems

2008/2/27 10

USE OF OPCUSE OF OPC’’s In DEVICE s In DEVICE TRIALSTRIALS

Given that the use of OPC represents a significant departure from the standard scientific approach to the design and analysis of medical device clinical trials, the question now becomes: when might it be appropriate to use this type of non-control comparison in the medical device approval process?

2008/2/27 11

CITED ADVANTAGES Of OPCCITED ADVANTAGES Of OPC’’ss

Smaller Sample SizeStandard Value for All SponsorsSave Time and MoneyEasier to ExecuteI.E. – “Least Burdensome”

2008/2/27 12

DISADVANTAGES Of OPCDISADVANTAGES Of OPC’’ss

All the Problems Associated with Historical ControlsProblems with Validity of Data & AnalysisProblems with Advances in Practice of MedicineMay be Disagreement on Final OPC ValueProblems with Single Arm TrialsSelection BiasMay Not See Pre-Post Benefits

2008/2/27 13

DISADVANTAGES Of OPCDISADVANTAGES Of OPC’’ss

May Not Be Least BurdensomeSmaller Sample Size? N=100 or 150?Time and Resource Intensive to DevelopSets a Minimum StandardSuperiority?Older and Older DataProtect and Promote the Public Health?

2008/2/27 14

WHEN MIGHT OPCWHEN MIGHT OPC’’s Be USED?s Be USED?

History of OPC’s in Medical Devices

Non-Experimental or Quasi-Experimental Designs May be Valid in Certain Situations

2008/2/27 15

MINIMUM REQUIREMENTS MINIMUM REQUIREMENTS WHEN OPCWHEN OPC’’s MIGHT Be USED s MIGHT Be USED

Great Deal is Known About the Natural History of the Disease or ConditionUnderlying Patient Population is Well Described & Relatively StableExtensive Clinical History & Experience with This Device

2008/2/27 16

MINIMUM REQUIREMENTS MINIMUM REQUIREMENTS WHEN OPCWHEN OPC’’s MIGHT Be USED s MIGHT Be USED (cont.)(cont.)

Stable and Well Known Standard of CareAppropriate Ancillary Technology is Relatively StableNo Significant New Questions of Safety or Effectiveness Consensus Among FDA, Industry, Clinical, Academic and Patient CommunitiesExpectation of Significantly Positive Treatment Effect

2008/2/27 17

HOW SHOULD OPCHOW SHOULD OPC’’s BE s BE DERIVED? DERIVED?

Data DrivenRigorous And Scientifically Valid Methodologies Valid DatabasesAppropriate Statistical ModelingAppropriately Designed and Powered Pivotal Trial Periodical Re-Evaluation and Updating the OPC’s

2008/2/27 18

OTHER IMPORTANT OPC OTHER IMPORTANT OPC ISSUESISSUES

All Parties Must Have a Clear Understanding of the OPC in Medical Device Trials – Success and FailureOPC’s Already in Limited Use in Device

TrialsNo Current General Policy of OPC Use in CDRHNeed for Comprehensive, Coherent, and Consistent OPC Policy for Device Trials

2008/2/27 19

WHAT SHOULD THIS POLICY WHAT SHOULD THIS POLICY CONTAIN?CONTAIN?

Clear Definitions of All Appropriate TermsEstablished Guidance on Minimum Standards for a Device or Device ClassSpecific Roles Played by FDA, Industry, Clinical Community, Academia and Other Interested PartiesProvisions for Periodic Updating of OPCSpecific Guidance on Methodology to Derive an OPCUnambiguous Policy Regarding Failure to Meet OPC

2008/2/27 20

SUMMARYSUMMARY

An introduction to the history and use of Objective Performance Criteria (OPC) in medical device evaluationAdvantages and disadvantages of OPC’sWhen and how OPC might be used & developedDiscussed the need for developing a CDRH policy regarding use of OPC in medical device trialsGeneral outline of this policy is presented in an effort to begin dialogue between CDRH, industry, clinical community & other interested parties

2008/2/27 21

REFERENCES REFERENCES

Please see accompanying brief commentary

Request electronic copy from george.koustenis@fda.hhs.gov

2008/2/27 22

Q&AQ&A