The Failing Clinical Trial Enterprise in the US: Efforts of CTTI to Improve our National and Global Evidence Generating Capability

Robert M Califf MDVice Chancellor for Clinical Research, Duke University

The Official Talk

3

US Clinical trials in crisis

Trial start-up times lengtheningEnrollment slowingCosts increasingMany investigators pulling out of clinical research

4

5

Clinical Trial Cost Estimates

$0

$50

$100

$150

$200

$250

$300

$350

$400

$450

T o tal

Co o rd in atin g Cen ter

S ite Paym en ts

Oth er

Full Cost Industry

Streamlined Industry

More Streamlined

$ In US 2007 Millions

6

The Globalization of Clinical Investigators

80% 77%70%

62%57%

9% 10%

11%

13%14%

5%12% 13%

19%25% 29%

86%

9%

0%

25%

50%

75%

100%

1997 1999 2001 2003 2005 2007P

US-Based Western Europe Rest of WorldPercent of Total1572s Filed

Sources: Tufts CSDD

7

Which Treatment is Best for Whom?High-Quality Evidence is Scarce

7 Tricoci P et al. JAMA 2009;301:831-41Tricoci P et al. JAMA 2009;301:831-41

ACC/AHA Grading SchemaACC/AHA Grading SchemaClassification of Recommendations and Level of EvidenceClassification of Recommendations and Level of Evidence

Class of Recommendation: Class of Recommendation: value judgment by guidelines authorsvalue judgment by guidelines authors

Level of Evidence: Level of Evidence: objective description of existence/types of supporting studiesobjective description of existence/types of supporting studies

Level of Evidence ALevel of Evidence ACurrent Guidelines*Current Guidelines*

*Guidelines expressing *Guidelines expressing Level of EvidenceLevel of Evidence

*Guidelines expressing *Guidelines expressing Level of EvidenceLevel of Evidence

11.7%11.7%

26.4%26.4%

15.3%15.3%

13.5%13.5%

12.0%12.0%

22.9%22.9%

6.4%6.4%

6.1%6.1%

23.6%23.6%

0.3%0.3%

9.7%9.7%

11.0%11.0%

19.0%19.0%

4.9%4.9%

4.8%4.8%

0%0% 10%10% 20%20% 30%30%

AFAF

Heart failureHeart failure

PADPAD

STEMISTEMI

PerioperativePerioperative

Secondary preventionSecondary prevention

Stable anginaStable angina

SV arrhythmiasSV arrhythmias

UA/NSTEMIUA/NSTEMI

Valvular diseaseValvular disease

VA/SCDVA/SCD

PCIPCI

CABGCABG

PacemakerPacemaker

Radionuclide imagingRadionuclide imaging

Level of Evidence CLevel of Evidence CCurrent Guidelines*Current Guidelines*

*Guidelines expressing *Guidelines expressing Level of EvidenceLevel of Evidence

*Guidelines expressing *Guidelines expressing Level of EvidenceLevel of Evidence

58.6%58.6%

54.3%54.3%

25.1%25.1%

47.2%47.2%

32.0%32.0%

8.3%8.3%

54.5%54.5%

56.5%56.5%

29.6%29.6%

70.6%70.6%

58.5%58.5%

47.8%47.8%

20.0%20.0%

58.2%58.2%

26.3%26.3%

0%0% 20%20% 40%40% 60%60% 80%80%

AFAF

Heart failureHeart failure

PADPAD

STEMISTEMI

PerioperativePerioperative

Secondary preventionSecondary prevention

Stable anginaStable angina

SV arrhythmiasSV arrhythmias

UA/NSTEMIUA/NSTEMI

Valvular diseaseValvular disease

VA/SCDVA/SCD

PCIPCI

CABGCABG

PacemakerPacemaker

Radionuclide imagingRadionuclide imaging

11

It’s a “Systems Problem”

All members of the clinical research enterprise have played a part in this problem

Fixing it will require a collaborative effortFDA/global regulatorsIndustryAcademia/NIHInvestigators in clinical practiceConsumers

12

A collaborative effort to find solutions

U.S. FDA’s Office of Critical Path Programs established a public-private partnership:

The Clinical Trials Transformation Initiative (CTTI)

All stakeholders involved

Through a memorandum of understanding with FDA, Duke University serves as the host of CTTI

13

Executive Committee

Co-Chairs: Rob Califf(Duke) and Rachel Behrman(FDA)

Academia: David DeMets

At-large representative: Ken Getz

FDA: Bob Temple, CDER and Bram Zuckerman, CDRH

Industry: Glenn Gormley, Jay Siegel, Susan Alpert, Alberto Grignolo

Patient representative: Nancy Roach

NIH liaison: Amy Patterson

Non-US regulatory liaison: Hans-Georg Eichler, EMEA

Ex-Officio Members: James Ferguson and Briggs Morrison, Steering Committee Co-chairs; Judith Kramer, Executive Director

14

Steering Committee Representation

Category # organizations

Academic institutions 9

Pharmaceutical companies 8

Clinical research organizations 7

Professional societies 6

Government 6 (FDA [OC,CDER, CBER, CDRH] AHRQ, CMS, NIH, OHRP, VA)

Device companies 5

Biotechnology companies 3

Clinical investigator groups 3

Trade organizations 3

Patient representatives/at-large 3

Private equity firm 1

Regulatory law firm 1

*Began recruiting members May 2008

15

Mission

To identify practices that through broad adoption will increase the quality and efficiency of clinical trials

16

Scope

CTTI will generate evidence about how to improve the design and execution of clinical trials

CTTI will foster widespread change based on evidence

CTTI was created to address a crisis for US clinical research, however…

Trials and issues are globalCTTI seeks to identify practice improvements that can be applied internationallyCTTI is engaging international collaborators

17

Strategy to Accomplish our Mission

Aggressively pursue development of evidenceConduct “research on research”Pursue collaborative activities with other organizations sharing similar goals

Parallel activities:Systematically analyze the clinical trials process and potential impact of our activitiesMaintain awareness of other effortsPromote adoption of CTTI recommendations

18

Initial Priority Areas* for Research on Research

Design principles

Data quality and quantity (including monitoring)

Study start-up

Adverse event reporting

*Defined by CTTI’s Executive Committee

19

Status of CTTI Projects

2 Ongoing Projects (with 7 workstreams)

Effective and efficient monitoring as a component of quality assurance in the conduct of clinical trials

Improving the system of reporting and interpreting unexpected serious adverse events (SAEs) to investigators conducting research under an IND

2 Project plans in development

Executive and Steering Committees held a brainstorming session in September

3 Collaborations established

20

Effective and Efficient Monitoring Project

Goal Identify best practices and provide sensible criteria

to help sponsors select the most appropriate monitoring methods for a trial, thereby improving quality while optimizing resources

Specific objectives1. Describe the range of current monitoring practices

and examine factors that drive their adoption

2. Define key quality objectives for clinical trials

3. Illustrate strengths and weaknesses of the various monitoring practices in meeting quality objectives for a range of clinical trial settings

21

Effective and Efficient Monitoring Project Organization

Effective andEfficient Monitoring…

Rachel Behrman, MD, MPH

FDACo-Team Leaderand WS 2 Leader

Martin Landray, PhD, MRCP

University of OxfordCo-Team Leaderand WS 3 Leader

Briggs Morrison, MD Pfizer Inc.

Co-Team Leader and WS 1 Leader

David NickersonProject Manager

Pfizer Inc.

Melissa RobbSenior Program Mgr.

FDA

Workstream 1 Team

Workstream 3 Team

Workstream 2 Team

Project Mgmt

22

Improving SAE Reporting to IND Investigators

Goal: To generate empirical evidence about the current

US system for reporting SAEs to investigators under an IND

Consider potential modifications of the current system to more efficiently and effectively inform investigators of these events

23

Improving SAE Reporting to IND Investigators

Subprojects1.Document current range of sponsor practices for:

a. Reporting unexpected SAEs to investigators; b. Oversight of product safety (eg DSMBs, safety

committees)2. a. Quantify investigators’ time spent receiving,

interpreting, and communicating individual expedited reports

b. Assess perceived value to investigators of individual expedited reports in updating product’s risk profile

3. Compare current practice of submitting individual SAEs with an alternative approach based on European Commission’s guidance

4. Convene an expert group to integrate results and recommend ways to optimize reporting of SAEs to investigators and assure subject protection

24

Collaborations

Use of clinical trials in evaluation of comparative effectiveness

Collaboration between the Center for Medical Technology Policy (CMTP), Pragmatic Approaches to Comparative Effectiveness (PACE), and CTTIExpert meeting held May 6, 2009—directed to policy-makers

— New approaches to clinical trials will make them more attractive for comparative effectiveness research

— Manuscript proposing increased operational efficiency, analytical efficiency, and generalizability of clinical trials published in Aug. 4th issue Annals of Internal Medicine

25

Collaborations (continued)

FDA-initiated training course directed to clinical investigators

Collaborative effort to standardize definitions and data collection methods/case report forms for cardiovascular trials

FDA-initiated effort involving academics, industry, CDISC, and HL7

26

How do we effect widespread change?

Problems with clinical trials widely recognized

Need to go beyond elucidation of problems to effect change

Current behavior often driven by incentives and fears not consistent with the goal of increased efficiency Decision makers at different levels have variable understanding of the “big picture”

— Organizations engage in activities that may not add value

— Trade off of cost vs. value not explicitWide spread perception that clinical trials just take that long and cost that much!

27

How do we effect widespread change?

CTTI’s Approach

Involve all sectors in selection, conduct, and interpretation of projects

Explore the business case for change from different perspectives

Keep dialogue open across sectors

Provide evidence that can influence regulatory guidance

Attempt to create a “level playing field” when recommending change (i.e. don’t place a single organization or sector at risk)

28

For more information….

CTTI Website-Homewww.trialstransformation.org

Projectswww.trialstransformation.org/projects

Member organizationswww.trialstransformation.org/members/member-organizations/

How to joinwww.trialstransformation.org/join

Clinical Trials 2008; 5: 38-39 Sensible Guidelines Conference January 25-26, 2007

Sensible guidelines for the conduct of large randomized trials

Large trials are importantLarge trials are important

Complexity and cost Complexity and cost increasingincreasing

GCP ≠ good, or clinically GCP ≠ good, or clinically relevant, or even relevant, or even practicalpractical

Excess data, number of Excess data, number of visits, onsite monitoringvisits, onsite monitoring

Layers of ethics Layers of ethics approvalsapprovals

“For a scientific method that is at the heart of evidence-based medicine, no good evidence that the layers of complexity, approvals, processes, and laws to protect subjects have actually achieved their purpose.

What is clear is that such processes are extremely expensive and delay studies.

Goal: stimulate reform and simplification of clinical trials procedures, while enhancing patient safety and autonomy, improving the scientific validity and integrity of trials and making them more affordable.”

Clinical and Translational Science Award (CTSA)

At peak a $500 million investment by the NIH in translating scientific

discoveries to better human health

My Opinion—This does not represent CTTIMy Opinion—This does not represent CTTI

Disruptive innovation is needed to create a very Disruptive innovation is needed to create a very different system based on electronic data different system based on electronic data collection in practice with quality built in through collection in practice with quality built in through a systematic approach.a systematic approach.

The Cycle of Quality: Generating Evidence to The Cycle of Quality: Generating Evidence to Inform PolicyInform Policy

Califf RM et al, Califf RM et al, Health Affairs, 2007Health Affairs, 2007

Measurement Measurement andand

EducationEducation

Measurement Measurement andand

EducationEducation

Early Translational

Steps

Early Translational

Steps

ClinicalTrials

ClinicalTrials

ClinicalPractice

Guidelines

ClinicalPractice

Guidelines

PerformanceMeasures

PerformanceMeasures

OutcomesOutcomes

Discovery ScienceDiscovery Science

DataDataStandardsStandards

DataDataStandardsStandards

NetworkNetworkInformationInformation

NetworkNetworkInformationInformation

EmpiricalEmpiricalEthicsEthics

EmpiricalEmpiricalEthicsEthics

PrioritiesPrioritiesand Processesand Processes

PrioritiesPrioritiesand Processesand Processes

InclusivenessInclusivenessInclusivenessInclusiveness

Use forUse forFeedbackFeedback

on Prioritieson Priorities

Use forUse forFeedbackFeedback

on Prioritieson Priorities

Conflict-of-interestConflict-of-interestManagementManagement

Conflict-of-interestConflict-of-interestManagementManagement

Evaluation of SpeedEvaluation of Speedand Fluencyand Fluency

Evaluation of SpeedEvaluation of Speedand Fluencyand Fluency

Pay forPay forPerformancePerformance

Pay forPay forPerformancePerformance

TransparencyTransparencyto Consumersto ConsumersTransparencyTransparencyto Consumersto Consumers

FDAFDACritical PathCritical Path

FDAFDACritical PathCritical Path

NIH RoadmapNIH RoadmapNIH RoadmapNIH Roadmap12 3

4

5

6

7

8

910

11

12

The Learning Health SystemThe Learning Health System

Articulated goal of the Institute of MedicineArticulated goal of the Institute of Medicine

By implementing electronic health records, By implementing electronic health records, data warehouses and disease registries, every data warehouses and disease registries, every patient’s data will be used to further knowledgepatient’s data will be used to further knowledge

This means that all places of practice will This means that all places of practice will become research sitesbecome research sites

Research must become a normal part of clinical Research must become a normal part of clinical practice, not something done separately from practice, not something done separately from clinical practice (except for very special early clinical practice (except for very special early phase and highly controlled types of studies)phase and highly controlled types of studies)

% of Randomized Discontinued in A

0.00

5.00

10.00

15.00

20.00

25.00

30.00

35.00

Disc. May 06Disc. September 06

Protocol Adherence and Adverse EventsProtocol Adherence and Adverse Events

Non-Non-AdherenceAdherence

Withdrawal:Withdrawal:AE’sAE’s

Withdrawal: Withdrawal: Subject Subject preferencepreference

Withdrew Withdrew ConsentConsent

UKUK 39.7%39.7% 12%12% 17%17% 5555

USUS 35.5%35.5% 7%7% 16%16% 166166

Poland Poland 14.8%14.8% 2%2% 9%9% 22

ChinaChina 6.1%6.1% 5%5% 5%5% 11

ArgentinaArgentina 13.6%13.6% 2%2% 8%8% 11

Image Source: http://www.pandora.ca/pictures21/900666.jpg

The Demise of EmpiresThe Demise of Empires

Dominance at a point in timeDominance at a point in time

Arrogance about superiorityArrogance about superiority

Failure to pay attention to quality of workFailure to pay attention to quality of work

Leaders content to “ride the wave”Leaders content to “ride the wave”

Entrenched interests can buy stability through Entrenched interests can buy stability through controlling laws and regulationscontrolling laws and regulations

Inability to create or respond to innovationInability to create or respond to innovation

Cost of transactions exceeds cost of Cost of transactions exceeds cost of actually doing the work!actually doing the work!

Disruptive Innovation in Clinical TrialsDisruptive Innovation in Clinical Trials Electronic health recordsElectronic health records

Data warehouses in integrated health systemsData warehouses in integrated health systems

Learning health systemsLearning health systems

Use same information for clinical care and Use same information for clinical care and research research

Electronic permissions systems for Electronic permissions systems for participation in researchparticipation in research

Evaluation of RESEARCH SITES on a periodic Evaluation of RESEARCH SITES on a periodic basis with constant electronic surveillance and basis with constant electronic surveillance and periodic audits for causeperiodic audits for cause

Thank youThank you