Catrin Tudur Smith

Carrol Gamble

North West Hub for Trials Methodology Research

Outline Purpose of monitoring

Risk proportionate approach

Empirical example from cancer

Survey of current practice

Future work

Purpose of monitoring

Verify that :

(a) The rights and well-being of human subjects are protected.

(b) The reported trial data are accurate, complete, and

verifiable from source documents.

(c) The conduct of the trial is in compliance with the currently

approved protocol / amendment(s), with GCP and with the

applicable regulatory requirement(s).

(ICH GCP 5.18)

Approaches to trial monitoring

Oversight of the quality of the trial

o Central monitoring - use of centralised procedures for

quality control of trial dataIncludes checks for missing data across sites, exploring

unusual patterns in the data, checking accuracy of data with

external sources

o On-site monitoring – use of procedures for quality

control of trials data undertaken during on-site visits Includes source data verification: procedure used to check

that the data contained in the Case Report Form match the

primary source (e.g. medical record)

ICH GCP‘.... In general there is a need for on-site monitoring

before, during and after the trial; however in exceptional

circumstances the sponsor may determine that central

monitoring in conjunction with procedures such as

investigators’ training and meetings … can assure

appropriate

conduct of the trial in accordance with GCP’

(ICH GCP 5.18.3)

Risk assessment Process of identifying potential hazards, their

likelihood and harms

Risks associated with IMP

Safety risks from clinical procedures in the protocol

Risks related to participant rights

Risks to the reliability of trial results

Plans to mitigate or manage risks

trial management and monitoring plans

Determine the intensity and focus of monitoring

Risk proportionate approaches Joint working group (http://www.mhra.gov.uk/home/groups/l-ctu/documents/websiteresources/con111784.pdf)

Develop a risk assessment that gives practical guidance on risk adaptations.

Considered in relation to what is known about the IMP and standard medical care (Brosteanu et al. Clinical Trials 2009:585-596)

Identify lower risk trials where simplification is possible for regulatory approvals and trial conduct

IMP risk category and safety monitoring plan be submitted to the MHRA with the CTA

Type A: no higher than that of standard medical care

• Medicinal products licensed in any EU Member State if they relate to the licensed range of indications, dosage and form

• Off-label use if this off-label use is established practice and supported by published evidence/guidelines

Type B: somewhat higher than that of standard medical care Medicinal products licensed in any EU Member State

if : used for a new indication* (patient pop/disease group)• Substantial dosage modifications for licensed indication Used in combinations for which interactions are suspected

Medicinal products NOT licensed in any EU Member State if :The active substance is part of a medicinal product licensed in the EU

*Grading of Type A may be justified if there is extensive clinical experience with the product and no reason to suspect a different safety profile

Type C: markedly higher than that of standard medical care

Trials involving a medicinal product not licensed in any EU Member State

A grading other than TYPE C may be justified if there is extensive class data or pre clinical and clinical evidence

Guidance on focus and intensity of monitoringConcerns identified in the risk assessment with the design, methods or conduct of the trial (excl IMP) which remain after mitigations are in place

Risk assoc with IMP

No Yes

Type A Low intensityCentral monitoringNo requirement site visits

Low+Plus appropriate monitoring to address specific vulnerabilities

Type B Moderate intensityCentral monitoringTriggered visits

Moderate +Plus appropriate monitoring to address specific vulnerabilities

Type C Higher IntensityMore intense monitoring to include effective site visits

Higher+Plus appropriate monitoring to address specific vulnerabilities

PILOT:

http://www.netscc.org.uk/news/PDFs/MHRAPilotRept_FINALPUBL

ICATION.PDF

Empirical example from cancer*

Non-commercial cancer trial designed and initiated pre-2004 UK regulations

Parallel, open-label, multicentre (UK), phase III, superiority RCT comparing control chemotherapy with experimental chemotherapy

At the close of recruitment the drug company initiated 100% SDV of all data collected

All source verified data entered onto a ‘new’ database

Comparison of original data and source verified data

*Catrin Tudur Smith, Deborah Stocken, Paula Ghaneh, Trevor Cox, David Cunningham, John Neoptolemos

Strengths/limitations Strengths

Independent review of data

Independent database

Rare for 100% SDV to be performed in non-commercial trials

Limitations

Original ‘un-monitored’ data may not represent current practice

SDV may have changed trial conduct towards end of the trial

Outcomes Primary outcome

Overall Survival (OS)

Secondary outcomes

Progression Free Survival (PFS)

Objective Response

Serious Adverse Events

Overall survival

VariableDiscrepancies n (%)

Control Experimental Total(n=266) (n=267) (n=533)

Date of death all 21 (7.9) 22 (8.2) 43 (8.1)

Date of death excluding missing

5 (1.9) 8 (3.0) 13 (2.4)

Death status 15 (5.6)* 14 (5.2) 29 (5.4)

*1 additional patient had missing date of death in original data but date of death in SVD but death status agreed

Overall survivalNon-

monitored

data

Source

verified data

HR (95% CI)* 1.19

(0.99 to 1.42)

1.18

(0.99 to 1.41)

Number of

patients

533 533

Deaths 469 498

Log-rank

statistic

3.33 3.44

Log-rank p-

value

0.068 0.064

*HR>1 indicates benefit to E

Central monitoring for OS Discrepancies in death data – not clear whether SDV

accurate

Central collection of death data from ONS

Provides a ‘third’ data set for comparison

Central monitoring for OS 56 (11%) discrepancies between SDV and ONS date of

death

2 patients still alive in SDV but dead in ONS

2 dates were discrepant by 1 year

52 dates were discrepant by a few days

Central monitoring for OSNon-

monitored

data

Source verified

data

Central

monitored

data

HR (95% CI)* 1.19

(0.99 to 1.42)

1.18

(0.99 to 1.41)

1.18

(0.99 to 1.40)

Number of

patients

533 533 533

Deaths 469 498 499

Log-rank

statistic

3.33 3.44 3.22

Log-rank p-

value

0.068 0.064 0.073

*HR>1 indicates benefit to E

Overall survival

0. 00

0. 25

0. 50

0. 75

1. 00

t i me

0 200 400 600 800 1000 1200

STRATA: dat a=ONS Censor ed dat a=ONS

dat a=SDV Censor ed dat a=SDV

dat a=un- moni t or ed Censor ed dat a=un- moni t or ed

*HR>1 indicates benefit to E

Empirical example conclusions

SDV identified errors, BUT

Errors did not impact analysis of primary outcome OS

Central monitoring suggested possible errors in source verified data

Central monitoring (via ONS) more efficient and accurate in this example

Empirical example conclusions

SDV did impact RECIST response data and SAE data

Data collection difficult/subjective for these outcomes

More scope to introduce systematic error

Suggests a need to focus training research staff

Further investigation required

SDV resource intensive and may not necessarily provide error free data

End of trial ‘checklist’ of critical data to site staff may be an alternative more efficient approach for some data?

Current practice - CTTI

Electronic survey of over 200 organizations in academia, government, industry, and clinical research – 65 (30%) responders

• Clinical Trials Transformation Initiative (CTTI)

• On-site monitoring is routinely performed by industry and CROs but less frequently

and less extensively by academic coordinating center/cooperative group/government

organizations.

• The scope of on-site monitoring visits varies by organization

-some procedures, source documents and regulatory documents reviewed more often by CROs and

industry sponsors than by academic/cooperative group/government organizations.

Current practice - CTTI The rationale for using a specific monitoring approach does not appear

to be based on empirical evidence

little empirical evidence to determine which, if any, onsite monitoring practices lead to improved patient safety and data quality.

substantial lack of information to guide the planning of trials that include this commonly used and expensive quality assurance procedure.

... more research is needed to assess the potential impact of the variations in monitoring practices we observed.

Current practice – CTU survey

Norah Mawoko and Catrin Tudur Smith 2011

On-line survey of 48 UKCRC registered Clinical Trials Units

Only 19 (40%) complete responses

All 19 CTUs involved with CTIMPs

Monitoring plan: 37% all trials, 32% most trials, 32% some trials

Risk assessment to determine level of monitoring: 53% all trials, 16% most trials, 31% some trials

All CTUs used some level of central monitoring, either with or without on-site monitoring

Current practice – CTU survey

Current practice – CTU survey

Final remarks

Variation in monitoring practice across organisations

Evidence based guidelines for monitoring not available

Resource intensive approaches such as source data verification are often inefficient and unnecessary

Further empirical evidence required to evaluate alternative methods

Approaches to monitoring should be flexible and risk proportionate

Training in monitoring needs to reflect this

Monitoring workshop Clinical trial monitoring workshop is being developed

cross-hubs initiative (North West (lead hub), London, Bristol, Oxford, Leeds CTU)

collate and present a practical summary of current practice through consultation with UKCRC registered CTUs

focus on risk-based and central approaches to monitoring

include MHRA perspective

share best practice

establish a network of methodologists from hubs and CTUs with interest in monitoring