Data Integrity
Denial Ain’t Just a River in Egypt:
Panel Discussion
John Avellanet, Managing Director & Principal, Cerulean Associates LLC
Beverly Lorell MD, Senior Medical & Policy Advisor, King & Spalding LLP
David Elder, Vice President – Management, Parexel International
Steve Niedelman, Lead Quality Systems & Compliance Consultant, King & Spalding LLP
Vicky C. Stoakes, President, IntegRx, Inc.
Data Integrity Definitions
• Definition used by FDA for internal training:
“Data are of high quality if they are fit for their intended uses in operations, decision-making and planning . . . as data volume increases, the question of internal consistency within data becomes paramount….”
• ALCOA acronym used by FDA to define expectations: Attributable Legible Contemporaneous Original Accurate
2
FDA Enforcement Action
• FDA will take action against companies that commit data fraud or provide false information to the agency
• “Companies must provide truthful and accurate information in their marketing applications…The American public expects and deserves no less.”
Janet Woodcock, M.D., Director, CDER
February 25, 2009 FDA News Release
3
Importance of Data Integrity
• Continues to be an FDA priority – Investigators receive specialized training to detect data
integrity, data manipulation, and fraud
• Regulators must be able to rely upon the accuracy and completeness of data / information generated to meet applicable regulatory requirements
• Assurances of product safety, identity, strength, purity, and quality are dependent on the validity of data and information obtained
• Data integrity violations – Erode public confidence
– Impugn product quality and patient safety
– Have a devastating impact on implicated organizations 4
Data Integrity Violations
Come in Many Forms
• Geographical areas ranging from domestic to international
• Firms ranging from small to large, new to well established
• Involvement ranging from an individual employee to a conspiracy
• Frequency of occurrence ranging from isolated to pervasive
• Observed in data generated premarket and postmarket
• Data types ranging from medical data to production data
• Acts of commission as well as omission
• Cases involve all types of regulated products
• Identified via regulatory inspections ,Whistleblowers, internal
findings, other…
5
• Using FDA’s findings as a standard, has your firm
evaluated internal data integrity systems to determine if
similar underlying cGMP/QSR deficiencies exist and
whether data manipulation may have occurred?
• Is your company’s prevailing mindset that data integrity
problems only happen to “other people”?
Is your firm in denial?
6
Panel Discussion Topics
• The types of data integrity violations identified during
recent FDA Inspections
• The regulatory, civil, and criminal penalties associated with
data integrity violations
• FDA expectations for review of electronic data and
information
• How to conduct internal and external audits from a data
integrity perspective
• Actions to take if data integrity concerns are identified
within your company or at a contractor
7
John Avellanet
Managing Director & Principal, Cerulean Associates LLC
Data Integrity Considerations
8
FDA Investigator Instructions
“If a firm is keeping electronic records, determine if they are in compliance with 21
CFR Part 11. At a minimum, ensure that:
(1) the firm has prepared a plan for achieving full compliance with part 11
requirements and is making progress toward completing that plan in a timely
manner
(2) accurate and complete electronic and human readable copies of electronic
records, suitable for review, are made available
(3) employees are held accountable and responsible for actions.
If initial findings indicate the firm’s electronic records may not be
trustworthy and reliable, or when electronic recordkeeping systems
inhibit meaningful FDA inspection, a more detailed evaluation may be warranted.”
- FDA Enforcement Compliance Policy Manual, Attachment A
http://www.fda.gov/ICECI/EnforcementActions/BioresearchMonitoring/ucm133927.htm
9
ICH Data Integrity
“…Quality control should be applied to each
stage of data handling to ensure that
all data are reliable and have been processed
correctly….”
- 2008 FDA GCP Training for Investigators, Drs. David Lepay & Jean Toth-Allen, OGCP
2011 FDA presentation on PAIs of Biologics, Anastasia G. Lolas, CDER
quote from ICH E6 § 5.1 Quality Assurance and Quality Control
10
Example Considerations
• Have personnel been trained on good documentation and good
data integrity practices?
• How does the firm ensure that analysts enter ALL test data, not
just the passing test results?
• For transcribed data, what verification processes are in place?
• When data is scanned, how does the firm ensure the evidentiary
admissibility of the scan (e.g., “certified copy”)?
• Has the system been validated and under change control?
Data Integrity at Creation
11
Example Considerations
• Did the firm verify computerized calculations prior to usage on the
data?
• Does the firm claim to use “paper records only” but then actively
use e-records to release batches, make safety and efficacy
decisions, etc.?
• How does the firm ensure that previously recorded SUSAR data
cannot be altered when reviewed?
Data Integrity during Active Usage
12
Example
PDF corruption of 6 day old batch record discovered when QC double-checked the PDF prior to releasing the last of the batch
As of September 2014, firm remains unable to print or re-save the file despite direct help from Adobe
13
Example Considerations
• Does the firm retain raw lab data/digital clinical source data
along with context (e.g., metadata)?
• What were the process checks undertaken prior, during, and
after clinical trial database lock? Transmittal to the sponsor?
• Does the firm have traceability on its complaint records to
ensure that none of the data is left out of any later analysis
(such as for an APR or QSMR) or when transmitted?
Data Integrity while Semi-Active
14
Example
Database corruption from
SQL-based clinical EDC at
CRO after trial lock
As of October 2012, only
the descriptors (i.e., “text”
“number”, etc.) of data that
should have been there
could be recovered
15
Example Considerations
• If the firm uses a storage vendor, is the vendor qualified?
• How often does the firm sample its long-term archives to
ensure continuing storage suitability and prevent data
deterioration?
• What controls does the firm have on retained record
destruction to prevent inadvertent loss of required data?
• Does the firm have a digital media migration strategy?
Data Integrity of Archives
16
“Documents and e-data spend
more than 80% of their
lifespan in an archived (e.g.,
stored) state. ” - ARMA International
17
Example
Graphical product insert from
2005 with detailed risk
information on it
As of December 2012, this is all
that the company could
recover from its approved
electronic proof
18
Data Integrity Lifecycle Controls
19
Translation:
Good data integrity requires
cross-functional framework
risk-based philosophy
pragmatic mentality
20
What Controls Do You Have So…
FDA can rely upon your data?
Physicians can rely upon your data?
Patients can rely upon your data?
Investors can rely upon your
data?
21
Picture Credits
Photos, images and clip art that appear on these slides have been used to enhance this presentation and
may NOT be used for commercial or promotional purposes without permission from copyright holders.
Do not remove or copy from this presentation.
Contact:
iStockphoto.com
Fotolia
Flickr
Microsoft Corporation
Cerulean Associates LLC
22
Beverly Lorell, MD, FACC
Senior Medical & Policy Advisor, King & Spalding LLP
Clinical Data Integrity Violations –
A GCP Perspective
23
Scientific Data Integrity:
A Hot Issue
• Many scandals with intense media attention
– Pharmaceutical and medical device clinical studies, as well as cutting edge basic science
• Globalization of clinical trials and complexity
• Eroding public confidence
• Far-reaching repercussions
– Delay in regulatory review and approval, retractions of public data, impact on clinical guidelines/standards
As examples,
24
“Rise of the Retractions”
• Investigator misconduct accounts for ~50% of retractions
25
Clinical Data Integrity:
Recent Scrutiny
26
Bloomburg
Clinical Data Integrity:
Recent Scrutiny
27
Clinical Data Integrity:
A Journey to Yoda
28
FDA Bioresearch Monitoring
Objectives
• To protect the rights, safety, and welfare of subjects involved in FDA-regulated clinical trials
• To verify the accuracy and reliability of clinical trial data submitted to FDA in support of research or marketing applications, and
• To assess compliance with FDA’s regulations governing the conduct of clinical trials
29
BIMO Metrics FY’13
• Most common Sponsor deficiencies – Inadequate monitoring
– Failure to bring investigators into compliance
– Inadequate accountability for investigational product
– Failure to obtain FDA and/or IRB approval
• Most common Investigator deficiencies – Failure to follow investigational plan and/or regulations
– Protocol deviations
– Inadequate recordkeeping
– Inadequate accountability for investigational product
– Failure to comply with IRB requirements
– Inadequate human subjects protection (failure report AEs and obtain informed consent)
30
Recent Trends - Warning Letters
“Advanced Magnetic Research Institute – Jan. 16, 2014
• Failure to maintain records and accurately report serious adverse events
• Violative promotion of investigational device
•
31
Recent Trends - Warning Letters
“Advanced Interventional Pain Center” – Mar. 14, 2014
• FDA determination that device is Significant Risk (SR) – enrolling patients before submitting IDE to FDA
• Failure to maintain accurate and complete records, including source documents and financial disclosure under Part 54
• Failure to report deviations from investigational plan
32
Many Portals for Scientific
Integrity Allegations
• FDA advisory committee meetings
– Clinical Event Committee – event adjudication; Data Monitoring Board – sponsor interactions
– Geographic region data disparities
• Clinical investigators and medical journals
– Medical journals (e.g., BMJ, NEJM, Lancet, NEJM)
– Direct interaction with media and/or members of Congress
• Media and other watchdogs
– Forbes, WSJ, NYT
– ProPublica, Retraction Watch, and more!
•
33
FDA Initiatives
• Falsification of data/omission of material facts
– “Reporting Information Regarding Falsification of Data” - Final Rule pending
• Globalization of clinical trials
– “Human Subject Protection; Acceptance of Data from Clinical Studies for Medical Devices” - Final Rule pending
• Sharing of GCP inspectional data
– “FDA-EMA GCP Initiative” and other programs
• Risk-based monitoring
– “Oversight of Clinical Investigations – A Risk-Based Approach to Monitoring” - New guidance
•
34
Challenges for Sponsors
“Sponsors are responsible for selecting qualified investigators… and ensuring that any reviewing IRB and FDA are promptly informed of significant new information about an investigation.” 21 C.F.R. 812.40
• During an ongoing clinical investigation, when and how should internal GCP audits be conducted?
• When do issues of potential data integrity rise to level of “significant new information about an investigation?”
• When and how should voluntary disclosures be made to FDA Office of Compliance?
•
35
David Elder
Vice President – Management, Parexel International
Data Integrity Violations –
A GMP Perspective
36
Data Integrity
• Issues with data integrity have been the downfall of
several companies. There are few things in business
(or in life) that are more important than integrity.
When any one issue is found, ALL other actions,
answers, documents, records, and people become
questionable.
• A few examples of recent findings….
37
Data Integrity
WL, May 2014, Sun Pharma.
• “Your firm frequently performs “unofficial testing” of
samples, disregards the results, and reports results from
additional tests. For example, during stability testing, your
firm tested a batch sample six times and subsequently deleted
this data”
• “Similar unacceptable data handling practices were observed
in your laboratory’s conduct of gas chromatography (GC)
analyses. The FDA investigators reviewed what appear to be
data from “unofficial” injections …Therefore, it appears that
out-of-specification data … was considered to be “unofficial,”
while passing data were reported as the "official" results for
the batch”
38
Data Integrity
WL, November 2013, Wockhardt
• “our investigators identified your practice of performing
“trial” sample analysis for high performance liquid
chromatography (HPLC) analyses at your Chikalthana
and Waluj facilities prior to acquiring the “official”
analytical data for release and stability testing”
39
Data Integrity
WL, Smruthi Organics Ltd, March 2014
• Our investigators identified calibration and media preparation records that were not authentic in that the persons that signed each record as having performed the activity were not at work on the day the work was accomplished.
• Our investigators identified the practice of performing trial injections for HPLC analyses prior to running the release and stability tests that are then reported. There was no justification for the practice of the trial preparations and
injections
40
Data Integrity
FDA-483, Ranbaxy-Toansa, January 2014
• Paraphrased observations (leading to extension of Consent
Decree and IA to this additional manufacturing site)
• Overwriting raw data files that had failing results
• Test/Sample injections with failing results
• Reported results differed from raw data
• Backdating of records; pre-filled records
• Samples/Records found in drawer in laboratory
labeled “blank paper”
41
Data Integrity
December 16, 2013
• The MHRA is setting an expectation that pharmaceutical manufacturers, importers and contract laboratories, as part of their self-inspection programme must review the effectiveness of their governance systems to ensure data integrity and traceability.
• This aspect will be covered during inspections from the start of 2014, when reviewing the adequacy of self inspection programmes in accordance with Chapter 9 of EU GMP.
• It is also expected that in addition to having their own governance systems, companies outsourcing activities should verify the adequacy of comparable systems at the contract acceptor.
• The MHRA invites companies that identify data integrity issues to contact: [email protected]
Food for Thought: Does the self-inspection/internal audit and supplier management function of a company include trained individuals capable of detecting data integrity issues if they exist?
42
Data Integrity Is Not Isolated To
Lab Operations (Real Examples Noted)
• Applications filed with Regulatory Agencies:
– Records in FDA Applications Contained Different Signatures for the Same Person
– CMC Section in NDA Submitted to FDA Was Different from Copy on File at the Company. Company Unable to Verify Which Version Was Correct
– CMC Section Contained Validation Data Reported to Be from Production Autoclave, But FDA Inspection Revealed Production Autoclave Had Not Yet Been Installed on Dates Recorded
– Certain testing data submitted with application could not be verified from review of laboratory notebooks or electronic raw data
43
Data Integrity Is Not Isolated To
Lab Operations (Real Examples Noted)
• Quality System Documentation: • Observed employee fraudulently signing weighing record for activity
he did not perform
• Supervisor created logbooks after the fact for two years (after FDA requested to review)
• Cleaning employee was suspected of lying, so 5 FDA investigators returned to the plant during the next cleaning to simultaneously watch actual practices in five different areas. FDA confirmed employee had lied
• Employee discarded project file (raw data, chromatograms, analytical worksheets) to hide records from detection by auditor (& company management) Entire completed original batch records for several lots were found in supervisors possession, duplicate records had been created for the document retention files
• Entries for “tomorrow’s” batch had already been filled out
• Desk drawer at analyst workbench contained multiple OOS worksheets that had not been reported to QA
• Log book contained dated entries that were out of sequence
• In-process batch record with blanks for significant steps, post-it-note created by supervisor instructing employee to fill in the blanks
44
Data Integrity Is Not Isolated To
Lab Operations
WL, July 2014, Zhejiang Jiuzhou Pharmaceutical – Failure to document manufacturing operations at the time they are
performed.
• When reviewing the entries in your… logbook for the days immediately prior to the inspection, our investigator found missing entries. Your operators stated that lines were left blank to later add information about cleaning events that may have occurred during a previous shift. During the inspection, our investigator found other similar instances of missing data or belated data entry in your manufacturing records. These practices are not consistent with CGMP.
• In addition, during the inspection, one of your quality unit employees presented the investigator with a batch record containing his signature, stating that he had performed the review of this batch record. The employee later admitted that he had falsified this CGMP record and stated that he in fact had not performed the review, despite having signed the batch record as the QA reviewer and having released the batch. This data falsification and the record-keeping deficiencies described above raise doubt regarding the validity of your firm’s records.
45
Acknowledgement/References
• Presentation, FDLI Pharmaceutical GMP training program, July 2014,
D. Elder
• Presentation/Training, R. Tetzlaff, PhD
• FDA Training Course, Drug Inspection Training, Data Integrity Section
46
