OOPS! What Happened?
Investigating Anomalous Results in a GLP/Bioanalytical Environment
Jacquelynn Karau, QA Supervisor-BioanalyticalMiddleton, WI
Goals and Objectives:
• Why investigations are needed (including repercussions of not investigating)
• When to conduct an investigation• What comprises an investigation• How to document an investigation
What This Presentation is Not:
• An endorsement of one particular way of investigating
• A summary of PPD’s processes• Trying to state what the FDA, or anyone
else for that matter, is thinking about this topic
• An indication that you MUST have an investigation/anomaly review procedure
• A forum to discuss the merits of having an investigation procedure
“We’ve Never Investigated Before; Why Should We Do it Now?”• FDA warning letters have been issued
citing failures to “systematically investigate” contamination and anomalous results, repeated lack of conformance, etc.
• The white paper on bioanalytical method validation indicates that documentation of “deviations and unexpected events” as well as “investigation and impact assessment of unexpected events” be retained at the analytical site as well as discussed in the final study report
What’s the Worst That Could Happen if We Don’t Investigate?• The same anomaly could occur in
other projects/studies (i.e., contamination, carryover, misdosing) resulting in delays and possibly repeating projects/studies
• Loss of client confidence and business
• Possible increased FDA inspections• Submissions may not approved
“Okay, I Get It. Now What?”
• The prospect of writing an SOP on investigations/anomaly review can be very intimidating
• Chances are- “investigations/anomaly reviews” are already being done, all that’s missing is a formally defined process
• The easiest way to start an SOP is to state what you’re currently doing as well as to develop a list of possible “anomalies” that may require a review
How to Define “Anomaly”
• Determining what is an anomaly is the most difficult part of establishing a procedure for anomaly review
• Webster’s on-line dictionary defines anomaly as “deviation from the normal or common order or form or rule”
• Anomalies, by definition, are not easily defined
How Do We Establish Investigation Guidelines?• Make a list of common problems/
anomalies encountered in the past (e.g., mislabeling of animals/sample tubes, unexpected concentration results, inconsistencies between original and reassay results)– Note: Any compiled list will probably not be all
inclusive
• Any established guidelines need to be consistently applied to all projects/studies
Anomaly Review/Investigations Should Be:
• Thorough• Timely• Unbiased• Scientifically defensible• Well documented
Investigations in a Nutshell:
1. Problem definition2. Scope3. Possible causes4. Investigations and examinations5. Assess results6. Corrective and preventative action7. Follow-up
Step#1: Problem Definition
• Define the anomaly as thoroughly as possible in technical terms– What is the anomaly?– When and where did it happen?– What is the significance of the problem?
• A clearly defined/identified anomaly will aid in the investigation
Step #2: Scope
• Determine the extent of the anomaly– How much of the project/study is affected?– How many projects/studies affected (past,
present and possibly future)?– Has the anomaly happened before?
• Be sure to look for “ripple effects”; evaluate what else could be involved or affected
Step #3: Possible Causes
• Develop a list of all possible causes– Include the most probable as well as the least
likely– All causes and effects are part of an infinite
continuum• For example: An injury was caused by a fall, which
was caused by a wet surface, which was caused by a leaky valve, which was caused by a seal failure, which was caused by improper/inadequate maintenance
Step #4: Investigations/Examinations
• Determine how to evaluate each possible cause identified in step #3– Were all methods, policies, procedures
followed? If not, how did that affect the anomaly?
• Based on the initial evaluation, develop a working hypothesis of the most probable cause of the anomaly
Step #4 cont’d
• Design experiments to test if the working hypothesis is the cause of the anomaly– Break the situation down into smaller “chunks”
if needed
• The working hypothesis may change based on the experiments
• Be sure to test every possible cause
Step #5: Assess
• Review the results of step #4– Are certain causes eliminated?– Were other possible causes identified during
the course of the investigation?– What is the impact on the project/study?
• Was the root cause identified? Is there more than one root cause? Keep looking, digging, investigating until a clear picture emerges
Step #6: Corrective/Preventative Actions (CAPA)
• Once the cause(s) is/are identified, establish and implement CAPA items– How should the current anomaly be
addressed? (corrective)– How can the anomaly be prevented in the
future? (preventative)
Step #6 cont’d
• Be sure that proposed preventative actions are:– Within your realm of control– Going to meet regulatory/SOP requirements,
deliverable timelines, etc.– Going to actually prevent recurrences of the
anomaly• Avoid the “quick fix” such as
training/retraining or “analyst will be more careful in the future”
Step #7: Follow-up
• Assign a specific person the responsibility (and the authority!) to follow up on any CAPA items– Assure that CAPA measures are appropriate
and are being implemented/followed– Assign a specific date for CAPA completion
“That Sounds Great, Jacqui. Does it Really Work?”
• Yes, it does! I’ll review an example in the next few slides
• These seven steps can be used to “investigate” just about anything in your facility from trending analysis of protocol/SOP deviations to misdosing of test animals to, well, just about anything else!
Example: SOP Deviations
• QA was making a number of findings that the laboratory analysts were not following a section of an SOP that stated “The number of calibration standard replicates analyzed at each calibration level must be determined prior to study initiation and must be the same for both the performance qualification and study runs.”
• The performance qualification runs had n=1 and the study runs had n=2
Example cont’d
• As a result of the audits, SOP deviations were written
• Between March 2005 and February 2006 there were twelve SOP deviations: eight were a result of the difference in calibration standard replicates
• QA wanted to reduce this number, so we investigated
Example- Let’s Investigate!
• Steps #1 and 2- Problem Definition and Scope:– Two thirds of the deviations are for the issue
of different calibration replicates– There were multiple lab analysts involved,
including experienced personnel– All had been trained on the SOP and passed
the associated quiz
Example Investigation cont’d
• Step #3- Possible Causes:1. Analysts are misinterpreting the SOP
requirement2. QA is misinterpreting the SOP requirement3. Confusion between the requirements of a
partial validation (n=2 curves, n=6 QCs) and a performance qualification (n=1 curve- unless more are used in the study, n=2 QCs)
Example Investigation cont’d
• Step #4- Investigations/Evaluations1. Misinterpreting the SOP wasn’t plausible; the
language is really clear2. QA began to ask the specific analysts why
they think they deviated from the SOP3. The overwhelming response was:
When the analysts looked at the SOP to make sure all the required components were included, they didn’t “read far enough” into the SOP
Example Investigation cont’d
• Step #5- Assess– Possible causes #1 and 2 are ruled out due to
the clarity of the SOP language– Possible cause #3 is a possible cause however
the most likely cause was how the SOP was structured: the particular section of the SOP the analysts were reviewing did not reinforce the need to keep the number of calibration curve replicates consistent between qualification and sample analysis.
Example Investigation, Almost Done!
• Step #6- CAPA– Corrective actions- SOP deviations were
written for the specific projects involved.– Preventative action- Revise the SOP to
emphasize that the same number of curve replicates had to be used in the qualification as in the study sets (We accomplished this by moving the exact same language to the required components section)
Example Investigation- Final Wrap-up
• Step #7- Follow-up– The SOP revision went into effect 15 February
2006, prompting all laboratory analysts to read the SOP and pass the associated quiz, which had a specific question about the number of required curve replicates
– Between February 2006 through January 2008 there has been one SOP deviation
Document the Investigation• Investigations need to be thoroughly
documented– All pertinent information (study numbers,
methods/equipment used, original and experimental results, etc.) must be present
– Follow pre-established documentation procedures– Document decision processes as well as who
authorizes each step/section of the investigation– All forms used, any communications/interviews as well
as all data generated during the course of the investigation must be retained as part of the study record
– The investigation summary report must be clear, concise, and easy to understand
Investigation Reports
• The investigation report should contain, at a minimum:– A clear summary of the anomaly– Description of the extent/scope– A list of all possible causes as well as all
evaluations/experiments testing those causes– Facts to support any and all conclusions– Clear recommendations for CAPA items– Non-judgmental language– Approval signatures from all pertinent personnel,
including the Study Director, Principal Investigator, QA and sponsor as applicable
In Summary:
• Investigations need to be conducted when an anomaly occurs
• Establish a process for potential anomaly identification and subsequent investigation
• Keep a central repository of investigations for future reference
• Investigations are not an indication of systemic problems; they are opportunities for systemic improvements
Any Questions???
Thank you!!!