OOPS! What Happened?

Investigating Anomalous Results in a GLP/Bioanalytical Environment

Jacquelynn Karau, QA Supervisor-BioanalyticalMiddleton, WI

Goals and Objectives:

• Why investigations are needed (including repercussions of not investigating)

• When to conduct an investigation• What comprises an investigation• How to document an investigation

What This Presentation is Not:

• An endorsement of one particular way of investigating

• A summary of PPD’s processes• Trying to state what the FDA, or anyone

else for that matter, is thinking about this topic

• An indication that you MUST have an investigation/anomaly review procedure

• A forum to discuss the merits of having an investigation procedure

“We’ve Never Investigated Before; Why Should We Do it Now?”• FDA warning letters have been issued

citing failures to “systematically investigate” contamination and anomalous results, repeated lack of conformance, etc.

• The white paper on bioanalytical method validation indicates that documentation of “deviations and unexpected events” as well as “investigation and impact assessment of unexpected events” be retained at the analytical site as well as discussed in the final study report

What’s the Worst That Could Happen if We Don’t Investigate?• The same anomaly could occur in

other projects/studies (i.e., contamination, carryover, misdosing) resulting in delays and possibly repeating projects/studies

• Loss of client confidence and business

• Possible increased FDA inspections• Submissions may not approved

“Okay, I Get It. Now What?”

• The prospect of writing an SOP on investigations/anomaly review can be very intimidating

• Chances are- “investigations/anomaly reviews” are already being done, all that’s missing is a formally defined process

• The easiest way to start an SOP is to state what you’re currently doing as well as to develop a list of possible “anomalies” that may require a review

How to Define “Anomaly”

• Determining what is an anomaly is the most difficult part of establishing a procedure for anomaly review

• Webster’s on-line dictionary defines anomaly as “deviation from the normal or common order or form or rule”

• Anomalies, by definition, are not easily defined

How Do We Establish Investigation Guidelines?• Make a list of common problems/

anomalies encountered in the past (e.g., mislabeling of animals/sample tubes, unexpected concentration results, inconsistencies between original and reassay results)– Note: Any compiled list will probably not be all

inclusive

• Any established guidelines need to be consistently applied to all projects/studies

Anomaly Review/Investigations Should Be:

• Thorough• Timely• Unbiased• Scientifically defensible• Well documented

Investigations in a Nutshell:

1. Problem definition2. Scope3. Possible causes4. Investigations and examinations5. Assess results6. Corrective and preventative action7. Follow-up

Step#1: Problem Definition

• Define the anomaly as thoroughly as possible in technical terms– What is the anomaly?– When and where did it happen?– What is the significance of the problem?

• A clearly defined/identified anomaly will aid in the investigation

Step #2: Scope

• Determine the extent of the anomaly– How much of the project/study is affected?– How many projects/studies affected (past,

present and possibly future)?– Has the anomaly happened before?

• Be sure to look for “ripple effects”; evaluate what else could be involved or affected

Step #3: Possible Causes

• Develop a list of all possible causes– Include the most probable as well as the least

likely– All causes and effects are part of an infinite

continuum• For example: An injury was caused by a fall, which

was caused by a wet surface, which was caused by a leaky valve, which was caused by a seal failure, which was caused by improper/inadequate maintenance

Step #4: Investigations/Examinations

• Determine how to evaluate each possible cause identified in step #3– Were all methods, policies, procedures

followed? If not, how did that affect the anomaly?

• Based on the initial evaluation, develop a working hypothesis of the most probable cause of the anomaly

Step #4 cont’d

• Design experiments to test if the working hypothesis is the cause of the anomaly– Break the situation down into smaller “chunks”

if needed

• The working hypothesis may change based on the experiments

• Be sure to test every possible cause

Step #5: Assess

• Review the results of step #4– Are certain causes eliminated?– Were other possible causes identified during

the course of the investigation?– What is the impact on the project/study?

• Was the root cause identified? Is there more than one root cause? Keep looking, digging, investigating until a clear picture emerges

Step #6: Corrective/Preventative Actions (CAPA)

• Once the cause(s) is/are identified, establish and implement CAPA items– How should the current anomaly be

addressed? (corrective)– How can the anomaly be prevented in the

future? (preventative)

Step #6 cont’d

• Be sure that proposed preventative actions are:– Within your realm of control– Going to meet regulatory/SOP requirements,

deliverable timelines, etc.– Going to actually prevent recurrences of the

anomaly• Avoid the “quick fix” such as

training/retraining or “analyst will be more careful in the future”

Step #7: Follow-up

• Assign a specific person the responsibility (and the authority!) to follow up on any CAPA items– Assure that CAPA measures are appropriate

and are being implemented/followed– Assign a specific date for CAPA completion

“That Sounds Great, Jacqui. Does it Really Work?”

• Yes, it does! I’ll review an example in the next few slides

• These seven steps can be used to “investigate” just about anything in your facility from trending analysis of protocol/SOP deviations to misdosing of test animals to, well, just about anything else!

Example: SOP Deviations

• QA was making a number of findings that the laboratory analysts were not following a section of an SOP that stated “The number of calibration standard replicates analyzed at each calibration level must be determined prior to study initiation and must be the same for both the performance qualification and study runs.”

• The performance qualification runs had n=1 and the study runs had n=2

Example cont’d

• As a result of the audits, SOP deviations were written

• Between March 2005 and February 2006 there were twelve SOP deviations: eight were a result of the difference in calibration standard replicates

• QA wanted to reduce this number, so we investigated

Example- Let’s Investigate!

• Steps #1 and 2- Problem Definition and Scope:– Two thirds of the deviations are for the issue

of different calibration replicates– There were multiple lab analysts involved,

including experienced personnel– All had been trained on the SOP and passed

the associated quiz

Example Investigation cont’d

• Step #3- Possible Causes:1. Analysts are misinterpreting the SOP

requirement2. QA is misinterpreting the SOP requirement3. Confusion between the requirements of a

partial validation (n=2 curves, n=6 QCs) and a performance qualification (n=1 curve- unless more are used in the study, n=2 QCs)

Example Investigation cont’d

• Step #4- Investigations/Evaluations1. Misinterpreting the SOP wasn’t plausible; the

language is really clear2. QA began to ask the specific analysts why

they think they deviated from the SOP3. The overwhelming response was:

When the analysts looked at the SOP to make sure all the required components were included, they didn’t “read far enough” into the SOP

Example Investigation cont’d

• Step #5- Assess– Possible causes #1 and 2 are ruled out due to

the clarity of the SOP language– Possible cause #3 is a possible cause however

the most likely cause was how the SOP was structured: the particular section of the SOP the analysts were reviewing did not reinforce the need to keep the number of calibration curve replicates consistent between qualification and sample analysis.

Example Investigation, Almost Done!

• Step #6- CAPA– Corrective actions- SOP deviations were

written for the specific projects involved.– Preventative action- Revise the SOP to

emphasize that the same number of curve replicates had to be used in the qualification as in the study sets (We accomplished this by moving the exact same language to the required components section)

Example Investigation- Final Wrap-up

• Step #7- Follow-up– The SOP revision went into effect 15 February

2006, prompting all laboratory analysts to read the SOP and pass the associated quiz, which had a specific question about the number of required curve replicates

– Between February 2006 through January 2008 there has been one SOP deviation

Document the Investigation• Investigations need to be thoroughly

documented– All pertinent information (study numbers,

methods/equipment used, original and experimental results, etc.) must be present

– Follow pre-established documentation procedures– Document decision processes as well as who

authorizes each step/section of the investigation– All forms used, any communications/interviews as well

as all data generated during the course of the investigation must be retained as part of the study record

– The investigation summary report must be clear, concise, and easy to understand

Investigation Reports

• The investigation report should contain, at a minimum:– A clear summary of the anomaly– Description of the extent/scope– A list of all possible causes as well as all

evaluations/experiments testing those causes– Facts to support any and all conclusions– Clear recommendations for CAPA items– Non-judgmental language– Approval signatures from all pertinent personnel,

including the Study Director, Principal Investigator, QA and sponsor as applicable

In Summary:

• Investigations need to be conducted when an anomaly occurs

• Establish a process for potential anomaly identification and subsequent investigation

• Keep a central repository of investigations for future reference

• Investigations are not an indication of systemic problems; they are opportunities for systemic improvements

Any Questions???

Thank you!!!