OOSsess7 1215 1
Develop Effective Strategies for Resampling
and Retesting
OOS SESSION 7
Steven S. Kuwahara, Ph.D.
GXP BioTechnology6336 N. Oracle Rd. #326-313
Tucson, AZ 85704-5480
E-mail: [email protected]
Guidance for Industry
Investigating Out-of-Specification (OOS) Test
Results for Pharmaceutical Production
• U.S. Department of Health and Human Services
Food and Drug Administration
• Center for Drug Evaluation and Research
(CDER)
• October 2006 Pharmaceutical CGMPs
OOSsess4 1215 2
Resampling and Retesting. B. Additional Laboratory
Testing
• A full-scale OOS investigation may include additional
laboratory testing. A number of practices are used during the
laboratory phase of an investigation. These include (1)
retesting a portion of the original sample and (2) resampling.
• 1. Retesting
• Part of the investigation may involve retesting of a portion of
the original sample. The sample used for the retesting should
be taken from the same homogeneous material that was
originally collected from the lot, tested, and yielded the OOS
results. For a liquid, it may be from the original unit liquid
product or composite of the liquid product; for a solid, it may
be an additional weighing from the same sample composite
prepared for the original test.
OOSsess7 1215 3
B.1. Retesting. Comments I.
• Note that it is important to define what is the
original sample.
– In some laboratories an original sample is taken and
aliquots for many tests are taken from it. Is this the
original sample or is it the aliquot taken for the test that
produced the OOS?
• The aliquot for the test that produced the OOS should be large
enough to allow for at least one retest.
• The guidance mentions composite samples, but 21 CFR
211.84(c)(4) forbids compositing certain types of samples.
– Compositing is really not a good idea except to save testing costs. It
is a form of physical averaging and results in a loss of information
on variability.
OOSsess7 1215 4
B.1. Retesting. Comments II.
• Compositing is often allowed by commercial or other
guidance, but should be treated with caution.
• Compositing final product samples causes a loss of
information on the variability of the samples.
– If it must be done, do not composite more samples than the amount
that will be used in the next stage.
– This is also true for final products.
– Never composite material where you are concerned about the
variability of the samples.
• Be sure that the dates show that the retesting did not start
before the initial laboratory investigation concluded.
– Otherwise, it will be considered to be reflexive retesting which is a
relative of testing into compliance.
OOSsess7 1215 5
7UCSCGMP2014 6
Sec. 211.84(b) Testing and approval or rejection of
components, drug product containers, and closures.
• (b) Representative samples of each shipment of each lot
shall be collected for testing or examination. The number
of containers to be sampled, and the amount of material
to be taken from each container, shall be based upon
appropriate criteria such as statistical criteria for
component variability, confidence levels, and degree of
precision desired, the past quality history of the supplier,
and the quantity needed for analysis and reserve where
required by Sec. 211.170.
7UCSCGMP2014 7
Sec. 211.84(c)(4 - 6) Testing and approval or
rejection of components, drug product containers,
and closures.
• (4) If it is necessary to sample a component from the top,
middle, and bottom of its container, such sample
subdivisions shall not be composited for testing.
• (5) Sample containers shall be identified so that the
following information can be determined: name of the
material sampled, the lot number, the container from
which the sample was taken, the date on which the
sample was taken, and the name of the person who
collected the sample.
• (6) Containers from which samples have been taken shall
be marked to show that samples have been removed from
them.
Resampling and Retesting. B. Additional
Laboratory Testing. 1. Continued A.• Situations where retesting is indicated include investigating
testing instrument malfunctions or to identify a possible
sample handling problem, for example, a suspected dilution
error. Decisions to retest should be based on the objectives of
the testing and sound scientific judgment. It is often important
for the predefined retesting plan to include retests performed
by an analyst other than the one who performed the original
test. A second analyst performing a retest should be at least as
experienced and qualified in the method as the original
analyst.
• The CGMP regulations require the establishment of
specifications, standards, sampling plans, test procedures, and
other laboratory control mechanisms (§ 211.160).
OOSsess7 1215 8
B.1. Retesting. Comments III.
• In 21 CFR 211.84(c)(5) the individual containers
from which samples were taken were identified.
– If the samples are composited, that information is
wasted and a determination of component variability
(21 CFR 211.84(b)) cannot be made.
• The second analyst should be equally qualified
and there should be data that proves this.
– If the second analyst is not as competent as the original
analyst, and the second analyst obtains a passing result,
how do you know that it is not due to a Type II error?
– During test training, information should have been developed
showing that all analysts are equally qualified.
OOSsess7 1215 9
Resampling and Retesting. B. Additional
Laboratory Testing. 1. Continued B.• The maximum number of retests to be performed on a sample
should be specified in advance in a written standard operating
procedure (SOP). The number may vary depending upon the
variability of the particular test method employed, but should
be based on scientifically sound principles. The number of
retests should not be adjusted depending on the results
obtained. The firm's predetermined retesting procedures
should contain a point at which the additional testing ends and
the batch is evaluated. If the results are unsatisfactory at this
point, the batch is suspect and must be rejected or held
pending further investigation (§ 211.165(f)).
OOSsess7 1215 10
B.1. Retesting. Comments IV.
• Retesting cannot continue forever. There must be
a stopping point, and this point should be written
into the test procedure.
– The point at which retesting ends should depend on the
known variability of the test method (the information
should have been developed during assay validation.)
– There are statistical methods that can define the end
point.
– When the limit is reached, if the OOS cannot be
removed, you must err on the side of caution and
proceed with the idea that the OOS is real.
OOSsess7 1215 11
Resampling and Retesting. B. Additional Laboratory
Testing. 1. Continued C.
• Any deviation from this SOP should be rare and done
in accordance with § 211.160(a), which states that
any deviations from written specifications, sampling
plans, test procedures, or other laboratory control
mechanisms shall be recorded and justified. In such
cases, before starting additional retesting, a protocol
should be prepared (subject to approval by the QCU)
that describes the additional testing to be performed
and specifies the scientific and/or technical handling
of the data.
OOSsess7 1215 12
Resampling and Retesting. B. Additional Laboratory
Testing. 1. Continued D.
• In the case of a clearly identified laboratory error, the retest
results would substitute for the original test result. All original
data should be retained, however, and an explanation
recorded. This record should be initialed and dated by the
involved persons and include a discussion of the error and
supervisory comments. (See section III of this guidance for
more details on a laboratory investigation.)
• If no laboratory or calculation errors are identified in the first test, there
is no scientific basis for invalidating initial OOS results in favor of
passing retest results. All test results, both passing and suspect, should be
reported 9 and considered in batch release decisions.
• 9 In other words, all data are reported in, for example, quality control
reports, batch records, Certificates of Analysis, in accordance with §§
211.188 and 211.192.
OOSsess7 1215 13
Resampling and Retesting. B. Additional
Laboratory Testing • 2. While retesting refers to analysis of the original,
homogenous sample material, resampling involves analyzing a
specimen from any additional units collected as part of the
original sampling procedure or from a new sample collected
from the batch, should that be necessary.
• The original sample from a batch should be sufficiently large
to accommodate additional testing in the event an OOS result
is obtained. In some situations, however, it may be appropriate
to collect a new sample from the batch. Control mechanisms
for examination of additional specimens should be in
accordance with predetermined procedures and sampling
strategies (§ 211.165(c)).
OOSsess7 1215 14
B.1. Retesting. Comments V.
• Note that there are tests that are not retests.
– A test that does not follow the original test method is
not a retest.
• Product cannot be released using this test, but it can provide
information that may be considered in thinking about the OOS.
– For instance, a U.V. test for protein, could be checked
by a dye-binding test, or vice versa.
– A failed electropherogram, could be checked using a different
electrolyte solution at a different pH, or ionic strength.
– If a dilution series was used, testing one of the intermediate
solutions or re-introducing an aliquot of the final dilution, will be a
re-run, not a retest.
OOSsess7 1215 15
B.2. Resampling. Comments VI.
• Note that in some cases such as with in-process
samples, a resampling may not be possible.
– In these cases, you have only the retesting results to go
on.
• In some cases you may be confronted with an ugly
choice. You may not resample unless you have
information indicating that the original sample
was bad, but you may not be able to prove this
without resampling.
– If the resample indicated that you original sample was
bad, you must conduct studies to find out why. (Note
the next slide.)
OOSsess7 1215 16
Resampling and Retesting. B. Additional
Laboratory Testing. 2. Continued• When all data have been evaluated, an investigation might
conclude that the original sample was prepared improperly
and was therefore not representative of the batch quality (§
211.160(b)(3)). Improper sample preparation might be
indicated, for example, by widely varied results obtained from
several aliquots of an original composite (after determining
there was no error in the performance of the analysis).
Resampling should be performed by the same qualified,
validated methods that were used for the initial sample.
However, if the investigation determines that the initial
sampling method was inherently inadequate, a new accurate
sampling method must be developed, documented, and
reviewed and approved by the QCU (§§ 211.160 and
211.165(c)).
OOSsess7 1215 17
B.2. Resampling. Comments VII.
• Note: If a new sampling method is introduced, this
will invalidate the previous sampling method.
– Otherwise, why are you developing a new method?
– If the previous sampling method was a source of
problems, you must review all of the previous tests that
used that sampling plan.
– The new sampling method needs to be validated. If the
sampling method is an inherent part of a test method, it
may be necessary to revalidate the whole method.
– If the old sampling method was used in several test
methods, be sure to evaluate the new sampling method
with all affected test methods.
OOSsess7 1215 18
CAPA and Other Corrective Actions. I.
• The efficacy of the training and re-training of an
analyst should be demonstrated by comparative
statistical tests.
• The supervisor should be monitoring OOX to
know if an analyst is committing too many errors.
– Re-training may be nothing more than more practice.
– New equipment or procedures may lead to simple OOX.
– If OOX are the result of difficult or complex actions, the
Test Method SPO should be re-written to simplify it.
– OOX that arise from statistical variation are very
difficult to detect due to their random nature.
OOSsess4 1215 19
CAPA and Other Corrective Actions. II.
• All actions taken in response to OOX should be
recorded and placed in the documentation for all
affected test methods.
– For instance, weighing and dilution errors may affect
more than one test method.
– Any OOX beyond a simple lab error will need a CAPA.
– Repetitive simple errors will also require a CAPA.
• Simple lab errors still require re-runs that add costs.
– CAPAs need completion dates and records.
– Explain the reason for a CAPA and how it will correct
the problem.
OOSsess4 1215 20
Starting Samples
• The regulations (21 CFR 211.170 (a & b) only
require holding retention samples for the API and
the final product, so many companies do not hold
retention samples from excipients, intermediates
or other products.
– “The reserve sample consists of at least twice the quantity
necessary to perform all the required tests, except those
for sterility and pyrogens. . . . .”
– Consequently, cheap companies do not take large
enough samples to allow for OOS retesting.
– This is justified only if OOS are rare events.
– Hold extra material for any test that can lead to an
OOS.
OOSsess4 1215 21
Sample Size
– The basic sample size is the minimum amount needed to
perform all of the tests that will be required for that
sample.
• Always allow some overage as there will be some loss due to the
transfer process, and more than the exact, calculated, sample
quantity will be needed.
– For regulatory purposes, the retention sample should be
twice the basic sample size.
• Set these aside as “regulatory retains.”
– Take another amount of twice the basic sample size as
the testing sample.
• One part is for actual testing and the second for OOS retesting.
• The OOS should only involve one test, and you must not “test
into compliance.”
OOSsess4 1215 22
From a Warning LetterClaim of a Dilution Error
• Specifically, Pedia D Suspension, failed the assay specification for CCC Tannate (Specification (b4)) with an averaged result of (b4). The other active ingredient used in the product, PPP Tannate, was within assay specification. Your firm's investigation revealed that laboratory error did not contribute to the OOS Assay result and that the probable cause was a insufficient quantity of water added during the QS phase of mixing, resulting in a higher concentration of all ingredients.
• However, the investigation did not document any explanation for the normal assay values observed for the second active ingredient, PPP Tannate.
OOSsess4 1215 23
Write Up Your Reasoning. 1.
• Some people find that documenting an OOS
investigation is hard because they lack a defined
format for the write up.
– If nothing else, the testing group should develop a
format for recording the investigation.
– You will want two formats. One for laboratory level
investigations, and one for full scale investigations.
• Resampling and retesting can be done either during a
laboratory level investigation or a more extensive one.
• If necessary, call on a good recorder for the investigation.
• Integrate these formats and the reports into your laboratory
documentation system, so they are coupled to the test method
and the record of the OOS.
OOSsess7 1215 24
Write Up Your Reasoning. 2.
• 1. Record the OOS.
– The test method SOP, sample number, OOS result,
acceptance range, analyst, date and time the OOS was
noted, location of the laboratory record with the
original findings.
– Who was notified, when, and how.
– Analyst comments and supervisor’s review and
comments with dates and signatures.
– Initial findings. (A simple (obvious) OOS does not even
need to go this far. I can be recorded in the test record.)
– Recommendations for the next step.
OOSsess7 1215 25
Write Up Your Reasoning. 3.
• 2. Re-testing. Say why retesting is being done.
– Where is the sample coming from?
• What test method will be used? It should be identical.
– Justify the number of replicates that will be used.
• There are statistical calculations that can be made.
• Explain the reasoning for the number of replicates.
• 3. If the retest confirms the OOS.
– The OOS must still be investigated to find its cause.
– A full-scale investigation will be required.
• 4. If the OOS is invalidated.
– The new result can replace the OOS, but the data
should be strong and the reasoning recorded.
OOSsess7 1215 26
Write Up Your Reasoning. 4.
• 5. Resampling.
– Why are you resampling? The reasoning should be
strong enough that, if correct, the reason will invalidate
the original sampling.
– Or you have a strong reason to believe that the sample
was problematic.
• The guidance talks about variation in samples taken from a
composite, but this is not reasonable in that your original test
method validation should have shown that your compositing
procedure would produce a homogenous composite.
• It does not make sense to take several aliquots from a
composite. Why did you composite them in the first place? The
sampling and test method precision should have been worked
out during the test method validation.
OOSsess7 1215 27
Write Up Your Reasoning. 5.
• 5.a. Resampling
– Resampling will require retesting.
– The retesting should be done under the same conditions
as the initial testing.
– If the resampling confirms the OOS, proceed to a full
scale investigation to determine the cause of the OOS.
– If the OOS shows that there was a problem with the
original sample, the investigation must continue to
determine the cause of the problem.
• A CAPA will be needed.
– If the sampling method was inherently bad, it must be
invalidated and re-designed.
OOSsess7 1215 28
Write Up Your Reasoning. 6.
• If the OOS is confirmed, but the decision is made
to release the material, the documentation and
reasoning must be very strong.
– In one instance, a product was released despite an OOS.
The argument was made that the OOS did not
compromise patient safety.
– If that was true, why did the specification exist in the
first place?
OOSsess7 1215 29
483’s A.
• a) Your firm's 2007 OOS investigation into high levels of (b)(4) found
in (b)(4) lot (b)(4) was not completely documented, nor was the
investigation extended to other lots. You indicated that the cause of the
high levels of (b)(4) within (b)(4) lot (b)(4) was related to a component
used in the manufacture of (b)(4) lot (b)(4) namely (b)(4) lot (b)(4)
The investigation did not document the testing of (b)(4) lot (b)(4) for
(b)(4), nor could the results be located or provided during our inspection.
There were (b)(4) lots of (b)(4) and (b)(4) used in the production of (b)(4)
lot but only one lot (b)(4) of (b)(4) was tested. This lot of (b)(4) was also
used in the manufacture of (b)(4) lot (b)(4), but you did not conduct
further investigation into (b)(4) lot (b)(4). Also, you did not evaluate (b)(4)
lot (b)(4) for high levels of (b)(4) as a potential additional source of (b)(4)
in (b)(4) lot (b)(4) .You also used (b)(4) lot (b)(4) in the manufacture of lot
(b)(4).
OOSsess7 1215 30
483’s B.
• b) Your firm's OOS investigation relating to
impurity levels for (b)(4), lot (b)(4) , concluded
that the root cause was a laboratory error, but the
investigation did not identify what specific
laboratory error occurred. Initial results for both
Highest Individual Impurity (specification NMT
(b)(4)%) and Total Impurities (specification
(b)(4)%) were OOS at (b)(4)% and (b)(4)%
respectively. The investigational checklist initially
indicated that no problem was found with the
analysis.
OOSsess7 1215 31
483’s C.
• The investigational checklist you currently use is
insufficient to detect and evaluate instrument problems
and standard/sample preparation errors.
• You authorized retesting of (b)(4), lot (b)(4), without
identifying a possible root cause. Instead, a new sample
preparation was used to retest the product, which was
found within specification.
• You used the passing retest results to invalidate the
original OOS results, with no laboratory error attributed
in obtaining the original result.
• This retesting approach lacks scientific justification.
OOSsess7 1215 32
483’s D.
• The 00S SOP provides little or no detail regarding
retesting. The retesting section of the SOP should be
very detailed regarding the specific course of action
to be pursued when an 00S event is encountered.
• We believe this statement leaves open the possibility
that averaging of results may be used in some
circumstances.
• Averaging of passing and failing analytical results to
obtain passing results Not an acceptable practice.
OOSsess7 1215 33
483’s E.
• Numerous inconsistencies were noted in the handling of data
and the decisions made in response to these 00S results. You
have failed to maintain adequate documentation to substantiate
the invalidation of 00S results. This was noted during content
uniformity, assay, and dissolution testing. The inspection
noted instances of the failure to follow procedure, substitution
of standards, discarding of 00S results without an
investigation, and reporting of only passing results. Your
procedure for the Handling of Out of Specification Results
allowed for the improper discarding of 00S results without an
investigation. The procedure allowed for the discarding of
results without sufficient documentation of the failures and
any corrective actions taken to prevent reoccurrence.
OOSsess7 1215 34
483’s F.
• Your firm has failed to properly investigate and document 00S
results obtained from malfunctioning laboratory equipment.
High and missing dissolution results were noted reportedly due
to problems in the operation of the XXXX workstations.
• Similarly high atypical values were generated in the
dissolution and content uniformity testing due to problems
with the operation of the YYY workstation. This data was
invalidated without any review by a supervisor or other
responsible official. The lack of an investigation or appropriate
documentation of these instrument failures makes the trending
of these problems impossible. This allows recurring and
persistent instrument failures to go uncorrected.
OOSsess7 1215 35
