Understanding the uses, and
limitations, of attributes sampling
plan
Tom Ross
Food Safety Centre, University of Tasmania and
International Commission on Microbiological Specifications for Foods
Overview
• attributes sampling plans
• statistics of sampling plans
• limitations of testing
• other complications
• sources of advice on sampling and sampling plan
design
Sampling plans
• “attributes” plans
– “2 class” or “3 class”
– test against a (probability of a) specific level of the hazard
– used by buyers, regulators
• “variables” plans
– incorporate all measured levels of the hazard to assess compliance
– must know mean and standard deviation of hazard levels, i.e. a history of quantitative results
“attributes” sampling plans:
• intended to tell what proportion of the units in
the lot meet some criterion (i.e., the
“attribute” we are interested in)
• how many are “acceptable”; how many “pass”,
or “fail”
attributes sampling plans
defined by three (or four) quantities:
• ’m’ – the microbiological limit (e.g., <1 per 25g, 100 cfu/g) desired (= the “attribute”)
– may implicitly dictate the sample size
• ‘n’= the number of samples from the lot that must be tested
• ‘c’ = the number of samples that may exceed ‘m’
• ‘M’ – applied in some cases to recognise occasional deviations, a level higher than ‘m’ but one which must never be exceeded
2 and 3-class attributes plans
2 class:
the criterion is absolute, sample units either “pass” or “fail”
described only by m, c, n
3 class:
the criterion includes a third category, i.e., “marginally acceptable (“M”)
described by M, m, c, n
log count / g
Rela
tive p
rop
ort
ion
of
sam
ple
un
its in
lo
t
log count / g
Rela
tive p
rop
ort
ion
of
sam
ple
un
its in
lo
t
Two- and three-class attributes plan m m
2-class plan 3-class plan M
acceptable
unacceptable
acceptable
unacceptable
marginally acceptable
Wrong!
In fact, we can never demonstrate ‘zero tolerance’, only a
certain level of confidence that the contamination is
below a certain level
Common Misconception about
Presence/Absence Testing
“If the test is negative, the batch is free from
pathogens”
Sampling and the Probability of Detection
<50% defective?
<10% defective?
<1 % defective?
Sampling and the Probability of
Detection
• probability of detection depends on
– actual proportion of samples that are unacceptable
– number of samples examined
– it is easily described mathematically (a binomial sampling process)
- (in fact, the ‘hypergeometric’ distribution is more correct, but for low contamination values, the results are nearly equivalent)
Pacceptance = (1 - Pdefective)number of samples examined
Binomial Distribution
• Pacceptance often = 0.05 (i.e., 95% confidence that a ‘bad’
batch will be detected and rejected
• Pdefective = real proportion defective or the upper limit of
our tolerance of units that don’t meet our criterion
Pacceptance = (1 - Pdefective)number of samples examined
“attributes” sampling plans
confidence in the result of testing is based on:
– number of samples taken
– true (or required) frequency of compliance
plan performance (confidence) is summarised as
‘operating characteristic curve’
0
0.2
0.4
0.6
0.8
1
0 0.2 0.4 0.6 0.8 1
Proportion defective sample units in a lot
Pro
ba
bil
ity o
f a
cce
pti
ng
th
e l
ot
n=5
Operating Characteristic Curve
(n=5, c=0)
Probability of accepting a lot (Operating Characteristic Curves)
0
0.2
0.4
0.6
0.8
1
0 0.2 0.4 0.6 0.8 1
Proportion defective sample units in a lot
Pro
ba
bil
ity o
f a
cce
pti
ng
th
e l
ot
n=5
Probability of acceptance with 5 negative
samples depends in true ‘defect’ rate
Pacceptance = 0.05 with 5 samples (all negative); depends on true ‘defect’ rate
n = 5, c = 0
• means that we can be 95% confident only that, on
average, less than half the units in the lot exceed
the ‘attribute’, m.
0.05 = (1 - Pdefective) 5
=> Pdefective = 0.46
what’s acceptable (“appropriate level of protection”)
• 50 % defective (=contaminated)?
• 1 % contaminated?
• 0.1% contaminated?
• 0.01% contaminated (1 in 10,000 chance)?
samples required for 95% confidence of rejection of a non-compliant lot
• no sampling plan can ever guarantee absolute safety of the food
• the sampling plan can only provide a certain level of confidence
• it is virtually* impossible to use sampling and testing to determine whether a lot of product is acceptable when the tolerable level of failure is very low, or zero
• better to rely on preventive approaches
Limitations of Testing
• methods may not recover injured cells
• microbiological condition of the food changes over time
• microbial contamination isn’t uniformly distributed
throughout the lot
• even if microbes were completely evenly distributed the
samples would not contain identical numbers of cells
Limitations of Testing
• methods may not recover injured cells
• microbiological condition of the food changes over time
• microbial contamination isn’t uniformly distributed
throughout the lot
• even if microbes were completely evenly distributed the
samples would not contain identical numbers of cells
Variability and acceptable levels
Variability and acceptable levels
Limitations of Testing
• methods may not recover injured cells
• microbiological condition of the food changes over time
• microoorganisms aren’t uniformly distributed throughout the lot
• even if they were completely evenly distributed the samples would
not contain identical numbers of cells
Poisson processes
• random sampling from a population when assessing
a (binomial) attribute is a Poisson process
• the chances of making a particular number of
observations, in a given time, or space, based on
some true average density, or frequency, is defined
by the Poisson distribution
• for example….
• if the concentration of a pathogen in a food:
– were exactly 1 cell per cm3, and
– the cells were perfectly evenly distributed, and
– our sample size was exactly one cm3
• our samples of 1 cm3 would not always contain 1
cell….
Poisson processes
Poisson sampling “error”
i.e., mean concentration is one cell per sample unit
i.e. but a sample could, by chance, contain four
cells …
Poisson sampling “error”
Poisson sampling “error”
or could contain two cells …
Poisson distribution
• The probability that a certain number of ‘events’
(e.g., number of positive samples) will be observed,
when the true average of those ‘events’ is known, is
given as:
P(observing n positives) =
(true_averagen ) ´ e-( true _ average )
n!
ICMSF Sampling Plans – ‘risk’ basis
incre
asin
g s
everity
of c
on
seq
uen
ce
increasing probability of unacceptable hazard
Examples of Sampling Plans
ICMSF: Microorganisms in Foods, 2. Sampling for Microbiological Analysis. Principles and Specific Applications,
2nd Edition, Blackwell Scientific Publications, 1986 (ISBN-0632-015-675).
Advice on Sampling Plan Design and Interpretation (http://www.icmsf.org)
Attributes Sampling Plans
• are like the results of MPN tests
• if we can quantify, or estimate, the variability of
contamination levels in a lot, we can calculate the
sensitivity and confidence of the sampling plan
using the equations and understanding just
presented
• its complicated!
Food Control, 20 (2009): 967 - 979
ICMSF Sampling Plan Spreadsheet (www. icmsf.org)
Conclusions
• in the absence of non-destructive testing, we can never
guarantee that a lot contains no unacceptable units
• transparent, science-based criteria and sampling plans
can be developed and their performance compared based
on knowledge of the SD of contamination within a lot
• correct design, and interpretation, of sampling plans is
complex
• user-friendly tools are available, for free, to assist
Useful Reading
van Schothorst, M., Zwietering, M.H., Ross, T., Buchanan, R.L and Cole, M.B. International
Commission on Microbiological Specifications for Foods. (2009). Relating
microbiological criteria to food safety objectives and performance objectives. Food
Control, 20: 967-979.ICMSF
R.C. Whiting, A. Rainosek, R.L. Buchanan, M. Milioti, D. LaBarre, W. Long, A. Ruple and S.
Schaub. (2006). Determining the microbiological criteria for lot rejection from the
performance objective or food safety objective. International Journal of Food
Microbiology, 110: 263–267.
ICMSF (International Commission on Microbiological Specifications for Foods), 2002.
Microorganisms in Foods, Microbiological Testing in Food Safety Management, Vol. 7.
Kluwer Academic/Plenum Pub, NY. 362 pp.
Acknowledgements
SAAFoST
International Commission on Microbiological
Specifications of Foods
