Date post: | 09-Mar-2018 |
Category: |
Documents |
Upload: | doannguyet |
View: | 245 times |
Download: | 10 times |
1
AOAC – NMKL - NordVal
International Symposium 21/22 Mai 2015 in Stockholm
Harmonization of the NordVal International
Validation Protocol with ISO 16140:2015
for the validation of alternative methods.
Sven Qvist
Chairman
NordVal International
www.nmkl.org
2
Recognised international validation organisations have participated in the elaboration of the new ISO/FDIS 16140-2:2015.
AFNOR Certification
AOAC
MicroVal
NordVal International
ISO 16140 method validation
program consists of 6 parts 1. Vocabulary
2. Protocol for validation of alternative methods
3. Protocol for verification of reference and
alternative methods in a single laboratory
4. Protocol for in-house method validation
5. Protocol for interlaboratory validation
6. Protocol for validation of confirmation methods
3
4
Objective of an international protocol for validation of alternative methods
Guidance for applicants, kit producers, expert laboratories, steering groups and technical committees on the procedure of validation of alternative microbiological methods
Obtain international recognition of validations carried out according to ISO 16140:2015
5
Present legal basis for the use of alternative microbiological methods.
EC 2073/2005 states that the use of alternative methods are acceptabel when validated against an authorised reference method and certified in accordance with ISO 16140:2003 or other internationally similar protocol.
6
Importance of validation
• To ensure reliable analytical results
• To ensure a high level of food safety
• Demonstrate equivalence with ref. method
• Demonstrate fit for purpose
• To be in accordance with EU regulations
• To be in compliance with the manufacturer's
claims
7
Validation protocol contain sections for both qualitative and quantitative methods - both divided into:
comparison study of the alternative method
against a reference method
(performed by an expert laboratory)
collaborative study of the alternative method.
Valid results from at least 10 laboratories
(inclusion of 12-15 labs is advisable).
NordVal International criteria for
approval of expert laboratories
• Economical independent of the applicant.
• Accredited to perform the ref. method.
• Be familiar with the principle of the
method to be validated.
•A competent person shall be in charge.
•A detailed protocol shall be approved.
8
9
Method Comparison Study - Qualitative methods
For food: at least 5 relevant food matrices
For other categories: a relevant number of matrices.
For each matrix 60 samples are analysed
Inclusivity, exclusivity, sensitivity, accuracy,
specificity, limit of detection and agreement
between methods are studied
Method comparison study on
inclusivity and exclusivity
Inclusivity: Select 50 pure cultures of the
target microorganisms relevant to the
alternative method and the food matrices
(for Salmonella select 100 pure cultures)
Exclusivity: Select 30 pure cultures of
non-target microorganisms known to
cause interference with the target
microorganism
10
11
Method comparison study for accuracy, sensitivity and specificity Meat matrices in the present NordVal protocol
comprises one main category and 5 submatrices:
1.1 Raw red meat
1.2 Raw white meat
1.3 Raw smoked and salted products
1.4 Heat treated products
1.5 Fermented products
12
Method Comparison Study – Matrices for meat
New ISO 16140:2015 validation protocol – 4
categories
1. Raw meat and ready-to- cook meat products
2. Ready-to-eat and ready-to-reheat meat products
3. Raw poultry and ready-to-cook poultry products
4. Ready-to-eat and ready-to-reheat poultry products
13
Method Comparison Study – Matrices for meat ISO 16140:2015 validation protocol - Category 2:
2.1 Cooked meat products
2.2 Fermented or dried meat products
2.3 Raw cured (smoked) products (aw > 0,92)
2.4 Raw cured (smoked) products (aw < 0,92)
ISO Interpretation of results
between ref. and alt. method
Reference
Method
Alternative
Method
+
+
-
-
+
-
-
+
-
+
Confirmed
Alt. Method
Interpretatio
n
Not needed
PA
Not needed
NA
Not needed
ND
+
PD
-
NA 14
15
Method Comparison Study - Qualitative methods - Screening
Responses Reference method
positive (+/ )
Reference method
Negative (-/)
Alternative
method
Positive
+/+ positive
agreement
(PA)
-/+ positive
deviation
(PD)
Alternative
method
Negative
+/- negative
deviation (ND)
-/- negative
agreement (NA)
16
Method Comparison Study - Qualitative methods - Confirmation
Responses Reference method
positive (+/ )
Reference method
Negative (-/)
Alternative
method
Positive
+/+ positive
agreement
(PA)
-/+ true positive
(TP)
Alternative
method
Negative
+/- false negative
(FN)
-/- negative
agreement (NA) +
false positives (FP)
17
Estimation of accuracy, sensitivity, specificity and agreement - Confirmation Matrix PA
NA
ND
PD
Sum Relative Accuracy
AC (%)
Relative
Sensitivity SE (%)
Relative
Specificity SP
(%)
N (PA+NA+FP) x 100
N
PA + TP x 100
PA+FN
NA x 100
NA+FP
Food cat 1
Food cat 2
Food cat 3
Food cat 4
Food cat 5
Other
18
NordVal International acceptance
criteria for qualitative methods
The overall sensitivity should 95%
The agreement between method, kappa
should indicate very good agreement,
i.e. κ > 0.80.
19
Relative level of detection study of the alternative method against the reference method- ISO 16140:2015
Level 1 Level 2 Level 3
Concentration negative low higher
No of replicates 5 20 5
No of positives 0 25-75% 100%
RLOD =
LOD - alt.
LOD - ref.
Method collaborative study
of a qualitative method
Determine the variability of the results obtained by the alternative method in different laboratories using identical samples
The expert and 10 participating laboratories are using both the alternative and the reference method for the study (10 valid results required)
One food matrix, 8 replicates and three different levels of contamination are analysed in the study
20
Organisations and locations of
the participating laboratories
The 10 collaborators shall come from a minimum of 5 different organisations (advisable with at least 12 collaborators)
A maximum of 3 data sets can be produced by one organisation with laboratories in different locations
T
21
22
Performance characteristics for
the study of quantitative methods
Inclusivity
Exclusivity
Reliability
Repeatability
Reproducibility
Uncertainty
23
Comparison Study-Quantitative methods
For each matrix, calculate the mean and the SD of
the results obtained by the reference method and
its confidence level 2 ∙ SD
Calculate the mean and the SD of the results
obtained by the alternative method.
Plot the results into a designed diagram.
24
Quantitative Methods
Milk powder
0
1
2
3
4
5
6
0 1 2 3 4 5 6
levels
log
cfu
/g
reference method
alternative method
25
Acceptance criteria for
quantitative methods
If the mean of the alternative method is included in the confidence interval of the reference method (i.e. refmean ± 2 SD), there is no significant deviation between the methods.
Repeatability of less than 0.4 log cfu/g illustrates satisfactory precision.
26
NordVal International Steering Group
Denmark: Sven Qvist (chairman)
Finland: Annika Pihjalasaari
Iceland: Franklin Georgsson,
Norway: Kjell Hauge
Sweden: Charlotta Engdahl Axelsson
Secretary: Hilde Skår Norli, NMKL
NordVal International Organisation
• NMKL national committees appoint the steering
group members - one representative from each
of the Nordic countries.
• NordVal International appoints experts in ad
hoc technical committees.
27
NordVal International policy
• The persons involved in the NordVal
International validation programme shall be
independent and cannot have special
interests in the production and distribution
of test kits, special reagents or instruments.
• The members must have the required
competence in microbiology and validation.
28
29
30
31
32
33
Future status of validated and
certified alternative methods
• Capacity, rapidity, cost effectiveness
and reliability have resulted in
increased and dominating role as
preferred methods in the food industry.
• Acceptance by authorities as official
standard reference methods should be
considered as a possibility in future
legislation. 34
NordVal International policy
• The persons involved in the NordVal
International validation programme shall be
independent and cannot have special
interests in the production and distribution
of test kits, special reagents or instruments.
• The members must have the required
competence in microbiology and validation.
35
Increased scope for items to be
covered by ISO 16140-2:2015
• Present validation protocol: Food, animal feed,
environmental samples and samples from the
primary production stage.
• Future items for consideration: Water, virus,
confirmation methods, microtoxins, parasites.
36