Food Quality and Safety
Diagnosis of your food safety
management system
performance : output
Pieternel Luning & Willem Marcelis (WUR),
Liesbeth Jacxsens (UGent)
Food Quality and Safety
Topics
• Introduction
• Food Safety Management System (FSMS)
diagnostic instrument
• Food Safety Performance Indicators (SPI)
diagnose
• Example of case company X
Food Quality and Safety
Introduction
Safe products
Food safety assurance requirements
PRP, CODEX, GMP,HACCP, BRC, ISO22000, SQF, etc………
Company specific Food Safety Management System
Stakeholders
Government, retailers, branch organisations, sector boards, etc
Dynamic environment
Socio-demographic changes, technological developments,
emerging pathogens, global sourcing, etc
Food Quality and Safety
This slide illustrates that companies have to operate in a dynamic environment, and
that stakeholders (that are government, retailers, branch organisations, etc) put
assurance demands on the company’s FSMS.
They require that the set-up of the company’s FSMS is according to e.g. PRP, HACCP
guidelines and or BRC standard, and or ISO22000 standard, etc.
So the stakeholders affect the company’s FSMS set up by demanding the
implementation of certain QA guidelines/standards.
Keep in mind however, that each company has a UNIQUE FSMS which is a
translation of all the QA requirements into their own specific system
Food Quality and Safety
Food Safety Management System
Food Safety Management System
Stakeholder
requirements
Food safety assurance activities
to control safety management system and to provide
evidence and confidence to stakeholders about
meeting safety requirements
System requirements feedback
Assurance
on
Product
Safety
Food safety control activities
aim at keeping product properties, production
processes, and human processes between certain
acceptable tolerances
Product
safety
Food Quality and Safety
This slide is to show the companies that there are actually to outputs of their
FSMS.
One output is the realisation of safe products, which is the result of all actual
control activities as executed by equipment and people during daily operation
under the given contextual situation.
The other output is assurance of product safety, which means being able to give
the assurance that product as made under the company’s conditions are safe.
Control activities contribute to the first output whereas assurance (indirectly affects
the first output) but results in assurance.
So it does not mean that if a company is not/scarcely elaborating assurance
activities that products are not safe….but it induces a risk!!!
Food Quality and Safety
Topics
• Introduction
• Food Safety Management System (FSMS)
diagnostic instrument
• Food Safety Performance Indicators (SPI)
diagnose
• Example of case company X
Food Quality and Safety
FSMS diagnostic instrument :
general picture
Product
characteristics
Process
characteristics
Organisational
characteristics
Environmental
characteristics
Product
safety
feedbackSystem requirements
Food safety assurance
Food safety control
Preventive measures design
Intervention processes design
Monitoring system design
Operation control strategies
Setting system requirements
Validation
Verification
Documentation and record keeping
Assurance
on product
safety
Food Quality and Safety
FSMS Diagnostic Instrument :
control activities
CORE CONTROL ACTIVITIES
Preventive measures design
•Sophistication hygienic design equipment & facilities
•Specificity of sanitation program
•Extent personal hygiene requirements
•Raw material control, etc….
Intervention processes design
•Adequacy intervention equipment
•Specificity maintenance program
•….etc
Monitoring system design
•Appropriateness CCP analysis
•Adequacy analytical equipment
•…..etc
Operation control strategies
•Appropriateness and compliance to procedures
•Actual performance cooling, intervention, measuring equipment ,..etc
Food Quality and Safety
FSMS Diagnostic Instrument :
assurance activities
Food safety control system
CORE ASSURANCE ACTIVITIESDefining system set-up
•Sophistication translating external requirements
•Extent of systematic use of feedback information
Validation
•Sophistication validation
preventive measures
•etc
Verification
•Extent of verification of people
related performance
•Extent of verification equipment
related performance
Documentation and record-keeping
Food Quality and Safety
This slide shows the core assurance activities. The primary objectives of assurance
are to control the system, check its effectiveness, check whether it works in practice
as designed, and adapt the system when changes are necessary. All in order to
provide evidence and confidence to stakeholders that your system is able to comply
with the safety and assurance requirements.
An important assurance activity is defining the specific system set up for the company
(based on stakeholder requirements and internal feedback information about the
FSMS performance).
Validation is aimed ate checking effectiveness in advance, whereas verification is a
check afterwards (when to check if the system is functioning properly in practice).
Documentation and record keeping support above activities and play a major role in
providing evidence and confidence to stakeholders.
Food Quality and Safety
Example of grid
Indicator: Specificity cleaning and disinfection program
Mechanism
Complete, tailored
sanitation program
with appropriate
cleaning agents,
supported with
appropriate user
instructions better
prevents
contamination
positive contribution
to food safety
Level 1
•Incomplete program
equal for all
•Equipment & facilities.
•Common cleaning
agents not specific
for production system.
•Cleaning instructions
derived from
information on label
or company
experience
Level 2
•Complete programme
equal for all equipment
and facilities.
•Cleaning agents
selected based on
advices of suppliers;
•specific agents for
typical product
applications.
•Idem for instructions
Level 3
•Complete programs,
tailored different
equipment/facilities
•Cleaning agents
•specifically modified
•and tested on
effectiveness for
specific food
production system.
•Instructions based
on test results.
Food Quality and Safety
Criteria to differentiate FSMS
(=FSC + FSA) levelsControl activities
• Level 3 high i.e. scientifically underpinned accurate, complete, stable, predictable, tailored
• Level 2 medium, i.e. Best practice knowledge, sometimes variable, not fully predictable, generic information
• Level 1 low i.e. lack of scientific evidence, use company experience, history, variable, unknown, unpredictable, common materials, equipment
Assurance activities
• Level 3 high i.e. systematic, pro-active, scientific evidence, independent, experimental trials, well-documented
• Level 2 medium, i.e. regular, reactive, expert knowledge/opinion, internal independent, no actual testing, restricted documentation
• Level 1 low i.e. ad hoc, reactive, history/ company knowledge, dependent, no testing, no documentation
Food Quality and Safety
Product characteristics
Organisational characteristics
• Lack of technical workforce
• Variability in workforce composition
• Insufficiency operators competence
• Lack of commitment
• Deficiency of employee involvement
• Absence of formalisation
• Insufficiency information support
systems
Process characteristics
Environmental characteristics
Food safety assurance
system
Food safety control
systemProduct
safety
Assuranc
e on
product
safety
CONTEXTUAL FACTORS FOOD SAFETY MANAGEMENT SYSTEM OUTCOME
Food Quality and Safety
Criteria to differentiate context situations
• Situation 1 : not critical, not vulnerable
• Situation 2 : potentially critical, potentially vulnerable
• Situation 3 : highly critical, highly vulnerable
• Situation 1: high ability, advanced, specific
• Situation 2:constrained ability, restricted advanced, specific
• Situation 3: low ability, not advanced not specific
•Situation 1: low dependency, not vulnerable
•Situation 2: restricted dependency, potentially vulnerable
•Situation 3: highly dependent, highly vulnerable
Product/process Organisational Environmental
Food Quality and Safety
FSMS diagnostic instrument :
interpretation of results
• Addresses:
– Core control activities
– Core assurance activities
– The contextual factors of company
• Assesses
– at which level control & assurance activities are executed (0,1,2,3, simple sophisticated)
– in which risky contextual situation the FSMS has to operate (1,2,3 not demanding very demanding and riskfull)
• Aggregates
– Insights in relationships between context, activities and food safety performance
Food Quality and Safety
• Output of the assessment gives indication of:
– Insights in relationships between context, activities
and food safety performance
– At which level the specific control and assurance
activities of food safety control and assurance are
now
– How to go – what is needed for the higher level in
control and assurance activities
– Middle or long term improvements can be identified
• NOT:
– Judgement tool no audit is performed
FSMS diagnostic instrument :
interpretation of results
Food Quality and Safety
Interpretation results of contextual factors
• The contextual situations are represented in spiderweb diagrams
A more coloured spiderweb diagram is associated with a more demanding and risky contextual situation
A more risky contextual situation is expected to result more easily in food safety problems, which will put higher demands on the FSMS.
• To obtain an overall picture of the contextual situation, overall scores have been assigned For this purpose the mean scores for a certain set of characteristics are calculated by dividing the sum of situation numbers by the number of situation assessments for this specific set of characteristics (see the legend of a spiderweb diagram).
• The overall score has been assigned by interpreting the mean score of all contextual characteristics as:– Overall score 1: if mean score of characteristics is 1 - 1.2
– Overall score 1-2: if mean score of characteristics is 1.3-1.7
– Overall score 2: if mean score of characteristics is 1.8 - 2.2
– Overall score 2-3: if mean score of characteristics is 2.3 - 2.7
– Overall score 3: if mean score of characteristics is 2.8 - 3.0
Food Quality and Safety
Interpretation results of Food Safety Control and
Assurance Activities• The levels of the core safety control and assurance activities are
represented in spiderweb diagrams
The spiderweb diagrams represent the detailed levels of the core safety control activities and the core safety assurance activities
A more coloured spiderweb diagram is associated with a higher/more sophisticated level of control activities and assurance activities.
• To obtain an overall picture of the FSMS activities, overall scores have been assigned For this purpose the mean scores of levels for a certain set of activities were calculated by dividing the sum of activity levels by the number of activities of this specific set.
• The overall score has been assigned by interpreting the mean score:– Overall score 1: if mean score of activities is 0 - 1.2
– Overall score 1-2: if mean of score of activities is 1.3-1.7
– Overall score 2: if mean score of activities is 1.8 - 2.2
– Overall score 2-3: if mean score of activities is 2.3 - 2.7
– Overall score 3: if mean score of activities is 2.8 - 3.0
Food Quality and Safety
Assumption behind diagnosis
• High risks products and/or processes (= higher overall score in contextual factors) put higher demands on FSMS to achieve safety requirements
• Less supporting organisational conditions and/or higher chain dependency (= higher overall score in contextual factors) result in higher impact on FSMS to achieve safety requirements
Which FSMS levels is necessary?
It depends !!
Food Quality and Safety
Assumption of output
Context FSMS FS
3
1
2
1
2
3
1
2
3
3=most dangerous 3=highest level 3=best performance
Food Quality and Safety
Topics
• Introduction
• Food Safety Management System (FSMS)
diagnostic instrument
• Food Safety Performance Indicators (SPI)
diagnose
• Example of case company X
Food Quality and Safety
Food safety performance indicators
• The microbiological food safety output of a companies Food Safety Management System can be measured via microbiological analysis of raw materials, intermediated products and final food products
• But the food safety output can as well be measured via Food Safety Performance Indicators (SPI)
• Indicators are measurable, objective, quantitative measures of key system elements performance
• They indicate the extent upto which a certain Food Safety Management System meets the needs and expectations towards microbiological food safety
• They can be applied for self-evaluation or for benchmarking
• They are applied in this work to measure the food safety output
Food Quality and Safety
Food safety performance indicators
For each indicator, a food safety level is attributed :
• Level 0 : not conducted, not present, not appropriate
• Level 1 : low performance of food safety problems
regarding food safety can be expected and are not
under control by the current food safety management
system
• Level 2 : medium performance of food safety problems
regarding food safety can be expected and are under
control by the current food safety management system
• Level 3 : high performance of food safety no problems
regarding food safety can be expected
Food Quality and Safety
Food safety performance indicators
• To obtain an overall picture of the food safety performance, overall scores have been assigned.
• For this purpose the mean scores of levels for a certain set of food safety performance indicators need to be calculated by dividing the sum of performance levels by the number of indicators of this specific set.
• The overall score has been assigned by interpreting the mean score:– Overall score 1: if mean score of activities is 0 - 1.2
– Overall score 1-2: if mean of score of activities is 1.3-1.7
– Overall score 2: if mean score of activities is 1.8 - 2.2
– Overall score 2-3: if mean score of activities is 2.3 - 2.7
– Overall score 3: if mean score of activities is 2.8 - 3.0
Food Quality and Safety
Assumption behind Food Safety
Performance Indicators diagnosis
• More sophisticated FSMS would be better
able to realise products with lower
contamination levels and less deviation in
contamination loads
Different FSMS level/context
situations
may result in good FS !!
Food Quality and Safety
Assumption of output
Context FSMS FS
3
1
2
1
2
3
1
2
3
3=most dangerous 3=highest level 3=best performance
Food Quality and Safety
Topics
• Introduction
• Food Safety Management System (FSMS)
diagnostic instrument
• Food Safety Performance Indicators (SPI)
diagnose
• Example of case company X
Food Quality and Safety
Assumption of output
Context FSMS FS
3
1
2
1
2
3
1
2
3
3=most dangerous 3=highest level 3=best performance
Food Quality and Safety
Profiles contextual situation
company XProduct and process characteristics
0
1
2
3Risk level raw materials
Risk level most criticalproduct groups
Safety contribution ofpackaging concept
Intervention steps
Level of production
processchanges
Rate of product / process
design changes
1-2 Environmental characteristics
0
1
2
3
Safety contribution in chainposition
Power in supplier
relationship
Authority in customerrelationship
Strictness
of legislativerequirements
2
Organizational characteristics
0
1
2
3Technical workforce
Variability workforcecomposition
Operators' competence
Management commitmentEmployee involvement
Formalisation
Supporting information
systems
2-3
• Overall context score:
2
Food Quality and Safety
Profiles of core control activities
company XPreventive measures design
0
1
2
3
Product specific measures
Hygienic design
Critical cooling facilitiesSanitation program
Personal hygienerequirements
1-2 Monitoring system design
0
1
2
3
CCP analysis
Standards and tolerancesassessment
Analytical methods
Measuring equipmentCalibration program
Sampling design
Corrective actions
2
Operation of FSCS
0
1
2
3
Appropriateness ofprocedures
Compliance to
procedures
Actual hygienicperformance of equipment
and facilities
Actual cooling capacity
Actual process
capabilityof intervention process
Actual performance of
measuring equipment
Actual analytical
equipment performance
2Intervention process design
0
1
2
3
Intervention equipment
Maintenance programIntervention methods
0
Overall score control activities 1-2
Food Quality and Safety
Profile of core assurance activities
company X
Food safety assurance system
0
1
2
3
Translating externalrequirements
Use of feedback
information
Validating
preventive measures
Validating
intervention systems
Validating monitoring system
Verifying people related
performance
Verifying equipment andmethods related
performance
Documentation and recordkeeping system
Overall score
assurance activities
1-2
System req.: 1-2
Validation: 1
Verification: 2-3
Documentation: 2
Food Quality and Safety
Assigning overall scores to mean scores
of control and assurance activity levels
• Overall score 1 if mean score of activities 0 - 1.2
• Overall score 1-2 if mean of score of activities 1.3-
1.7
• Overall score 2 if mean score of activities 1.8 - 2.2
• Overall score 2-3 if mean score of activities 2.3 - 2.7
• Overall score 3 if mean score of activities 2.8 - 3.0
• To calculate the overall score for FSMS one should
take all original scores as basis!!!
Food Quality and Safety
Assigning overall scores to mean scores
of contextual situations
• Overall score 1 if mean score of activities 1 - 1.2
• Overall score 1-2 if mean of score of activities 1.3-
1.7
• Overall score 2 if mean score of activities 1.8 - 2.2
• Overall score 2-3 if mean score of activities 2.3 - 2.7
• Overall score 3 if mean score of activities 2.8 - 3.0
• To calculate the overall score for context one should
take all original scores as basis!!!
Food Quality and Safety
Food safety performance indicators
company x
Overall score 2 (because average of the score is 2)
Food Quality and Safety
Assigning overall scores to mean scores
of food safety performance indicators
• Overall score 1 if mean score of activities 0 - 1.2
• Overall score 1-2 if mean of score of activities 1.3-1.7
• Overall score 2 if mean score of activities 1.8 - 2.2
• Overall score 2-3 if mean score of activities 2.3 - 2.7
• Overall score 3 if mean score of activities 2.8 - 3.0
• To calculate the overall score for FSMS one should take all original scores as basis!!!
Food Quality and Safety
Overall results company X
Context FSMS FS
3
1
2
1
2
3
1
2
3
3=most dangerous 3=highest level 3=best performance
1-2
Food Quality and Safety
Conclusion : overall results company X
• Assumption for this company : In context 2 we expect FSMS level 2 and FS level 3
• so, FS level lower than expected probably due to FSMS level being lower than 2
• With FS level 2 there is a potential risk on microbiological food safety
• When FS level perceived as unacceptable by the company, measures could be discussed in FSMS activities (towards level 2/3) and or in the context (towards level 1)
• Possible measures/interventions can be discussed from a detailed analysis of the spider-webs
Food Quality and Safety
Detailed discussion FSMS interventions
for company X• Score 0: Product specific measures; intervention strategies
• Score 1: Extent of personal hygiene requirements; corrective actions; appropriateness of procedures; use of feed back information and validation
• Score 2: Sophistication hygienic design, specificity sanitation program, compliance to procedures, sampling design and tolerances design (important for control raw materials)
• Resulting in discussions about:
-preventing raw material contamination by product specific measures and or raw material control (including sampling design, tolerances design of monitoring system)
-possible techological intervention strategies (because now score 0)
-personal hygiene, hygienic design, hygiene control, and cleaning and disinfection program
-use of information for feedback and corrective actions
-adequacy of procedures and instructions and compliance to procedures
-validation of preventive measures and monitoring system
Food Quality and Safety
Detailed discussion context interventions
for company X• Score 3: Risk level of raw materials and products; lack of
technological staff; lack of information systems; lack of formalisation/procedures; lack of management commitment; strictness of legislative requirements
• This results in discussions about:
-reducing the risk level of raw materials (i.e.. Reducing initial load and variation in initial load) in collaboration with suppliers
-redesigning their information system (to support decision making)
-the quality management system with its procedures (introduce some extent of formalisation)
-people in quality staff positions (technological staff, people responsible for FSMS activities), enhance competences/skills (e.g. by training)
-management commitment
Food Quality and Safety
Thanks for your attention
and
for contributing to our research
2 2 3