IAEA International Atomic Energy Agency
IAEAs Approach to Safety Culture Assessments
Monica Haage – [email protected] Expert on Safety Culture and Human & Organizational Factors
IAEA
The IAEA Safety Culture Framework
Safety Culture is that assembly of characteristics and attitudes in organizations and individuals which establishes that, as an overriding priority, protection and safety issues receives the attention warranted by their significance. The 2007 IAEA glossary
The internationally agreed IAEA normative framework defines strong safety culture into: - 5 characteristics - 36 attributes
Recognized that safety culture is an essential component of the defence in depth and need to be addressed proactively
IAEA
Safety is a clearly recognized value Attributes
• High priority to safety: shown in documentation, communications and decision- making
• Safety is a primary consideration in the allocation of resources
• The strategic business importance of safety is reflected in business plan
• Individuals are convinced that safety and production go ‘hand in hand’
• A proactive and long-term approach to safety issues is shown in decision-making
• Safety conscious behavior is socially accepted and supported (both formally and informally)
GS-G-3.1
IAEA
Accountability for safety is clear Attributes
• Appropriate relationship with the regulatory body exists, which ensures that the accountability for safety remains with the licensee
• Roles and responsibilities are clearly defined and understood • There is a high level of compliance with regulations and procedures • Management delegates responsibilities with appropriate authority to
enable accountabilities • Ownership for safety is evident at all organizational levels and by all
individuals
GS-G-3.1
IAEA
Safety is learning driven Attributes • A questioning attitude prevails at all organizational levels • An open reporting of deviations and errors is encouraged • Internal and external assessments, including self-assessments are used • Organizational and operating experience (both internal and external to
the facility) is used • Learning is enabled through the ability to recognize and diagnose
deviations, formulate and implement solutions and monitor the effects of corrective actions
• Safety performance indicators are tracked, trended, evaluated and acted upon
• There is a systematic development of staff competencies
GS-G-3.1
IAEA
Safety is integrated into all activities Attributes
• Trust permeates the organization • Consideration for all types of safety, including industrial and
environmental safety and security, is evident • Quality of documentation and procedures is good • Quality of processes, from planning to implementation and review, is
good • Individuals have the necessary knowledge and understanding of the
work processes • Factors affecting work motivation and job satisfaction are considered • Good working conditions exist with regards to time pressures, work
load and stress • Cross-functional and interdisciplinary cooperation and teamwork are
present • Housekeeping and material condition reflect commitment to
excellence
GS-G-3.1
IAEA
Leadership for safety is clear Attributes
• Senior management is clearly committed to safety • Commitment to safety is evident at all management levels • Visible leadership showing involvement of management in safety related
activities • Leadership skills are systematically developed • Management assures that there is sufficient and competent staff • Management seeks the active involvement of staff in improving safety • Safety implications are considered in the change management process • Management shows a continuous effort to strive for openness and good
communications throughout the organization • Management has the ability to resolve conflicts as necessary • Relationships between management and staff are built on trust
GS-G-3.1
IAEA
IAEA Safety Culture Publications http://www.iaea.org
Document Title
Safety Fundamentals No. SF-1 Fundamental Safety Principles
Safety Requirements No. GS-R-3 The Management System for Facilities and Activities
Safety Requirements GS-R-Part 2 (supersedes Safety Requirements No. GS-R-3)
The Management System for Facilities and Activities – draft DS-456 – to be published 2015
Safety Guide No. GS-G-3.1 Application of the Management System for Facilities and Activities
Safety Guide No. GS-G-3.5 The Management System for Nuclear Installations
Safety Series No. 75-INSAG-4 Safety Culture
Safety Series No. 75-INSAG-15 Key Practical Issues in Strengthening Safety Culture
Safety Report Series No. 11 Developing Safety Culture in Nuclear Activities
Safety Report Series No. 42 Safety Culture in the Maintenance of Nuclear Power Plants
Safety Report Series: No 74 Safety Culture during Pre-Operational Phases – Published Sept 2012
Safety Report Series: Performing Safety Culture Self-Assessments –– approved and expected to be published 2014/2015
Safety Report Series: How to Continuously Improve Safety Culture – draft – to be published 2015
TECDOC-1321 Self-assessment of safety culture in nuclear installations
TECDOC-1329 Safety culture in nuclear installations
TECDOC-1707 Regulatory Oversight Of Safety Culture In Nuclear Installations
IAEA
• Culture is seen as something we can influence, rather than something we can control
• Culture work needs to encompass the whole organization – not only as a top-down process
• Edgar Schein’s well established iceberg metaphor helps to understand and how to continuously improve safety culture • observe visible aspects (artefacts and
behaviour, “above surface”) and; • interpret this information to reveal the cultural
reasons behind (found in attitudes, values and basic assumptions, “below surface”)
• For safety culture improvement, the IAEA emphasises human interactions (shared space) including trust, mindful communication, learning attitude, inquiring attitude, self-accountability, diversity, self-reflection etc.
IAEAs Approach to Safety Culture Continuous Improvement
Artefacts, Behaviour
Values
Attitudes
Basic Assumptions
IAEA
Shared Space • Safe performance depends on the ability and
willingness of individuals to continuously think, engage, and demonstrate safe behaviours. It is shaped by • Personal motivation • Knowledge and understanding • Shared space
• A good shared space is characterized by • Working relationships that support trust • Decrease of power dynamics • Mutual respect • Openness – free flow in sharing of
thoughts and ideas • Enables individuals to express views related
to their inner thoughts and feelings about a particular issue without fear of recrimination or exclusion
• Shared space goes deeper than sharing facts • Dialogue instead of discussion/argumentation
Me
Shared space
External space
IAEA
IAEA Safety Standards
The image cannot be displayed. Your computer may not have enough memory to open the image, or the image may have been corrupted. Restart your computer, and then open the file again. If the red x still appears, you may have to delete the image and then insert it again.
IAEA
Safety Standard GS-G-3.5: Assessment of Safety Culture
Safety Culture Self-Assessement should: • Include the entire organization • Several different self-assessment tools should be used (e.g. interviews, focus
groups, questionnaires, observations and document reviews) • A designated team representing all organizational levels and functions at the
installation should carry out the self-assessment • A specialist in safety culture should be included in the team • The self-assessment team should receive training • The self-assessment team should summarize the results and identify areas for
improvement and may suggest actions to be taken • The results should be reported to the management at an appropriate level • A follow-up assessment should be performed The independent assessment of safety culture should follow a similar approach
IAEA
• Member States requests to IAEA to provide practical guidance • IAEA Safety Standards • Behaviour and social science • Past experiences
Basis of the safety culture assessment methodology
IAEA
Overall characteristics of the IAEA safety culture methodology
• Multiple-methods approach • Explorative, open approach • Raw material for interpretation • Data in itself say little about culture (tip of
the iceberg)
IAEA
• Using several assessment methods
The IAEA Safety Culture Assessment Methodology
Document Review Interviews
Surveys
Focus Groups
Observations
IAEA International Atomic Energy Agency
Interviews
IAEA
Interview technique
S t r u c t u r e d S e m i - s t r u c t u r e d N o n - s t r u c t u r e d
• Open-ended questions • Striving towards a non-structured interview technique • Explorative – more like a conversation
IAEA International Atomic Energy Agency
Focus Groups
IAEA
Focus Groups
• The purpose of focus groups is to develop a broad and deep understanding rather than a quantitative summary
• 7 – 10 participants (2 Facilitators) • Various cross-section of the organisation
• Job, Grade, Age, Department, etc. • A highly effective method for “listening” to others’ views
and gain insight into group dynamics • Used to draw out attitudes, feelings, beliefs, experiences
and reactions in a way that is not feasible using other methods
IAEA International Atomic Energy Agency
Questionnaires/Surveys
IAEA
Why surveys?
• To capture attitudes and impressions of a large population
• To make sure that everyone in a organization has had an opportunity to make his/her voice heard
• To establish a baseline and be able to track changes over time
IAEA
Disadvantages of surveys
• Surveys identify symptoms rather than causes
• The information collected is about what employees think they think – this is not the same as how they really act!
• Surveys are subject to response bias, e.g. respondents may feel that they should respond in certain ways
• Questionnaires say more about what the person asking questions thinks is important than what the respondent feels is important!
Safety culture only make sense in a qualitative perspective and caution should be made when using quantitative methods
IAEA
IAEAs Safety Culture Perception Questionnaire
IAEA International Atomic Energy Agency
Observations
IAEA
Why observations?
• Visible manifestations of culture • What you see is factual – whether it should have
happened or not! • Should involve observations in different areas and
with different people across the organization to ensure the validity of the findings
IAEA International Atomic Energy Agency
Document Review
IAEA
Why document review?
• Documents communicate the organizations values and expectations
• Reveal approaches/beliefs related to ensuring compliance, e.g. how positional power authority is distributed, degree of formality, approaches to corrective actions, etc.
• May reveal actual work practices, e.g. event reports.
• Can show how the organization thinks, e.g. in documents with analytical content.
• A document says a lot about how the organization presents itself – to itself.
IAEA
• Separation of descriptive and normative
The IAEA Safety Culture Assessment Methodology
Descriptive
‘is’ Based on data and a theory of culture
Normative
‘should’ Based on data, a theory of culture and a norm
IAEA
The IAEA Safety Culture Assessment Methodology
• Performed in silos – each assessment method treated separate
Survey Interviews Focus groups Observations
IAEA
The IAEA Safety Culture Analysis Process e. g. self-assessment
Final Issues; Normative, evaluative analysis
Overarching themes: Image(s) of culture
Interview data
Cultural expressions
Cultural themes
Survey data
Cultural expressions
Cultural themes
Focus group data
Cultural expressions
Cultural themes
Document data
Cultural expressions
Cultural themes
Observation data
Cultural expressions
Cultural themes
IAEA
Example of the Descriptive Analysis Process
Focus group data
Cultural expressions
Cultural themes
Observation data
Cultural
expressions
Cultural themes
Theme “Relaxed attitude towards radiological risks ” Cultural expressions • Several incident of
body contamination and one internal
• People express they did not go to RP re-training for last five years
• People say that RP-techs are sitting mostly in their office
• Managers have identified problems and taken decisions but nothing have changed
Theme “Contamination risk is not considered” Cultural expressions • Skipping
contamination area borders
• Not properly marked boarders
• Chewing gum in RCA
• Not wearing gloves when working in the RCA where its required
Overarching theme; Risk awareness related to actions for preventing contamination spread
IAEA
Normative Analysis
Overarching theme; Risk awareness related to actions for preventing contamination spread
Final Issues; Normative, evaluative analysis
IAEA International Atomic Energy Agency
IAEAs Approach to
ISCA Independent Safety Culture Assessment
within Operational Safety Review Team (OSART)
IAEA
Safety Culture Independent Assessment should: The independent assessment of safety culture should follow a similar approach as
self-assessment • The independence and qualification of the members of the assessment team
should be considered crucial for the success of the assessment • The team should be staffed with sufficient diversity of experience and should
include specialists in behavioural science, with knowledge of statistical methods of analysis
• The independent assessment team should aim at identifying strengths and areas for improvement
Safety Standard GS-G-3.5: Assessment of Safety Culture
IAEA
The ISCA Teams Area of Expertise • Psychology • Cognitive science • Sociology • Social Psychology • Organizational theory • Cultural theory • Leadership and management theory • Human Factor / Human Factor Engineering • Organizational Factors • Resilience Engineering • ITO (interaction between Individuals, Technology and
Organizations) Basic knowledge; Nuclear technology, nuclear organizations,
regulatory framework
IAEA
OSART Findings and Safety Culture
IAEA
Sources of information:
The IAEA ISCA Assessment Methods
Document Review Interviews
Surveys
Focus Groups
Observations
Team Findings
IAEA
Core of IAEA Safety Culture Analysis Process e. g. independent assessment within OSART
Interview data
Cultural expressions
Cultural themes
Survey data
Cultural expressions
Cultural themes
Focus group data
Cultural expressions
Cultural themes
Document data
Cultural
expressions
Cultural themes
Observation data
Cultural
expressions
Cultural themes
Overarching themes: Image(s) of culture
Final Issues; Normative, evaluative analysis
Team findings
data
Cultural
expressions
Cultural themes
IAEA
Example of amount of safety culture facts
• Team findings: Circa 220 facts • 25 interviews: Circa 150 facts • 7 observations: Circa 30 facts • 7 focus groups: Circa 80 facts • Survey 389 participated 97 questions In total about 480 fact excluding the survey material
IAEA
Reporting of the results
• Recommendations for areas in need of attention
• Three senior managment workshops to reach a shared understanding
• Exit meeting • Report
…Thank you for your attention