+ All Categories
Home > Documents > Guidelines on auditing a Safety Management System - · PDF file1.3 ISAGO Audit Scope ......

Guidelines on auditing a Safety Management System - · PDF file1.3 ISAGO Audit Scope ......

Date post: 14-Mar-2018
Category:
Upload: dangnguyet
View: 220 times
Download: 2 times
Share this document with a friend
33
Guidelines on auditing a Safety Management System June 2017
Transcript
Page 1: Guidelines on auditing a Safety Management System - · PDF file1.3 ISAGO Audit Scope ... factors will have to be taken into consideration when determining the scope of the audit and

Guidelines on auditing a Safety Management System

June 2017

Page 2: Guidelines on auditing a Safety Management System - · PDF file1.3 ISAGO Audit Scope ... factors will have to be taken into consideration when determining the scope of the audit and

NOTICE

DISCLAIMER: The information contained in this publication is subject to constant review in the light of changing government requirements and regulations. No subscriber or other reader should act on the basis of any such information without referring to applicable laws and regulations and/or without taking appropriate professional advice. Although every effort has been made to ensure accuracy, the International Air Transport Association shall not be held responsible for any loss or damage caused by errors, omissions, misprints or misinterpretation of the contents hereof. Furthermore, the International Air Transport Association expressly disclaims any and all liability to any person or entity, whether a purchaser of this publication or not, in respect of anything done or omitted, and the consequences of anything done or omitted, by any such person or entity in reliance on the contents of this publi-cation.

© International Air Transport Association. All Rights Reserved. No part of this publication may be reproduced, recast, reformatted or transmitted in any form by any means, electronic or me-chanical, including photocopying, recording or any information storage and retrieval system, without the prior written permission from:

Senior Vice President Safety and Flight Operations

International Air Transport Association 800 Place Victoria

P.O. Box 113 Montreal, Quebec

CANADA H4Z 1M1

Page 3: Guidelines on auditing a Safety Management System - · PDF file1.3 ISAGO Audit Scope ... factors will have to be taken into consideration when determining the scope of the audit and

i

Table of Contents

Foreword .................................................................................................................................................................................. ii Use of this Document .............................................................................................................................................................. iv Section 1 – Safety Management in ISAGO .............................................................................................................................. 1 1.1 Introduction .................................................................................................................................................................... 1 1.2 SMS Implementation ...................................................................................................................................................... 1 1.3 ISAGO Audit Scope ....................................................................................................................................................... 2 Section 2 – SMS Audit Aims, Focus & Planning ....................................................................................................................... 4 2.1 Audit Aims ..................................................................................................................................................................... 4 2.2 Audit Focus .................................................................................................................................................................... 4 2.3 Audit Planning................................................................................................................................................................ 5 2.4 The Safety Office ........................................................................................................................................................... 6 Section 3 – SMS Audit by GOSARP ........................................................................................................................................ 7 3.1 Introduction .................................................................................................................................................................... 7 3.2 Organization & Accountability ........................................................................................................................................ 7

3.2.1 The Accountable Executive ...................................................................................................................................... 7 3.3 Safety Policy & Objectives ............................................................................................................................................. 8

3.3.1 SMS ......................................................................................................................................................................... 8 3.3.2 (The Safety) Manager .............................................................................................................................................. 8 3.3.3 Safety Roles & Responsibilities ................................................................................................................................ 9 3.3.4 Corporate Safety Policy (Safety Objectives) ............................................................................................................. 9 3.3.5 Safety Reporting Policy ...........................................................................................................................................10 3.3.6 Emergency Response Plan (ERP) ..........................................................................................................................11 3.3.7 SMS Documentation (SMS Manual) ........................................................................................................................11 3.3.8 SMS Implementation Plan .......................................................................................................................................12

3.4 Safety Risk Management ..............................................................................................................................................12 3.4.1 Hazard Identification ...............................................................................................................................................13 3.4.2 Safety Reporting System ........................................................................................................................................13 3.4.3 Safety Risk Assessment & Mitigation ......................................................................................................................14 3.4.4 Accident/incident Investigation & Reporting.............................................................................................................15 3.4.5 Ground Damage Reporting .....................................................................................................................................15

3.5 Safety Assurance ..........................................................................................................................................................15 3.5.1 Safety Assurance Program .....................................................................................................................................16 3.5.2 Safety Performance Metrics ....................................................................................................................................16 3.5.3 Management of Change ..........................................................................................................................................17 3.5.4 Continuous Improvement of the SMS ......................................................................................................................17 3.5.5 Management Safety Decision Making .....................................................................................................................18

3.6 Safety Promotion ..........................................................................................................................................................18 3.6.1 Safety Awareness ...................................................................................................................................................18 3.6.2 Safety Information ...................................................................................................................................................18 3.6.3 Safety Training ........................................................................................................................................................19

3.7 SMS Checklist ..............................................................................................................................................................20 Section 4 – Audit Summary Report - Assessment of the SMS ................................................................................................22 4.1 Introduction ...................................................................................................................................................................22 4.2 Audit SMS Summary .....................................................................................................................................................22 Appendix A – QA Provisions & SMS Training Tables ..............................................................................................................24

Page 4: Guidelines on auditing a Safety Management System - · PDF file1.3 ISAGO Audit Scope ... factors will have to be taken into consideration when determining the scope of the audit and

ii

Foreword

A Safety Management System (SMS)1 is a framework of policies, processes, procedures and techniques

for use by an organization to monitor and continuously improve its safety performance. Improvements

are made by making informed decisions on the management of operational safety risks. Annex 19 to the

Convention on International Civil Aviation (ICAO Annex 19, Safety Management) details the global

regulations for SMS that are applicable to specified air operators, air traffic service providers and certified

airports and other operational services.

The principle method of safety management prescribed by ICAO is similar for all types of operator and

service provider, based on a common framework of processes and procedures contained in 4 discrete

components that are further sub-divided into a total of 12 elements, as illustrated in figure 1 below.

Figure 1 – The 4 Components of the ICAO SMS Framework (Annex 19)

Guidance on the ICAO SMS regulations and their implementation is provided in ICAO Doc 9859, Safety

Management Manual.

The ICAO SMS regulations do not currently apply to ground service providers (Providers) but those

applicable to aircraft operations encompasses ground operations where aircraft safety is concerned.

Ground handling personnel are mentioned in the regulations in the context of reporting safety events or

issues. Providers therefore play an important role in safety management at an airport. Furthermore, by

implementing SMS, Providers would gain considerable credibility from air operators, airports and

regulatory authorities worldwide by acknowledging the contribution and influence that ground operations

has in improving the safety of aircraft operations and the airport environment in general.

IATA has already recognized the global regulations and the importance placed on the implementation of

SMS by aircraft operators. The IATA Operational Safety Audit (IOSA) program is an internationally

1 A systematic approach to managing safety, including the necessary organizational structures, accountabilities, policies and procedures.

(ICAO Annex 19)

Page 5: Guidelines on auditing a Safety Management System - · PDF file1.3 ISAGO Audit Scope ... factors will have to be taken into consideration when determining the scope of the audit and

iii

recognized and accepted evaluation system designed to assess the operational management and control

systems of an airline. All IATA members are IOSA registered and must remain registered to maintain

IATA membership. The IOSA standards are published in the IOSA Standards Manual (ISM). The current

edition, ISM Edition10, includes standard, ORG 1.1.102, that establishes the management of the safety

risks associated with aircraft operations.

The IATA Safety Audit for Ground Operations (ISAGO) is an industry audit and registration scheme aimed

primarily at creating safer ground operations and cost benefits by reducing the risk of aircraft damage,

reducing delays, and eliminating redundant audits by airlines. The GOSM Ed 5 and Ed 6 included a

review of the existing SMS provisions, elevating some to Standard level as the first and second phase of

a SMS Strategy (SMS Implementation - Strategic Plan for Upgrading ISAGO SMS Provisions 2nd Edition

September 2017). The strategy upgrades all SMS recommended practices to Standard level over a three

year period.

Auditing the SMS, internally by the Provider and by an external body (such as in the case of ISAGO), is

an essential activity as part of assurance that the SMS is, or could be made to be, effective and meets

expectations.

A specific focus on making safety management the principle component of the Organization and

Management Section of the GOSM, as well paving the way for the introduction of the ISAGO new

operational audit model in 2017, will require further refinement and amendment of the SMS provisions,

to reduce duplication, account for any changes in global regulations and define more clearly the ISAGO

audit scope and content. The annual review of the GOSM will therefore include a review of these auditing

guidelines.

2 ORG 1.1.10 The Operator shall have an SMS that is implemented and integrated throughout the organization to ensure management of the safety risks associated with aircraft operations. Note: Conformity with this ORG standard is possible only when the Operator is in conformity with all standards that are identified by the [SMS] symbol.

Page 6: Guidelines on auditing a Safety Management System - · PDF file1.3 ISAGO Audit Scope ... factors will have to be taken into consideration when determining the scope of the audit and

iv

Use of this Document

The GOSARPs are the basis for an ISAGO audit of a Provider. This document provides guidelines on

what to look for when auditing the SMS of a Provider, as a whole, against the SMS GOSARPs contained

in Section 1, Organization and Management (ORM) of the GOSM. Suggested recommended actions,

questions, checklists and audit summary text are also provided. It is not a definitive guide and hopefully

not a condescending one.

The guidelines do not replace formal auditing procedures and should be considered as an aid to the Auditor Actions described in the GOSM, checklists and incorporated in Q5AIMS. More detailed checklists (and hence more appropriate for a well-established SMS) may be found in the ICAO Doc 9859, Safety Management Manual. Refer also to ACI SMS Handbook Step A.

The IOSA SMS standards and associated guidance material that is developed would provide useful

reference material complementary to the ISAGO provisions.

The ORM is now a section including the previous three sections (ORM-H, ORM-HS and ORM-S). The

SMS related GOSARPs have also been included in the new ORM. This document will be updated, as

necessary, when changes are made to the GOSARPs and audit procedures, or through practical

experience. Suggestions for improvements are always welcome.

Page 7: Guidelines on auditing a Safety Management System - · PDF file1.3 ISAGO Audit Scope ... factors will have to be taken into consideration when determining the scope of the audit and

1

Section 1 – Safety Management in ISAGO

1.1 Introduction From the start it is important to keep in mind that a SMS is foremost a decision making tool. The SMS

provides the organization with information on operational and other safety risks, such that actions to

eliminate, mitigate and/or control the safety risks can be determined and, if accepted by the decision-

makers, implemented. A SMS does not normally provide immediate solutions. Although immediate action

could (and probably should) be needed to address an unexpected unsafe situation, the SMS is not

intended to cater for these situations.

Instead, processes and procedures gather safety data and information, and, once there is sufficient or

relevant data and information, formal assessments are conducted and, if necessary, measures are

implemented to prevent an identified hazardous condition escalating into an accident scenario. Done

properly, this takes time and effort. Where significant effort would be needed, such as in terms of people,

finance, equipment or change, the SMS provides senior management with the information to make

informed decisions on what to do and, if necessary, when. These decisions, when accepted as necessary,

are then translated into safety action plans to implement safety risk controls and as safety objectives. The

ultimate aim of the audit of the SMS should therefore be (in addition to assessing the organization’s

implementation and conformity with the SMS GOSARPs) whether the SMS is, or will be, effective in

achieving the safety objectives of the Provider.

Installing a SMS doesn’t happen overnight. Gradual implementation in an easy-to-do manner seems to

be the way that many aviation organizations are going about it. The IATA Strategic Plan for Upgrading

ISAGO SMS Provisions applies the same principle in a structured schedule of upgrading the SMS

GOSARPs over a three year period. The audit guidelines in this document do not differentiate between

a standard and a recommended practice as their implementation is essentially the same. Until required

and implemented, the audit would therefore have to take appropriate account of a Provider that is not

able to demonstrate full conformance with a specific GOSARP and the consequences on other

GOSARPs. Until all SMS GOSARPs are at standard level, an important feature of the audit would be an

assessment of a Provider’s implementation of the SMS (see 4.1).

1.2 SMS Implementation A new GOSARP was introduced in GOSM Edition 5 (ORM 3.1.8 –) requiring the Provider to have an SMS

implementation plan. The SMS implementation plan should detail the way the Provider will structure its

entire organization (including all stations), resources and processes to effectively manage safety in

operations.

Considering that a safety and quality control program, with some risk assessment procedures, may

already be in place, certain aspects can be directly transferable to the SMS. It would therefore be prudent

of the Provider to follow the ICAO guidance and conduct a gap analysis to identify what changes or new

processes would be needed to comply with the ISAGO SMS implementation strategy. The

implementation plan should therefore show which SMS elements (or equivalent processes) are already

implemented, and those in the process of being or planned to be implemented. The plan should also

describe how the SMS will be based at a corporate (headquarters) level and implemented throughout the

organization.

Page 8: Guidelines on auditing a Safety Management System - · PDF file1.3 ISAGO Audit Scope ... factors will have to be taken into consideration when determining the scope of the audit and

2

Recommendation – Obtain a copy of the Provider’s SMS Implementation Plan prior to the audit to determine the audit scope and expectations for the SMS aspects.

It is possible that implementation progress rates may vary within a Provider, especially at stations as part

of an international organization and where local regulations may have an impact. If encountered, these

factors will have to be taken into consideration when determining the scope of the audit and the

assessment of the overall implementation of the SMS in an organization.

The SMS implementation plan should, in the way that GOSARP recommended practices are treated, give

a good indication of the Provider’s commitment and recognition of current safety practices in aviation that

are becoming the norm if not a requirement for an organization to conduct business. Credit, in the audit

report, should therefore be given when a recommended practice is implemented by the Provider as if it

were a standard. Where the SMS is already implemented, and functioning and the Provider is measuring

its effectiveness, then ORM 3.1.8 might be assessed as not applicable (N/A).

Whilst conformance with each individual GOSARP should be assessed, the SMS functions may be

integrated with other management systems and/or distributed throughout the organization. The

requirement to have a SMS (ORM 1.1.3) is not, however, met until all the SMS GOSARPs are

implemented.

1.3 ISAGO Audit Scope The extent of the SMS activities to be included in the audit is outlined in the framework (Figure 1) as

specified in ICAO Annex 19, Safety Management, and is captured in the GOSARPs. The amount of

activity, once implemented (see also 3.3.1), would depend to a large degree upon the size of the

organization or the extent of its operations.

In many States there are civil aviation regulations that require the establishment of an SMS within aircraft

and airport operators. The services provided by Providers can have a direct influence on aircraft and

airport operations and hence, even if not explicitly applicable in the regulations, the SMS of those

operators should acknowledge many of the Provider’s operational and management activities. Similarly,

the SMS of a Provider should have established links to the SMS of the airport operator and those of

customer airlines. This aspect is very important. It makes little sense for the SMSs of all the organizations

that operate on an airport (and there can be many) to be developed or work in isolation or, in the worst

case scenario, in conflict. Indeed, there could be additional safety risks created by the actions resulting

from the SMS of an individual organization without considering the safety impact on other operators and

their operations. For a typical Provider’s operation, with multiple customer airlines, this could very likely

be the outcome if there were no measures in place for collaboration on safety management.

The interface between the SMSs of Providers, aircraft and airport operators, perhaps with the air traffic

services too, may sometimes be part of an airport collaborative decision-making initiative. This is

particularly relevant to safety reporting (as already a regulatory requirement in some cases) and the

development of safety action plans (as a result of a safety risk assessment), and the development of an

emergency response plan (ERP). The actions of a Provider may also have a direct impact on safety

performance indicators, as may be set for aircraft and airport operators by the regulatory authorities.

Some of the SMS GOSARPs (especially ORM 3.1.6 and 3.3.3) include such interactions with other

organizations on the airport. It is therefore important during a SMS audit to seek evidence of the existence

and effectiveness of external relationship procedures and communications, and to be assured that it is

bi-directional.

Page 9: Guidelines on auditing a Safety Management System - · PDF file1.3 ISAGO Audit Scope ... factors will have to be taken into consideration when determining the scope of the audit and

3

Recommendation – Identify the external organizations that may need to be contacted to verify conformance where interaction with the Provider is specified in a process or procedure.

A similar relationship, sometimes referred to a “bridge”, would be in effect between the Provider’s

headquarters and each station(s). The SMS audit should seek evidence of effective communication,

consistent implementation of corporate processes and procedures and clear lines of safety

responsibilities between the two. The aim of the GOSM is to ensure that the Provider has a thorough and

robust corporate management of the services it provides at each station. This is why the GOSARPs often

refer to implementation throughout the organization. Implementation in a GOSARP sense means that the

process, procedure or otherwise required action or activity at a station is directed by headquarters and

there is continuous oversight at a headquarters level to ensure correct implementation. A station audit

would therefore have to use the most recent and a valid headquarters audit as a baseline reference for

the implemented processes and procedures, and the effectiveness of the bridge should be tested for

each relevant GOSARP.

It is entirely plausible that management and communications between headquarters and the stations can

become estranged. The emphasis must be on a top (headquarters)-down approach to SMS

implementation, management and oversight - not bottom (station)-up or disconnected. The SMS audit

therefore has to verify that implementation and compliance at each station is coordinated by headquarters

and checked on a regular basis.

Page 10: Guidelines on auditing a Safety Management System - · PDF file1.3 ISAGO Audit Scope ... factors will have to be taken into consideration when determining the scope of the audit and

4

Section 2 – SMS Audit Aims, Focus & Planning

2.1 Audit Aims The aim of the headquarters audit would be to determine the extent of implementation of the SMS

throughout the organization and the effectiveness of the corporate management aspects. The audit

summary would provide a detailed description of the Provider’s conformance with the relevant SMS

GOSARPs as implemented, see 4.2.

Similarly, the aim of the station audit would be, in addition, to determine the effectiveness of the corporate

SMS at the station through assessment of the implementation of procedures, oversight and the

deployment of SMS safety risk management and safety assurance activities.

2.2 Audit Focus The primary focus of the SMS audit should be, where implemented, to seek evidence of:

due diligence and competence in the assigned safety roles;

the development, implementation of and conformance with documented processes and procedures;

effective safety reporting systems, safety communications and awareness (hopefully, reflecting a

positive safety culture);

coordination and cooperation with other relevant SMSs at the airport, including customer airline(s);

and

the monitoring and measurement of SMS outcomes and effectiveness (quality assurance).

A headquarters audit would be based predominantly upon an assessment of documentation. For auditing

purposes the SMS documentation should provide a complete picture of how the SMS should work and

all the SMS activities that have taken place. A fully implemented SMS should be rich with processes and

procedures, assessments, reports, and other documentation that can be assessed against corresponding

GOSARPs; checking for content, completeness, consistency and currency. It does, however, take time

to develop a ‘safety library’ of safety reports, safety assessments, action plans and documented

decisions, which should be taken into consideration. Similarly, smaller organizations or those with limited

ground operations may not produce large volumes of documentation. However, this situation should not

prevent the organization from taking account of or using safety data and information shared or made

available publicly.

GOSARPs upgraded to Standards in GOSM Edition 5 and 6 relate to administrative processes for which

documentation should be available. As a result, there should at least be verifiable evidence of some

development and, if in advance of the SMS strategy timeline, possible implementation of an internal safety

reporting system in operation. There should be documented evidence of reports and other management

oversight records that demonstrate that processes and procedures are implemented and followed.

There should also be ample opportunities to talk with the organization's personnel, from the very top level,

and test their awareness of and whether or not they actually carry out their SMS duties and

responsibilities.

A station audit would be based on an assessment of implementation. On-site, Interviews with key people

and the observation of a procedure in operation should be undertaken as the opportunity to do so arise

Page 11: Guidelines on auditing a Safety Management System - · PDF file1.3 ISAGO Audit Scope ... factors will have to be taken into consideration when determining the scope of the audit and

5

or is requested. It should also be possible to review procedures that have been developed and evidence

of being correctly followed or reports produced and acted upon as required.

2.3 Audit Planning The audit of a SMS would normally consist of:

A review of documented processes, procedures, reports, assessments and records;

An assessment of evidence of implementation of processes and procedures;

Interviews with key safety personnel; and

Observations of operational procedures (at a station).

While the use of computer networks (internet, intranet etc.) should render the physical location of

documentation (and its development or management) of little consequence, the verification of use and

access to SMS documentation and document management systems might depend upon the location of

the Provider’s headquarters and station(s). The organization could be spread across several countries,

and activities could vary from place to place. The documentation could also be held locally in a different

language and translation/interpretation services may need to be considered.

Interviews with the nominated key safety personnel are needed to verify conformance with corresponding

GOSARPs, that the SMS processes and procedures are implemented and used correctly, and that

everyone is aware of their SMS roles and responsibilities. Some of these personnel may be located at a

station; hence the headquarters audit should identify these personnel for when the station audit is

conducted.

Recommendation – Establish where, if different, the management and administration of each SMS function is conducted and the location of key safety personnel.

There are few, if any, SMS activities that can be observed in the same way as a ground operations

procedure. Even if, say, a safety assessment was taking place during the audit, there would be little

benefit in observing it. It would be more worthwhile seeking evidence that the safety assessments were

recorded properly and have produced tangible outcomes in accordance with the SMS safety risk

management and safety assurance GOSARPs. In this respect, there should be a record of the risk

assessment activity, discussions that took place and any decisions made by management.

The headquarters documentation review could, by way of records of safety events and safety risk

management/safety assurance actions, reveal the extent of the SMS activities at each station. Based on

reasoned judgment, queries could be raised and explored if one station appears to be less safe than

others, or if there is a marked difference in the number of safety reports generated at each station or how

safety issues are operationally addressed. This situation could indicate a lack of conformity with

processes and procedures at the headquarters or the station, and raised with the Provider for immediate

clarification or attention. If the reason for the anomalies is an issue at a station, the next planned audit at

that station should verify that corrective action has been successfully completed by the Provider. A

finding, however, has to be raised if the Provider’s oversight of effective SMS implementation is at fault.

Where a Provider has an extensive network of stations, perhaps 20 or more, a pragmatic approach should

be taken during the headquarters audit when assessing conformity of implementation and headquarters

oversight. A sample of stations may be chosen as a rational indication that GOSARP conformance at the

other stations is likely to be at least as good as those in the sample. In this respect, the number and

Page 12: Guidelines on auditing a Safety Management System - · PDF file1.3 ISAGO Audit Scope ... factors will have to be taken into consideration when determining the scope of the audit and

6

location of stations chosen by the auditor for the sample should consider the Provider’s ISAGO history

(in terms of results) and if potential weaknesses or failures of management oversight of station activities

are apparent. Where station sampling is used, justification, including the methodology and evidence used,

must be documented by the auditor in the headquarters audit report.

2.4 The Safety Office Depending upon the size of the organization, the administrative aspects of the SMS (such as safety risk

management) may be undertaken by a dedicated team, perhaps a centralized Safety Office, managed

by a person normally with the title Safety Manager. This will probably mean that a station will play only a

participative role and therefore all the documentation needed for review would be accessible from the

Safety Office. The Safety Manager would be a key person in the audit.

It would be unusual for more than one Safety Office to exist in an organization but there could be more

than one Safety Manager, dependent upon the delegation of responsibilities and possibly one at each

station if the operation is large enough. The roles and responsibilities of the Safety Office and Safety

Manager(s) have to be clearly defined. The Safety Office may be located anywhere provided that effective

lines of communication with operational subject matter experts and responsibility for establishing safety

action plans are in place.

The Safety Office is also the place where safety issues (safety reports) identified at a station should be

forwarded to for processing, including review and recording, and analysis and distribution as necessary.

The Safety Office is where the administrative center of the organization and the “safety library” exist.

The Safety Office would normally be responsible for the following:

safety reports are received and, with other safety information, are processed according to the

procedures

safety risk assessment outcomes are handled correctly and efficiently

actions to control safety risks are implemented and monitored

safety performance is monitored and measured

reviews of the SMS performance take place.

The Safety Office may also be responsible for the dissemination of safety information and facilitator of

safety training.

As the SMS becomes more established the Safety Office should increase its presence and its influence

over the safety activities throughout the organization. In future, the Safety Office and the Safety Manager

might become the focal point for the ISAGO SMS audit.

Page 13: Guidelines on auditing a Safety Management System - · PDF file1.3 ISAGO Audit Scope ... factors will have to be taken into consideration when determining the scope of the audit and

7

Section 3 – SMS Audit by GOSARP

3.1 Introduction This section aims to provide, where perhaps necessary, some guidance on the audit of each SMS

GOSARP.

The SMS GOSARPs follow a similar format to that of the ICAO SMS framework illustrated in Figure 1. Of

the four ICAO SMS framework components, the safety policy and objectives aspects are mostly

administrative and may not change significantly over a period of time. The fourth framework component,

safety promotion, is also administrative in nature but will most likely have regular tangible outcomes and

outputs that can be audited.

The two main SMS functional areas, involving routine activities, are safety risk management and safety

assurance. These are two functions expected to be administered by the Safety Office or, in kind, by a

person with safety responsibilities at the station. If addressed at the station (or the Safety Office is located

at the station) the audit should verify that the associated GOSARPs (ORM 3.2 and 3.3) are implemented

and that there is effective management control by the Provider at a headquarters level.

Documentation is needed in nearly all cases to verify conformance with the corresponding GOSARPs but,

in general, interviews with the nominated key safety personnel may be useful and, where practicable,

observations may take place.

3.2 Organization & Accountability Although only one GOSARP in this part of the ORM is directly linked to SMS, all the GOSARPs have an

association and therefore the SMS should be taken into consideration in the context of a management

system.

3.2.1 The Accountable Executive

ORM 1.1.2 The Provider shall identify one senior management official as the Accountable Executive who is

accountable for performance of the management system as specified in ORM 1.1.13 and:

(i) Irrespective of other functions, has ultimate responsibility and accountability on behalf of the Provider for the implementation and maintenance of the safety management system (SMS) throughout the organization;

(ii) Has the authority to ensure the allocation of resources necessary to manage safety risks to ground operations;

(iii) Has overall responsibility and is accountable for ensuring operations are conducted in accordance with applicable regulations and standards of the Provider. [SMS]

3 ORM 1.1.1 The Provider shall have a management system that ensures:

i. Management key policies, systems, programs, processes, procedures and/or plans are determined and implemented throughout the

organization;

ii. Lines of accountability for operational safety and security are defined throughout the organization;

iii. Resources necessary to conduct Operations in accordance with standards of the Provider, applicable regulations and requirements

of the customer airline(s) are granted at all times (GM)

Page 14: Guidelines on auditing a Safety Management System - · PDF file1.3 ISAGO Audit Scope ... factors will have to be taken into consideration when determining the scope of the audit and

8

A SMS is designed to be driven from the highest level of the organization, with clearly defined roles,

responsibilities and lines of authority and communication. At the top is the person nominated as the

Accountable Executive. An interview with this person, if possible, would be useful to ascertain the

management commitment, verify senior management involvement (in decision making) and awareness of

the SMS and its outcomes. The interview should establish whether the level of commitment typically

indicated in safety policies is in fact put into place. The answers given to simple questions can reveal a lot

and prepare the auditor for the rest of the audit.

Question – Is the Accountable Executive made fully aware of the level

of operational safety of the organization, including all the stations?

Question – What has been done to address safety issues, improve

safety and improve the SMS?

The Accountable Executive is the only person with accountability for the safety performance of the

organization and therefore should be fully aware of the SMS outputs and effectiveness.

Recommendation – Ask the Accountable Executive what the Provider’s top, say, 3 safety risks are and verify that they are represented by safety performance indicators/targets (and possibly safety risk mitigation plans).

If it is not possible to arrange an interview, verify through documentation and questioning those persons

with SMS responsibilities that the Accountable Executive takes an active role in the SMS and for allocating

resources. The Accountable Executive should not be just a signature.

3.3 Safety Policy & Objectives The first component of the SMS framework mainly addresses the administrative aspects of the SMS that

would also mainly be within the scope of the headquarters audit.

3.3.1 SMS

ORM 1.1.3 The Provider should have an SMS that is implemented and integrated throughout the organization to ensure management of the safety risks associated with ground operations. [SMS] Note: Within 2019, this recommended practice will be upgraded to a standard. Conformity with ORM 1.1.3 is possible only when the Provider is in conformity with all standards and recommended practices that are identified by the [SMS] symbol.

All components and elements of the SMS framework have to be in place for the SMS to function properly.

Conformance with this GOSARP would depend upon conformance with all other SMS GOSARPs. In many

cases this will not be the case; hence, the reason why this GOSARP has a standard upgrade date of 2019,

after all the other GOSARPs have been upgraded too, and the requirement for the SMS implementation

plan (ORM 3.1.8). Note that it is intended that ORM 3.1.8 will be removed once the IATA strategic

implementation plan is completed.

3.3.2 (The Safety) Manager

ORM 1.1.4 The Provider shall appoint a manager who is responsible for the implementation, maintenance and the

day-to-day administration and operation of the SMS at the corporate level and throughout the organization on behalf of the AE. [SMS]

Page 15: Guidelines on auditing a Safety Management System - · PDF file1.3 ISAGO Audit Scope ... factors will have to be taken into consideration when determining the scope of the audit and

9

Another key safety role is that of the Manager assigned to administer the SMS – usually called the Safety

Manager. The role and responsibilities of this person (or persons depending upon if the role is spread

across several stations) should be clearly defined and there should be documented evidence of the person

performing the role. If more than one Safety Manager (or other defined job title) exists then there should be

defined lines of authority and communication such that there is no ambiguity or interference with performing

the safety responsibilities within the organization.

Question – Ask the Safety Manager or person responsible at a station

the same questions suggested to the Accountable Executive.

Question – Can the Safety Manager explain the organization’s safety

hierarchy and the lines of responsibility?

The Safety Manager should be able to demonstrate that the SMS policy and objectives, and the associated

processes and procedures are implemented at all stations. The Safety Manager should also be able to

provide evidence of the way that safety reports and safety information is processed (through the safety risk

management process) and that records and other documentation is controlled.

Recommendation – As time allows, trace a safety report through the safety risk management process (see 3.4). Wherever possible, choose at least one that results in a safety recommendation that required a management decision, and mitigation measures with documented performance indicators and targets.

3.3.3 Safety Roles & Responsibilities

ORM 1.6.1 The Provider shall define the safety responsibilities of management and non-management personnel

throughout the organization and specify the levels of management with the authority to make decisions that affect the safety of ground operations. [SMS]

.

Other key safety roles would probably be those of station personnel with direct management or supervisory

responsibilities for ground operations. Apart from documented details of the roles and responsibilities of

named persons, there should be evidence of their involvement in safety risk management and safety

assurance activities, usually as an operational expert. An interview should test the awareness and

knowledge of the assigned roles and responsibilities, and confirm recent activity.

Recommendation – Look for evidence of the named persons responsible for ground operational safety involvement in the implementation and monitoring of safety risk mitigation or control activities.

3.3.4 Corporate Safety Policy (Safety Objectives)

ORM 1.2.2 The Provider shall have a corporate safety policy that:

(i) Reflects the organizational commitment regarding safety;

(ii) Includes a statement about the provision of the necessary resources for the implementation of the safety policy;

(iii) Includes safety reporting procedures as specified in ORM 3.2.2;

(iv) Indicates which types of behaviors are unacceptable and includes the circumstances under which disciplinary action would not apply as specified in ORM 3.1.5;

(v) Is signed by the Accountable Executive of the organization;

(vi) Is communicated, with visible endorsement, throughout the organization;

(vii) Is periodically reviewed to ensure it remains relevant and appropriate to the Provider. [SMS]

Conformance is determined mostly as a straightforward verification exercise. Documentation should be in

conformity to all seven items listed in the GOSARP.

Page 16: Guidelines on auditing a Safety Management System - · PDF file1.3 ISAGO Audit Scope ... factors will have to be taken into consideration when determining the scope of the audit and

10

It should be clear from the safety policy that it is relevant to the Provider and there should be evidence of

implementation. Implementation in this sense means that it is clear that safety activities exist or actions are

taken directly as a result of the policy.

Question – Is the safety policy generic (indicating a possible lack of

detail/sincerity) or contains policies specific to the organization, or to a

station?

Resources (ORM 1.2.2(ii)) would usually be in the form of funding, people and equipment. Time, allocated

to undertake SMS activities could also be included. A typical indication of inadequate resourcing is where

SMS activities are delayed or if safety recommendations are postponed. Another indicator of inadequate

resourcing is where key safety personnel posts remain vacant for a prolonged period of time or are assigned

to people with inadequate credentials and time to undertake the extra responsibilities.

Recommendation – Examine the CVs and SMS training records of key personnel.

For ORM 1.2.2(iii) see ORM 3.1.5 and ORM 3.2.2.

A behavior policy (ORM 1.2.2(vii)) should be clear, comprehensive and communicated to all employees.

There could be evidence provided of an example where the policy was invoked and action was taken but

be careful to respect sensitive information. Note that a non-punitive behavior policy may require approval

from a regulatory body. For example, in aviation there are instances where a mandatory report is required

but the organization may be authorized to investigate and, subject to the outcome, address the issue without

recourse to the regulatory authority.

Periodic review of the safety policy (ORM 1.2.2(vii)) would normally be covered by ORM 3.4.1 (the Quality

Assurance program – see Appendix A) but may also depend upon the implementation of ORM 3.3.4. It is

typical for a two-year review period to apply.

Although not explicitly required as a GOSARP at this time, the Provider’s safety objectives should be

documented with the safety policy. There is, however, inference of the requirement for safety objectives in

ORM 2.1.3 and 3.1.8 in documentation and the implementation plan respectively. In any case, safety

objectives should be linked to the safety policy and the safety assurance component.

Question – Are safety objectives relevant to the organization stated (or

related to safety mitigation activities)?

Safety objectives should be derived as a consequence of ORM 3.3.2. The safety objectives should reflect

any high level safety performance indicators and targets that the Provider sets and, once the safety

assurance component is fully functional, may include significant safety objectives, i.e. set as a direct result

of the Provider’s safety risk management and related to an assessment or, perhaps, set by a regulatory

authority as part of a national issue or safety campaign.

Recommendation – Look for evidence of the safety objectives incorporating an objective that reflects an established safety performance indicator(s) associated with a significant safety risk mitigation or control activity.

3.3.5 Safety Reporting Policy

ORM 3.1.5 The Provider shall have a corporate safety reporting policy that encourages personnel to report

hazards to ground operations and, in addition, defines the Provider's policy regarding disciplinary action, to include:

(i) Types of operational behaviors that are unacceptable;

(ii) Conditions under which disciplinary action would not be taken by the Provider. [SMS]

Page 17: Guidelines on auditing a Safety Management System - · PDF file1.3 ISAGO Audit Scope ... factors will have to be taken into consideration when determining the scope of the audit and

11

The safety reporting policy supports the policy outlined in ORM 1.2.2(iii) and the behavior policy in ORM

1.2.2(iv) but has to specifically address safety reporting. In this respect, the policy should outline clearly

what should be reported, by whom and when. The behavior policy should reflect the “non-punitive”

requirement in ORM 3.2.2.

The safety reporting policy should also address the data protection aspects of ORM 3.2.2(iv), which may

be subject to applicable national regulations or guidelines.

Question – Is the Provider aware of the data protection regulatory

guidance provided by ICAO in Annex 19?

3.3.6 Emergency Response Plan (ERP)

ORM 3.1.6 The Provider should have a corporate emergency response plan (ERP) that includes provisions for:

(i) The central management and coordination of all the Provider's activities should it be involved in or it is necessary to respond or react to an aircraft accident or other type of adverse event that could result in fatalities, serious injuries, considerable damage and/or a significant disruption to operations;

(ii) The appropriate coordination or be compatible with the ERPs of other applicable organizations relevant to the event. [SMS]

Note: Within 2018, this recommended practice will be upgraded to a standard.

The Provider's ERP should describe in a suitable document who does what, when and how for all perceived

emergency situations. The ERP should address the emergency procedures that maintain operational safety

from the time that an emergency is declared until normal operations are resumed. ERP should also address

security events.

The ERP should be made available and be known to all relevant personnel. Named persons or those in

named posts should be interviewed to test their knowledge and understanding of the ERP and their roles

and responsibilities. Personnel should also be trained and equipped to deal with their roles and

responsibilities.

While the Provider should develop its own ERP, specifying what its staff should do, it is highly likely that

the Provider’s station personnel will play a participative or perhaps a coordination role in the ERP of the

airport with some supervisory roles and functions (particularly for passenger handling). Look for the

association of the Provider with the ERP or other such contingency plans of the customer airline(s) and,

importantly, that of the airport authority. There should be evidence of collaboration in the ERP development

as required in ORM 3.1.6(ii).

Recommendation – Confirm that the Provider actively participates in the development, maintenance and testing of the ERP of the airport.

3.3.7 SMS Documentation (SMS Manual)

ORM 2.1.3 The Provider shall have SMS documentation that includes a description of:

(i) The safety policy and objectives, SMS requirements, SMS processes and procedures, the accountabilities, authorities and responsibilities for processes and procedures, and the SMS outputs;

(ii) Its approach to the management of safety, which is contained in a manual as a means of communication throughout the organization. [SMS]

A SMS Manual is the recommended method of collating and documenting all the administrative SMS

policies, processes, procedures when developed. ICAO and regulatory authorities produce guidelines on

the typical content of the manual. There should be a full description of the Provider’s SMS that details what

it entails, its objectives, and the roles and safety responsibilities throughout the organization. There should

Page 18: Guidelines on auditing a Safety Management System - · PDF file1.3 ISAGO Audit Scope ... factors will have to be taken into consideration when determining the scope of the audit and

12

also be a description of how safety management activities are coordinated between the organization’s

corporate and station entities. A diagram (organizational chart) of the roles and reporting lines would be

useful. There should also be details of the corporate safety policies (management commitment to provide

resources and conformance with regulations and standards); the safety reporting system; staff behavior

and punitive actions; personnel training and safety communications.

The SMS Manual may have referenced sub-parts or other documents. It is possible that the SMS Manual

and other documents are not paper-based, but are digital files or recorded and visible only on an

internet/intranet based application. This should not make any difference but obviously the way in which a

document review is conducted could differ, perhaps in a positive way as digital media is more efficiently

transferable or transportable.

Other SMS documentation would include safety reports, safety risk management records of assessments

and decisions, safety assurance reports, internal reviews, training material and records, safety

notices/communications and other SMS products and outcomes.

In reviewing the documentation, the auditor should look not only for conformance with the GOSARP but

also for completeness and continuity of the content/information. Whatever the format, the documentation

must show evidence of document/version control or being part of a document/record management system

(as per ORM 2.1.1)4 and distribution (as per ORM 1.4.1)5.

The distance between the corporate and station locations should not result in the organizations failure to

comply with its own processes and procedures, and this is an important aspect to test when conducting a

station audit.

3.3.8 SMS Implementation Plan

ORM 3.1.8 The Provider shall have an SMS implementation plan, formally endorsed by the organization that

defines the Provider's approach to the management of safety in a manner that meets the organization's safety objectives. [SMS]

See 1.2.

3.4 Safety Risk Management The second component of the SMS framework addresses the management of safety risks and is associated

with GOSARPs specified in the sub-paragraphs of ORM 3.2. Safety risk management involves

administrative processes and procedures that provide for the identification and assessment of hazards

(usually raised in safety reports) that may result in a recommendation of mitigating action to reduce the

safety risk to a tolerable/acceptable level.

Where a Safety Office exists, it should be possible to conduct the audit of the safety risk management

aspects in one location. All the documentation – details of the procedures, safety reports, assessment

results and recommendations – should be available there. If not, the documentation should in any case be

4 ORM 2.1.1 The Provider shall have a system for the management and control of the internal and external documentation and/or data used

directly in the conduct or support of operations. Such system shall comprise the elements specified in Table 1.1 and shall include documentation provided to external entities, if applicable. 5 ORM 1.4.1 The Provider shall have a communication system that:

(i) Enables and ensures an exchange of information that is relevant to the conduct of ground operations; (ii) Ensures changes that affect operational responsibilities or performance are communicated as soon as feasible to applicable management and front line personnel.

Page 19: Guidelines on auditing a Safety Management System - · PDF file1.3 ISAGO Audit Scope ... factors will have to be taken into consideration when determining the scope of the audit and

13

accessible within the Provider’s document control system. Operational personnel may be enlisted to assist

in conducting assessments and therefore should be familiar with the processes or procedures involved.

Continuity is a key aspect of the safety risk management function. An audit may take an example safety

report and “follow” it through the process of hazard identification, safety risk assessment and, as necessary,

the development of recommended risk reduction actions. There should be a record and description of each

step taken, the decisions made and their rationales. If a significant action is recommended (i.e. one that

involves budgeting, resources and planning) and is referred to senior management, the audit of the process

could be extended to the concluding events covered by the SMS safety assurance function, whereby the

action is implemented, monitored and measured. The documentation for these activities may only be

available on request, at the Safety Office or at the relevant station; however, the documentation should be

consistent.

3.4.1 Hazard Identification

ORM 3.2.1 The Provider should have a hazard identification program that is implemented and integrated

throughout the organization to include:

(i) A combination of reactive and proactive methods for safety data collection;

(ii) Processes for safety data analyses that identify existing hazards and predict future hazards to operations. [SMS]

Note: Within 2018, this recommended practice will be upgraded to a standard.

There are a number of techniques and tools available to identify hazards from safety data and safety

information derived from safety reporting systems, safety reports, external sources, etc. The audit should

verify that there is a process in place for the collection of the safety data and information and procedures

for the use of whatever hazard identification technique or tool is used. In addition, personnel involved in

safety data and information gathering and hazard identification should be adequately trained (see 3.6.3).

Safety data analysis that predict future hazards refers mainly to processes that analyze performance data

and trends. For example, an operation may be subject to a continuous safety monitoring program to identify

hazardous aspects (latent conditions) that as isolated events may be acceptable but in combination and

under certain circumstances could result in an accident (the “Swiss cheese” model).

3.4.2 Safety Reporting System

ORM 3.2.2 The Provider shall have a non-punitive operational safety reporting system that is implemented

throughout the organization in a manner that:

(i) Encourages personnel to report any incident or hazard to ground operations, identify safety hazards, expose safety deficiencies or raise safety concerns;

(ii) Complies with applicable mandatory reporting regulations and requirements;

(iii) Includes analysis and management action as necessary to address safety issues identified through the reporting system;

(iv) Specifies the measures to protect safety data from being used for any purpose other than the improvement of safety and SMS. [SMS]

.

An effective safety reporting system is arguably the most important element of the SMS. Without it, there

would be little or no safety data directly relevant to the Provider’s ground operations to base safety risk

management on. Even so, the type and amount of safety reports that are received would be dependent

upon many factors - operational, logistical and cultural - and meaningful data (such that might show trends

or latent conditions) may take some time to accumulate. Nevertheless, the safety reporting system

documentation should describe what it is, its purpose and method of operation. There should be records of

each safety report submitted and, in other processes, what happened to it. Ideally, there would be evidence

Page 20: Guidelines on auditing a Safety Management System - · PDF file1.3 ISAGO Audit Scope ... factors will have to be taken into consideration when determining the scope of the audit and

14

of the safety report using a taxonomy that was consistent with that of the safety reporting systems of the

other organizations on the airport or the regulatory authority.

The safety reporting policy (as outlined in 3.3.5) should encourage the reporting of reactive (has happened),

proactive (may happen) and, possibly, predictive (looks likely with an estimated degree of certainty to

happen) events or situations. Conformance with ORM 3.2.2(i) would include evidence of the processing of

mandatory safety reports through the appropriate internal channels.

Question – Is there a means of submitting voluntary or confidential

reports?

Internal mandatory safety reports may also be mandatory in a legal or regulatory sense. Verification that

the Provider’s safety reporting system complies with local requirements would show conformance with ORM

3.2.2(ii).

ORM 3.2.2(iii) is also related to and may be addressed by ORM 3.2.3.

Verification of a process and/or procedure to implement the safety data protection policy (see 3.3.5), to

protect the reporter or dissemination of the safety data, would demonstrate conformance with ORM

3.2.2(iv).

A selection of safety reports should be examined to verify compliance with the safety reporting policy and

procedures.

3.4.3 Safety Risk Assessment & Mitigation

ORM 3.2.3 The Provider should have a safety risk assessment and mitigation program that includes processes

implemented and integrated throughout the organization to ensure:

(i) Hazards are analyzed to determine corresponding safety risks to ground operations;

(ii) Safety risks are assessed to determine the requirement for risk mitigation action(s);

(iii) When required, risk mitigation actions are developed and implemented in operations. [SMS]

NoteWithin 2019, this recommended practice will be upgraded to a standard.

Conformance with ORM 3.2.3(i) would be demonstrated by the implementation of a process and/or

procedure that addresses the outcome of the hazard identification process (ORM 3.2.1) and the

determination of the operational consequences and safety risks.

Conformance with ORM 3.2.3(ii) would be demonstrated by the implementation of a process and/or

procedure for the analysis of the outcome of ORM-H/HS/S 3.2.3(i) and develops recommendations for a

management decision on the implementation of safety risk mitigation.

Conformance with ORM 3.2.3(iii) would be demonstrated by the implementation of a process and/or

procedure for the development of a safety action plan for an agreed implementation of a safety risk

mitigation.

In addition to the verification of the safety risk assessment and mitigation processes and procedures,

records of the assessments, meeting reports and decisions taken should be examined.

The outcome of a safety risk assessment should result in either a recommendation that no further action is

necessary (the safety risk is tolerable/acceptable) or that some form of mitigation measure is needed (to

make the safety risk tolerable/acceptable). Evidence should be sought to verify that the decision was taken

in accordance with the Provider’s procedure and criteria.

Question – Are the recommendations accompanied by a project-based

(SMART) action plan?

Page 21: Guidelines on auditing a Safety Management System - · PDF file1.3 ISAGO Audit Scope ... factors will have to be taken into consideration when determining the scope of the audit and

15

3.4.4 Accident/incident Investigation & Reporting

ORM 3.2.4 The Provider should have a process:

(i) To conduct and/or participate in an investigation of an incident/accident where its services were involved, to include reporting of events, in accordance with requirements of the costumer airline(s), the Airport Authority, and/or State, as applicable;

(ii) For identifying and investigating irregularities and other non-routine operational occurrences that might be precursors to an accident or incident. [SMS]

Note: Within 2018, this recommended practice will be upgraded to a standard.

Conformance with ORM 3.2.4(i) requires the Provider to specify what it does, when required, in the event

of an investigation. The process or procedure(s) should outline the roles and responsibilities of key

personnel, including for coordination with other organizations that may be involved or leading the

investigation. The audit should verify not only the relevance and implementation of the procedure(s) but

also that these personnel are identified and fully aware of what they will have to do (or not have to do if that

is the case). There may be differences in the procedures according to State or airport, and with respect to

an airline if relevant.

Conformance with ORM 3.2.4(ii) should be verified by evaluation of the procedure(s) and any recorded

instance of where the outcome of an investigation may have had safety implications (perhaps safety

recommendations or lessons learnt) for the Provider.

3.4.5 Ground Damage Reporting

ORM 3.2.10 The Provider should have a process to ensure aircraft ground damages are reported, if not prohibited

by the customer airline(s), to IATA for inclusion in the Ground Damage Database (GDDB). Such reports should be submitted in accordance with the formal IATA ground damage reporting structure. [SMS]

.

Sharing of safety information is a fundamental aspect of SMS. Many others may benefit from the

experiences or misfortunes of a Provider. The safety information may prove useful in predicting trends or

potential safety risks, and could prove the effectiveness of the SMS in a safety assurance sense. Instances

of ground damage should always be reported at least to the Provider concerned. The report should then,

subject to whatever measures are needed to protect the identity of the reporter, be shared with other

interested parties, including the airline concerned.

IATA is one the interested parties and conformity with ORM 3.2.10 requires the Provider to submit a report

to the Ground Damage Database. The audit should verify that this process is in place and occurs unless

there is an exception applied where an airline specifically prohibits such action. A valid signed contract

between the Provider and IATA should be available and, unless evidence of a prohibition from each

customer airline is provided; evidence of submissions (not internal reports) to IATA (in the prescribed

format) should be verified. Conformance may also be not applicable where the Provider performs only cargo

operations (no transport to and from the aircraft) or only check-in and boarding activities (no boarding bridge

maneuver activities).

Recommendation – Confirm with IATA that an alleged GDDB report has been submitted by the Provider.

3.5 Safety Assurance The third component of the SMS framework addresses the monitoring of implemented actions to mitigate

safety risks, assessment of the effectiveness of the SMS and general management of potential safety

issues associated with change.

Page 22: Guidelines on auditing a Safety Management System - · PDF file1.3 ISAGO Audit Scope ... factors will have to be taken into consideration when determining the scope of the audit and

16

Although the functional aspects of safety assurance might be performed by the Safety Office or other

administrative personnel, the data or information provided would normally be provided by operational

personnel or specific operational monitoring activities.

3.5.1 Safety Assurance Program

ORM 3.3.1 The Provider should have a safety assurance program, including a detailed audit planning process

and sufficient resources that provides for the auditing and evaluation of the effectiveness of the management system and ground operations at all stations to ensure the Provider is:

(i) Complying with applicable safety regulations and requirements of the customer airline(s);

(ii) Identifying hazards to operations;

(iii) Monitoring effectiveness of safety risk controls;

(iv) Verifying safety performance in reference to the safety performance indicators and safety performance targets. [SMS]

Note: Within 2019, this recommended practice will be upgraded to a standard.

In a similar manner to that of existing quality assurance GOSARPs, the SMS safety assurance program

comprises internal processes and procedures for the evaluation of the safety risk management function

and monitoring an implemented safety risk management recommendation.

Conformance with ORM 3.3.1(i) would be demonstrated by records of periodic or on-demand assessments

of internal procedures with external requirements, which may have changed during the course of ISAGO

certification or since the last audit. This aspect should also confirm the interaction that the Provider has with

all external parties at the airport.

Conformance with ORM 3.3.1(ii) would be demonstrated by records of periodic assessments of the

effectiveness of the hazard identification process in ORM 3.2.1.

Conformance with ORM 3.3.1(iii) would be demonstrated by records of periodic assessments of the

appropriateness of the in-progress or completed safety action plans for the implementation of safety risk

controls developed in ORM 3.2.3(iii).

Conformance with ORM 3.3.1(iv) would be demonstrated by an examination of the process or procedure

for the development of the safety performance metrics (safety indicators, targets etc.) and the periodic

measurement of the metrics with respect to the in-progress or completed safety action plans for the

implementation of safety risk controls developed in ORM 3.2.3(iii). A procedure should also be in place in

the event of loss of safety performance or failure to meet a specified safety performance target, which might

be to re-initiate the safety risk management process.

3.5.2 Safety Performance Metrics

ORM 3.3.2 The Provider should have processes for setting performance objectives and measures as a means to

monitor the operational safety performance of the organization and to validate the effectiveness of safety risk controls. [SMS]

Note: Within 2019, this recommended practice will be upgraded to a standard.

The development of safety performance metrics is a consequence of the implementation of safety risk

controls, which in the case of ground operations, where interaction with other operations is likely, could be

developed in collaboration with the airport or the airline. If such collaboration exists, and hopefully it does,

the safety performance metrics may not be developed specifically or only by the Provider. However, the

contribution made by the Provider in monitoring and assessing safety performance is a useful indicator for

assessing the effectiveness of the SMS.

Page 23: Guidelines on auditing a Safety Management System - · PDF file1.3 ISAGO Audit Scope ... factors will have to be taken into consideration when determining the scope of the audit and

17

Question – Has an external organization (airport, airline or regulatory

body) requested an input or information relating to the safety performance

metrics?

Question – Are the safety performance metrics realistic? Has there been

any change made as a result of a review?

Established safety performance metrics should be translated into safety objectives for the organization, see

3.3.4.

3.5.3 Management of Change

ORM 3.3.3 The Provider should have a process to identify changes within or external to the organization that

have the potential to affect the level of safety risk of ground operations, identify, and to manage the safety risks that may arise from such changes. [SMS]

Note: Within 2019, this recommended practice will be upgraded to a standard.

A process or procedures should be in place to detect and assess any change that may occur within or

external to the organization that has the potential to affect operational safety.

The process should identify the means of detection (monitoring, assessment etc.) and action to be taken.

Records of change monitoring activities and decision making meetings should be examined.

Change management should also address changes in organizational structure, personnel and cultural

issues.

3.5.4 Continuous Improvement of the SMS

ORM 3.3.4 The Provider should have processes to review and ensure continual improvement of the SMS

throughout the organization to include:

(i) Identification of the cause(s) of substandard performance of the SMS;

(ii) Determination of the implications of substandard performance of the SMS in operations;

(iii) Elimination or mitigation of such cause(s) of substandard performance. [SMS]

Note: Within 2019, this recommended practice will be upgraded to a standard.

This GOSARP follows a typical 3-step quality assurance process to review and seek improvements. In this

respect conformance would be demonstrated by the application of an internal quality assurance program

to assess the SMS throughout the organization.

Prior to SMS many aviation operators evolved their quality activities to include safety assurance. The

processes and procedures are similar and it is possible to integrate some safety and quality (and security)

assurance activities. This integration is scalable to the size and complexity of the organization, and would

be of particular advantage for a small, non-complex organization. There are also similarities with Workplace

or Occupational Health and Safety in which integration may occur. Integration is acceptable provided that

there are clear roles and lines of responsibilities for safety management in conformance with the relevant

GOSARPs.

Recommendation – Determine the extent to which the SMS incorporates QA procedures and GOSARPs. There should be correlation.

Page 24: Guidelines on auditing a Safety Management System - · PDF file1.3 ISAGO Audit Scope ... factors will have to be taken into consideration when determining the scope of the audit and

18

3.5.5 Management Safety Decision Making

ORM 3.3.5 The Provider should have a process for management consideration of and decision-making to ensure significant issues arising from:

(i) The safety risk assessment and mitigation program, and;

(ii) The safety assurance program are subject to management review in accordance with ORM 3.3.4 and ORM 1.5.1, as applicable. [SMS]

Note: Within 2019, this recommended practice will be upgraded to a standard.

As stated, the SMS is foremost a management decision making tool. It follows that there should be a

GOSARP that requires the Provider to have in place a process or procedure for management review of the

SMS outcomes, specifically the safety risk management and safety assurance functions. GOSARP ORM-

1.5.16 is the requirement for a periodic review of the effectiveness of a management system, which would

include the SMS, and complementary to ORM 3.3.4 if implemented in full.

Conformance would be demonstrated by examination of the processes and records of meetings, decisions

taken etc. The decision making aspect should clearly indicate who made the decision and why.

3.6 Safety Promotion The fourth component of the SMS framework provides processes and procedures for the communication

of safety information and the training of personnel.

3.6.1 Safety Awareness

ORM 1.4.2 The Provider shall have processes for the communication of safety information throughout the organization to ensure personnel maintain an awareness of the SMS and current operational safety issues. [SMS]

The SMS communication aspects are covered specifically in ORM 3.5 (as opposed to organization

communications in general); however, the means of communication may be the same. Whatever the

communication means, conformance is demonstrated by evidence of the intended audience receiving and

understanding or complying with the message or required action. Random samples should be requested

or interviews with selected personnel could reveal whether messages are being received and complied with

and the effectiveness of the safety communication process overall within the organization.

3.6.2 Safety Information

ORM 3.5.2 The Provider should have a means for disseminating information from:

(i) The safety risk assessment and mitigation program, and;

(ii) The safety assurance program to management and non-management operational personnel as appropriate to ensure an organizational awareness of compliance with applicable regulatory and other safety requirements. [SMS]

Note: Within 2019, this recommended practice will be upgraded to a standard.

Similar to ORM 1.4.2, the Provider should be able to demonstrate the processes in place to disseminate

safety information. The outputs of the safety risk management and safety assurance functions may be of a

specialist nature, sensitive or for a specific purpose. The communication process should detail precisely

how the various SMS information is handled and examples of each should be demonstrated by the Provider.

6 ORM 1.5.1 The Provider shall have a process to review the management system at intervals not exceeding one year to ensure its continuing

suitability, adequacy and effectiveness in the management and control of ground operations. A review shall include assessing opportunities for improvement and the need for changes to the system, including, but not limited to, organizational structure, reporting lines, authorities, responsibilities, policies, processes, procedures and the allocation of resources.

Page 25: Guidelines on auditing a Safety Management System - · PDF file1.3 ISAGO Audit Scope ... factors will have to be taken into consideration when determining the scope of the audit and

19

Question – How often are safety communiqués transmitted?

Question – Does the organization promulgate safety performance

information to its personnel?

3.6.3 Safety Training

ORM 5.7.2 The Provider should have a program that ensures personnel throughout the organization are trained

and competent to perform SMS duties. The scope of such training should be appropriate to each individual's involvement in the SMS as detailed:

(i) In Table 1.2 for all personnel, and

(ii) In Table 1.16 for personnel with specific assigned duties in the safety management system. [SMS]

Note: Within 2018, this recommended practice will be upgraded to a standard.

Tables 1.2 and 1.16 are detailed at Appendix A to this document.

Personnel safety training is covered on two separate levels. All personnel within the organization should

receive basic safety awareness training and familiarization of the SMS and their safety roles and

responsibilities.

Conformance with ORM 5.7.2(i) could be demonstrated by training course records and schedules (the

content of which should cover the subjects listed in Table 1.2) and the training records of personnel with a

range of duties.

ORM 5.7.2(ii) refers to training for personnel needing specific SMS skill-sets as listed in Table 1.16.

Conformance could also be demonstrated by training records/schedules.

While generic safety training could be handled in-house, the more specific training would most likely be

delivered by expert training service providers.

Page 26: Guidelines on auditing a Safety Management System - · PDF file1.3 ISAGO Audit Scope ... factors will have to be taken into consideration when determining the scope of the audit and

20

3.7 SMS Checklist The following checklist may help summarize the main aspects of the SMS audit. The checklist could be

used as a summary of the organization as a whole, or for a corporate/station pair where there are multiple

stations. The GOSM (6th Edition) ORM Section is used as a reference.

More detailed checklists (and hence more appropriate for a well-established SMS) may be found in the

ICAO Doc 9859, Safety Management Manual and ACI SMS Handbook Step A.

GOSARP Subject

ORM 3.1.8 There is a SMS implementation plan

ORM 1.1.3 An effective SMS is in place

ORM 1.1.2 There is a nominated Accountable Executive that has final authority over all the aviation activities of the organization

ORM 1.1.4 There is a person/Safety Manager who performs the role of administering the SMS

ORM 1.6.1 The SMS roles and responsibilities of all personnel are clearly defined

ORM 1.2.2 There is a safety policy statement relevant to the scope and complexity of the organization’s operations, and there is evidence of the policies resulting in safety activities or actions

ORM 3.1.5

ORM 3.2.2

There is an effective safety reporting system

ORM 3.1.6 Procedures are in place for an emergency response plan

ORM 2.1.3 All SMS information is recorded in an organization-wide document control management system

ORM 3.2.1 Procedures are in place for the assessment of safety reports

ORM 3.2.3 Procedures are in place for the identification of hazards, evaluation of safety risks, and development of recommendations for safety action plans to control unacceptable safety risks

ORM 3.2.4 Procedures are in place for the participation in incident/accident investigations

ORM 3.2.10 Procedures are in place for the reporting of ground damage to the IATA GDDB

ORM 3.3.1 Procedures are in place for the monitoring of safety risk controls

ORM 3.3.2 Procedures are in place for setting safety performance metrics

ORM 3.3.3 Procedures are in place for the assessment of safety issues related to change management

ORM 3.3.4 Procedures are in place for the review of SMS processes and procedures

ORM 3.3.5 Procedures are in place for the management review of SMS outputs and the effectiveness of the SMS

ORM 1.4.2 Safety information is promulgated throughout the organization

Page 27: Guidelines on auditing a Safety Management System - · PDF file1.3 ISAGO Audit Scope ... factors will have to be taken into consideration when determining the scope of the audit and

21

ORM 3.5.2

ORM 5.7.2 All personnel receive relevant safety training

Page 28: Guidelines on auditing a Safety Management System - · PDF file1.3 ISAGO Audit Scope ... factors will have to be taken into consideration when determining the scope of the audit and

22

Section 4 – Audit Summary Report - Assessment of the SMS

4.1 Introduction The assessment should be produced that provides a “snap-shot” of the SMS within the organization. While

the checklists provide a high-level overview of the SMS functions that correspond to the GOSARPs,

consider again the audit focus and the 4 qualities that sum up the main objectives of a SMS:

due diligence and competence in the roles;

the development, implementation of and conformance with documented processes and procedures;

coordination and cooperation with other relevant SMSs; and

the monitoring and measurement of SMS outcomes and effectiveness.

If evidence was produced at the audit that proved that all 4 qualities were fully embedded within the

organization then it could be said that the SMS was fully functional. However, it is unlikely at this time that

an organization will have implemented all the SMS processes and procedures in full, throughout the

organization, and producing results. It is more likely that some functions of the SMS were in place and

working but, perhaps in a multi-station organization, not all in the same way or to the same extent.

The assessment should therefore also consider, and evaluate, the level of implementation of the SMS. An

indication of the level of implementation should be provided in the audit report, in terms of the extent to

which each applicable GOSARP is in conformity. A measure of conformity could be the number of

framework elements implemented in full or partially per station and whether they are effective, or some

other form of gap analysis.

4.2 Audit SMS Summary The following text could be used as a basis for a (executive) summary of the SMS aspects of an audit

report. Where a deficiency is reported, full details should be included in the main body of the report.

Evidence was provided that [demonstrated/did not demonstrate] full conformance with the ORM SMS

GOSARPs. [A full list of non-conformities is provided in the report].

For example: The SMS is implemented at a basic level. The corporate SMS documentation

contains a full set of processes and procedures and safety roles and responsibilities have been

established; however, only the safety risk management process is currently in place and

functioning correctly, and only at some stations (station A, station B, station C and station D). The

safety risk management is partially implemented at station E and is expected to be fully functional

within 6 months. The other stations at which the Provider currently operates have yet to implement

any SMS function and the plan is to commence implementation in 20XX. Full details of the

Provider’s SMS implementation plan are included in the report.

Or

The SMS is implemented in full at station A and station B. Ground operations have only recently

commenced at station C and an implementation plan for the establishment of the corporate SMS

processes and procedures at station C has been developed. The Provider is conducting a gap

analysis to determine if and how the processes and procedures will need to be adapted to the

Page 29: Guidelines on auditing a Safety Management System - · PDF file1.3 ISAGO Audit Scope ... factors will have to be taken into consideration when determining the scope of the audit and

23

local operation. Full implementation of the SMS at station C and integration with the corporate

SMS is expected in Q3/20XX.

The SMS documentation provided was assessed as [satisfactory/unsatisfactory and details of the

identified deficiencies were notified to the Provider]. The SMS documentation [provided/did not provide]

a clear description of the:

SMS functionality

For example: The description of SMS functionality in the corporate SMS Manual had not been

updated to include the new Safety Office that had been established in the corporate headquarters.

As a consequence virtually all of the processes and procedures for the safety risk management

and safety assurance functions were invalid and the effectiveness of the SMS could be brought

into question.

Roles and responsibilities

For example: Evidence of job descriptions and lines of responsibility was provided to verify that all

roles and responsibilities for the SMS were identified and described in full.

Lines of communication for SMS activities

For example: Although the implementation of the SMS is on-going and the lines of communication

are in the process of being established, there were serious deficiencies with the transition.

Evidence was found of safety instructions issued by the corporate headquarters that did not reach

the intended destination and recipient.

All staff involved in key SMS roles [demonstrated/did not demonstrate] satisfactory knowledge of their

roles and responsibilities.

For example: Despite the existence of an adequate training procedure for the induction of new

operational employees, there was no evidence, verified by questioning, of the use of the procedure

since the last audit.

A complete set of written processes and procedures [were/were not] included in the SMS

documentation.

For example: A complete set of processes and procedures were included in the SMS

documentation; however, the documentation viewed at station C was of a previous version. The

latest versions were received by the station over 3 months ago but were yet to be implemented.

------------------------------------

Page 30: Guidelines on auditing a Safety Management System - · PDF file1.3 ISAGO Audit Scope ... factors will have to be taken into consideration when determining the scope of the audit and

24

Appendix A – QA Provisions & SMS Training Tables

ORM

ORM 3.4.1 The Provider shall have a quality assurance program, including a detailed audit planning process and sufficient resources that provides for the auditing and evaluation of the management system and ground operations at all stations to ensure the Provider is:

i. Complying with applicable regulations and requirements of the customer airline(s);

ii. Satisfying stated operational needs;

iii. Identifying undesirable conditions and areas requiring improvement.

iv. Monitoring effectiveness of safety risk controls

ORM 3.4.2 The Provider shall have a station quality control program that provides for scheduled and unscheduled inspections and/or evaluations of ground operations at the station for the purpose of ensuring compliance with standards of the Provider, quality assurance program as specified in ORM 3.4.1, applicable regulations, and requirements of the customer airline(s).

ORM 3.4.3 The Provider shall have a process for addressing findings that result from audits conducted under the quality assurance program and station quality control program, as specified in ORM 3.4.1 and ORM 3.4.2, which ensures:

(i) A determination of root cause(s);

(ii) Development of corrective action as appropriate to address findings;

(iii) Implementation of corrective action in appropriate operational area(s);

(iv) Monitoring and evaluation of corrective action to determine effectiveness.

ORM 3.4.4 The Provider shall have a process to ensure significant issues arising from the quality assurance and station quality control program are subject to management review in accordance with ORM 1.5.1.

ORM 3.4.5 The Provider shall have a means for disseminating information from the quality assurance program and station quality control program, as specified in ORM 3.4.1 and ORM 3.4.2, to management and non-management operational personnel as appropriate to ensure an organizational awareness of compliance with applicable regulatory and other requirements.

ORM 3.4.6 The Provider shall ensure the quality assurance program utilizes auditors that:

(i) Have been trained and are qualified;

(ii) Are impartial and functionally independent from operational areas to be audited.

Page 31: Guidelines on auditing a Safety Management System - · PDF file1.3 ISAGO Audit Scope ... factors will have to be taken into consideration when determining the scope of the audit and

25

Table 1.2 Safety Training Specifications

Functional Groups

For the purpose of determining the applicability of airside safety training subject areas, ground handling personnel are grouped according to

operational function as follows. Note 1

Function 1: Personnel whose duties require access to airside areas.

Function 2: Personnel whose duties require operation of basic GSE (e.g., tractors, belt loaders).

Function 3: Personnel whose duties require: (1) operation of specialized equipment (e.g., aircraft movement units, container/pallet loaders, de-icing vehicles, catering vehicles), (2) exercise of control during aircraft movement operations, or (3) performance of lead responsibility over other personnel.

Function 4: Personnel in first level management, to include supervisors having responsibility for: (1) directing staff and/or equipment resources, or (2) controlling an operational activity.

Function 5 Personnel in station management having responsibility for resource issues, health and safety, incident management and budgetary control.

Function 6 Personnel with duties in ticketing, check-in and boarding activities.

Function 7 Personnel operating within Cargo warehouse

Note 1: Functional definitions may be varied as determined by local requirements or considerations

Training Subject Areas

Safety training shall address, according to assigned operational function(s).

1.1.1 Safety Philosophy

a) Company safety policy and program [SMS] All Functions

b) Employer/employee responsibilities [SMS] All Functions

1.1.2 Safety Regulations

a) International aviation regulations [SMS] All Functions

b) State aviation regulations [SMS] All Functions

c) Airport airside regulations [SMS] All Functions

d) Safe working and operating practices [SMS] All Functions

1.1.3 Hazards Note 2

a) Vehicle movements All Functions

b) Pedestrian movements All Functions

c) Aircraft movements All Functions

d) Jet engines All Functions

e) Propeller-driven aircraft and helicopters All Functions

f) Aircraft antennae and other protrusions All Functions

g) GSE Functions 2-5

h) Aircraft fuelling and fuel spills All Functions

i) Adverse and seasonal weather conditions All Functions

j) Night operations All Functions

k) Working at height All Functions

l) Slips, trips and falls All Functions

m) Noise All Functions

n) Manual handling All Functions

o) Confined Spaces All Functions

p) Office Equipment All Functions

q) Display Screen Equipment (DSE) All Functions

r) Violence (physical & verbal attack and public disorder) All Functions

s) Lone working All Functions

Note 2: Subject areas a) through s) are applicable to personnel as appropriate to specific function and types of operations conducted.

Page 32: Guidelines on auditing a Safety Management System - · PDF file1.3 ISAGO Audit Scope ... factors will have to be taken into consideration when determining the scope of the audit and

26

Table 1.2 Safety Training Specifications (cont’d)

1.1.4 Human Factors

a) Motivation and attitude All Functions

b) Human behavior Functions 4, 5

c) Communication skills All Functions

d) Stress All Functions

e) Ergonomics All Functions

f) Effects of psychoactive substances (drugs and alcohol) All Functions

g) Fatigue All Functions

h) Time pressure All Functions

i) Peer management pressure All Functions

j) Situational awareness All Functions

k) Teamwork All Functions

1.1.5 Airside Markings and Signage Functions 1 to 5

1.1.6 Emergency Situations Note 3

a) Reporting [SMS] All Functions

b) Injuries All Functions

c) Security threats All Functions

d) Spillage Functions 1 to 5

e) Alarms and emergency stops Functions 1 to 5

f) Fuel shut-offs Functions 1 to 5

g) Ground-to-flight deck emergency hand signals Functions 1 to 5

h) Fire All Functions

i) Severe weather Functions 1 to 5

j) Aircraft stand emergency procedures Functions 1 to 5

Note 3: Subject areas a) through j) are applicable to personnel as appropriate to specific function and types of operations conducted.

1.1.7 FOD prevention Functions 1 to 5

1.1.8 Personal protection Note 4

a) Personal protective equipment All Functions

b) Occupational health and safety All Functions

c) Musculoskeletal injury prevention All Functions

d) Weather exposure Functions 1 to 5

Note 4: Subject areas a) through d) are applicable to personnel as appropriate to specific function and types of operations conducted.

1.1.9 Accidents, Incidents, Near Misses Note 5

a) Personnel injuries [SMS] All Functions

b) Damage to aircraft, GSE, facilities Functions 1 to 5

c) Reporting [SMS] All Functions

d) Investigation Functions 4, 5

e) Prevention [SMS] All Functions

f) Cost of accidents, incidents [SMS] All Functions

g) Risk assessment All Functions

Note 5: Subject areas a) through g) are applicable to personnel as appropriate to specific function and types of operations conducted.

Page 33: Guidelines on auditing a Safety Management System - · PDF file1.3 ISAGO Audit Scope ... factors will have to be taken into consideration when determining the scope of the audit and

27

Table 1.2 Safety Training Specifications (cont’d)

1.1.10 Airside Safety Supervision

a) Creating an open reporting culture [SMS] Functions 4, 5

b) Performance monitoring Functions 4, 5

c) Coordination of airside activities Functions 4, 5

d) Workload management Functions 4, 5

e) Decision making Functions 4, 5

f) Planning Functions 4, 5

Table 1.16 Specific SMS Training Specifications

Training for personnel with assigned duties in the safety management system (typically within the Safety Office) shall address the following subject areas, as applicable to assigned function(s):

i) Safety Risk Assessment:

a) management of safety reports;

b) hazard identification;

c) hazard analysis;

d) safety risk assessment;

e) safety mitigation and risk management;

f) Development of safety action plans.

ii) Safety Assurance:

a) Development of safety performance indicators;

b) Safety performance monitoring and measurement;

c) Safety auditing methodologies and techniques.

End


Recommended