System Safety - M2 System Safety Planning V1...System safety e ort needs to be de ned for multiple...

System SafetyM2 System Safety Planning V1.2

Matthew Squair

UNSW@Canberra

8 April 2016

1 Matthew Squair M2 System Safety Planning V1.2

Except for images whose sources are specifically identified, this copyright work islicensed under a Creative Commons Attribution-Noncommercial, No-derivatives 4.0International licence.

To view a copy of this licence, visit http://creativecommons.org/licenses/by-nc-nd/4.0/


1 Introduction

2 Overview

3 Acquirer safety planning

4 Supplier safety planning

5 Contracting for safety

6 Tailoring the Safety Program

7 The System Safety Program Plan (SSPP)

8 Conclusions


Introduction

1 Introduction

2 Overview






8 Conclusions


Introduction

Learning outcomes

How to:

plan an effective safety program that is integrated into all life cyclephases

establish definitive system safety program requirements

define safety requirements clearly in system specifications andcontractual documents

prepare a SSPP that reflects how the total program is to be conducted

review and approve for implementation contractor SSPPs

establish work breakdown structure elements at appropriate levels forsystem safety management and engineering


Overview

1 Introduction

2 Overview






8 Conclusions


Overview

Overview

“The seeds of problems are laid down early. Initial planning is themost vital part of a project. The review of most failed projects orproject problems indicate the disasters were well planned to happenfrom the start.”

— One Hundred Rules for NASA Project Managers


Overview

Planning, the precursor, and most critical task

Inadequate specification of safety requirements can lead to cost andschedule impacts. Poor upfront understanding of specialist design risks cancompound this problem

Effective safety planning must

Describe the safety effort required and intended

Document the safety effort for common understanding

The challenge, to plan cost-effectively in the face of uncertainty

Iterative planning is necessary for all but smallest projects

Acquirer must define needs program requirements

Supplier must define solution safety program


Overview

The planning balancing act

Successful safety programs emerge at the balance point of cost, scheduleand safety

We cannot afford accidents

We also cannot afford systems that cannot perform because ofoverstated safety requirements

Defining what is acceptable risk is essential for cost/effectiveness

Safety effort needs to be balanced across the program

Safety risks can drive cost and schedule, but cost and schedule can drivesafety risk


Overview

Limitations of discussion

We will primarily discuss planning in the context of a genericMIL-STD-882 program

Noting that:

MIL-STD-882D replaced 882C and was performance based

MIL-STD-882E reintroduced task descriptions, & additional tasks!

We won’t get bogged down in detailed task descriptions and nuances...


Overview

Karl Weick’s view on planning

Planning can be antithetical to understanding and addressing the realissues

Plans are always based on the past, and are thus a retrospective activity.In circumstances of high uncertainty they may be of little use and at worsta cookie cutter response that tries to makes the problem fit the solution

The planning paradox

If a situation is uncertain we should plan less and talk, think, act more,while if it’s more certain (but complex) we should plan more and talk,think less. In practice we tend to do the reverse


Acquirer safety planning

1 Introduction

2 Overview






8 Conclusions


Acquirer safety planning Is it necessary?

Assess the necessity

A review of the following issues will normally clarify the need for a formaland funded safety program

What is the corporate safety or WHS policy?

What regulatory requirements for safety apply?

What is the precedence of corporate policy and regulation?

What are the mandatory tasks and deliverables?

Who accepts (assumes) safety risk on behalf of the customer orperson conducting the business?

What regulatory approvals (certification) are required?

Does the system contain safety critical elements or functions?

You can also conduct a PHL to establish a list of hazard sources andaccidents of concern


Acquirer safety planning Safety goals, objectives and requirements

Establish the overall safety goals

A safety goal is simply an expression of a high level desire

Usually (but not always) qualitative in nature

The originating requirement for the safety program

Apollo program safety goal

”That within the decade this nation shall land a man on the moon andreturn him safely to earth.” President John F. Kennedy

Ideally the goal should be & endorsed by upper management

May be supported by a statement as to how the project will be managed(a credo or policy statement)


Acquirer safety planning Safety goals, objectives and requirements

Define program requirements and objectives

Objectives are what we seek to achieve to satisfy our safety goal

Program requirements are external program requirements levied on theproject (such as legislation, regulations, specifications etc) that we need toaddress:

Define the precedence hierarchy of such requirements

Identify source documents for the above hierarchy

Cross walk the requirements and identify discrepancies

Resolve disconnects, overlaps and conflicts

Consider the safety goal, and objectives should we modify them?


Acquirer safety planning Define program scope

Defining program scope

We next need to define how much we’re going to do (and why):

Review the safety requirements & objectives

What do we have to do to achieve these objectives?

The answer represents the program scope

Document in the project WBS

If it’s not in the scope, we don’t do it. So it’s important that all partiesunderstand and agree that the scope is acceptable


Acquirer safety planning Safety interfaces

Identify safety interfaces

For effective risk management information from many sources must beintegrated

System safety integrates these into a coherent program

Each source (or discipline) may have specific regulatory requirements

Such interfaces are normally documented in the SSPP

Safety interfaces and human factors

A supplier may have little insight into human factors issues, failing toestablish the interface between end users and the supplier’s safety programmay lead to the human factors aspects of system safety being neglected



Identifying safety interfaces

At the acquirer level

Certifying authorities

The accepting authority for the system from the project

Centers of expertise for specialist/operational safety aspects

The end user!

At the supplier level

Program management

Design & Test engineering

Subcontract management

Production

Define interfaces in the SSPP, work tasks in the WBS



Safety interfaces (example)


Acquirer safety planning Roles, responsibilities and authorities

Authority, responsibility & accountability for safety

Seems simple enough but it is actually one of the most difficult andnecessary elements of the safety program. Ideally integrated in a singleindividual or small group. For a project the project manager is the logicalchoice

For a running system or standing organisation a senior executive should beresponsible (UK HSE practice)

Some organisations split operational & technical safety

Hint, look at who has financial control e.g. budget

Hint, look for technical and operational responsibilities

The WHS Act identifies the Person Conduction the Business orUndertaking (PCBU), as well as Officers (executives) assisting the PCBU



The system safety manager

A person responsible to project management for setting up and managingthe system safety program (per MIL-STD-882)

Acts on behalf of the duty holder (PM, customer, DAR et al.)

Inform the duty holder of risks and hazards before decisions are madeimpacting safety

Ensure the duty holder is aware of what they’re accepting

Assure appropriate due diligence is applied to safety

An implicit goverance role

The challenge of a governance role

The role of system safety is often something that line management doesnot ’get’, and can find confronting



Establish independence

We also know a bit about what is needed from an organisationalperspective to make a successful system safety organisation. These aresummarised in the 3 ’I’s:

Influence. Directly report to decision makers

Independence. To be independent of project or management and oftheir parochial concerns

Insight. Unchallenged visibility of what’s going on

Lack of independence

If the organisation falls short of these requirements then the system safetyactivities will likely be ineffective to a greater or lesser extent


Supplier safety planning

1 Introduction

2 Overview






8 Conclusions




“The Rule of Cs, Contractors are cunning and ye can’t trust em.”

— Anon



Contractual roles

The role of the contractor may vary:

Prime contractor responsible to the customer

Supplier of a specific item or equipment, to the prime

The prime contractor’s role is to:

Flow head contract safety requirements to subcontracts

Manage the safety engineering work of subcontractors

Make a profit, meet schedule & satisfy customer requirements

The customer has no contract with subcontractors

Primes come in all shapes and sizes

Item supplier may have low levels of expertise in safety engineering




Supplier planning tasks

Translate customer objectives into a viable program

Define work-scope (work breakdown structure)

Define inputs, outputs and interfaces to safety program

Time sequence the work (master and intermediate schedule)

Assign resources

Make/buy decisions for key analyses

Cost program and agree budget with project manager

Address management challenges e.g. budget reductions

Identify program risks to project management based on above



Defining the work-scope

Use the project Work Breakdown Structure (WBS) and dictionary todefine and document the work scope

System safety effort needs to be defined for multiple WBS

Prime contractor systems engineering (PHA, SSPP etc)

Subcontractor (SSHA, SSWG support)

Test & Evaluation (safety verification)

It is an integral part of developing subcontracts

Good liaison with the subcontract managers is essential

Active involvement of prime’s system safety in negotiations is essential



Example prime contractor WBS dictionary contents

Includes:

Safety planning effort and subcontract definition

Safety management of work including subcontract effort

System level hazard analyses (PHA, SHA & O&SHA)

Excludes:

Subcontractor effort (subcontract SSPPs, SSHA & SAR)

Safety verification effort (T&E WBS element)

Work standards and specifications

System specification (safety requirements)

Design safety standards (O&HS regulations, PV codes etc)

Once defined?

Map tasks & milestones to schedule



Example prime contractor WBS dictionary contents

Includes:

Safety planning effort and subcontract definition

Safety management of work including subcontract effort

System level hazard analyses (PHA, SHA & O&SHA)

Excludes:

Subcontractor effort (subcontract SSPPs, SSHA & SAR)

Safety verification effort (T&E WBS element)

Work standards and specifications

System specification (safety requirements)

Design safety standards (O&HS regulations, PV codes etc)

Once defined? Map tasks & milestones to schedule


Supplier safety planning Planning the safety analysis program

Planning the analysis program

“Engineering advances by proactive and reactive failure analysis, andat the heart of the engineering method is an understanding of failurein all its real and imagine

— H. Petroski



Planning the analysis program

Planning the analysis program involves answering four questions:

Program. What’s the answer you need, and by when?

Methodological. What techniques will you use?

Scope. What will each analysis cover, and why?

Assignment. Who will do it, specialists, supplier, the PMO?

The analysis plan is an integral and core part of the safety program

Safety programs and Collingridge’s dilemma

System safety programs always face a trade-off between fidelity of answerand timeliness



Analysis techniques

Analysis searches for temporal/structural relationships between events

Trade off accuracy against timeliness

Completeness problem still exists

Uncertainty (Q3 epistemic risk) needs to be evaluated

Don’t ’boil the ocean’, start, and stop rules are needed

Have you considered

Software, operators and the environment as hazard sources?

Have you considered whether certain hazards may interact?

If combining probabilities what of the law of comparative crudeness?



Hazard analysis search strategies

Figure: Search strategy weaknesses (from Safeware 1993)



Example analysis strategies

Different analysis techniques use differing search strategies

Fault tree. Backward/top down from a known Top Event via a logicdiagram to unknown causal factors

Event tree. Forward from a known Initiating Event to look at howthe system responds to challenges and unknown end state or states

FMECA. Forward/bottom up from a known component failure tolook at next level effects which are unknown

PHA. Exploring which causal factors could lead to what consequences

Analyses can be inventory or logic tree in style



Hazard analysis techniques

An incomplete list of how’s:

Structured What If (SWIFT)

FMEA

Fault Tree

Zonal Hazards

Functional Hazards

HFPMEA

Fire Smoke & Toxicity

Probabilistic Risk Analysis

SW Requirements Inspection

Event trees

Pressure Vessel Hazard Analysis

Cause Consequence Analysis

HAZOP/CHAZOP

Ontological Hazard Analysis

Interface Hazard Analysis

Some techniques (such as PRA) are meta-techniques, i.e they involve morethan one technique



Hazard analysis types

An incomplete list of the why or what for:

Type of analysis

Preliminary Hazard List

Preliminary Hazard Analysis

Subsystem Hazard Analysis

System Hazard Analysis

Operating & Support HazardAnalysis

Why do it?

Identify hazard sources

Identify system hazards

Identify subsystem hazards

Identify interface hazards

Identify hazardous tasks



Hazard analysis Interrelationships



Analysis techniques

How do we really decide which technique to use and when?

Some organisations mandate one technique

Some analysts are ’one trick ponies’

Perception that a one hour ’brainstorming’ session will do it

Procedural ’safety checklist’ approach

How much time and effort we’re willing to spend

Who’s available...

Remember that the technique you apply will tacitly frame (i.e bias) yourresults

Structuring the analysis program is essential

Develop a System Mishap Model (SMM) to provide structure for theinitial safety analyses that will be focusing on hazard identification



Practical issues

The objective of a credible safety analysis program is to influence thedevelopment of the system with the least amount of pain

The most effective way to do this is via a continuous, iterative feedback ofanalysis results into design

Earlier accommodation to findings

Man-hours and calendar time conserved

Fewer surprises or performance-threatening retrofits

Fewer awkward compromises and a more coherent design



The safety schedule

The prime contractor normally develops the project schedule based on highlevel customer milestones (and interfaces)

See project specific example.



Requirements flowdown

The prime flows down appropriate SOW tasks and data requirements notresponsibility

Prime needs to tailor tasks and CDRL items (a balancing act) the PHA isan essential tool for this

Subcontractors tend to want to tend to their knitting and may not haveexpertise (or intent) to do paperwork

But you need to work these issues as part of tender process, if you don’tthen scope and cost creep will occur

Safety must compete for attention

The point at which subcontracts are let is one of the busiest in a project,it is easy for safety to become ’just another issue’



Requirements flowdown (cont’d)

Partial solution is to develop a tailoring matrix

If time permits use to develop subcontractor CDRLs and ProcurementSpecs

If not provide matrix to subcontractor with statement as to what categorytheir effort is and request quote on scope

Either way subcontractor evaluates (and buys into) scope of effort

Cover from COTS, build to print to blue sky development

AND ensure project management endorse it


Contracting for safety

1 Introduction

2 Overview






8 Conclusions




“If you dont spec it, you won’t get it.”

— Anonymous Project Manager



Basic contracting principles

The devils principles

1 A contractor will not include any uncalled for effort in a proposalwhich will increase cost to a level which might not be competitive

2 A contractor will not accomplish a task unless specifically andcontractually required to do so

3 A contractor will pay more attention to a requirement when thepenalties for noncompliance are clear

To contract without consideration of the above is to invite disaster



Basic contracting principles (cont’d)

Clear specification of safety requirements is essential for

A reasonable proposal to be prepared

A straightforward evaluation

A transparent tender process

Mistakes in safety requirements can be expensive

A mistake in safety requirements is usually less amenable to correction dueto the complex relation with system life-cycle phases



Basic contracting principles (cont’d)

There are four elements involved in achieving the system safety objectives:

1 The statement of work must clearly define the required tasks

2 The CDRL must define the data required as a result of those tasks3 Bidders instructions, accompanying the RFP, must specifically define

the response required so that there is

Proper evaluation of each response from the same baselineProper understanding of the task by bidders

4 The specifications must include all necessary requirements documentsto define safety design requirements



Defining the contracted effort (task or performance)

Safety programs are normally implemented via contracted effort

MIL-STD-882C was developed as a detailed ’task’ standard to allowtailoring of the safety effort to suit the program size and life-cycle phase

MIL-STD-882D or ISO 61508 are in comparison not intended for tailoring,they are intended to be placed on contract in toto as a performancestandard

There are pros and cons to both approaches...


Tailoring the Safety Program

1 Introduction

2 Overview






8 Conclusions



Tailoring the safety program

“Never mind the width, feel the quality.”

— Anon



Tailoring the safety program

If using a task based standard such as MIL-STD-882 select tasks:

that materially aid in achieving safety requirements

that are most cost-effective

that meet cost & schedule constraints

Tailor selected safety tasks

to fit the level of effort for the project

to ensure coordination with other technical disciplines

to eliminate overlap in activities

Specify when is the task to be accomplished (date/milestone) and definewhat actions should/must be carried out on completion



Tailoring the safety program (cont’d)

Hints on saving time and budget

Phase reports relative to major milestones rather than by ED

Ask for updates to existing reports rather than new

If design still open express tasks in the SOW to allow flexibility e.g.’the fault tree analysis will consider X number of topics to be agreedat the SSWG..’

If many analyses but few reports are required ensure thatcomprehensive backup files are held of supporting data

Require access to data, not the document/file itself

Integrate safety reporting into the program status report

Use a checklist to review the scope of work



Abuses of the tailoring process

Be aware of abuses of the tailoring process when reviewing SSPPs:

Boilerplate or verbatim quoting of the standard

Referencing an out of date versions of the standard

No insight into the actual program specifics

Smorgasbord, or multiple tasks are included regardless of need

’Invisible’ tailoring, deleting acquirer responsibilities

Cut-up tailoring, safety requirements are tailored out



A small scale safety program

Small scale safety programs may be required when

The project budget is limited

Scope is small

The system/project is clearly low risk

The following is a minimum effort system safety program:

Conduct a Preliminary Hazard Analysis (PHA)

Assign a priority (risk based) to the recommended actions

Evaluate interfaces between actions & rest of the system

Take the recommended actions to modify the system

Prepare a system safety assessment report as a wrap-up



Customer Furnished Equipment (CFE)

CFE may be specified by the acquirer as part of the contract as providingnecessary inputs or receiving outputs from the system

Safety tasks for CFE:

If hazard data is available, identify what is needed and when

Supplier performs safety analyses needed on interfaces

Review field history if available

Acquirer to advise of unwanted/hazardous characteristics

Supplier mitigates hazards through interface design

For CFE with no provenance the supplier does hazard analyses

Prepare a system safety assessment report as a wrap-up



A safety program for COTS or NDI Items

Can save development costs but introduce safety issues!

The safety effort is dependent on available data & history

If procured as part of a system perform a failure mode analysis

If procured stand-alone

poor safety data makes justifying safety more difficult

Difficult to establish that it is safe to operate in the intended missionif it is different from commercial use

Supplier may not have capability to evaluate mission delta



A safety program for COTS or NDI Items (cont’d)

USAF Safety lessons learned for COTS acquisitions

1 Establish requirements for up-front mission/usage analysis

2 Define safety analysis requirements for unique config/ops

3 Once operational, restrict equipment to the missions intended unlessin-depth analysis is done to assess new missions



A safety program for COTS or NDI Items (cont’d)

For a small COTS/NDI acquisition project carry out a safety program andprepare a Safety Assessment or Safety Case

For a larger COTS/NDI acquisition project additionally prepare a systemsafety program plan, analyse requirements for hazards and provide supportto the SSWG to identify and address safety issues

Market investigation by the acquirer

To which safety (or other standards) the system was designed?

What is the extent to which the system is certified?

Do these certifications mean it meets mission requirements?

If modifying consider additional hazard analyses

Operate within the supplied/extended certification basis


The System Safety Program Plan (SSPP)

1 Introduction

2 Overview






8 Conclusions



The safety program plan

“The act of planning is everything, the plan itself is nothing.”

— Gen. Dwight Eisenhower



The Purpose of the SSPP

The SSPP documents the planning process it is not an end in itself

Along with the PHA the SSPP forms the safety baseline

Is the basis of ongoing understanding amongst stakeholders

There is little point in writing a plan that has no chance of implementation

Nor is there much point in parroting the words of a standard

The SSPP must reflect the way things will be done, ’warts and all’

Essential that PM & sponsor provide the ’buy in’

’Buy in’ is another reason why the project safety manager should reportdirectly to the project manager



Who authors the SSPP? Contractor prepared

Advantages:

Proposal can be evaluated before contractor selection

The contractor assumes major responsibility for the program

The contractor has more staff, the customer may not

You get a good indication how well the contractor understands yourobjectives

Disadvantages:

A contractor SSPP just restates what is written in the RFP

It may take considerable review and approval cycle time to get asatisfactory SSPP

It costs money



Who authors the SSPP? Customer prepared

Advantages:

SSPP is released with RFP, early impact on program

Defines the minimum program acceptable to the customer

The contractor can then accurately bid this effort

Saves time needed to prepare & approve a contractor SSPP

Forces the customer, before contract award, to agree on whatconstitutes an acceptable safety effort

Saves money the contractor would spend on the SSPP

Disadvantages:

The customer assumes responsibility for the safety program

Customer must have adequate staff to write the SSPP



Evaluating a SSPP

The acquirer should evaluate how well a supplier’s SSPP shows anunderstanding of the system and inherent hazards:

Is the level of effort consistent with program objectives?

Does it show a good grasp of the concept of assumed risk andacceptable versus unacceptable risk?

A supplier’s PHA, if provided with the SSPP, will enable the acquirer tocompare it with his own assessment (e.g a PHL)

Judge the supplier on her provisions for:

Identifying the hazards

Eliminating or reducing them to an acceptable level

Verifying their elimination or control



Evaluating a SSPP (cont’d)

Does the acquirer have the right to look at the suppliers procedures? Forexample the hazard analysis worksheets

Pay particular attention to

The qualifications of safety personnel

Their place in the hierarchy, and

The three I’s

Scope drives cost and should be evaluated in that light

Is it possible to combine some activities without adding risk?

Data costs; there should be just enough for an effective job

How does the proposer intend to support safety reviews?

How is the hazard log compiled for acquirer review & approval?



Format of an SSPP

A number of formats are available

DMO Acquirer Safety Management Plan (ASMP) template

Oriented towards a major prime contractor effort

Problematic if a large in-house safety effort is anticipated

US DoD DID DI-SAFT-81626 (Active)

Reflects safety plan task under MIL-STD-882

Contractor oriented, but can be used in house

Safeware safety plan format (Nancy Leveson)

Function should define form

The form of a plan should suit the program



Tailoring the plan

Not every project requires a standalone plan:

Major projects. Document in a formal SSPP

Medium sized projects. Document in the PMP or SEMP

Small projects. Use SOP’s & a safety file

Note the working group or program review at which program scope isendorsed

Check your lower tier procedures reflect the projects definitions

Remember it is an ongoing process


Conclusions

1 Introduction

2 Overview






8 Conclusions


Conclusions

Conclusions

This module has addressed the management issues of system safetyplanning

Improper planning is often at the heart of ineffective safety programs, aplan is in part a statement of ”what we’ll pay attention to..”, they need tobe the right things

It is an iterative, top down process, that must be performed by bothacquirer and supplier, you are not finished at contract signature

Ensure that safety activities are embedded in the project SOW,specifications, schedule and WBS, otherwise they will not be resourcedand executed


Date post:	15-Aug-2021
Category:	Documents
Upload:	others
View:	0 times
Download:	0 times

System Safety - M2 System Safety Planning V1...System safety e ort needs to be de ned for multiple...

Documents