Identifying Risk via via Confidential Reporting Confidential Reporting

SystemsSystems

RAe Soc HF Group EMSG RAe Soc HF Group EMSG Conference, 10Conference, 10thth October 2007 October 2007

Dr Mike Rejman Dr Mike Rejman Director, CIRASDirector, CIRAS

Recent Career 1988 - present Recent Career 1988 - present

Head of Human Factors Unit, UK Army Air Corps Head of Human Factors Unit, UK Army Air Corps (accident investigator, set up their confidential (accident investigator, set up their confidential reporting system) reporting system)

Principal Consultant, DERA, QinetiQ,Principal Consultant, DERA, QinetiQ,

Assistant Director of Patient Safety at National Assistant Director of Patient Safety at National Patient Safety Agency (accident investigation Patient Safety Agency (accident investigation training, reporting system, risk assessment) training, reporting system, risk assessment)

Director of CIRAS, the confidential reporting Director of CIRAS, the confidential reporting system for the UK railwayssystem for the UK railways

Understanding the ProblemUnderstanding the Problem• ~ 80% of accidents are attributable to human factors 80% of accidents are attributable to human factors

issues, at the individual level, the organisational level, issues, at the individual level, the organisational level, or more commonly bothor more commonly both

• This is probably a conservative figure, and is This is probably a conservative figure, and is irrespective of domainirrespective of domain

• To manage this we need to identify and understand the To manage this we need to identify and understand the risksrisks

• Without this understanding we can’t put appropriate Without this understanding we can’t put appropriate remedial action in placeremedial action in place

Reason’s Swiss Cheese ModelOrganisation and processes- Deficiencies “Latent failures” - precursors, pre-

existing enabling conditions

“Active failures” (skill, rule and knowledge errors)

Multiple Defences Accident or incident

The Accident Iceberg (Heinrich, Bird, and others)

1

?

?

?

accidentsaccidents

serious incidentsserious incidents

incidents incidents

near misses near misses

& concerns& concerns

often often unreportedunreported

Prior Indicators of RiskPrior Indicators of Risk

• Herald of Free Enterprise, 1987Herald of Free Enterprise, 1987

– Numerous accounts of previous sailings with the Numerous accounts of previous sailings with the bow doors openbow doors open

– Economic pressuresEconomic pressures to spend to spend

less turnaround time in port less turnaround time in port

Prior Indicators of RiskPrior Indicators of Risk

• Challenger Space ShuttleChallenger Space Shuttle

– Ample previous evidence of seals shrinking Ample previous evidence of seals shrinking in cold temperaturesin cold temperatures

– Voiced by some engineersVoiced by some engineers

– Political pressure to launchPolitical pressure to launch

Prior Indicators of RiskPrior Indicators of Risk

• Kings Cross Fire, 1987Kings Cross Fire, 1987

– Numerous fires, bundles of rubbish and Numerous fires, bundles of rubbish and wooden escalators with greased tracks, some wooden escalators with greased tracks, some records but nothing done records but nothing done

– Complacency and no system to evaluate risk Complacency and no system to evaluate risk

Prior Indicators of RiskPrior Indicators of Risk

• Hillsborough, 1989Hillsborough, 1989

– Two Police Officers on horseback prevented Two Police Officers on horseback prevented similar incident the previous year by blocking similar incident the previous year by blocking access to the same terrace areaaccess to the same terrace area

– No organisational memoryNo organisational memory

Investigating Accidents with the Army Air Investigating Accidents with the Army Air CorpsCorps

• Experience while conducting investigations Experience while conducting investigations indicated that the workforce held lots of indicated that the workforce held lots of important information that could have been important information that could have been used to prevent the accidentused to prevent the accident

• The culture didn’t encourage reporting and The culture didn’t encourage reporting and there was no ‘safe’ route for this information there was no ‘safe’ route for this information to be passed on and assessedto be passed on and assessed

AAC views on confidential reportingAAC views on confidential reporting

• (Some) Senior Command (Some) Senior Command – Blame & punishmentBlame & punishment– Would subvert the chain of commandWould subvert the chain of command

• Others Others – System requiredSystem required– RAF-run Condor system inappropriateRAF-run Condor system inappropriate

AAC maintenance issues AAC maintenance issues

• Gazelle maintenance team changeGazelle maintenance team change– habit, slip habit, slip

• Lynx tail rotor gear box problem and Lynx tail rotor gear box problem and cannibalisationcannibalisation– culture, knowledge error, perceptual problem, culture, knowledge error, perceptual problem,

assumptions (some of these issues could have assumptions (some of these issues could have emerged via a confidential reporting system)emerged via a confidential reporting system)

The situation in the NHSThe situation in the NHS

Prof Sir Liam Donaldson, CMO EnglandProf Sir Liam Donaldson, CMO England

• ‘‘Organisation with a Memory’ (2000)Organisation with a Memory’ (2000)

• National Patient Safety Agency formed (2001)National Patient Safety Agency formed (2001)

• NHS Staff views – e.g. Primary CareNHS Staff views – e.g. Primary Care– PharmacistsPharmacists– GPsGPs– DentistsDentists

Patient safety risk

0

2

4

6

8

10

12

14

16

18

% of acute admissions

Australia 16.6%

England 10.8%

Denmark 9%

New Zealand 12.9%

Canada 7.5%

Japan 11%

France 14.5%

The situation in the NHSThe situation in the NHS

• The NHS is a The NHS is a labour-intensive industry labour-intensive industry – leading to a large number of human leading to a large number of human

interactionsinteractions– increasing the risks of decision-making errorsincreasing the risks of decision-making errors– and increasing the risk of communication and increasing the risk of communication

errors errors

The national reporting and learning system The national reporting and learning system

• Positive:- the system now has over Positive:- the system now has over well over one million reports well over one million reports

• Negative:- however the quality of Negative:- however the quality of much of the data is not high much of the data is not high

• The system was originally set up to The system was originally set up to be anonymous, rather than be anonymous, rather than confidential confidential

NHS exampleNHS example

• ~ I30 types of infusion pump available ~ I30 types of infusion pump available • Study revealed that 47 different types Study revealed that 47 different types

were present in one Trustwere present in one Trust• And 6 different types were found on And 6 different types were found on

one wardone ward• Leading to high level of performance Leading to high level of performance

errors errors

NHS examples NHS examples

• Removal of the wrong kidneyRemoval of the wrong kidney

• Chemotherapy drug vincristine Chemotherapy drug vincristine delivered intrathecally instead of delivered intrathecally instead of intravenouslyintravenously

Rail Confidential Incident and Analysis Rail Confidential Incident and Analysis System System

• Pilot study in Scotland 1996Pilot study in Scotland 1996• Rolled out nationally after Rolled out nationally after

Ladbroke Grove accident and Ladbroke Grove accident and Cullen Inquiry 2000Cullen Inquiry 2000

• 3 regional offices run by 3 regional offices run by contractors, now one in-house contractors, now one in-house team in Londonteam in London

• In 10 years of operation there In 10 years of operation there has never been a breach of the has never been a breach of the confidentiality guarantee for confidentiality guarantee for reportersreporters

UK Rail UK Rail

• Is one of the safest Is one of the safest forms of transport forms of transport both for travellers both for travellers and staffand staff

• But it could be even But it could be even safer as the safer as the accident in Cumbria accident in Cumbria in February showedin February showed

Forms of non-compliance across all sectors (2004 - 2006)

51.0%

31.4%

17.6%

Intentional rule violation

Best practice ignored

Rule violation

CIRAS reports on non-compliance CIRAS reports on non-compliance with rules with rules

Non-compliance by sectorNon-compliance by sector Forms of non-compliance by sector (2004 - 2006)

0

1

2

3

4

5

6

7

8

9

10

NR Subcontractor TOC LUL

Sector

No

. o

f re

po

rts

Intentional rule violation

Rule violation

Best practice ignored

Causal factors for each sectorCausal factors for each sector

SectorSector Cause 1 Cause 1 Cause 2Cause 2 Cause 3Cause 3

Network RailNetwork Rail Performance Performance before safetybefore safety

46.7%46.7% Poor Poor planningplanning

20.0%20.0% Poor Poor managementmanagement

13.3%13.3%

SubcontractSubcontract- or to NR- or to NR

Inadequate Inadequate trainingtraining

33.3%33.3% Staff Staff shortagesshortages

22.2%22.2% Performance Performance before safetybefore safety

22.2%22.2%

TOCTOC Performance Performance before safetybefore safety

28.0%28.0% OtherOther 22.2%22.2% Inadequate Inadequate trainingtraining

16.7%16.7%

LULLUL Performance Performance before safetybefore safety

55.6%55.6% Staff Staff shortagesshortages

22.2%22.2% Poor Poor managementmanagement

22.2%22.2%

27

Quotes from reportersQuotes from reporters

“ We’ve refused to do the job on the grounds of safety … …. and we’ve been threatened with disciplinary action...”

“I’ve been told that if I didn’t want to do the job [on safety

grounds] then I shouldn’t

bother coming to work

tomorrow”

How can we best identify risks?How can we best identify risks?

• Reactive/retrospectiveReactive/retrospective – Accidents and incident investigation Accidents and incident investigation – Root Cause Analysis (what, who, how, Root Cause Analysis (what, who, how, WHYWHY))– Reporting systems: incidentsReporting systems: incidents

• But we need to move people from ‘fire-fighting’ But we need to move people from ‘fire-fighting’ the last error to trying to prevent the next onethe last error to trying to prevent the next one

• Proactive/prospective Proactive/prospective – Confidential Reporting systems: near misses, and safety Confidential Reporting systems: near misses, and safety

concernsconcerns– Hazard identification, and prospective risk assessments Hazard identification, and prospective risk assessments

Barriers to ReportingBarriers to Reporting• Fear (including concerns re confidentiality)Fear (including concerns re confidentiality)

– blame culture, job loss, commercial issues blame culture, job loss, commercial issues • PracticalitiesPracticalities

– time to reporttime to report– complex formscomplex forms

• Ignorance Ignorance – what to report (definitions e.g. ‘near miss’)what to report (definitions e.g. ‘near miss’)

• Apathy Apathy – lack of perceived benefit to individual vs lack of perceived benefit to individual vs

potential cost potential cost – lack of faith in the system to change things lack of faith in the system to change things – lack of any feedback (‘black hole’ syndrome)lack of any feedback (‘black hole’ syndrome)

Reporting rates and triangulating riskReporting rates and triangulating risk

• For these reasons, incidents, near-misses & For these reasons, incidents, near-misses & safety concerns safety concerns will always be under-reportedwill always be under-reported whatever the systemwhatever the system

• But there are ways to increase reporting, by But there are ways to increase reporting, by targeting specific issues, and seeking other targeting specific issues, and seeking other ways to triangulate risk including surveys ways to triangulate risk including surveys

Reporting rates and triangulating riskReporting rates and triangulating risk

• More reports = good news, not bad news. With More reports = good news, not bad news. With more information to analyse we can act as an more information to analyse we can act as an Early Warning SystemEarly Warning System

• The industry must therefore encourage and The industry must therefore encourage and reward reporting, not penalise itreward reporting, not penalise it

• CIRAS has recently moved beyond the model of CIRAS has recently moved beyond the model of the ‘passive post-box’ and is seeking to be more the ‘passive post-box’ and is seeking to be more ‘proactive’ in engaging industry groups e.g. with ‘proactive’ in engaging industry groups e.g. with workshops and surveys workshops and surveys

Confidential Reporting Systems Confidential Reporting Systems

In an ideal world we wouldn’t need In an ideal world we wouldn’t need confidential reporting systems. Staff confidential reporting systems. Staff would be happy to volunteer information would be happy to volunteer information about errors they made, or safety about errors they made, or safety concerns they had, without fear of blame concerns they had, without fear of blame or victimisation, and management would or victimisation, and management would willingly address all the issues raised by willingly address all the issues raised by their staff………… their staff…………

Confidential Reporting Systems Confidential Reporting Systems

But we don’t live in an ideal world ! But we don’t live in an ideal world !

Consequently in virtually all safety-critical Consequently in virtually all safety-critical industries, here and abroad, it has been industries, here and abroad, it has been necessary to incorporate confidential necessary to incorporate confidential reporting systems as part of the suite of reporting systems as part of the suite of reporting systems available and operating reporting systems available and operating within safety management systemswithin safety management systems

Identifying Risk by Identifying Risk by Donald RumsfeldDonald Rumsfeld

As we know,As we know,

there are known knowns.there are known knowns.

There are things we know we know. There are things we know we know.

We also know there are known unknowns. We also know there are known unknowns.

That is to say That is to say

We know there are some things we do not know.We know there are some things we do not know.

But there are also unknown unknowns, But there are also unknown unknowns,

The ones we don’t know we don’t know. The ones we don’t know we don’t know.

Confidential reporting systemsConfidential reporting systems

• Are there to access all these Are there to access all these categories categories

• Are an indispensable part of the Are an indispensable part of the process of identifying risk in any process of identifying risk in any safety-critical industry safety-critical industry

• Should be an integral part of any Should be an integral part of any safety management systemsafety management system

mike.rejman@ciras.org.ukmike.rejman@ciras.org.uk