1
A Systemic Model of ATM Safety: The Integrated Risk Picture (IRP)
7th USA – EUROPE ATM R&D SeminarJuly 02-05, 2007
Barcelona, SPAIN
� Eric PERRIN, Barry KIRWAN, & Ron Stroup*
EUROCONTROL Experimental Centre (EEC)*FAA ATO Safety, Washington
European Organisation for the Safety of Air Navigation
2
Outline of Presentation
� Why we need an integrated risk picture
� What it is
� How it works
� What it gives us
� What it doesn’t
� Who else is using this approach
� How it leads to a roadmap (STAR) to manage safetymanage safety
through a period of change [SESAR / NGATS]
3
Matching the safety approach to the needs…
� If your industry is stable, you have operational ongoing safety
cases and monitor accidents, incidents and precursors
� If you are making small changes, you carry out a safety
assessment of the individual Operational Improvement (OI)
� However, if you are changing the overall system, this is much
more complicated…
EEC Project
Project
OK, you’resafe - this time
4
But what if multiple parallel changes? What is the safety level of the overall system?
ATM - a set of inter-related systems
??
??
??
??
Total safety= ???
Interactionsunderstood?
Where issafety strong or weak?
5
Information Management
Demand& Capacity
Management
Traffic Management
Separation Management
Airport
ATC/FMPAOC - Military
CFMU - AOC - ATSP Military - Airport
Airspace Organization
20 minutesTake-off1 year
ATC Sector
Pilot
What is the safety
assessment of the
overall ATM system?
How might these new
elements interact?
Are there negative
interactions that can be
avoided?
Are there positive
interactions that could
lead extra safety?
Where are the strong
and weak safety areas in
the overall system?
Why do we need an Integrated Risk Picture?
6
Solution – An Integrated Risk Picture
� Look at current safety (use what we know now)
� From accidents to incidents to precursors to errors to influences
� Model how we stay safe now, and where our vulnerabilities are
� Extrapolate for the future system (predict as best we can)
� Identify safety priorities for improvements during the design phase
� Specify the safety requirements needed to keep us safe
� Consider how the transition will be managed safely
� Recognise it is not a ‘big bang’ transition
� Plot anticipated safety improvements required to keep us safe
� Measure if they are achieved, and update / modify if needed
7
So – what do we know now?
Building an Integrated Risk Picture
8
ATM Direct Contribution to Accidents –2005 (Baseline)
-1.6E-8Structural accidents
* Potential ATM contributions to these accident categories have not yet been estimated.
1.1E-082.9%3.8E-07Total aircraft accidents
--2.2E-08Fire/explosion
--6.4E-08Loss of control in landing*
--4.8E-08Loss of control in take-off*
--1.3E-07Loss of control in flight*
2.3E-106.9%3.3E-09Wake turbulence accident
8.3E-101.5%5.4E-08CFIT
3.1E-109.2%3.4E-09Taxiway collision
6.3E-0918.9%3.3E-08Runway collision
3.5E-0964.50%5.4E-09Mid-air collision
FREQUENCY OF FATAL
ACCIDENT DIRECTLY CAUSED BY
ATC
(per flight)
ATC
DIRECT
CAUSES
(%)
FATAL ACCIDENT
FREQUENCY (per
flight)ACCIDENT CATEGORY
� ATM contribution to direct causes:
� Fatal accidents – 2.9%
� ICAO accidents – 2.0% (i.e. 4.5 x 10-8 per flight)
9
Swiss Cheese Model – Runway Collisions
Visual warning
RIMCAS warning
Runway separation
Runway configuration
Intermediate runway entry
ATCO failure to recognise conflict
RIMCAS not installed
Operation in low visibility
Causal factors
Strategic conflict
Collision
Runway incursion
Precursors
Imminent runway collision
Potentially conflicting runway approach
10
Precursor Frequencies
Precursorsfrequencies
Accident frequency
Exposure frequency
RF1 Fatal runway collision involvement 3.3E-08 per flightRF2 Fatal runway collision 2.8E-08 per flightRF3 Runway collision 3.8E-08 per flightRP1 Imminent runway collision 4.3E-07 per flightRP2 Runway conflict 4.5E-07 per flightRP3A Aircraft runway incursion 2.0E-05 per flightRP4 Potentially conflicting runway approach 0.33 per flightRP5 Runway approach 1.10 per flight
RB1 Ineffective collision avoidance 0.22 per imminent collisionRB2 Ineffective conflict warning 0.95 per conflictRB4 Ineffective runway entry procedures 5.9E-05 per potentially conflicting approachRB5 Potentially conflicting runway configuration 0.30 per approach
Barrier failure probabilities
Caution: Preliminary results only. Confidence ranges being determined.
11
RB4.1 Operat ional
error in runway ent ry;
29.2%
RB6.1 Operat ional
error in t ake-of f ; 3.4%
RB4.2 Pilot deviat ion
in runway ent ry; 61.0%
RB6.2 Pilot f ailure t o
f ollow t ake-of f
inst ruct ions; 6.4%
Expanded Barrier Failure Causes
RB6.1.1 Take-off
instruction error by
ATCO; 2.0%
RB4.1.1.2.1 ATCO
failure to recognise
runway conflict;
5.9%
RB4.1.1.2.2 ATCO
misjudgement of
runway separation;
5.9%
RB4.1.1.1 Inadequate
aircraft position
information to
ATCO; 4.3%
RB4.1.2 Inadequate
communication with
pilot; 13.2%
RB6.1.2 Inadequate
communication with
pilot; 1.4%RB4.2.1 Pilot failure
to follow taxi route;
19.5%
RB6.2 Pilot failure
to follow take-off
instructions; 6.4%
RB4.2.2 Pilot failure
to follow runway
entry instructions;
41.5%
Runway entry procedures
Barrier Causes of barrier failure
Caution: Preliminary results only. Confidence ranges being determined.
12
IRP: Risk Model Overview
Taxiw
ay collision
Mid-air
collision
Runw
ay
collision
Wake
turbulence
CF
IT
Accident categories
Causal factors(technical failures, human errors)
Influences(safety management, operating environment, etc.)
Risks (frequencies of accidents)
� Fault tree model:
� Widely understood.
� Combination of multiple causes.
� Transparent quantification.
� Top-down approach:� Calibrated against accident &
incidents experience.� 5 main ATM-influenced
accident categories.
� Influence model
� Modifications of base events.
� Same for all accident categories.
13
Influence Model Structure
Task performance
Base event in fault tree
Quality of task inputs
Performance of other tasks
Performance of safety management
Performance of actors
Performance of equipment
PolicyPlanning
AchievementAssurancePromotion
ResourcesCompetence
HMIReliability
ProceduresTeamwork
FunctionalityIndependenceTransparencyRedundancy
MaintainabilityIntegrity
Operating environment
TrafficWeatherTerrain
etc
Quality of airport/airspace
design
14
0.0 0.2 0.4 0.6 0.8
Di r ect AT C causes
RB4.1.1.1.2.2 Inadequate ai r por t AT CO coor di nat i on
RB4.1.1.2.2 AT CO mi s j udgement of r unway separ at i on
A T C pr event i on f ai l ur es
RB4.1.1.1.1 Inef f ect i ve gr ound r adar sur vei l l ance
RB4.1.2 Inadequate communi cat i on wi th pi l ot
RB2.3 Cont r ol l er f ai l ur e to r espond to RIM CAS war ni ng
RB1.1.3 Rest r i cted vi ew f r om tower pr events conf l i c t detect i on
RB1.1.5 AT CO f ai l ur e to r esol ve conf l i ct i n t i me
AT M pr event i on oppor tuni t i es
RB5.2 Runway ent r y at i nter medi ate l ocat i on
RB2.1 RIM CAS not pr esent
RB1.1.2 Dar kness pr events conf l i c t detect i on
CONTRIBUTION
(maximum pot ent ial reduct ion as f ract ion of f at al runway collision f requency)
ATM Contribution to Runway Collisions
Direct causes
Failures to prevent accidents
Opportunities to prevent accidents
Caution: Preliminary results only. Confidence ranges being determined.
15
IRP for Safety Assurance of Future Systems
16
ATM CONOPS
CONOPS assessment…
ATM Direct cause of accidents pfh
Y/N
Step 6
If criteria cannot be met
(or is exceeded),
implementation
assumptions must be
changed.
17
TRANSITION TO SESAR END-STATE
SWIM
DLK
NAVIGATION
18
The Safety Target Achievement Roadmap (STAR)
Up
Risk Prediction for
Individual ATM
Change
OI # 2
OI # 3OI # i
THEN
OI # j
OR
j THEN i
Risk Prediction for
group of ATM
Changes
Risk Predictions for
different order of
implementation
19
Safety Target Achievement Roadmap (STAR): a potential safety monitoring tool
15%
5%
40%
10%
30%
Up
2005 2010 2015 2020
Accidents Frequency
OI # 1
OI # 2
OI # i
THEN
OI # j
OR
j THEN iOI # n
OI # 3
Risk Prediction for
Individual ATM
Change
Risk Prediction for
group of ATM
Changes
Risk Predictions for
different order of
implementation
When
OI # k?Risk Predictions for
different
implementation dates
Risk Predictions
considering
interactions
20
Summary - IRP – What it is, and what it is not
� How the IRP can help you
� Show overall safety target
compliance of a CONOPS
� Identify strategic directions for
safety improvements
� Determine specific safety
requirements
� Support the creation of a safe
implementation roadmap for a
CONOPS (STAR)
� Support safety performance
monitoring
�Where it needs to be
complemented…
�HAZID Method (IRP-ESSI
FAST White Paper)
�Substitute to SAM
(complementary)
�Dynamic Risk Modelling
�Substitute to advanced
Human Reliability
Assessment (HRA)
21
A coordinated & shared vision
22
Development till late 07 (started 2004)
� 2005 Initial IRP finished and being refined
� Uncertainty analysis (confidence ranges using Monte Carlo simulation)
� Sensitivity analysis
� 2012 IRP completed
� 2020 (SESAR) will follow the finalised Concept of Operation
(CONOPS)
� (STAR) Roadmap developed for end 2007
� STAR is sensitive to implementation timescale & sequence
� User Guidance will be developed to make the tools more
‘accessible’
23
Conclusions
� EUROCONTROL and FAA developing parallel toolsets to
� Model the safety of current operations
� Predict overall safety of future ATM system
� Identify safety vulnerabilities
� Identify safety opportunities
� Plan a safe way forward
� Enable monitoring of safety progress as new system
elements are implemented
Thank YouThank You
25
The need for a safe transition…
� In parallel with the SESAR CONOPS and Architecture
� Need for a roadmap for achieving the operational improvements
� I.e. showing how and when different pieces of the SESAR “system”
will fall into place.
� Safety “roadmap”:
� Highlights a transition to the end state and underscores the safety benefits along
the way
� Ensures transition remains within the safety envelope
� Roadmap for achieving operational improvements
� How & When different pieces fall into place – their ‘safety contribution’
� Need for new safety defenses
26
Integrated Risk Picture in Practice
Accidents
Airspace Design
Flow Management
Deconfliction
ATC Tactical
a/c Tactical
ATC Recovery
Pilot Recovery
Providence
Accidents
0.00 0.05 0.10 0.15 0.20 0.25 0.30
Airspace organisation & management
Air traffic flow & capacity management
ATC - traffic synchronisation
ATC - traffic separation
ATC - conflict resolution
ATC systems
Communications
Surveillance
ATM avionics - ACAS
Airport operations
Information management
ATM CONTRIBUTION(maximum potential reduction / total fatal accident frequency)
CFITMid-air collisionRunway collisionTaxiway collisionWake turbulence accident
27
END USERS
IRP Method
Risk R
esultsSa
fety R
equirements
Safety R
ecommendations
CONOPS Designers
System Designers
Safety Managers
28
0.0 0.2 0.4 0.6 0.8 1.0 1.2
AO&M
ATFCM
Traffic synchronisation
Traffic separation
Conflict resolution
ATC systems
Communications
Surveillance
ATM avionics
Airport operations
Flight planning
Information management
CONTRIBUTION (maximum potential reduction as fraction of fatal runway collision frequency)
Direct cause
Prevention failure
Prevention opportunity
Indirect influence
Contributions of ATM Elements to Runway accidents for Commercial Flights in 2005
Caution: Preliminary results only. Confidence ranges being determined.
29
SESAR Definition Phase
Air
Transport
Framework:
The
Current
Situation
Air
Transport
Framework:
The
Current
Situation
ATM
Performance
Targets
ATM
Performance
Targets
ATM
Target
Concept
ATM
Target
Concept
ATM
Deployment
Sequence
ATM
Deployment
Sequence
ATM
Master Plan
of Action
ATM
Master Plan
of Action
Work
Programme
for
2008-2012
Work
Programme
for
2008-2012
07/06 12/06 06/07 11/07 02/08 03/08
D 3 D 4
WP 2.2.2 / 2.2.4: Operational Concept
WP 2.3.1 / 2.3.2: Models & Validation Needs
D1 & D2: Bottlenecks & Performance Targets
30
Prediction and Validation
1990 1992 1994 1996 1998 2000 2002 2004 2006 2008 2010 2012
FA
TA
L A
CC
IDE
NT
FR
EQ
UE
NC
Y (
per
fligh
t)
IRP 2005:
� Top-down prediction from trended accident experience.
� Deduced base event probabilities.
IRP 2012:
� Bottom-up prediction from planned ATM changes.
� Judged changes to base events.
IRP 1990:
� Bottom-up prediction from historical ATM changes.
� Validated against accident experience.
31
EEC Objectives
To validate the SESAR target concept, i.e. providing the evidence, or otherwise, that the SESAR 2020 concept:
� Is specified to be acceptably safe (Holistic Risk modelling, and 1st steps of
SAM/ED78A safety analysis)
� Is operationally viable (through prototyping and R/T simulations)
� Can attain the required level of performances (ECAC wide modelling)
� Is environmentally efficient (Emission and Noise modelling)
Federating European Research actors to run this validation programme
based on the European Commission’s FP6 project – Episode 3).
RTA
AUTHORIZEDRBTA
Current Pos
EXECUTEDRBTX
PLANNEDRBTP
RTA
AUTHORIZEDRBTA
Current PosCurrent Pos
EXECUTEDRBTX
PLANNEDRBTP
32
Scope of Risk Estimates
Risks estimated
Risks not estimated
Fatal accident frequencies
Group risks of fatalities
External risks (people not on board aircraft)
Individual risks
ICAO-defined accident frequencies
Precursor incident
frequencies
Safety net reliability