PROBABILISTIC SAFETY ANALYTICS
FOR UAS INTEGRATED RISK MODELING
James T. Luxhøj, Ph.D.
Industrial and Systems Engineering
Rutgers University
The Mid-Atlantic Symposium on Aerospace,
Unmanned Systems and Rotorcraft
Villanova University
April 10, 2014
Outline
2
UAS System Safety and Hazard Identification
Probabilistic Safety Risk Analytics
• Concepts of the safety risk modeling approach.
Notional UAS Pipeline Inspection Scenario
Concluding Remarks
3
• Decomposes the
UAS domain.
• Identifies the main
sources or clusters
of hazards for UAS.
• HCAS is
comprehensive,
but not necessarily
exhaustive.
Hazard Classification and Analysis System (HCAS)
Components of the Hazard Taxonomy:
Source: Luxhøj and Oztekin, 2009
UAS Hazard Classification and Analysis System (HCAS) – version 4.2
• Aircraft
• Aerodynamics
• Airframe
• Payload
• Propulsion
• Avionics Hardware and Software
• Sensors / Antennas
• Communication Link
• Onboard Emergency Recovery
• Detect, Sense and Avoid
• Other Aircraft Systems
• Control Station
• Classification
• Mobile
• Fixed
• Multiple
• Combinations
• Hardware and Software
• Communications Link
• Data Link Framework
• Infrastructure
• Signals
• Organizational Human Factors
• Aircraft Design Organization
• Control Station Design Organization
• Regulatory Agency • Certification
• Licensing
• Oversight
UA
S
OP
ER
AT
ION
S
• Flight Operations
• Flight Planning
• Phase of Flight
• Emergency Recovery
• Type of Operations • Line of Sight / Beyond Line of Sight
• VFR / IFR
• Operational Control
• Instrument Procedures and Navigational Charts
• Continued Airworthiness
• UAV
• Control Station
• Maintenance Source
• Communication Interface
• ATC Communications
• Radio
• Data Transmission
• Visual
• Airspace
• Established
• Temporary
• Personnel (including Oversight Personnel and ATC)
• Organizational Human Factors
• Operator
• Regulatory Agency
• Certification
• Oversight
EN
VIR
ON
ME
NT
• Terrain
• Electromagnetic Activity • Weather (includes wind)
• Particulates
• FOD
• Wildlife
• Bird Strike
• Animals
• Obstacles
• Others Traffic
• External Influences
• International Regulatory Differences
• Airports (i.e., takeoff/landing areas)
• Navigation Network
• National Security
AIR
ME
N
• Individual Human Factors
• Pilot
• Maintenance Technician
• Service and Support Personnel
• Organizational HF
• Operator • Training
• Supervision
• Regulatory Agency • Certification
• Licensing
• Oversight
• Individual Licensing • Pilot
• Maintenance
• Service and Support Personnel
Operations
Hazards
Environment
Hazards
Airmen
Hazards
UAS
Hazards
Source: adapted from Luxhøj and Oztekin, 2009
Hazards
related to…
4
Analytics: Bayesian Belief Networks (BBNs)
Decision Nodes(i.e., Mitigations)
The approach uses qualitative, probabilistic
reasoning about the interactions of risk
factors (chance nodes) and mitigations
(decision nodes) to make inferences.
Bayes Theorem:
P(X2|X1) = P(X1|X2)P(X2) / P(X1)
Directed Causal Link(i.e., with underlying
Conditional Probability
Table (CPT) – indicates
influence “strength”)
Chance Nodes(i.e., Causal Factors)
X1
D2
D3
X2
X7
X3
X5
X4
X6
D1
UE
5 Source: Luxhøj et al., 2012
Chance Nodes: These are the Random Variables (i.e., the hazard causal factors - could be discrete or continuous). Each node has states (usually binary but could be more than two).
Decision Nodes: These are the Mitigations or Controls. Directed Causal Links: Depict the direction of the causality. Where do the Conditional Probability Tables (CPTs) come from?
- Multiple disparate data sources:
- histograms, reliability models, fault and/or event trees - simulations - Knowledge Elicitation (KE) sessions with subject matter experts (SMEs)
BBN Components
6 Source: https://www.metavr.com/casestudies/insitu_uas.html
7
Analytical ApproachDescribe Case-
Based Scenario
Identify Hazards
(HCAS)Construct Influence
Diagram
Build Belief
Network
Insert Mitigations/
Value Functions
Assess Relative Safety
Risk Reduction
Conditioning
Context
Analytic
Generalization
Causal
Structure
Risk Modeling Steps
M1
M3
M2
V1
V2
V3
Source: Adapted from Luxhøj, 2003 7
Aviation System Risk Model (ASRM)
A Notional Scenario – Pipeline Inspection Monitoring
• Scenario: This UAS flight involves a trans-continental gas pipeline inspection
monitoring. The UAS launches from a remote location airspace and follows a pre-
programmed flight path. The UAS is to fly toward the pipeline, intercept, and then fly
along the pipeline. The UAS is equipped with infrared (IR) sensors and electro-optical
(EO) sensors. The Operator is a UAS Company that selects the UA, flight profile and
operations team.
Develop a causal narrative from scenario by exploring “what ifs”. What if there are local radio frequencies (RF)/power levels that interfere with the
continuous connectivity required of the communication and control links?
What if there is a General Aviation (GA) piloted aircraft in the vicinity of the airport?
What if there is a loss of data link from the Ground Control Station (GCS) to the UAS?
What if there are strong wind gusts (> 40 knots) that contribute to the loss of
separation between the UAS and the manned aircraft?
What if the Automatic Dependent Surveillance-Broadcast or ADS-B Out transmission
from the UAS is disrupted by RF interference? (Note: ADS-B will replace radar.) 8
23
UAS Pipeline Scenario
2.1.1 AIR -GA pilot fails
to see & avoid visually
or with ADSB-IN
UAS/GA
in-flight collision
1.2.3 VEH-UAS
Data link
transmission
disruption
from GCS
4.3 ENV –
Wind gusts
4.2 ENV–
Electromagnetic
activity
4.8 ENV –
Other traffic in
Class E airspace
(near airport)
2.1.1 AIR- GA pilot –
Inexperienced
Aeronautical DM &
struggles to maintain
stability of the aircraft
1.2.1.1 VEH-
UAS pilot fails to
regain control
of UAS due to
signal latency1.1.9 VEH-UAS
While flying in
autonomous mode
back to
recovery point, UAS
veers off course
1.1.7 VEH-UAS
Data link
transmission
disruption
to GCS
3.3 OPS–
ATC Comms./
transmission
disruption
M2: Advanced
EMI testing
M1: NextGen
Enhanced
4D weather
cube wind
predictor
M4: Mixed or Hybrid
UAS control
M6: Virtual Environment (VE)
with predictive graphics displays
M3: GA Sense
and Avoid
Technology
M5: NextGen
Enhanced DSA
Technology
3.2.3 OPS–
Main Source
deficient
3.2.2 OPS–
GCS Main
improper
1.2.2 VEH–
GCS locked
1.1.5 VEH-
ADSB-OUT
on UAS failsM7: GCS/UAS
Link Software
Design Upgrade
2.0 Airmen
3.0 Operations
1.0 Vehicle
4.0 Environment
9
10
HUGIN Model with Conditional Probability Table (CPT)
0.01
10
HUGIN BBN Software Tool
Baseline Scenario Probability = 0.000357 (3.57 x 10-4)
*Consider exposure per 10-4 or 10-5 flight hours so risk/flight hour in the range of 10-8 or 10-9. 11
0.0357
Note:
HUGIN
output is in
percentages
Probability Ang & Buttery (1997) Verbal Descriptor
1
0.9999 extremely likely (i.e. almost certain)
0.9 very likely
0.7 likely
0.5 indeterminate
0.1 probable (i.e. credible)
0.01 unlikely
0.001 very unlikely
0.0001 extremely unlikely
0
Probability Elicitation: Degree of Belief (DoB) Approach
12
“The purpose of computing is insight, not numbers.” - Richard Wesley Hamming
Hazard Clusters
0.0000
100.0000
200.0000
300.0000
400.0000
500.0000
600.0000
Airmen Vehicle Operations Environment
Likelihood Multiplier
Airmen
Vehicle
Operations
Environment
560.7
299.5
195.6
14.0
Baseline Scenario Undesired Event (UE) Probability = 0.000357 (3.57E-4)
13
Specific Causal Factors
0.0000
100.0000
200.0000
300.0000
400.0000
500.0000
600.0000
Likelihood Multiplier
Baseline Scenario Undesired Event (UE) Probability = 0.000357 (3.57E-4)
14
Instance nodes
Sub-net S2
Sub-net S1
Output node
Output node
OOBN Modeling
Approach – System
of Systems (SoS)
Top-Level Model
UE
Mishap
Key Properties:
-Abstraction
- Inheritance
-Encapsulation
Object-Oriented Bayesian Networks (OOBNs)
15
23
UAS Pipeline Scenario
2.1.1 AIR -GA pilot fails
to see & avoid visually
or with ADSB-IN
UAS/GA
in-flight collision
1.2.3 VEH-UAS
Data link
transmission
disruption
from GCS
4.3 ENV –
Wind gusts
4.2 ENV–
Electromagnetic
activity
4.8 ENV –
Other traffic in
Class E airspace
(near airport)
2.1.1 AIR- GA pilot –
Inexperienced
Aeronautical DM &
struggles to maintain
stability of the aircraft
1.2.1.1 VEH-
UAS pilot fails to
regain control
of UAS due to
signal latency1.1.9 VEH-UAS
While flying in
autonomous mode
back to
recovery point, UAS
veers off course
1.1.7 VEH-UAS
Data link
transmission
disruption
to GCS
3.3 OPS–
ATC Comms./
transmission
disruption
M2: Advanced
EMI testing
M1: NextGen
Enhanced
4D weather
cube wind
predictor
M4: Mixed or Hybrid
UAS control
M6: Virtual Environment (VE)
with predictive graphics displays
M3: GA Sense
and Avoid
Technology
M5: NextGen
Enhanced DSA
Technology
3.2.3 OPS–
Main Source
deficient
3.2.2 OPS–
GCS Main
improper
1.2.2 VEH–
GCS locked
1.1.5 VEH-
ADSB-OUT
on UAS failsM7: GCS/UAS
Link Software
Design Upgrade
2.0 Airmen
3.0 Operations
1.0 Vehicle
4.0 Environment 16
Sub-net
Sub-net
17
Concluding Remarks
Just as UAS technology is advancing, the analytical methods for probabilistic safety risk modeling need to similarly advance.
BBNs facilitate the modeling and uncertainty investigation of the complex interactions of the UAS, Airmen, Operations and the Environment for an integrated safety risk assessment.
OOBNs offer the potential of modular network development with reusable and portable sub-nets.
The modeling approach can assist in “vulnerability discovery” (i.e., recognize new risks and system-level precursors) where mitigations may not yet exist.
