A New Approach for Designing SaferCollision Avoidance Systems
Mykel J. KochenderferJames P. Chryssanthacopoulos
Roland E. Weibel
14 June 2011
This work is sponsored by the Federal Aviation Administration under Air Force Contract #FA8721-05-C-0002.Opinions, interpretations, conclusions, and recommendations are those of the authors and are not necessarilyendorsed by the United States Government.
ATM Seminar - 1MJK - 14Jun11
Aircraft Collision Avoidance
Traffic Alert and Collision Avoidance System (TCAS)
Green arc indicatesrequired vertical rate
• TCAS uses range, altitude, andbearing measurements of local traffic
• Traffic alerts are issued to pilots toassist in visual acquisition
• Resolution advisories (RAs) advisepilots to climb or descend
• Required decades of development
ATM Seminar - 2MJK - 14Jun11
Problem Statement
• NextGen procedures will require changes to TCAS logic– Improvements to surveillance will likely be insufficient– Revising the TCAS logic is difficult and may be inadequate– A different approach to logic development may be required
• Challenges to collision avoidance logic development– Uncertainties due to sensor noise and aircraft behavior– Aircraft performance and operational constraints– Need to maximize safety while minimizing alerts
• Solution is a decision-theoretic approach– Uses explicit models of sensor and dynamic uncertainty– Optimizes logic according to performance measure– Leverages advances in computation and algorithms
ATM Seminar - 3MJK - 14Jun11
Approaches to Collision Avoidance
• Nominal (green) used by TCAS– Does not consider hazardous low-probability events
• Worst case (red)– Potential for high rate of unnecessary course deviation
• Probabilistic (blue)– Improved robustness due to accounting for relative
likelihood of all possible future trajectories
ATM Seminar - 4MJK - 14Jun11
Approaches to Logic Development
Legacy TCAS Development Cycle
EncounterModel
PerformanceMetrics
LogicPseudocode
Simulation Evaluation
manual pseudocode revision
Logic Optimization Approach
EncounterModel
Optimization LogicTable
PerformanceMetrics
ATM Seminar - 5MJK - 14Jun11
Timeline
• FY08: Initial concept for UAS (Report ATC-356)– Internal seed funding– Applied approach to sensor systems and platforms
• FY09: Initial exploration for TCAS (Report ATC-360)– Funding from TCAS Program Office– Experimented with different methodologies on 2D model
• FY10: Development and analysis (Report ATC-371)– Developed 3D model, incorporated sensor noise– Focus primarily on single, unequipped intruder
• FY11: Further enhancement (ongoing)– Enhanced multithreat and coordination– Compare to alternative approaches
ATM Seminar - 6MJK - 14Jun11
Logic Development and Usage
Logic Development
EncounterModel
PerformanceMetrics
ModelDiscretization
DiscreteModel
Optimization
LogicTable
Logic Usage
LogicTable
SensorData
StateEstimation
StateDistribution
Advisory Selection
Advisory
ATM Seminar - 7MJK - 14Jun11
Simple 2D Encounter Model
own aircraftintruder aircraft
Miss distance atclosest point of approach
Near mid-air collision (NMAC) when miss < 100 ft
Model Assumptions
• Head-on encounter
• No turning
• Constant closure rate
• Random verticalaccelerations
• Advisory may be changed
State Variables
• Relative altitude
• Time to closest approach (τ )
• Own vertical rate
• Intruder vertical rate
• Advisory state (for 5 secondpilot delay)
ATM Seminar - 8MJK - 14Jun11
Cost Metric
• NMAC: 1– Accrued when an NMAC occurs
• Alert: 0.01– Accrued when an initial advisory is issued
• Strengthening: 0.009– Accrued when an advisory is strengthened
• Reversal: 0.01– Accrued when an advisory is reversed
• Clear of Conflict: −0.0001– Accrued every time step when no advisory is on display
These costs were chosen arbitrarily but can be adjusted toaccommodate safety or operational requirements
ATM Seminar - 9MJK - 14Jun11
Logic Development and Usage
Logic Development
EncounterModel
PerformanceMetrics
ModelDiscretization
DiscreteModel
Optimization
LogicTable
Logic Usage
LogicTable
SensorData
StateEstimation
StateDistribution
Advisory Selection
Advisory
ATM Seminar - 10MJK - 14Jun11
Model Discretization
• Discretize state spaceusing a (5-D) grid
• Vertices correspond todiscrete states(8.7 million states)
• Grid coarseness affectsthe accuracy of the model
• Use encounter model todetermine state transitionprobabilities
State Action Next State
0.8
0.1
0.1
0.2
0.7
0.1
0.5
0.2
0.3
No Alert
Climb
Descend
ATM Seminar - 11MJK - 14Jun11
Dynamic Programming
• The expected cost table contains the expected cost foreach action (and then continuing with the optimal logic)from each state
• Dynamic programming efficiently computes this table
• Table used in real time to choose actions
Notional expected cost table
State Expected cost
Rel.alt.
Time toclosest
Ownvert.
Int.vert.
RAstate
No alert Descend Climb
100 19 1500 −1000 No alert 0.0144 0.4215 0.0190200 20 0 0 No alert 0.0449 0.0339 0.4251. . . . . . . . . . . . . . . . . . . . . . . .
ATM Seminar - 12MJK - 14Jun11
Logic Development and Usage
Logic Development
EncounterModel
PerformanceMetrics
ModelDiscretization
DiscreteModel
Optimization
LogicTable
Logic Usage
LogicTable
SensorData
StateEstimation
StateDistribution
Advisory Selection
Advisory
ATM Seminar - 13MJK - 14Jun11
Optimal Action Plot
Descend
Climb
No advisory
0 5 10 15 20 25 30 35 40−1000
−500
0
500
1000
τ (s)
Rel
ativ
eal
titu
de
of
intr
ud
er(f
t)Both aircraft initially level, no advisory issued yet
ATM Seminar - 14MJK - 14Jun11
Optimal Action Plot
Climb
Descend
No advisory
0 5 10 15 20 25 30 35 40−1000
−500
0
500
1000
τ (s)
Rel
ativ
eal
titu
de
of
intr
ud
er(f
t)Own climbing 1500 ft/min, intruder level, no advisory issued yet
ATM Seminar - 15MJK - 14Jun11
Optimal Action Plot
Strengthen
Reverse
Continue descent
Discontinue advisory
0 5 10 15 20 25 30 35 40−1000
−500
0
500
1000
τ (s)
Rel
ativ
eal
titu
de
of
intr
ud
er(f
t)Own climbing 1500 ft/min, intruder level, descend in 3 seconds
ATM Seminar - 16MJK - 14Jun11
Results
Metric DP Logic TCAS Logic
NMACs 3 169Alerts 690,406 994,317Strengthenings 92,946 40,470Reversals 9569 197,315
Performance on metrics can be traded by adjusting costs
ATM Seminar - 17MJK - 14Jun11
Safety curve
0 0.2 0.4 0.6 0.8 1
0.9
0.95
1
Pr(alert)
Pr(
safe)
DP logicTCAS logic
• DP curve generated by varying alert cost
• TCAS curve generated by varying sensitivity level
ATM Seminar - 18MJK - 14Jun11
Example Encounter
Clim
b
Des
cen
dD
esce
nd
Incr
ease
des
cen
t
−40 −30 −20 −10 0 10 20 30 40−600
−400
−200
0
200
400
Time (s)
Alt
itu
de
(ft)
DP logicTCAS logicNo logicIntruder
ATM Seminar - 19MJK - 14Jun11
NMAC and Alert Slices
0 10 20 30 40−1000
−500
0
500
1000
τ (s)
Rel
ativ
eal
titu
de
(ft)
Probability of NMAC
0 10 20 30 40
τ (s)
Probability of Alert
0
0.2
0.4
0.6
0.8
1
• Both aircraft level and no advisory issued
• Computed for full state space using DP algorithm
ATM Seminar - 20MJK - 14Jun11
Robustness to Modeling Errors
• Model used foroptimization will notperfectly match real world
• Example plots varyenvironment model, butkeeping the DP logic fixed(at 8 ft/s2)
• DP outperforms TCASeven with modeling error
0
2
4
6
×10−3
Pr(
NM
AC
)
0 5 10 150.4
0.6
0.8
1
Environment noise (ft/s2)
Pr(
Ale
rt)
DP
TCAS
ATM Seminar - 21MJK - 14Jun11
Collision Avoidance in 3-D
• When aircraft are maneuvering horizontally,τ cannot be known exactly
• Use probabilistic model to infer distribution over τ andweight cost appropriately
• Explored different methods for estimating τ
0 10 20 30 ≥ 400
0.2
0.4
0.6
0.8
1
τ (s)
Pro
bab
ility
DP
Monte Carlo
Straight Line
ATM Seminar - 22MJK - 14Jun11
Results
DP Logic
Metric DP Monte Carlo Straight Line TCAS Logic
NMACs 2 11 1 101Alerts 540,113 400,457 939,745 994,640Strengthenings 39,549 37,975 26,485 45,969Reversals 1242 747 129 193,582
DP logic significantly outperforms TCAS
ATM Seminar - 23MJK - 14Jun11
Safety Curve
0 0.2 0.4 0.6 0.8 1
0.96
0.98
1
Pr(Alert)
Pr(
Saf
e)
Monte CarloDPStraight LineTCAS
Using a distribution over τ is better than a point estimate
ATM Seminar - 24MJK - 14Jun11
Probabilistic Pilot Response
• Pilots do not respond according to the standarddeterministic response model (5 second delay, 1/4 gmaneuver)
• Modifying model to capture pilot variability leads tosignificantly improved performance
COCCOC
CL1500COC
CL1500CL1500
SDES1500COC
SDES1500SDES1500
COCCOC
5/6
1/6
3/4
1/4
3/4
1/4
1 /6
1
1
1 /4
Issue CL1500 Issue SDES1500 Issue COC
Ongoing research on more sophisticated models
ATM Seminar - 25MJK - 14Jun11
Sensor Noise
• TCAS (and other systems) usemost likely state (MLS) estimate
• Investigated use of full statedistribution in new logic
• Involves averaging expected cost
Mostlikelystate
State distribution
0 5 10 15 200
1 · 10−4
2 · 10−4
3 · 10−4
4 · 10−4
σχ (deg)
Pr(
NM
AC
)
0 5 10 15 200
0.2
0.4
0.6
σχ (deg)
Pr(
Ale
rt)
DP-Full
DP-MLS
TCAS
ATM Seminar - 26MJK - 14Jun11
Coordination
Coordination is critical when both aircraft are equipped
• Different views of the world can lead to incompatiblemaneuvers (e.g., climb/climb)
• Aircraft communicate the intended sense of the maneuver
• Simply restricting action set to compatible advisoriessignificantly improves safety
• Further work will investigate:– Incorporating broadcasts into model– Interoperability with legacy TCAS
ATM Seminar - 27MJK - 14Jun11
Multiple Threat Logic
• One of the most complex pieces of TCAS logic• Currently exploring different methods for extending
approach to handle multiple threats:– Command arbitration: choose one of the actions
associated with an intruder– Utility fusion: combine expected costs– Global strategy: find path that best avoids all intruders
Command Arbitration Utility Fusion
Closest TCAS-like Max-sum Max-min Global TCAS
Pr(NMAC) 9.662 · 10−3 4.810 · 10−3 2.206 · 10−3 1.676 · 10−3 2.714 · 10−3 2.960 · 10−3
Pr(Alert) 0.640 0.682 0.604 0.682 0.581 0.764Pr(Str.) 0.137 8.467 · 10−2 6.991 · 10−2 8.925 · 10−2 0.483 4.430 · 10−2
Pr(Rev.) 3.644 · 10−3 5.290 · 10−3 6.632 · 10−3 9.538 · 10−3 6.996 · 10−3 6.550 · 10−3
ATM Seminar - 28MJK - 14Jun11
Multiple Threat Stress Test
2 3 4 5 6 7 8 9
10−5
10−3
10−1 No system
Pr(
NM
AC
)
2 3 4 5 6 7 8 9
0.6
0.7
0.8
0.9
1
Pr(
Ale
rt)
2 3 4 5 6 7 8 90
0.2
0.4
0.6
Number of intruders
Pr(
Str
eng
then
ing
)
2 3 4 5 6 7 8 90
0.1
0.2
Number of intruders
Pr(
Rev
ersa
l)
Closest TCAS-like Max-sum Max-min TCAS
ATM Seminar - 29MJK - 14Jun11
Alternative Approaches
Path planning
• Find a conflict-free path,delay alert if possible
• Uses deterministic models
• Analytic solutions
• Insensitive todimensionality of model
• Does not account forfuture state information
Probability thresholding
• Computes probability ofconflict for differentactions
• Alert if probability exceedssome threshold
• Uses probabilistic models
• Does not account forchanging action in future
Performance can be poor with increased trajectory uncertainty
ATM Seminar - 30MJK - 14Jun11
Publications• M. J. Kochenderfer and J. P. Chryssanthacopoulos, “Robust airborne collision avoidance through dynamic
programming,” Massachusetts Institute of Technology, Lincoln Laboratory, Project Report ATC-371, 2010.• M. J. Kochenderfer, J. P. Chryssanthacopoulos, L. P. Kaelbling, T. Lozano-Perez, and J. K. Kuchar,
“Model-based optimization of airborne collision avoidance logic,” Massachusetts Institute of Technology,Lincoln Laboratory, Project Report ATC-360, 2010.
• J. P. Chryssanthacopoulos and M. J. Kochenderfer, “Decomposition methods for optimized collisionavoidance with multiple threats,” in IEEE/AIAA Digital Avionics Systems Conference, Seattle, Washington,2011.
• M. J. Kochenderfer, J. P. Chryssanthacopoulos, and R. E. Weibel, “A new approach for developing safercollision avoidance systems,” in USA/Europe Air Traffic Management Research and Development Seminar,Berlin, Germany, 2011.
• J. P. Chryssanthacopoulos and M. J. Kochenderfer, “Hazard alerting based on probabilistic models,” inAIAA Guidance, Navigation, and Control Conference, Portland, Oregon, 2011.
• J. P. Chryssanthacopoulos and M. J. Kochenderfer, “Analysis of open-loop and closed-loop planning foraircraft collision avoidance,” in IEEE International Conference on Intelligent Transportation Systems,Washington, DC, 2011 (under review).
• J. P. Chryssanthacopoulos and M. J. Kochenderfer, “Accounting for state uncertainty in collisionavoidance,” in Journal of Guidance, Control, and Dynamics, 2011 (in press).
• J. P. Chryssanthacopoulos and M. J. Kochenderfer, “Collision avoidance system optimization withprobabilistic pilot response models,” in American Control Conference, San Francisco, Calif., 2011.
• M. J. Kochenderfer and J. P. Chryssanthacopoulos, “Partially-controlled Markov decision processes forcollision avoidance systems,” in International Conference on Agents and Artificial Intelligence, Rome,Italy, 2011.
• M. J. Kochenderfer and J. P. Chryssanthacopoulos, “A decision-theoretic approach to developing robustcollision avoidance logic,” in IEEE International Conference on Intelligent Transportation Systems,Madeira Island, Portugal, pp. 1837–1842, 2010.
• M. J. Kochenderfer, J. P. Chryssanthacopoulos, and P. Radecki, “Robustness of optimized collisionavoidance logic to modeling errors,” in IEEE/AIAA Digital Avionics Systems Conference, Salt Lake City,Utah, 2010.
ATM Seminar - 31MJK - 14Jun11
Summary and Next Steps
• Automated optimization based on probabilistic models– Focuses human engineering effort on developing models
and metrics– Improves safety while reducing alert rate– Is sensor agnostic
• Logic specification in terms of expected cost table– Reduces implementation burden of manufacturers– Eases verification process– Simplifies the update process in response to evolution of
airspace
• Next steps:– Development: coordination and interoperability issues– Analysis: safety and operational acceptability– Certification: develop plan, engage safety community
ATM Seminar - 32MJK - 14Jun11
