1
Project on Unmanned Aircraft in the NASFinal Review Panel Meeting
2
Integration of Unmanned Aircraft into the National
Airspace System
A Project Course byCarnegie Mellon University
Dept. of Engineering and Public Policy
Dept. of Social and Decision Sciences
May 1, 2007
3
Expert Review Panel
Tom Curtin, AUVSI Bret Davis, AUVSI Lexa Garrett, America West Airlines Jim Geibel, GAO David Gerlach, FAA Tom Henricks, Aviation Week Ramon Lopez, Aurora Flight Sciences Edmond Menoche, GAO Rene Rey, FAA Melissa Rudinger, Aircraft Owners & Pilots
Assn. James Sizemore, FAA Larry Thomas, GAO Dyke Weatherington, DoD/OSD
4
Purpose of CMU Project Courses in Technology and Policy
Analyze a “real world” policy problem involving technology
Combine diverse information and analytic frameworks to derive policy insights
Learning objectives: Problem decomposition, structuring
and formulation Interdisciplinary problem solving Communication Teamwork
5
Examples of past project courses
Title Year
Safety and the Light Truck Craze: Who Wins? Who Loses? Who Cares?
2000
Environmental Impacts of E-commerce - A case study of book purchasing
2000
Sustaining Pittsburgh's Vital Services When the Power Goes Out
2004
Wireless Communications Systems for Emergency Responders
2004
Hybrids and Diesels in the American Automobile Fleet
2005
The Impact of Spyware 2005
Safety of Implanted Cardiac Devices 2007
6
Contributors to our UAS project
20 undergraduates majoring in: Engineering Social Science Business Administration
3 Ph.D. student managers 3 faculty advisors Expert review panel Other experts
7
Background for this Project
Increasing demand for UA Military (many current uses) Civilian (many potential uses)
Federal Aviation Administration (FAA) is developing a roadmap for integrating UA into the NAS
A few of the issues to be addressed: Safety and reliability Public acceptability Market viability
8
Analysis Areas Economics
How cost-effective are UA compared to alternative means of providing specific services?
Risk, Technology and Standards What are the regulatory implications of
different approaches to “equivalent level of safety?”
Public Awareness and Perceptions Are risks of UA of greater public concern than
risks of manned aircraft? Governance
How can the current system for deliberation and decision-making on UA access be improved?
9
Project Outcomes
~16 person-months of research completed across the four focus areas
Economic model of market viability Risk model of fatality implications of
UA introduction Better understanding of public
awareness & risk perception Actor & “roadblock” analysis yields
insight on deliberative process for UA integration
Regulatory & policy recommendations
1010
Economics
Team Members:Nathan Diorio-TothFeng DengReiko BaughamVictoria MortonBrad Brown
Team Manager:Ryan Kurlinski
1111
Purpose
Assess the market viability of UAS applications using relative cost effectiveness
Assess the effect of various regulatory measures on the market viability of UAS applications
12
Goals
12
Develop UAS cost model Cost components
Airframe Communications Insurance Pilot Etc.
Apply cost model to chosen applications and alternatives to compare cost
Examine sensitivity of overall cost to changes in each cost component
Estimate cost implications of different regulatory measures and technology improvements
13
UAS Applications
Weather Reconnaissance Alternative:
WC-130J Hercules: high-wing, medium range aircraft
Pipeline monitoring Alternative:
Concentric sensors: pressure sensitive sensors
Localized Surveillance Alternative:
Traffic Helicopter: e.g. Bell JetRanger
13
14
Analysis Method
Used triangular distributions to assign probable ranges to each input cost From this, generated a Probability
Density Function Probability Density Function shows
the entire range of possible costs with the associated likelihood of each cost
Allows analysis of the most probable cost advantages
14
15
Importance Analysis
Triangular probability
distributions of all input variables
Economic Model
Contribution of uncertainty in each input to uncertainty in total cost
16
Weather Reconnaissance
Analyzed the use of Aersonde UAS for Weather Reconnaissance vs. the use of the WC-130J Hercules
Aerosonde UAS currently in use for Weather Reconnaissance
Hercules WC-130J currently in use by Keesler Air Force Base
16
17
Pro
bab
ility
De
nsi
ty
Alt Cost-UAS Cost0 100K 200K 300K 400K 500K 600K 700K
0
2u
4u
6u
8u
10u
12u
Results: Weather Reconnaissance
17
($/flight hour)
Probability Density of UAS Cost Advantage
18
Imp
ort
an
ce i
n A
lt C
ost
-UA
S C
ost
Alt Cost-UAS Cost Inputs
Component costsOperational lifetime
ManpowerCom-Link Cost
Cost per gallonInsurance Rate
Hours per yearGallons per Hour
Safety Technology Cost0
0.2
0.4
0.6
0.8
1
Results: Weather Reconnaissance
18
Importance Analysis of Model Inputs
Mission Hours per Year
Com-Link CostOperational Lifetime
19
Results: Weather Reconnaissance
Key Results: UAS more cost effective than current
manned alternative Most important inputs in determining
overall cost effectiveness: Mission hours per year Com link cost Operational lifetime
Currently available sense-and-avoid equipment cause significant decrease in cost effectiveness, but does not cause the UAS to be more expensive than the manned alternative
19
20
Pipeline Monitoring
Analyzed the use of the Aero Environment AeroPuma vs. the use of concentric wire sensors ($6+/m)
Note the difference in monitoring style UAS monitors using thermal imaging with
each pass and relays pertinent leak info to docking stations
Concentric sensors constantly monitor pipeline and relay information
21
Results: Pipeline Monitoring
Key Results: UAS cheaper depending on number in
use Important to note difference in
monitoring styles between UAS and concentric sensor
Important inputs: Relay/Docking station cost Number of UASs in use
21
22
Localized Surveillance
Application based on the surveillance of a 1km2 area for a short time (~1-3 hours)
Considered the use of a Cyber Defense Systems CyberBUG vs. the use of a traffic helicopter
For model inputs, considered monitoring a large traffic accident over 2 hours
For policy considerations, analyzed the addition of mandated sense-and-avoid hardware to the UAS
22
23
Results: Localized Surveillance
23
PDF of Cost per Mission for UAS Compared with Manned Alternative
Cost per Mission ($)
Probability Density
2000 4000 6000 8000 10K8000 12K 14K 16K
Note: no meaningful overlap
2424
PDF of Cost per Mission for UAS Compared with Manned Alternative with High-Range
Fixed Cost Variance
Cost per Mission ($)
Probability Density
2500 5000 7500 17.5K 20K 22.5K
Note: still no meaningful overlap
Results: Localized Surveillance
10K 12.5K 15K
25
Results: Localized Surveillance
25
PDF of Cost per Mission for a Larger UAS Capable of Carrying Sense-and-Avoid Equipment Compared with the Cost of
Manned Alternative
Probability Density
0 10K
Cost per Mission ($)
20K 30K 40K 50K 60K 70K
Note: Significant overlap indicating that UAS would likely no longer be a viable alternative to manned craft
26
Results: Localized Surveillance
26
Importance of inputs.
Input CostsInput Importance for Cost Per Mission
Missions per Year
Mission Related Costs
Flight Hours Per Mission
27
Results: Localized Surveillance
Key Results: UAS less expensive in almost every case Levelized cost for manned more
sensitive than to utilization hours & discount rate than cost for unmanned
UAS cost effectiveness reduced significantly by requirement for sense-and-avoid hardware
Important inputs: Missions per year Discount rate Flight hours per mission
27
28
Policy Implications
Analyzed the effect of the following policies: Mandated insurance premiums Mandated use of A/N hardware
(Increased fixed cost)
Mandated record-keeping practices (Increased yearly cost)
Mandated airframe materials (Increased fixed cost)
Mandated minimum amount of safety equipment (Increased fixed cost)
Mandated pilot/operator training
28
29
Policy Implications: Results
All policies except mandated sense-and avoid hardware had little effect on the cost advantage of UAS over manned alternative
Required sense-and-avoid hardware greatly affects cost-effectiveness, however Localized Surveillance and Pipeline
Monitoring would no longer be viable as larger, much more expensive UAS would be necessary
29
3030
Risk, Technologies, & Standards
Team Members:Samiah AkhtarJonathan CornellNicole HaywardWill KimNick MisekDoug Robl
Team Manager:Keith Florig
31
Purpose Derive a risk model to explore how risk is related to UAS numbers, dimensions, and flight zones
Research on elements of risk mitigation such as human factors, sense and avoid
Exploration of alternative incident reporting systems
Predator
Source: http://www.fs.fed.us/psw/news/PSW_News/2005_09/ima
ges/uav.gif
32
Technology and Risk Outline
Goals Risk Modeling
Purpose Assumptions & Approach Findings
UAS Risk Mitigation
33
Risk Modeling Purpose
Provide a way of modeling that creates some groundwork for future modeling
Use model to compare relative risk calculations
Pointer to the future, not the answer Points of interest
Mid-air vs. single-craft crash Effect of sense and avoid technology UAS to displace manned aircraft
Source: http://www.maximog.com/images/sublevel/uav_left.jpg
34
Risk Modeling Assumptions Uniform national model Uniform traffic density Uniform ground population density Uniform aircraft per type Appropriate for:
VFR traffic Rural, less populated areas
NOT Appropriate for: Urban settings Airports High traffic densities
35
Risk Model
36
Risk Modeling Approach
UAV Picture Source: http://www.evworld.com/press/spider_lion_uav.jpg
Number of midair collisions:
N = total number of aircraft in defined airspaceρ = aircraft traffic densityD = diameter of plane (wingspan)S = average aircraft speedP(A) = probability of avoidance
(Used for calibration)
yearSDAPN sec/1031)](1[ 62
VFR operations only
37
Risk Modeling: UAs displacing Manned
Small risk from unmanned at lower extrema
Expected Annual Fatalities vs. % of Airspace Unmanned
0
2
4
6
8
10
12
14
0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%
% of Total Aircraft is Unmanned
Ex
pe
cte
d A
nn
ua
l M
ort
ali
ty
Manned Mid-AirsManned Single-Craft
Unmanned Mid-Airs P(Avoid)=0Unmanned Mid-Airs P(Avoid)=0.5
Unmanned Single-Craft 100 failures/100k flight hrsUnmanned SIngle-Craft 10 failures/100k flight hrs
Unmanned Single-Craft 500 failures/100k flight hrs
Risk from unmanned at low levels less than
decreased risk from manned
Single-craft crashes still present less risk than mid-
airs
38
Expected Annual Mortality vs. % UASs Added to NAS
0
5
10
15
20
25
30
35
40
0% 20% 40% 60% 80% 100% 120% 140% 160% 180% 200%
% UASs Added
Exp
ecte
d A
nn
ual M
ort
ality
Single-Craft 50 failures/100k flight hrsSingle-Craft 200 failures/100k flight hrsMid-Airs P(Avoid)=0Mid-Airs P(Avoid)=0.25Mid-Airs P(Avoid)=0.5Mid-Airs P(Avoid)=0.95Single-Craft From Substituting MannedMid-air from Substituting Manned
Risk Modeling: Mid-Air vs. Single-Craft
At some point, manned
risk surpasses
unmanned risk
Single-Craft generally less risk than mid-
air
At low numbers, sense and avoid has little effect
39
Risk Modeling Conclusions
Mid-air collisions generally have more risk than single-craft crashes
Displacing small to moderate amounts of manned craft represents decrease in risk
Smaller, less reliable UAs can present less risk than larger more reliable manned aircraft
For small numbers of UAs in low traffic densities, sense and avoid has small effect
40
Technology and Risk Outline
Goals Risk Modeling UAS Risk Mitigation
Human Factors Sense and Avoid
41
Human Factors Implications
Risks - Caused Most Number of Accidents “Sensory Isolation” [McCarley et al]
UAS operator does not receive same sensory cues as manned aircraft operator
Automation Malfunction of automated components
controlled by the UAS operator Operator Hand-Off
Issues with handing off control of vehicle from one operator or crew to another
42
Human Factors Implications
Recommendations Training and Procedures
Up to date training as new technology advancements arise
Ensure that operator has accurate knowledge of automated components within UAS
Multimodal displays Prevent sensory isolation Allow for audio, visual and speech
control Example: simulated cockpit
43
Detect, Sense and Avoid
Risks Market impact of single fatal
collision Lack of standardization among DSA
systemsSense and Avoid Technology
Cost* Size* Weight* Power Usage*
Pros Cons
IFF Transponder $500-3000 1.5m2 6 kg 30 watts, nominal
Low-cost, Transponder requirement
ADS-B Transponder
$2000-6000 2.0m2 8 kg 50 watts, nominal
Compatible with NGATS
High power, transponder
EO/IR Sensors $50,000 - 200,000
4.0m2 5-10 kg 10-20 watts
Detection in IFR and VFR, no metal requirement
Short detection range, high cost
Synthetic Aperture Radar
$10,000 - 50,000
3.0m2 20 kg 80 W All-weather Weight, high power
44
Detect, Sense and Avoid
Recommendations Create regulations specific to
size, weight, application etc Testing Periods Phased Integration
45
Technology and Risk Outline
Issues Goals Risk Modeling UAS Risk Mitigation
Reporting systems
46
Current Reporting Systems
Two Options NTSB Reporting (as required by FAR) -
Accident NASA ASRS Voluntary Reporting - Incident
Current Implementation NTSB mandates detailed information when:
Flight control system malfunction, Illness of crewmember, Turbine Engine Failure, In-flight fire, Mid-air collision or Damage in excess of $25,000 to other property
ASRS System is anonymous and does not have any reporting requirements
47
Reporting Recommendations Initially mandate reporting of all accidents
and incidents Re-evaluate strategy after testing period
NTSB
FAAUAS manufacturers
- NTSB provides useful information on UAS failures
- UAS responds with improved design and engineering
- NTSB information helps FAA to assess standards
- FAA responds with rules for reporting incidents.
Communication
Triangle
4848
Public Awareness & Perceptions
Team Members:Darian GhorbiJenny KimMark PetersonLaura SeitzPatrick Snyder
Team Manager:Pete Tengtrakul
49
Statement of Purpose
Add the element of public perception to the discussions of UAS in the NAS
Motivation: the fact that there has never been a formal presentation of public perception on the topic
Findings: useful for the creation of regulations and policy implications
50
Objectives
Compare public perceptions of the risks concerning manned and unmanned aircraft
Find demographic groups with certain risk and benefit patterns of UAs
Research implications of opinion of UAs
Create survey to aid in completing objectives
51
Hypotheses Perceived Risk
- Manned < Remotely Piloted < Autonomous Ground vs. Air
- More risk of UAS perceived in air Prior Knowledge vs. Risk Perception
- Prior knowledge, associate less risk Benefit vs. Risk Perception
- Higher benefit, lower risk Education vs. Risk Perception
- Technical education, associate less risk Age vs. Risk Perception
- Older participants more cautious Frequency of Flight
- Those that fly frequently, associate less risk
52
Layout of Survey First Page
Provide information about UAS Autonomous Remotely Piloted
Gauge previous knowledge Source
Last Page Demographics
Gender Age Education Frequency of Flight Voting (identify opinions of those that are
politically engaged) Pilot
53
Layout of Survey
Picture of UAS application
Application
•Traffic Monitoring
•Pipeline Monitoring
•Disaster Relief
•Border Patrol
Description of UAS
•Physical Information
•Current application
Questions
• Quick Response
• Benefit
• Stakeholder
• Public
• Risk
• Ground
• Air
• 7 Point Scale
• 1 - Much Less
• 4 - Same
• 7 - Much More
54
Obtaining Surveys
Method # of Surveys
Mon Valley NGO
79 (56%)
Word of Mouth
62 (44%)
Total 141
Coding: numerical code assignedScreening: data obtained from those under 16 years of age were not counted
55
Statistical Methods
Paired T-tests Across applications
ANOVA Significance of mean
Regression Correlations
Demonstrated the strength of the variables (risk and benefit)
56
Results:Descriptive Statistics
Demographic Variable Mean Scale Previous Knowledge 0.53 1= heard, 0 =never heard
Gender 0.44 1=Male, 0=Female Age 36.04 Years
Education 2.33 1=High School,4= Graduate Register to Vote 0.66 1=Registered,0=not
Frequency of Flight 2.51 1=never, 5=more than 12x Licensed Pilot 0,01 1=Licensed,0=not
Variable Remotely Piloted Autonomous Support .75
(0.45) .58
(0.50) Benefit to Stakeholders 4.65
(1.57) 4.29
(1.77) Benefit to Society 4.14
(1.58) 4.01
(1.66) Risk to people on Ground 4.36
(1.51) 4.81
(1.66) Risk to people in the Air 4.55
(1.39) 4.87
(1.68)
57
Perceived Relative Risks Between Remotely Operated vs. Autonomous
Autonomous applications are viewed to have more risk in comparison to remotely operated UAs.
DisasterBorderTrafficPipeline
0
1
2
3
4
5
6
7
UA Applications
Perc
eiv
ed
Ris
k (
1=
Lo
w,
7=
Hig
h)
Remotely operated UA
Autonomous UA
58
Relative Risks Across Applications
Pipeline TrafficBorder Disaster
0
1
2
3
4
5
6
7
UA Applications
Pe
rce
ive
d R
isk
(1
=L
ow
, 7
=H
igh
)
Remotely operated UA
Autonomous UA
Traffic Monitoring has the highest perceived risk.
59
Relative Benefit Across Applications
Pipeline*Traffic
Border Disaster
0
1
2
3
4
5
6
7
UA Applications
Pe
rce
ive
d B
en
efi
t (1
=L
ow
, 7
=H
igh
)
Remotely operated UA
Autonomous UA
The more risky the public perceived the application, the less benefit they associated with the application.
60
Relative Perceived Risk to People on Ground vs. Air
Disaster*Border*Traffic*Pipeline*
0
1
2
3
4
5
6
7
UA Applications
Perc
eiv
ed
Ris
k (
1=
Lo
w,
7=
Hig
h)
Risk to people on Ground
Risk to people in the Air
There is no difference between risk perceived on ground vs. air. Also, there is no difference between perceived benefit between stakeholders and society.
61
Demographics and Risk
Age N Mean StDev
Under 25 56 4.340 1.541
36-35 28 4.679 1.786
36-45 13 5.192 1.665
46-55 14 4.964 1.216
56-65 21 5.310 1.512
Over 65 9 2.333 1.871
Those over the age of 65 perceived UAs as least risky and least beneficial; the mean value is insignificant.
62
Risk Perception Conclusions
Unmanned aircraft risk > manned Autonomous risk > Remotely piloted No difference:
Risk: Ground vs. Air Benefit: Stakeholders vs. Society
54% heard of UAS Need education programs 78% of those that heard of UAS obtained information
from television The more familiar, the more comfortable
Traffic Monitoring - higher risk Fear of operating around high population density
areas
63
Impact on Policy
Limit flight path/area
Limited population density
Implement education/outreach programs
64
Future Insights
Limitations: Time, Resources, and Budget
Sample National scale-different regions
Future Surveys: Compare UAS to other risky
technologies Size of aircraft Privacy concerns Economics concerns Lengthened
6565
Governance
Team Members:Nora DarcherNorma EspinosaScott FortuneAndrea Fuller
Team Manager:Leonardo Reyes-Gonzalez
66
Purpose
Evaluate current system of governance for UAS integration against principles of good governance
Suggest measures that could improve the governance process
67
Analysis
Principles of Good Governance Rules for FAA governance Historical Technologies Actor Interactions Roadblocks Cost and Benefits for each actor
68
Characteristics of Good Governance
http://www.unescap.org/pdd/prs/ProjectActivities/Ongoing/gg/governance.asp
69
Governance requirements on FAA
OMB rule requires FAA standards adoption procedures to have the following: Openness Balance of interest Due process Appeal process Consensus
70
Historical Analysis
How did the governance system handle the introduction of new technologies?
Supersonic Transport(1960s)
Public Interest NGOs can have a large impact
Automation of Radar System(1970s)
Incremental changes are easier than changing entire system at once
Microwave landing system (1970s)
Intl. adoption of US standard is advantageous to US technology firms
Technology What We Learned
71
Actor Analysis
Objective: Provide a systematic assessment of the
actors involved in integrating UAs into the NAS
Process: Identified key actors, examined their
goals and looked at problem from each actor’s perspective
72
Actor AnalysisResistant
Neutral
For
Hesitant
73
Roadblock Analysis
Objective: prioritize problems inhibiting the integration of UAs
Categories Technological Organizational Infrastructural Public Concern
74
Roadblock Analysis
75
Roadblock Analysis
0
1
2
3
4
5
6# of actors
Complexity
Low High
Airspace Access
Equivalent/Acceptable Level of Safety
Data AcquisitionTransponders
76
Most obvious needs
Defining an equivalent/acceptable level of safety
Allowing UAS operations in scarcely-used airspace to facilitate testing and development for civil and commercial applications.
Potentially large public concern about UAS safety argues for proactive public involvement in deliberations
77
Conclusions
78
Summary of Conclusions
Economics Risk, Technologies, and
Standards Public Awareness and Perception Governance
79
Economics
Some civil UAS applications seem highly competitive with alternatives
Initial policy ought to be tailored to the most commercially viable applications
Cost models show that (i) costs are most sensitive to hours of utilization, (ii) safety equipment has modest cost effect, except for small systems using sense and avoid
Foreign UAS firms may develop an advantage if they gain airspace first
80
Risk, Technologies, and Standards
For some applications in some classes of airspace, unmanned aircraft result in fewer fatalities than manned aircraft used for the same task
Sense and avoid is important only in airspace with significant traffic density
Low risk areas could be used for experimentation and testing without posing a high risk to those on the ground or in other aircraft
A mandatory incident reporting system has potential to greatly improve both airworthiness and human factors reliability
81
Public Awareness and Perception
All UAS applications surveyed were considered more risky and less beneficial than the manned alternative
Traffic monitoring perceived as most risky (likely due to flight over dense population)
About half of participants had heard of UAs
Those more familiar with UAS technology perceive less risk
82
Governance
Integration problem is more complex than many people realize
Incremental approach allows for policy experimentation at low risk (e.g., sparsely populated areas/airspace)
Standards need to be established to provide benchmark and incentive for manufacturing
Attention to public perception and involvement can greatly influence unfolding of UAS issue
83
Thank You for Coming!
Questions?