MIT ICATMIT ICATNational Workshop on Aviation Software Systems:

Design for Certifiably Dependable Systems

Civil Aviation Context

Prof. R. John HansmanMIT International Center for Air Transportation

rjhans@mit.edu 617-253-2271

MIT ICATMIT ICATObjectives

Define Current State of the Art

Identify Key Issues and Needs

Identify Promising Research Approaches

Define Educational Needs and Approaches

MIT ICATMIT ICATSystem Scope

Software

Hardware Dependability problem can be fairly well defined with good specifications

Tractable with current methods

Hard

MIT ICATMIT ICATSystem ScopeEnvironment

Software

Hardware Environment

MIT ICATMIT ICATSystem Scope Human

Software

Human

Hardware Environment

My Bias

MIT ICATMIT ICATSystem ScopeInteractions

Software

Human

Hardware Environment

MIT ICATMIT ICATWhat is High Dependability ?

Civil Aviation Context Target Level of Safety Equivalent Level of Safety

MIT ICATMIT ICAT

MIT ICATMIT ICAT

MIT ICATMIT ICATProbability vs. Consequences Graph AC 25.1309-1A

Probable ImprobableExtremely

Improbable

Catastrophic Accident

Adverse Effect On

Occupants

AirplaneDamage

EmergencyProcedures

AbnormalProcedures

Nuisance

Normal

MIT ICATMIT ICATDescriptive Probabilities

1

10E-3

10E-5

10E-7

10E-9

FAR

Probable

Improbable

Extremely Improbable

JAR

Frequent

ReasonablyProbable

Remote

Extremely Remote

Extremely Improbable

Probability (per unit of exposure)

What is the correct unit of exposure : Flight hour, Departure, Failure

MIT ICATMIT ICATSoftware Criticality Levels

Level A - Anomalous behavior causes catastrophic failure Inability to continue safe flight and landing

Level B - Anomalous behavior causes hazardous/sever-major failure Large reduction in safety margins Inability of crew to perform Serious or fatal injuries to small number of occupants

Level C - Anomalous behavior causes major failure Reduced capability of aircraft (safety margins, functionality) Reduced crew performance Injuries or discomfort to occupants

Level D Anomalous behavior causes minor failure No significant reduction in aircraft safety

Level E - Anomalous behavior causes no-effect on aircraft operational capability

DO-178B “Software considerations in Airborne Systems and Equipment Certification”

MIT ICATMIT ICATCivil Aviation Applications

Commercial Aircraft Fly by Wire/Light Flight Management Systems

General Aviation Aircraft Very Light Jets

Unmanned Air Vehicles

Air Traffic Management Communication Navigation Surveillance Decision Support

Integrated Air-Ground Systems

MIT ICATMIT ICATCivil Aviation Applications

Commercial Aircraft Fly by Wire/Light Flight Management Systems

General Aviation Aircraft Very Light Jets

Unmanned Air Vehicles

Air Traffic Management Communication Navigation Surveillance Decision Support

Integrated Air-Ground Systems

MIT ICATMIT ICATBoeing 747-200Electro - Mechanical“Steam Gauge”

Boeing 777Fly by Wire/Light

Boeing 747-400CRT - LCD Displays“Glass Cockpit”

Cockpit Evolution to Higher Criticality

MIT ICATMIT ICATVehicle Control LoopsInner Loops More Critical

AutopilotAutothrust

Sensors

FlightControl

Pilot

FMS

Displays

ControlsMCPCDU

ManualControl

StateCommands

TrajectoryCommands

StateNavigation

Rate

MIT ICATMIT ICATSlats

Aileron

Flaps

* Rudder

Elevator

* Trimmable horizontal stabilizer

Speed brakesRoll spoilersGround spoilers

..

* * Rudder & stabilizer have back-up mechanical controlRudder & stabilizer have back-up mechanical control

Electrically controlled, hydraulically actuatedElectrically controlled, hydraulically actuated

Fly-by-wire SystemsFly-by-wire SystemsA-320 Example A-320 Example

Anomalies : eg Hard Over Failures, Redundancy Architectures, Software as Single Point of Failure

MIT ICATMIT ICATTypical Redundancy Architecture

MIT ICATMIT ICATHuman Interaction - Manual Control Aircraft Pilot Coupling (aka PIO)

AutopilotAutothrust

Sensors

FlightControl

Pilot

FMS

Displays

ControlsMCPCDU

ManualControl

StateCommands

TrajectoryCommands

StateNavigation

Rate

MIT ICATMIT ICATQuickTime™ and a

decompressorare needed to see this picture.

MIT ICATMIT ICATMode Awareness

Mode Awareness is becoming a serious issues in Complex Automation Systems automation executes an unexpected action

(commission), or fails to execute an action (omission) that is anticipated or expected by one or more of the pilots

Multiple accidents and incidents Strasbourg A320 crash: incorrect vertical mode

selection Orly A310 violent pitchup: flap overspeed B757 speed violations: early leveloff conditions

Pilot needs to Identify current state of automation Understand implications of current state Predict future states of automation

MIT ICATMIT ICATComplexity and Conditional Statements

Used extensively in Pilot Guides

“Through the FCU, an immediate climb/descent is initiated by selecting the desired altitude in the ALT SEL window and either pulling the set knob or pressing the LVL/CH P/B to engage the LVL CHANGE mode. Pressing the LVL/CH P/B also disengages PROFILE, however, if PROFILE is engaged, pulling the set knob does not disengage it, rather it initiates an immediate climb/descent to the altitude selected on the FCU. The exceptions are ...”

MIT ICATMIT ICATEvolution (code Reuse) leads to Lack of Underlying Model

There does not appear to be a simple, consistent global model of current Autoflight Systems Not apparent in flight manuals Flight manuals focus on crew interface and procedures Manufacturer could not supply functional model or logic/control diagram Hybrid Automation Model created to allow analysis

In absence of a simple consistent model, pilots develop their own ad-hoc models

These models may not accurately represent AFS operation Concern in some (future) aircraft Individual pilot models may not be accurate Training/Design implications Models are created during nominal flight conditions and may not hold

during abnormal or emergency situations

Entropic Growth of Complexity

MIT ICATMIT ICATOperator Directed ProcessSanjay Vakill Thesis

AutomationModel

TrainingMaterial

SoftwareSpecification

Software

ConfigurationManagement

Training material is derived from Automation Model. Training Representation is created.

Automation Model is derived from Functional Analysis, operator and expert user input.

Software specification is derived from Training Material.

System is certified againstAutomation Model.

Specification changes must be consistent with Automation Model.

Certification

Configuration Management verifies and maintains consistency with Automation Model.

FunctionalAnalysis

Iterative Human-Centered Prototype Evaluation Stage

MIT ICATMIT ICATNew Functionality (eg Required Navigation Performance)

MIT ICATMIT ICATNew Functionality

Requires FMS and Memory Upgrades

Cost Issues

Maintenance

MIT ICATMIT ICATMaintenance and Capability Expansion (eg Memory)

.

Honeywell A320 Pegasus FMSAdvanced Features Addition of LOC/VNAV autoflight capability GLS/MLS Precision approach FLS (ILS like) Non Precision approach Enhanced LOC capture Multiple same-type RNAV Runway Approaches Improved Offset entry and display Mixed QNH / QFE approach capability QNH range extended to 1100 HPA 2MB Navigation Database capability Expandable Through Software to 12+MB ARINC 615A Ethernet software and database loading

….

http://www.honeywell.com/sites/aero/Flight_Management_Systems

MIT ICATMIT ICATSome Airline Comments

However we have had lots of issues with the upgrades of the FMC software.

1. Magnetic course displays are a moving target. Each upgrade uses a different set of Magnetic variations so we have to revise our plates so that the FMC and the plates are in sync.

2. The vendor changes algorithms so the procedures that we or the FAA has designed are no longer flyable. Something like this is an unintended consequence of a "fix". Also the boxes are not regulated nor are specifications so this type of disconnect can occur.

3. The FAA is asking us and any other carrier approved to fly RNP SAAAR procedures to verify that the software is safe. I do not think that this is our job. But once again this goes back to lack of regulation. They have no way of assuring that the changes being made will be compatible with RNP SAAAR.

MIT ICATMIT ICATElectronic Flight BagInformation vs Navigation Requirements

•Source: Brian Kelly, Boeing

MIT ICATMIT ICATCivil Aviation Applications

Commercial Aircraft Fly by Wire/Light Flight Management Systems

General Aviation Aircraft Very Light Jets

Unmanned Air Vehicles

Air Traffic Management Communication Navigation Surveillance Decision Support

Integrated Air-Ground Systems

MIT ICATMIT ICATVery Light JetsSmall turbofan aircraft

Eclipse500Eclipse Aviation

MustangCessna

Adam700Adam Aircraft

Safire26Safire Aircraft

ProJetAvocet Aircraft

HondaJetHonda

D-JetDiamond Aircraft

Epic LTEpic

Phenom-100Embraer

Eviation EV-20Excel Sport Jet Spectrum 33

Aircraft characteristics* Passengers:

4 to 8 Acquisition price:

$m 1.4 to 3.6 Cruise speed:

340 to 390 kts Operating ceiling:

41,000ft to 45,000ft Range:

1100 to 1750 NM Take off field length:

2200ft to 3400ft

Orders Eclipse: 2300 Adam: 75 Mustang: 330+

* for twin-engine VLJs (excludes D-Jet)

MIT ICATMIT ICATEclipse 500 CockpitAvio Avionics Suite

MIT ICATMIT ICATNext Generation ?

George Jetson Car

MIT ICATMIT ICATCivil Aviation Applications

Commercial Aircraft Fly by Wire/Light Flight Management Systems

General Aviation Aircraft Very Light Jets

Unmanned Air Vehicles

Air Traffic Management Communication Navigation Surveillance Decision Support

Integrated Air-Ground Systems

MIT ICATMIT ICATSpectrum of Current UAVs

Aerovironment BlackWidow – 2.12 oz.

BAE SystemsMicrostar – 3.0 oz.

Sig Kadet II RCTrainer – 5 lb

AerovironmentPointer – 9.6 lb

Boeing/ Insitu Scaneagle – 33 lb

AAI Shadow 200 – 328 lbBoeing X-45A UCAV – 12,195 lb (est)

Bell Eagle Eye – 2,250 lb

Allied Aero. LADF – 3.8 lb

NOAA Weather Balloon 2-6 lb

Gen. Atomics – Predator B – 7,000 lb

Northrop-Grumman Global Hawk 25,600 lb

UAV Weight (lb)0 1 10 100 1,000 10,000 100,000

Micro Mini Tactical

High Alt / UCAV

Med Alt Heavy

MIT ICATMIT ICATHistorical Comparison of Accident Rates

Notes:1) UAV Accident Rates are averaged for Pioneer, Hunter, and Predator UAVs from OSD UAV Reliability Study.

February 20032) General Aviation data from AOPA historical data, http://www.aopa.org/special/newsroom/stats/safety.html, 2006.

Flight hours unavailable from 1943-1945.3) Commercial Aviation Accident Rates from Air Transport Association aggregated CAB and NTSB statistics.

Operating hours estimated from miles flown and average speed for 1927-1948. All air carriers operating under Part 121, including cargo.

4) Air Force Aviation Accident Rates from Air Force Safety Center – Includes UAV Accidents

0.1

1

10

100

1000

10000

1925 1935 1945 1955 1965 1975 1985 1995 2005

Year

Yearly Accident Rate(Accidents / 100,000 hr)

UAVs (Average)

General Aviation

Air Force Aviation

Commercial Airlines

MIT ICATMIT ICATCertification Considerations

CatastrophicAccident

Adverse Effecton Occupants

AirplaneDamage

EmergencyProcedures

AbnormalProcedures

Nuisance

Normal

Probable ImprobableExtremely

ImprobableAC 25.1309-1A

Consequences of Failure Changefor Unmanned Operation

MIT ICATMIT ICATPredator Crash, Nogales, AZ

From Steve Swartz, FAA UAS Program Office. 2006 CERICI Workshop.

MIT ICATMIT ICATBorder Patrol Predator B Accident

NTSB Accident #CHI06MA121 Nogales, AZ

April 25, 2006, 03:41 MST

Image © General Atomics

From: http://www.ntsb.gov/ntsb/brief.asp?ev_id=20060509X00531&key=1

Excerpts from Preliminary Report

The flight was being flown from a ground control station (GCS) located at HFU. The GCS contains two nearly identical consoles, pilot payload operator (PPO)-1, and PPO-2. During a routine mission, a certified pilot controls the UAV from the PPO-1 console and the camera payload operator (typically a U.S. Border Patrol Agent) controls the camera from PPO-2. The aircraft controls (flaps, stop/feather, throttle, and speed lever) on PPO-1 and PPO-2 are identical. However, when control of the UAV is being accomplished from PPO-1, the controls at PPO-2 are used to control the camera.

The pilot reported that during the flight the console at PPO-1 "locked up", prompting him to switch control of the UAV to PPO-2. Checklist procedures state that prior to switching operational control between the two consoles, the pilot must match the control positions on the new console to those on the console, which had been controlling the UAV. The pilot stated in an interview that he failed to do this. The result was that the stop/feather control in PPO-2 was in the fuel cutoff position when the switch over from PPO-1 to PPO-2 occurred. As a result, the fuel was cut off to the UAV when control was transferred to PPO-2.

The pilot stated that after the switch to the other console, he noticed the UAV was not maintaining altitude but did not know why. As a result he decided to shut down the GCS so that the UAV would enter its lost link procedure, which called for the UAV to climb to 15,000 feet above mean sea level and to fly a predetermined course until contact could be established. With no engine power, the UAV continued to descend below line-of-site communications and further attempts to re-establish contact with the UAV were not successful.

MIT ICATMIT ICATPredator Ground Control Station

MIT ICATMIT ICATCivil Aviation Applications

Commercial Aircraft Fly by Wire/Light Flight Management Systems

General Aviation Aircraft Very Light Jets

Unmanned Air Vehicles

Air Traffic Management Communication Navigation Surveillance Decision Support

Integrated Air-Ground Systems

MIT ICATMIT ICATATM System LevelOuter Loop Criticality

Vectors

AircraftF light

M anagementC omp ute r

S tate

N av igation

Flight PlanAmendments

Autop ilo tAuto thrust

MCP Controls

ATCF lightS trip s

Surveillance:Enroute: 12.0 sTerminal: 4.2 s

S tateC ommands

Tra jecto ryC ommands

InitialClearances

CDU

ADS : 1 sDisplays

AOC:AirlineOperationsCenter

Pilot

DisplaysManual Control

Voice

ACARS(Datalink)

Decision Aids

MIT ICATMIT ICATSimplified Enroute Architecture Legacy Software Issues - JOVIAL

Source: GAO/AIMD-97-30 Air Traffic Control

Host

MIT ICATMIT ICATTCAS Emergent Backup

MIT ICATMIT ICATADS-B GPS Backup Issue

Bob Hilb UPS/Cargo Airline Association

MIT ICATMIT ICATOEP and NGATS

10 Year PlanFAA

20 Year PlanMulti-AgencyFAA, DOD, CommerceDHS, NASA, DOT, OSTP

MIT ICATMIT ICATFor more detail see Operational Improvement Roadmap in the Tech Hanger section of JPDO Website www.jpdo.aero

Source: John Scardina JPDO

MIT ICATMIT ICAT

Source: John Scardina JPDO

High Criticality

Moderate Criticality

Software Criticality Exposure

MIT ICATMIT ICATCivil Aviation Applications

Commercial Aircraft Fly by Wire/Light Flight Management Systems

General Aviation Aircraft Very Light Jets

Unmanned Air Vehicles

Air Traffic Management Communication Navigation Surveillance Decision Support

Integrated Air-Ground Systems

MIT ICATMIT ICATGPS Wide Area AugmentationSystem (WAAS)

• Increased Safety

•Fuel and Time Savings

• Increased Efficiency and Capacity

•Cost Savings

MIT ICATMIT ICATWAAS Safety Architecture

Weaknesses in Current System Monitor (Safety Processor) At Times Safety Processor Doesn’t Monitor Data

Therefore, System Integrity Is Not Quantifiable Integrity Requirement Is No More Than One in 10 Million Chance of Hazardously Misleading Information (10 -

7)

Corrections Processor

Safety Processor Uplink/GEO User

Monitors data

(Level B)

CRC protects dataGenerates data (Level D)

Satellite Signals

Receiver

MIT ICATMIT ICAT