GA JSC SAT and Working GA JSC SAT and Working Group ProcessesGroup Processes

Corey StephensCo-Chair, GA JSC SAT

GA JSC SAT MeetingMarch 22, 2011

Washington, DC

GA Joint Steering Committee

• Evolve GA JSC to a CAST like Model– Voluntary commitments– Consensus decision-making– Data driven risk management– Implementation-focused

• The GA JSC is a means to… Focus Limited Government/Industry

Resources on Data Driven Risks and Solutions

What is CAST?

• Work began in 1997 after two significant accidents in 1996 (TWA 800 & ValueJet 592)

• CAST focus was set by:– White House Commission on Aviation Safety– The National Civil Aviation Review Commission

(NCARC)

• Opportunity for industry and government to focus resources on one primary aviation safety initiative

What is CAST?

Vision• Key aviation stakeholders acting cooperatively to lead the

world-wide aviation community to the highest levels of global commercial aviation safety by focusing on the right things.

Mission• Enable a continuous improvement framework built on

monitoring the effectiveness of implemented actions and modifying actions to achieve the goal.

Goal • Reduce the US commercial aviation fatal accident rate 80% by

2007 and • Maintain a continuous reduction in fatality risk in US and

International commercial aviation beyond 2007.

CAST Safety Strategy

Influence Safety Enhancements -

Worldwide

DataAnalysis

Set SafetyPriorities

Achieve consensus on

priorities Integrate into existing work and distribute

Implement Safety Enhancements -

U.S.

Agree onproblems and interventions

How CAST Works

CAST Safety StrategyOngoingAccident/

Incident/Studies

IncidentAnalysisProcess Emerging/

Changing Risk

Develop/ReviseEnhancements

& Metrics

10-28-05 CAST-064

CAST Plan

PerformanceTo PlanReview

Things to Watch

Industry/Government

Action

Safer System

Information on System Performance

Future Changes Analysis Process

Develop/ReviseEnhancements

& Metrics

Develop/ReviseEnhancements

& Metrics

Master Contributing

Factors

\

Approve TrainingProposal

5 8.25d

1/2/96 1/12/96

Select Accident/IncidentSets

1 15d

10/30/95 11/17/95

Review Accident/IncidentReports

2 5d

10/30/95 11/3/95

Analyze Accident/IncidentReports

3 5d

11/6/95 11/10/95

Develop PrioritizedIntervention Strategies

7 2h

1/10/96 1/10/96

Prepare Draft Report

6 3d

1/2/96 1/4/96

Revise Report

4 5d

11/13/95 11/17/95

Prepare FinalReport

8 1h

1/11/96 1/11/96

Approve FinalReport

9 1h

1/11/96 1/11/96

Select InterventionStrategies

10 1d

1/11/96 1/12/96

NTSB AccidentIncident Reports

21.3 Reports

Airclaims data

Implementation Strategy JSIT

Turbofans Installed on part 25 Aircraft

0

5

10

15

20

25

Un

cont

ain

ed

-d

isk

/spa

cers

Mul

ti -

com

mon

- en

v.

Ma

lfunc

tio

n +

cre

wer

ror

Un

cont

ain

ed

- bl

ade

s

Fire

(e.

g.,

un

der

co

wl)

Rev

erse

r

Mu

lti -

com

mo

n -

oth

er

Fue

lco

nta

m./

exh

aus

tion

Mu

lti -

unr

ela

ted

Cas

e ru

ptur

e

Cre

w e

rror

En

gin

e se

para

tion

Un

know

n

Mu

lti-r

ela

ted

Cow

l se

par

atio

n

CA

AM

Lev

el 3

an

d 4

Ev

ents

Level 4

Level 3

HistoricalData

Pareto PlotsJSAT

ASIAS data

Causal Analysis

CombinedThreat

Threat

Cause Cause Cause Cause

Accident

1.

2.

3.4.

5.

CAST Safety Analysis Process

Industry

JSAT

JSAT JSAT

Intervention Strategy

6.7.

Coordinated Plan

Measuring Progress to Goal

Industry

Government

SaferSkies

AvSP

5.3-23

• Safety enhancement development

• Master safety plan • Enhancement

effectiveness• Future areas of

study

• Data analyses

CAST

Joint Safety Analysis Teams (JSAT)

Joint Safety Implementation

Teams (JSIT)

Joint Implementation Measurement Data

Analysis Team (JIMDAT)

Commercial Aviation Safety Team (CAST)Commercial Aviation Safety Team (CAST)Commercial Aviation Safety Team (CAST)Commercial Aviation Safety Team (CAST)

CAST Accomplishments

• Forensic analysis of US and world accidents since 1987 (ongoing)

• Industry and government cooperative safety plan:– 72 Prioritized Safety Enhancements– 50 Complete and 22 underway– Projected 74% fatality risk reduction by 2020

• Development of proactive analytic processes for incident data

• CAST was the recipient of the 2008 Collier Trophy For achieving an unprecedented safety level in U.S.

commercial airline operations by reducing risk of a fatal airline accident by 83 percent, resulting in two consecutive years of no commercial scheduled airline fatalities

For this discussion…

• GA JSC = CAST

• Steering Committee = CAST ExCom

• Safety Analysis Team (SAT) = JIMDAT

• Working Groups (WGs) = JSAT/JSIT

GA JSC Groups and their CAST Counterparts

• Identify future areas of study/risk• Charter safety studies• Provide guidance and direction• Draw data from various areas• Develop a prioritized Safety Plan • Develop metrics to measure

effectiveness of safety solutions

• Data analyses• Safety enhancement• Mitigation development

• Strategic guidance • Management/Approval of Safety Plan• Provide direction• Membership Outreach• Provides linkage to ASIAS

Steering CommitteeCo-Chairs: Bruce Landsberg (AOPA/ASF) Tony Fazio (FAA/AVP)Government - FAA (AFS, AIR, ATO & ARP) - NASA (Research)

- NWSIndustry - GAMA, EAA, NBAA, NATA, & SAMA

Safety Analysis TeamCo-chairs: Corey Stephens (FAA) Jens Hennig (GAMA)Members: FAA, NTSB, AOPA, FSF, UAA, CGAR, FAST, NAFI, Insurance, Academia, SAFE

Working Groups(To include SMEs from various general

aviation segments, depending on study)

General Aviation Joint Steering Committee (GAJSC)

GA Safety Plan

GAJSC Safety Strategy

08-16-2011 GAJSC

GAJSC

SAT

WG

Time

OngoingFatal Accident

Studies

Accident Area

Proposed

Approves Priority / Assigns

Resources to WG

Detailed Accident Review and Propose

Mitigations

Approves Proposed

Mitigations

Amend Safety Plan

Develops Detailed Implementation

Plans (DIP)

Approves DIP & Assigns Industry

Government Responsibility

Industry Action

Government Action

Develop, Revise & Monitoring of Metrics

Monitor Effect

Identification of System Changes

General Aviation NAS Safety State

AviationSystem

EstablishSAT

FAA

NTSB

PilotsManufacturers

Academia

ASIASAccident Selection NASA

Evaluate Cost & Benefit

Review Proposed

Mitigations

GA JSC Working Group

Process

Step 1: Analysis

Typical CAST JSAT Membership

• ALPA/APA

• FAA (AIR, AFS, ASA, AAI, ATO)

• Airbus

• EASA

• ATA

• Transport Canada

• NASA

• Engine companies – (PW, GE, RR-Allison)

• Boeing

• RAA

• NACA

• AIA

• NATCA

5.5-24

GA JSC WG Process

CharterDevelopment

EstablishTeam

SelectData Set

ReviewData

IdentifyInterventionStrategies

AssignStandardProblem

Statements

Record Characteristics/

Indicators

DevelopEvent

Sequence

EvaluateIntervention

Effectiveness

PrioritizeInterventions

Technical Review &

ReportResults

IdentifyProblems

(what/why)

GlobalReview of

Characteristics/Indicators

EvaluateProblem Importance

Developed Event Sequence

• Facts and data• Pilot - controller voice events• Missed calls• Events that occurred or should have

• Time coded each event

# Time Event1015 21:53:28 ATC issued ATIS information Sierra: Ceiling

100’ overcast, 1/2 mile visibility and fog1016 21:53:28 F/O call 200’ above minimums1017 21:53:32 F/O calls ATC to report Marker Inbound1018 21:53:33 F/O call out 100’ above minimums1019 F/O fails to call out “runway not in sight” at the

minimums for the Decision Height

Develop Problem Statements

• Problem statements– What went wrong– Deficiency definition– Potential reason– Something which happened or didn’t happen

# Time Event/Data PointProblem (What)

Contributing Factors (Why)

1

8:53:00 Aircraft took off from Taipei Intl Airport

210:45:00 F/O briefed CAPT

on approach into

3

10:49:00 Capt gave very basic guidance to the F/O on aircraft control during approach and landing.

F/O was inexperienced; his actions were not commensurate with 1034 hours in type.

It is not normal practice at China Airlines for Capt and F/O to rotate takeoffs and landings. The FO is required to fly aircraft "in t/o and landing phases at least 3 times every 3 months" (3-28) (airline culture)

Sample Standard Problem StatementsCAST Examples

• 10 FLIGHTCREW – Failure of flight crew to follow established procedures (SOP)

• 39 AIRCRAFT EQUIPMENT – DESIGN NOT ERROR TOLERANT System design does not provide adequate redundancy to counteract errors or alerting of the effects of errors

• 44 FLIGHTCREW – Flight crew failure to recognize and correct unstable approach

• 100 REGULATORS – INSUFFICIENT AIR CARRIER OVERSIGHT . Insufficient regulatory oversight of air carrier operations including management and training practices

Identify Intervention Strategies

• Intervention strategies– Suggested solutions– Things to do to prevent or mitigate the problem– Etc.# Time Event/Data Point

Problem (What)

Contributing Factors (Why)

Standard Problem Statement

P1 A

1

8:53:00 Aircraft took off from Taipei Intl Airport

210:45:00 F/O briefed CAPT

on approach into

3

10:49:00 Capt gave very basic guidance to the F/O on aircraft control during approach and landing.

F/O was inexperienced; his actions were not commensurate with 1034 hours in type.

It is not normal practice at China Airlines for Capt and F/O to rotate takeoffs and landings. The FO is required to fly aircraft "in t/o and landing phases at least 3 times every 3 months" (3-28) (airline culture)

20 AIRLINE OPERATIONS - LACK OF TRAINING (FLIGHTCREW)

3 5

414 Airline operations – training failed to adequately develop FIRST OFFICER piloting skills. (SPS-20)

4 3

Intervention Effectiveness

• Power– Effectiveness of a specific intervention in reducing the likelihood that

a specific accident would have occurred (“Perfect World”)

• Confidence– Confidence that this specific intervention will have the desired effect

• Future Global Applicability– How well the intervention can be extrapolated to apply to a world-wide

fleet in the future

Effectiveness Rating Scales

POWER

This scale is to be used to judge the effectiveness of a specific intervention in reducing the likelihood that a specific accident would have occurred had the intervention been in place and operating as intended. (“perfect world”)

Hardly any effect

Slightly effective

Moderately effective

Quite effective

Highly effective

CONFIDENCE

This scale is to be used to define the level of confidence that you have that this specific intervention will have the desired effect.

Hardly any confidence

Slightly confident

Moderately confident

Quite confident

Highly confident

0 1 2 3 4 5 6

Not at all effective

Completely effective

0 1 2 3 4 5 6

Not at all confident

Completely confident

FUTURE GLOBAL APPLICABILITY

This scale is to be used to estimate how well the intervention can be extrapolated to apply to a world-wide fleet in the future.(for example: how often the situation it addresses occurs in accident scenarios; whether its impact is on present and future operations (equippage, traffic, regulatory differences); and whether it is applicable across airlines/airplanes/regions.

Hardly any applicable

Slightly applicable

Moderately applicable

Quite applicable

Highly applicable

0 1 2 3 4 5 6

Not at all applicable

Completely applicable

GA JSC Working Group

Process

Step 2: Implementation

GA JSC Feasibility Scales

• Technical • Financial • Operational • Schedule • Regulatory • Sociological

GA JSC Safety Enhancements

• Develop Safety Enhancements from Interventions

• Collect detailed resource information

• Prepare Detailed Implementation Plans (DIP’s)

GA JSC WG Reports

• Standard Problem Statements

• Interventions Prioritized

• Recommendations

• Detailed Implementation Plans (DIPs)

What’s a DIP?

SE 31

Loss of Control Joint Safety Implementation Team

Implementation Plan

for Training - Advanced Maneuvers

Statement of Work

Advanced Maneuvers Training (AMT) refers to training to prevent and recover from hazardous flight conditions outside of the normal flight envelope, such as, inflight upsets, stalls, ground proximity and wind shear escape maneuvers, and inappropriate energy state management conditions.

The purpose of this project is to collect and provide advanced maneuver training material and to encourage Part 121 operators to use these materials to implement advanced maneuver ground training and flight training using appropriate flight training equipment. Emphasis should be given to stall onset recognition and recovery, unusual attitudes, upset recoveries, effects of icing, energy awareness and management, and causal factors that can lead to loss of control. Additionally, research should be conducted to determine how existing flight simulation devices can be used effectively in AMT. Safety Enhancement: (SE-31) Pilots will be better trained to avoid and recover from excursions from normal flight and loss of control. Lead Organization for Overall Project Coordination (LOOPC): FAA, Flight Standards (AFS)

Score: 2007-(13.0) 2020-(13.0) 100%-(13.0) Resource Requirements: FAA AFS-400, Air Transport Association Training Committee, National Air Carrier Association (NACA), Regional Airline Association, manufacturers, pilot associations, Principal Operations Inspectors (POI’s), Directors of Safety, flight operations and training departments, NASA, aircraft manufacturers, flight simulation device manufacturers, training centers, existing training aids, and other materials.

GA JSC SAT (Safety Analysis Team)

ProcessSafety Plan Development

Develops a Prioritization Methodology (GA JSC SAT)

• Identifies the most effective solutions derived from all accident categories

• Considers effectiveness vs. resources

• Tests solutions against fatal and hull loss accidents

• Creates draft master strategic safety plan

• Identifies areas for future study/mitigation

Effectiveness

that an intervention has for reducing the accident rate if incorporated

Portion of world fleet with intervention implemented

( ),Accident Risk Reduction =

General Methodology for Calculating the Potential Benefit of a Safety Enhancing Intervention

Accident Date Location Aircraft Accident Description Portion Intervention Name

Class Type of EGPWS CFIT TRN

Description Accident Portion of World Fleet with Intervention Implemented by (2007)

Eliminated.600 .900

Intervention Effectiveness (%/100)

CFIT 1/2/1988 IZMIR, TURKEY 737 HIT MOUNTAIN ON APPROACH .657 .950 .226CFIT 2/8/1988 LUANDA, ANGOLA 707 HIT ANTENNA ON APPROACH .586 .800 .226CFIT 2/27/1988 KYRENIA MTS, CYPRUS727 HIT MOUNTAIN ON APPROACH .657 .950 .226CFIT 3/17/1988 CUCUTA, COLUMBIA 727 HIT MOUNTAIN DURING CLIMB .657 .950 .226CFIT 6/12/1988 POSADAS, ARGENTINA MD80 CRASHED ON FINAL APPROACH .203 .000 .226CFIT 7/21/1988 LAGOS, NIGERIA 707 CRASHED ON APPROACH .203 .000 .226CFIT 10/17/1988 ROME, ITALY 707 LANDED SHORT .203 .000 .226CFIT 10/19/1988 AHMEDABAD, INDIA 737 LANDED SHORT .586 .800 .226CFIT 2/8/1989 SANTA MARIA AZORES 707 TERRAIN IMPACT/DESCENT .657 .950 .226CFIT 2/19/1989 KUALA LUMPUR, MALAYSIA747 TERRAIN IMPACT/APPROACH .657 .950 .226CFIT 6/7/1989 PARAMARIBO, SURINAMEDC8 TERRAIN IMPACT/FINAL APPROACH.203 .000 .226CFIT 7/27/1989 TRIPOLI, LIBYA DC10 TERRAIN IMPACT/FINAL APPROACH.203 .000 .226CFIT 8/25/1989 ANKARA, TURKEY 727 HIT ILS ANT. ON TAKEOFF .000 .000 .000CFIT 10/21/1989 TEGUCIGALPA, HONDURA727 TERRAIN IMPACT/APPROACH .657 .950 .226CFIT 10/26/1989 HUALIEN, TAIWAN 737 TERRAIN IMPACT/DEPARTURE .657 .950 .226CFIT 2/14/1990 BANGALORE, INDIA A320 HIT SHORT (300 FT) .203 .000 .226CFIT 6/2/1990 UNALAKLEET, ALASKA 737 HIT HILL 7 MILES OUT IN FOG .657 .950 .226CFIT 11/14/1990 ZURICH, SWITZERLANDDC9 CRASHED 5 MILES SHORT .634 .900 .226CFIT 12/4/1990 NAIROBI, KENYA 707 HIT POWER LINE ON ILS FINAL .203 .000 .226CFIT 3/5/1991 MT LA AGUADA, VENEZUELADC9 HIT MOUNTAIN/APPROACH .657 .950 .226CFIT 8/16/1991 IMPHAL, INDIA 737 A/C HIT HILL 20 MILES OUT/APPROACH.657 .950 .226CFIT 1/20/1992 STRASBOURG, FRANCE A320 IMPACTED GROUND/FINAL APPROACH.586 .800 .226CFIT 2/15/1992 KANO, NIGERIA DC8 CFIT OUT OF PROCEDURE TRN-DARK.586 .800 .226CFIT 3/24/1992 ATHENS, GREECE 707 ABANDONED APPROACH-HIT MTN .657 .950 .226CFIT 6/22/1992 CRUZEIRO DO SUL, BRAZIL737 HIT SHORT,DRK NT,DISTRACTED .203 .000 .226CFIT 7/31/1992 KATMANDU, NEPAL A310 CFIT-HIT MTN-MISSED APPROACH .657 .950 .226CFIT 9/28/1992 KATMANDU, NEPAL A300 CRASHED SHORT DURING APPROACH.657 .950 .226CFIT 11/25/1992 KANO, NIGERIA 707 LANDED SHORT MISLEADING LIGHTS.538 .700 .226CFIT 11/26/1992 MANAUS, BRAZIL 707 HIT LIGHTS ON TO/RMLG COLLAPSE.000 .000 .000CFIT 4/26/1993 AURANGABAD, INDIA 737 HIT TRUCK AFTER TAKEOFF .000 .000 .000

Spreadsheet Example – Historical Airplane Accidents & Proposed Safety Enhancements – CAST Example

Basics of the Selection Spreadsheet

• EffectivenessEach safety enhancements is evaluated against each undesired condition in the set to

determine how effective the enhancement would be at eliminating these conditions if the enhancement were put in place.

• ImplementationImplementation level is based on the portion of the affected population with the

enhancement incorporated or predicted to be incorporated by a future date.

• Severity WeightingTo account for differences in severity or significance of the undesired conditions, a

weighting value can be entered so that the relative risk of the undesired conditions is realized.

• To account for differences in fatality risk associated with each accident in the data set, a severity value was applied. In this assessment, the severity value represented the portion of people onboard that perished in the given accident.

• Example: Comparison of two fatal accidents

757 CFIT accident, 98% perished. Weighting factor is .98

747 Turbulence accident, .6% perished. Weighting factor is .006

• Hypothetically assume an assessment showed that the chance of these accident occurring would have been reduced by proposed safety enhancements by 50%.

• The associated portion of fatality risk eliminated can be determined using the severity weighting factor as follows:

757 CFIT.98 x .5 = .49

747 Turbulence, .006 x .5 = .003

Severity Weighting OverviewCAST Example

Analysis Tool Output

• The spreadsheet output can be set up to show the The spreadsheet output can be set up to show the effect that an individual safety enhancement, or effect that an individual safety enhancement, or group of safety enhancements have on reducing group of safety enhancements have on reducing exposure to the undesired conditionexposure to the undesired condition..

SE1 SE2 SE3 SE1 &SE2

SE1 &SE3

Fatality RiskReduction

Robust CAST Methodology

• Detailed event sequence - problem identification from worldwide accidents and incidents

• Broad-based teams (45-50 specialists /team)

• Over 450 problem statements (contributing factors)

• Over 900 interventions proposed

• Analyzed for effectiveness and synergy- CAST Safety Enhancements

CAST Process Led to Integrated Strategic Safety Plan

• Part 121 or equivalent passenger and cargo operations studied

• Current CAST plan:

• 72 Prioritized Safety Enhancements

• 50 Complete and 22 underway

• Projected 74% fatality risk reduction by 2020

• Industry and Government implementing plan

Resource Cost Vs. Risk ReductionCAST Example

APPROVED PLAN

Completed + Plan (2007

Implementation Level)

Completed + Plan (2020

Implementation Level)

All JSIT Proposed Enhancements

(2020 Implementation

Level)

0

1000

2000

3000

4000

5000

6000

7000

8000

9000

10000

Re

so

urc

e C

os

t ($

Mil

lio

ns

)

Risk Reduction

Total Cost in $ (Millions)

2007 2020

0%

25%

50%

75%

100%

Ris

k E

lim

ina

ted

by

Sa

fety

En

ha

nc

em

en

ts

Completed

$$

$$ $$ $$

$$

Do

llars

/Flt

. Cy

cle

Part 121 Aviation Industry Cost Due to Fatal/Hull Loss Accidents

100

80

60

40

20

0

Historical cost of accidents per flight cycle

74% Risk reductionSavings ~ $74/Flight Cycle

Or

~ $814 Million Dollars/Year

Cost of accident fatalities following implementation of the CAST plan @ 2020 levels

2020

Cost SavingsCAST Example

2007

What the GA JSC can accomplish

• GA accident and incident data drives direction of GA JSC activities

• GA JSC to charge the SAT with chartering study groups on specific topics

• Working groups of SMEs formed to identify risks and develop mitigations

• Mitigations are assessed and prioritized• A cooperative industry/government GA

safety plan is developed and implemented

GA JSC SAT & WGs - Moving Forward

• History shows focused action and introduction of new capabilities have led to accident risk reductions

• Joint industry and government teams working together to a common goal can further enhance the safety of our very safe aviation system

• Full implementation will require a coordinated effort between industry and government

• The GA JSC is moving forward to meet the challenge