Turner.john

Taking Program Risk ManagementTo The Next Level

on NASA’s Constellation Program

John V. Turner, PhDConstellation Program Risk Manager

CxIRMA

NASA CxP John V. Turner, PMC 2009Page 2

Agenda

• CxP Overview• Pre-Historic Risk Management• Risk Informed Decision Making

– CRM Process and Tools

– Risk Informed Design

– Integration with Systems Safety

– Risk Informed Test Program

– Knowledge Management

• CxP RIDM Status – Where are we Really on this?• Areas for Improvement


Ascent Stage Expended

ED

S, A

ltair

Orio

n

EDSExpended

Altair Performs LOI

MOONMOON

EARTHEARTH

100 km Low Lunar Orbit

Direct EntryLand Landing

Service Module ExpendedLow

Earth Orbit

CxP Lunar Mission Overview


Constellation Systems

Ares I and Ares V Rockets

Orion Capsule

Altair Lander


Lunar Outpost Concept


CxP Risk Management

• The complexity of the CxP, the ambitious nature of our mission, and the significant constraints placed on our program make effective RM essential

• We have to more proactive identify and manage our risks than previous human spaceflight programs


Early Risk Management

IRMA

Continuous Risk Management (CRM)

A meeting….A scorecard…..A database…….Hierarchical risk roll-up


Risk Informed Decision Making (RIDM)

• NASA NPR 8000.4A Agency Risk Management Procedural Requirements

• Integration of RIDM and CRM into a coherent framework:– to foster proactive risk management: – to better inform decision making through better use of risk

information, – and then to more effectively manage implementation risks using

the CRM process - which is focused on the baseline performance requirements emerging from the RIDM process.

• Within an RIDM process, decisions are made with regard to outcomes of the decision alternatives, taking into account applicable risks and uncertainties;

• As part of the implementation process, CRM is used to manage those risks in order to achieve the performance levels that drove the selection of a particular alternative

• Proactive risk management applies to programs, projects, and institutional or mission support offices.


RIDM

Design Trades

Design Risk Acceptance

Operational Risk Acceptance.....

Mission Concept Definition

Mgt of Change

Requirements Definition

Establish Controls / Ops Safety Baseline

Budget Scrubs

What Kind of Decisions?Acquisition Strategy Selection

Boards and Panels

ATP Milestones

Flight/Test Readiness Reviews

Safety Review Panels

Tiger Teams

Where Are They Made?

Source Boards.....


IRMA

ATP MMRs

Risk Informed Decision Making (RIDM)

• Test Objectives• Readiness Reviews• Real Time

• Managing risk through change

Systems Safety

• Systematic Analysis• Formal Risk Acceptance• Establish Operational Safety

Baseline (OSB)

• Risks Reviewed at Authority to Proceed

• C/S/T Baseline Decisions

• Standards for risk characterization• CLAS for risks• Risk Communication and Reporting

Process• Prioritization of risk mitigation

proposals

Systems Engineering

• Requirements and TPM Achievability

• Analysis priorities• Iterative Design and

Analysis

Probabilistic Designand Analysis

• Standards of Practice• LOC LOM Reqts• Integrated Campaign,

Architecture, System, Element Analysis

Ops/Test

Boards/Panels

Continuous Risk Management (CRM)

Dynamic Information Linkages

KnowledgeManagement

• KBRs• PAL• Knowledge Capture


CRM

• The CxP follows the NASA Continuous Risk Management Paradigm


CRM

• The CxP has established Risk Management offices at the Directorate level, program level and project level– In some cases level IV (element) have a RM office as well

• RM policy is flowed from the agency to directorate, to program to project level, and in some cases to elements


CRM

• A Risk Management Working Group (~bi-weekly) has been established to ensure common practice and guide the development of RM policies, practices, and tools– Including the CxP RM database application – IRMA

• A program risk scorecard has been put in place to help establish consistency in risk priorities


CRM

• A Top Risk Review Process is used to escalate the most significant risks to higher levels for communication and action– Occurs ~ bi-monthly– Top Project risks are discussed– Risks requiring higher level awareness or action are escalated to the

directorate risk review

• The CxP Risk Team provides training to all program elements in order to promote awareness, consistent practice, improvement– Several hundred personnel trained


CRM - Risk Review Process

OrionProject

AresProject

EVAProject

ESMD

AltairProject

Lunar Surface Systems

CxP

Ground Ops Project

Mission Ops Project

OTISRQAPPCSEI


CRM - CxP Cost Threat Process

• The PP&C organizations at all program levels are responsible forensuring that the impact of risks on program reserves is identified

• This effort involves the Cx program and projects identifying andquantifying new cost impacts related to risk mitigation planning

• A threat is money required to mitigate a risk that is not currently in the Program or project budget

• Cost threats are documented and tracked in CxIRMA

• During the risk review, management considers risks with technical performance, operations, safety, cost and schedule impacts

– Balances requests for new mitigation funding identified in threats

– What is the best portfolio of risk mitigation options that can be funded based on threat profile and reserves?

Risk 2564

Identification Assessment Handling

Team Brainstorming

PRA Risk Drivers

Acc Risk Hazards (FMEA)

Requirements Risks (CARD)

Integrated Analysis (TDS)

Problem Reports (PRACA)

Integrated Analysis

(TDS)

PRA

Ares IMS

Project Control Data

Communication(stakeholders)

SEI

SRQA

Orion

OTI

.

.

.

Orion IMS

L2 IMS

.

.

.

Fully Characterize the Risk


CxIRMA

• The CxP uses the IRMA risk database application to document, track, and communicate CxP risks

• CxIRMA users guide and training available in the tool• IRMA is used in the ISS and Shuttle Programs and has been

modified to complement the Cx risk process • The CxIRMA database is accessed in the CxP through the ICE

environment• Users are assigned a role and to a Cx organization, and can be

assigned to multiple organizations– Permissions are set by user type:

• All risks are visible in CxIRMA regardless of organization affiliation• Candidates are only visible to those users assigned to the owning

organization


• CxIRMA is based on a “homepage” concept– Each org has it’s own riskl list or homepage

• Risk they own, risks for which they are stakeholder, escalated risks– Captures risk relationships– Easy to generate reports

CxIRMA


CxIRMA

• Significant updates in work– Update CxIRMA sw technology

• Database, middleware, interface– New user friendly interface– Data relationships with other data systems

• Requirements• Critical Analyses (TDS)• Schedule (IMS)• Hazards• PRACA

– Embedded in Program Control Data System– Improved mitigation planning capability

• MS Project type interface– Improved graphical reports

• Mitigation Gant or “Waterfall” charts


Risk Informed Design (RID)

• Risk Informed Design means that the design of the CxP architecture will consider risk as a critical design commodity so that the designsproduced most effectively balance risk against performance and cost.– The ESAS used risk analysis to prioritize various architecture approaches

based on risk– The establishment and allocation of LOC and LOM requirements applies

design pressure on architecture development at all levels– Various risk analysis methods are used to identify risk drivers and identify

the most beneficial use of design commodities (mass, power, budget, etc) to better meet LOC and LOM• Hazard Analysis• FMEA• PRA• Physics models and simulations

– Risk associated with Cost, Schedule, and other design commodities are also considered

– The Iterative Design Analysis Process provides regular integration forums where design insights can be made


• RID uses LOC and LOM requirements to provide top down allocations of risk based on generic design reference mission configurations,– LOC and LOM were initially defined at the generic DRM level per the ESAS and

architecture changes made after CxP startup• These mission risk requirements were allocated to the system and

subsystem level• PRA, simulation, and physics modeling methodologies were used to used

to evaluate adequacy of current designs and operational plans in meeting these requirements

• LOC and LOM analysis addresses hardware, software, environments,human reliability, external events, phenomenological events, etc.

• LOC and LOM analysis is part of the IDAC process– LOC and LOM is incorporated in diverse assessments and trade studies as

integrated abort system design, launch order, land vs water landing, etc • The program is developing a campaign analysis capability that will

allow us to evaluate the integrated effect of current designs and plans over a campaign of missions– Could result in a re-assessment of mission allocations and their allocations to

the subsystem level– Could result in new requirements to drive more specific design issues



• The program is using PRA to provide more robust risk characterization during the hazard analysis process– Significant hazards will be quantified, and these incorporated in the PRA

mission models– Functional Hazard Analysis performed to provide a top down, mission based

review of hazards to provide a basis for IHA and system HA allocations and a starting point for mission PRA models

– Mission PRA models and hazards will have a common basis• Integration of PRA and HA through FHA, and the quantification of

significant hazards, promotes better understanding and intelligent management of the operational safety risk baseline

• FMEA, Hazard Analysis, PRA• Controls, Verifications



Development of Mission Concepts and Architectures

0 .0 0 1 .0 0 2 .0 0 3 .0 0

R e fe re n c e M is s io n s

A rchite c ture 2A rchite c ture 9A rchite c ture 4A rchite c ture 7A rchite c ture 1A rchite c ture 3A rchite c ture 8A rchite c ture 5

A rchite c ture 1 0A rchite c ture 6

Ris

k FO

M

Mars M iss ion Arch itec ture R isk Assessm ent

S ys te m s R e lia b i li tyE ntry / L a nd ingM a rs O rb it Inse rtio nL a unch / Inte g ra tio nTra ns M a rs Inje c tio nM a rs A sce ntTra ns E a rth Inje c tio nO the r H a za rd s

Example Only – Not Real Data


Development of Mission Concepts and Architectures

Loss of crew due to Abort System separation jettison motor fails to function4.31E-058855

Loss of crew due to ground operations induced malfunction2.16E-054896

SRM case burst3.02E-055957

Loss of crew due to LV Upper Stage Engine Upper Stage Engine Catastrophic Failure

6.16E-0511784

Loss of crew due to Capsule software failure9.08E-0516673

Loss of crew due to MMOD impact1.20E-0422502

Loss of crew due to common cause failure of parachutes during landing1.60E-0429291

Cut SetsProb./Frequency% Cut Set% Cumul.Cut No.



Systems Safety and Risk Management

• The CxP Risk Management program differentiates between risk acceptance decisions made during early design and operations, and longer term acceptance decisions– The Safety Review Process considers residual risk hazards and makes initial

acceptance decision– These risk are captured in the program CRM process to decide if longer term

mitigation is needed– Periodic reviews are made of acceptance rationale to determine if further risk

mitigation is warranted based on new information, new capabilities, evolving risk vulnerabilities, changes to designs and operating plans, or new funding


“Top” residual Hazards are entered in CRM process (Defined by place on matrix)

Define And Characterize Risk

Hazard Acceptance

Implement Controls

Development

Implement Strategic Mitigation

Hazard Acceptance

Hazard Acceptance

HazardAcceptance

Operations

The Life of a Safety/Mission Risk

HA, FMEA, PRA

CSERP Ops MS

Cease Mitigation?

Ops MS Ops MS

Maintain Controls

CRM Risk Acceptance

Risk ReviewRes

idua

l CRM Process

Systems Safety Process


Integrated Risk Management: CRM is the Glue

DDTE Operations

AcceptanceSystems Safety• Define Risks and Controls • Residual Risk Acceptance• Establish Operational Safety

Baseline (OSB)

• Capture most significant AR hazards as IRMA risks

• Continue to mitigate accepted risk hazards as appropriate

Boards/Panels• Evaluate risks associated with

proposed changes• Conscious risk acceptance

assoicated with change

• Document risks associated with decisions in CR and mitigate

Continuous Risk Management

ATP milestones• Define risks as part of ATP

prep and consider these in decision

• Conscious risk acceptance• Identify new risks

• Document risks identified as part of MMR process and mitigate


Apollo Test Program

Apollo LES

2004 2005 2006 2007 2008 2009 2010 2011 2012

Saturn I

Saturn IB

1957 1958 1959 1960 1961 1962 1963 1964 19651 2 3 4 1 2 3 4 1 2 3 4 1 2 3 4 1 2 3 4 1 2 3 4 1 2 3 4 1 2 3 4 1 2 3 4

Sputnik

Saturn1 ATP

Kennedy Speech“…before this decade

Is out…”

5/25

10/27 4/25 11/16 3/28 1/29

5/28 7/305/252/169/18

SA-1SO

SA-2SO

SA-3SO

SA-4SO

SA-45SO

SA-6 SA-7 SA-9 8 10

11/7 5/13 6/295/1912/8

PA-1 A-001 A-002 A-003 PA-2

VisionSpeech

ESASRoll-Out

LAS1

DFT1

LAS2

LAS3

RRF1

RRF2

RRF3

ISS1

Saturn I flew 4 times before adding an upper stageSaturn I flew 6 times with S-IV before moving to S-IVBSaturn IB flew 4 times before first manned flightSaturn V flew 2 times before first manned flight


Constellation’s Integrated Flight Test StrategyLow Earth Orbit Servicing Capability

1/121/107/09 1/117/10 4/124/09 10/09 4/10 10/10 4/11 7/11 10/11 7/12 10/12 1/13 4/14 7/14 10/144/13 7/13 10/13 1/149/08

HighAltitude

Abort

Ares I-Y Orion 1 Orion 2 Orion 3

AA-1Max qAbort

AA-3TumbleAbort

PA-2

AA-2Transonic

Abort

Ares I-X Orion 4

CLV CDRCEV CDR

Validation Flight Tests(Production Systems)

Orion Prime

Development Flight Tests

OrionProject

CxP CDR

PA-1

FIRST FEIT MEIT


Risk Informed Test Planning

• Goals of Test Program– Validate requirements – Validate models– Enhance reliability growth– Better support Risk Acceptance

• Methodology1) Identify Hazards Early using Functional Hazard Analysis (FHA)

1) High level functional hazards vs Cause level2) Evaluate likelihood of occurrence using available knowledge and historical analogs3) Determine the capability of analysis, ground test, flight test to characterize risks and

reduce uncertainty4) Recommend analysis and test activities needed to balance uncertainty reduction and

achieve reliability growth5) As hazard analysis and PRA mature, re-assess

Pilot Project• Examine 10-12 hazards, evaluate the adequacy of current planned

activities


RM and KM Integration

• In pursuit of becoming a learning organization, CxP risk management will include the integration of knowledge management and risk management processes into the program/project life cycle

• Designing a complex architecture of hardware, software, ground and space-based assets to return to the Moon and then on to Mars will require an effective strategy to generate, capture and distribute knowledge

• Premise: Risk Managers, who already use lessons-learned as a source of information for risk identification, are in a unique position within the organization to effectively perform these functions

• Strategies– Knowledge-Based Risks – Pause and Learn (PAL) Events– Knowledge Capture/Integration


Cx Knowledge-Based Risks

• NASA’s Cx Program plans to create KBRs from pre-existing program risks (housed inside of CxIRMA) as well as incorporate KBRs into new program risks as they are identified.

• As the Cx Program evolves, KBRs will be integrated into the existing continuous risk management (CRM) process.

– Similar to CRM, the Cx KBR process includes Identification, Disposition, Documentation, and Distribution. KBR identification will become synonymous with risk identification.

– The process also interacts with all levels and members of the Cx Program including: Cx Orgs, Cx Risk Management Working Group (RMWG), KBR Owners (similar to risk owners), ESMD, and SE&I.

• If the Cx Program decides a KBR is “significant,” the program has identified the need for further exploration (including interviewing subject matter experts on the topic, collecting related documentation, etc…) into how this KBR relates to other NASA programs and projects. ESMD is responsible for significant KBR development.

• Once the KBR implementation process has been tested successfully within the Cx Program, other programs will have the ability to participate in the process, creating a continuous KBR operation across the agency.


• The CxP RM program is very strong– Established Program Risk Management plan, risk review process, RM

tool, RM working group, and RM training (over 500 trained)– All Cx Projects are actively identifying and mitigating risks and

participating in the top risk reporting process– Integration of RM process & tools between levels I, II, and III going well– Risk Management is integrated with project control and ATP Milestone

processes– Overall, level of detail and fidelity of mitigation planning is excellent for this

stage of the program’s life and improves monthly– Risk identification processes such as Reqts Design Compliance, HA,

FHA, Independent Cost Analysis, and PRA are in place to provide legs to the RM process

– Integration of Technical Requirements, TPMs, TDSs, Cost Threats, Safety Analysis, Cost and Schedule under way

– CxIRMA continues to develop improved capability to support new risk integration initiatives and ease of use

CxP RM Status


• Results are Evident– Risk is driving the design of Ares, Orion, and Altair to obtain a more optimal

balance of risk across the architecture and mission timeline– Significant decisions are informed by risk analysis, including technical,

safety, cost, schedule, and mission success factors– RM practice is present at all levels and in all decision making forums in the

CxP– The CxP has created a RIDM culture

• Having said that….there are areas where we can improve on this practice

– Policy / Practice • Streamline and focus risk reviews, Continue to improve the

quality of our risks. Integration of risks with other critical data elements

– Tools • Risk Informed Test Planning Methodology. IRMA

Enhancements. Knowledge Based Risks– Training

• Case based training

CxP RM Status


Backup



RIDM relies on being able to both: 1) compare risks to resolve design trades, and 2) aggregate risks to understand risk postureat the mission and campaign level

• The Risk Informed Design paradigm has been adopted by Ares, Orion, Lander, and CxAT to establish a more optimal use of design commodities to balance risk

– Adaptation of NESC recommended methodology (RP-06-108: Design, Development, Test, and Evaluation (DDT&E) Considerations for Safe and Reliable Human Rated Spacecraft Systems)

– Define Needs, Objectives, Constraints– Define Minimum Functionality– Make it Work– Make it Safe– Make it Reliable– Make it Affordable


Technical Risk Scenario

Mitigation EventsInitiating Event

Des

irabi

lity

of O

utco

me

Time

ANominal

MinorDamage

Catastrophic

Outcome

Initiating Event

Conditional Event 1

Conditional Event 3

LOC

LOM

LOM

NOM

Conditional Event 2

• Paradigm works well for safety risk scenarios where discrete probabilities can be assigned to specific events in an accident sequence

• Each sequence of events or risk trajectory, has a unique probability, derived from the combination of conditional probability events


Mission Success Depends Upon a Combination of Many Variables

Launch:• Time increment

between launches

• Launch Availability

• Launch Probability

• Order of Launches

LEO Loiter:• LEO Loiter Duration

• Ascent Rendezvous Opportunities

• TLI Windows

Vehicle Reliability:• LOM/LOC

Target Characteristics:• Redundant Landing Sites

• Multiple opportunities to access a select landing site

• Lighting constraints at target

Launch Strategy:• Two launch

• Single Launch

Vehicle Performance:• Orbital Mechanics Variation

Tolerance

• Additional Propulsive Capability

• Vehicle Life

• Launch Mass Constraints


Example – Functional Risk Timeline



Saturn / Apollo Development Testing

• Saturn “Block 1” Sub-Orbital Flights– First Stage Ascent Tests with Inert Upper Stages (no

separation)– Validation of ascent performance, structural loads,

functionality of gimbaled nozzles on the outboard engines for S&C.

– SA-4 flight included intentional “engine-out”checkout.

• Saturn Block II Flights– Functional S-IV Upper Stage– SA-6 through SA-10 flights carried prototype Crew

Modules– Test of nominal LES jettison on SA-6 and SA-7.

• Un-Crewed SI-B Flights– Functional SIV-B upper stage powered by J-2

Engine.– CM separated and returned to Earth.

• Launch Escape System Testing– Abort Test Booster to test the LES at transonic,

maximum dynamic pressure, low altitude, and power-on tumbling abort conditions.


MARS

Mars First?MOON

ISS

• Exploration Campaign Analysis: Identify the activities and architectures required to optimally produce mission success and crew safety within cost and schedule constraints

• The high risk associated with manned Mars exploration make risk informed design essential

• ISS and Lunar missions are also essential to accomplishing this goal – Technology demonstration– Reliability growth– Operational experience

Earth

Date post:	12-Jan-2015
Category:	Technology
Upload:	nasapmc
View:	13,376 times
Download:	0 times