Uncertainty Course Stanley Wilhite Part 1.Aspx

7/31/2019 Uncertainty Course Stanley Wilhite Part 1.Aspx

1/127

Georgia Inst i tu t e

Of Technology

Daniel GuggenheimSchool ofAerospace Engineering

Overview of Uncertainty inAerospace Design

Dr. Douglas StanleyGeorgia Institute of TechnologyNational Institute of Aerospace

[email protected]

Dr. Alan WilhiteGeorgia Institute of TechnologyNational Institute of Aerospace

[email protected]

With Special Thanks to:Dr. Dimitri MavrisandDr. Michelle Kirbyof Georgia Tech


2/127


Of Technology

Daniel GuggenheimSchool ofAerospace Engineering Outline

Definitions

Brief Review of Probability Random Events

Probability Distributions

Sampling

Functions of Random Variables

Overview of Risk and Continuous Risk Management Risk Identification

Risk Analysis

Risk Planning

Risk Tracking and Control

Expert Elicitation in Risk Management Uncertainty and Margin in Design Weight/Performance Margins and Uncertainty

Cost Margins and Uncertainty

Schedule Margins and Uncertainty

Technology Risk Mitigation


3/127


Of Technology

Daniel GuggenheimSchool ofAerospace Engineering Outline

Risk Leveling

Probabilistic Risk Assessment

Response Surface Methods

Probabilistic Design

Example of Probabilistic Design Under Uncertainty

Decision Making Process Characteristics

Common Biases

Figures of Merit

Multi-Attribute Utility Theory

Making Design Decisions Under Uncertainty Summary


4/127


Of Technology

Daniel GuggenheimSchool ofAerospace Engineering Definitions

Uncertainty

The state of being uncertain; Doubt

The estimated amount by which a calculated value may differ from thetrue value

Uncertain

Not known or established; Not determined, Not having sure knowledge

Risk

The possibility of suffering harm or loss; Danger; Hazard

The chance of loss; The degree of probability of loss

Probability of a non-desirable event

Probability

The relative possibility that an event will occur; Likelihood

The relative frequency with which an event is likely to occur

Ratio of number of occurrences to number of possible occurrences

We wish to use probabilityto measure uncertainty, in order to reduce

uncertaintyto mitigate risk or to make designs robust to uncertaintyDictionary.reference.com


5/127


Of Technology


Summary


6/127


Of Technology


Risk comes from uncertainty in the design, development, production and

operations processes If no uncertainty there is no riskreducing uncertainty reduces risk

Very high level of uncertainty in exploration systems due to lack of

development and operations experience base

Sources of uncertainty include:

Uncertainty in performance, safety, cost, and schedule models

Uncertainty/changes in customer requirements

Uncertainty in integration effects on performance

Uncertainty in manufacturing variation/tolerances

Uncertainty in test results

Uncertain operating environments (temperature, pressure, acoustics, etc.)

Uncertain responses to operating environments/failure modes

Uncertain component/subsystem/system life

Potential human errors in design, development, production or operation

Summary of Risk and Uncertainty


7/127


Of Technology


Judicious use of MARGIN is the most important risk mitigation strategy!

Includes cost, schedule, and performance/weight margins

Use of margin is necessary but not sufficient for risk management

Still need to identify and mitigate root causes of risk

Increasing margin decreases risk, but at the expense of other FOMs

May decrease performance/payload or increase cost at some point Eventually reaches diminishing returns in customer value proposition

Finding the right balance between margin/risk and other FOMs is, once

again, a multi-attribute decision problem

How do I know how much increasing margin decreases risk?

Through probabilistic analysis using historical data regression or expert

elicitation coupled with Monte Carlo simulation

How do I know how much increasing margin affects other FOMs?

Through integrated systems analysis capability or expert elicitation

Summary of Margin and Risk


8/127


Of Technology

Daniel GuggenheimSchool ofAerospace Engineering Summary of Design Under Uncertainty

The level of uncertainty modeled in the design process depends on

the nature of problem, available resources, and criticality of decision Risk identification, analysis, planning, tracking and control require

many decisions that require formal methods such as expert elicitation

Judicious use of performance, cost and schedule margin is the mostimportant risk mitigation strategy in dealing with uncertainty

Risk leveling prevents resources from being focused on risks that donot have a significant relative effect on the system

A set of complete, independent, and well-defined Figures of Merit areessential for good design decisions

Multi-Attribute Utility Theory provides the most analytically sound and

comprehensive process for design decision making An integrated systems analysis capability is essential to good design

Sensitivity analysis enables better decisions by testing assumptions

Probabilistic analysis enables better designs by quantifying risk

Monte Carlo Analysis and Response Surface Methods are key tools

that enable efficient methods for probabilistic design


9/127


Of Technology


Probability


10/127


Of Technology

Daniel GuggenheimSchool ofAerospace Engineering Probability and Random Events

A basic underlying assumption of probability theory is that

it deals with random events

A randomevent is one in which the conditions are suchthat each member of the population, N, has an equalchance of being chosen

A special and precise system of language and notation isused in probability theory

Two events, A and B, are said to be independentif theoccurrence of either one has no effect on the occurrence

of the other Two events that have no elements in common are said to

be mutually exclusiveevents


11/127


Of Technology

Daniel GuggenheimSchool ofAerospace Engineering Errors and Samples

The act of making any type of experimental observation

involves two types of errors:

Systematic errors (which exert a nonrandom basis)

Experimental, or random, errors

When a large number of observations are made from arandom sample, a method is needed to characterize thedata

Histograms

Frequency Distribution


12/127


Of Technology

Daniel GuggenheimSchool ofAerospace Engineering Probability Distribution

A probability frequency distributionis a characterization of the possible

values that a random variable may assume along with the probability ofassuming these values.

The probability functionhas the following characteristics

0 f(xi) 1; f(xi) = 1

A probability density function, f(x), is characterized by the probabilities ofvarious outcomes of continuous random variables. Probabilities are definedover intervals computed as the area under the density function between x1and x2.

A cumulative distribution functionspecifies the probability that a randomvariable X will assume a value less than or equal to a specified value, x

denoted as P(X 1).

f(x)

x6 7 8

P(X > 7.3) = .33


13/127


Of Technology


Measures of Central Tendency andDispersion

A probability frequency distributioncan be described with

numbers that indicate the central location of thedistribution and how the observations are spread out fromthe central location (dispersion)

Arithmetic mean, or average

Median

Mode

Variance

Standard Deviation


14/127


Of Technology


Types of Distributions Normal Distribution

Normal Distributions

Very important in sampling because of the Central Limit Theorem

Many physical measurements follow the symmetrical, bell-shaped curveof the normal, or Gaussian, frequency distribution

f(x) = (1/( (2 )

0.5

))exp(-0.5((x- )/ )

2

)

Normal Distribution Standard Normal ( = 0, = 1)


15/127


Of Technology


Types of Distributions Weibull Distribution

Weibull Distribution

Widely used for many engineering problems because of its versatility,since many random variables follow a bounded, nonsymmetricaldistribution, such as fatigue life of components

Used to include infant mortality in component life modeling (bathtub)

f(x) = ((m/ )/(x/ )

m-1

)exp(-(x/ )

m

), x > 0

Weibull Distribution for = 1and various values of m

m = Shape Parameter

= Scale Parameter


16/127


Of Technology


Types of Distributions Gamma Distribution

Gamma Distribution

Used to describe random variables that are bounded at one end

Measures time required for total of h independent events to take place ifevents occur at a constant rate of l

Used to model failures and in queuing theory

Chi-square and exponential distributions are special cases

f(x) = ( x -1e- x)/(0

x -1e-xdx), x > 0, > 0, > 0

Gamma Distribution for = 3and various values of

= Shape Parameter

= Scale Parameter


17/127


Of Technology


Types of Distributions Exponential Distribution

Exponential Distribution

Measures time required for first event to take place if events occur at aconstant rate of l (widely used to measure time to failure)

Special case of the gamma distribution for = 1

Special case of the Weibull distribution for m =1 and x0 = 0

Exponential Distribution for= 1/ where = Failure

Rate

f(x) = (1/ )e-x/ , x > 0

1/


18/127


Of Technology


Engineering Statistics Sampling Distributions

The central problem in statistics is relating the population

and the samples that are drawn from it

This problem is viewed from two perspectives:

What does the population tell us about the behavior ofthe samples?

What does a sample or series of samples tell us aboutthe population form which the sample came?

Central Limit Theoremtells us that:

If a sample size is sufficiently large, the mean of arandom sample from a population has a samplingdistribution that is approximately normal, regardless ofthe shape of the relative frequency distribution of thetarget population


19/127


Of Technology


Probability Sampling AnalysisExample

12.8 15.6 13.5 15.7 15.3 15.2 20.1 14.2 12.9 14.016.9 14.3 15.5 14.6 13.0 14.7 19.0 13.0 11.3 14.214.5 14.8 14.2 13.0 13.1 12.5 16.1 19.1 16.7 13.215.0 12.7 13.6 13.3 13.2 14.7 12.9 13.1 17.3 15.417.9 13.0 14.3 14.2 15.7 15.6 13.0 13.9 14.2 16.012.9 13.1 13.3 12.3 13.1 13.6 13.2 18.5 13.2 13.712.6 14.4 14.5 13.9 17.0 13.7 12.7 16.8 13.3 14.714.2 13.0 14.6 14.0 12.9 14.7 12.8 12.0 14.2 12.813.7 15.2 14.8 13.0 11.7 12.2 13.3 13.8 14.2 14.314.7 12.6 18.9 14.3 14.4 15.5 16.8 17.0 13.2 12.9

Sample Times (in Seconds) to Inspect Test Devices for Calibration


20/127


Of Technology


Probability Sampling AnalysisExample

Histogram for the frequency distribution

of inspection times.

Suggested shape of smoothed frequencycurve for the entire population of

inspection times.

Frequency polygon for thefrequency distribution of

inspection times.


21/127


Of Technology

Daniel GuggenheimSchool ofAerospace Engineering Probability Sampling Analysis Example

50%

90%

CumulativeDistribution

Function

ProbabilityDistribution

Function


22/127


Of Technology


Sampling Distributions andStatistical Intervals

Distribution of Sample Means (t Distribution)

Distribution of Sample Variances (2 and F Distributions)

Determination of Confidence Intervals Confidence Interval containing with probability 1-:

Where 1- is given by:


23/127


Of Technology

Daniel GuggenheimSchool ofAerospace Engineering Statistical Tests of Hypotheses

The statistical decision-making process can be put on a rational,

systematic basis by considering various statistically basedhypotheses

Null hypothesis Ho: = o

Alternative hypothesis H1: < o

Example: Use to test if mean of sample meets minimum

acceptable value Type I error, it was acceptable but we concluded it was not

Type II error, it was not acceptable but we concluded it was (oops!)


24/127


Of Technology

Daniel GuggenheimSchool ofAerospace Engineering Functions of Random Variables

Functions can be as simple or as complicated as desired

Random variables can be independent or correlated.Some closed form solutions exist for addition andsubtraction of random variables.

For closed form solutions, information on the input

random variables distribution is used to describe thedistribution of the output random variable given certainconditions.


25/127


Of Technology

Daniel GuggenheimSchool ofAerospace Engineering Addition of 2 RVs

Lets start with a simple case.

Uniform Distribution

Y1 = X1 + X2 Assumptions:

X1 ~ U(0,1) X2 ~ U(0,1)

Can you speculate on the distribution of Y1?

Shape

Mean Upper and lower bounds?


26/127


Of Technology


10,000 Monte Carlo runs

Theory

Bounds: [0.0, 2.0]

Mean: 1.0

Empirical data Bounds: [0.01 , 1.9]

Mean: 1.0

Looks like a triangular distribution. Do you understand

mathematically why it makes sense that it is? Monte Carlorandomly selects points from the distribution

and operates on them

Named after casino by Los Alamos scientists in 1947


27/127


Of Technology


Y2 = X1 + X2 + X3 Assumptions:

X1 ~ U(0,1)

X2 ~ U(0,1)

X3 ~ U(0,1) Can you speculate on the distribution of Y2?

Shape

Mean

Upper and lower bounds?


28/127


Of Technology



Theory

Bounds: [0.0, 3.0]

Mean: 1.5


Mean: 1.49

No longer looks like a triangular distribution, but its not

quite a normal distribution either. Its simply bell-shaped.


29/127


Of Technology


Y3 = X1 + X2 + X3 + X4 + X5 Assumptions:

X1 , X2 , X3 , X4 , X5 ~ U(0,1)

Can you speculate on the distribution of Y2?

Shape Mean

Upper and lower bounds?


30/127


Of Technology



Theory

Bounds: [0.0, 5.0]

Mean: 2.5


Mean: 2.49

Looks much more like a normal.

What is happening to the bounds of the empirical datawith respect to the theoretical data?

What would happen with infinity Xi ~ U( 0,1)


31/127


Of Technology

Daniel GuggenheimSchool ofAerospace Engineering Observations

This was a simple case:

Uniform distribution is the simplest one.

Distributions are symmetric

All distributions were equal

Distribution limits (0, 1) make theoretical estimationeasier.

Addition of RVs is very intuitive

Things to vary:

Number of Monte Carlo runs Number AND type of RVs

Ranges, means and other parameters of the RVs

The actual function of the RVs


32/127


Of Technology

Daniel GuggenheimSchool ofAerospace Engineering The Normal Distribution

De Moivre developed the normal distribution as an

approximation to the binomial distribution Used by Laplace in 1783 to study measurement errors

Used by Gauss in 1809 in the analysis of astronomicaldata

Normal distributions have many convenient properties, sorandom variables with unknown distributions are oftenassumed to be normal

Normal distribution is often a good approximation due to

a result known as the Central Limit Theorem Many common attributes such as test scores, height, etc.,

follow roughly normal distributions, with few members atthe high and low ends and many in the middle

D i l G h i


33/127


Of Technology

Daniel GuggenheimSchool ofAerospace Engineering Adding 2 Normal RVs

X1~N(0,1) X2~N(0,1)

Y1 = X1 + X2

= -0.01

= 1.43

2 = 2.04

D i l G h i


34/127


Of Technology

Daniel GuggenheimSchool ofAerospace Engineering Adding 2 Normal RVs

X1~N(0,1) X3~N(3,2)

Y2 = X1 +X3

= 3.01

= 2.26

2 = 5.12


35/127

Daniel Guggenheim


36/127


Of Technology

Daniel GuggenheimSchool ofAerospace Engineering Subtracting 2 Normal RVs

X1~N(0,1) X3~N(3,2)

Y2 = X1 - X3

= -2.96

= 2.23

2 = 4.96

Daniel Guggenheim


37/127


Of Technology

Daniel GuggenheimSchool ofAerospace Engineering More Complex Functions

Daniel Guggenheim


38/127


Of Technology

Daniel GuggenheimSchool ofAerospace Engineering More Complex Functions

Y1 = X1 + X2 + X3 + X4 + X5 Entire range is from -1.22 to 15.21

Mean = 6.26 Std. Dev.= 2.12 Kurt. = 3.13

Daniel Guggenheim


39/127


Of Technology


Continuous Risk Management

Daniel Guggenheim


40/127


Of Technology

Daniel GuggenheimSchool ofAerospace Engineering Continuous Risk Management Process

Make Decisions Under Uncertainty at Every Step in Process!

Daniel Guggenheim


41/127


Of Technology

Daniel GuggenheimSchool ofAerospace Engineering Continuous Risk Management Process


Daniel Guggenheim

Risk Identification Decisions


42/127


Of Technology

ggSchool ofAerospace Engineering

Risk Identification Decisions:Knowing the Unknown

Ref: EAI-632

As we know, there are known knowns; there are things we know we

know. We also know there are known unknowns; that is to say weknow there are some things we do not know. But there are also

unknown unknowns -- the ones we don't know we don't know."

-- Don Rumsfeld

Daniel Guggenheim

Risk Identification Decisions:


43/127


Of Technology

ggSchool ofAerospace Engineering

Risk Identification Decisions:Identify Risk Early

Early Risk Identification Enables Good Design Decisions.

Daniel Guggenheim



44/127


Of Technology

School ofAerospace Engineering

Study historic sources of performance, safety, cost, and schedule risks

NASA/DoD/Industry lessons learned databases (e.g., LLIS, REDSTAR)

CAIB Report and other event investigations

Systematically parse project/system WBS looking for risk drivers:

New or adapted technology/designs

Significant design challenges due to complexity or high level of integration Harsh or new operational environments

Optimistic design assumptions and inadequate design margins

Inadequate testing

Historic root sources of unreliability for your mission/system (RoSA)

Systematically examine Risk Breakdown Structure from ESMD Risk

Management Plan, Section 4.2.5, for issues, or other check list

Systematically examine mission timeline and major events (e.g., EDL,

rendezvous, deployments) for potential failures (perform PRA).

Risk Identification Decisions:How Do I Identify Risks?

Daniel Guggenheim



45/127


Of Technology


Systematically examine project schedule, comparing allocations to

historic norms (especially software and integrated test and evaluation.

Systematically examine project cost estimates, comparing allocations

to historic norms.

Systematically examine project staffing plan and labor estimates,

comparing allocations to historic norms Systematically examine all margins (e.g., mass, power, Isp), factors of

safety, flight performance reserves, manufacturing tolerances, etc. and

compare to historic norms.

Have manufacturing and operations personnel participate in and

systematically examine all aspects of the system design and conceptof operations to identify potential risks.

Perform concurrent design, create environment that fosters open

communication, and listen to all team members.

Risk Identification Decisions:How Do I Identify Risks?

Daniel Guggenheim


46/127


Of Technology

School ofAerospace Engineering Continuous Risk Management Process

Early Risk Identification Enables Good Design Decisions.

Daniel GuggenheimS h l f Risk Analysis Decisions:


47/127


Of Technology


Risk Exposure = Probability x Impact

Probabilityof

Occurrenc

e

Impact if Occurs

Low Medium High

Low

Medium

High

Risk Analysis Decisions:How Do I Assess and Prioritize Risks?

How do you decide level of

impacts for prioritization? Impact against what?

> FOMs!

This is multi-attributedecision problem

Level of analysis depends onresources and importance

> From pros/cons to MAUT

> From expert judgment toquantitative analysis

Need integrated systems

analysis capability How do you decide level of

probability for prioritization?

Experience/Databases

Expert Elicitation

QRA/PRA

Daniel GuggenheimS h l f Risk Analysis Decisions:


48/127


Of Technology


Risk Analysis Decisions:Probability Assessment

Probability

Rating

Ordinal

Value

Description

Very Low 1 Qualitative: Very unlikely to occur, management not required in most cases. Strong controls in place.

Quantitative: P< 10-5 (for risks with primary impact on Safety) or P


49/127


Of Technology


Risk Analysis Decisions:Impact Assessment

Daniel GuggenheimSchool of Risk Analysis Decisions:


50/127


Of Technology


Risk Analysis Decisions:Impact Assessment

Daniel GuggenheimSchool of Example: ELP Top Project Risks


51/127


Of Technology


RankIRMAID No. Risk Title (Owner)

RiskType

1 1154 Launch Vehicle Operability (J. Reuter) Perf

2 1118Ability for CLV to Meet PerformanceRequirements (J. Reuter) Perf

3 1128 J2X Development Schedule (J. Snoddy) Sch

4 1113 Requirements Maturation (J. Reuter) Sch

5 1155 Enhanced Flight Termination System (R. Burt)Cost/S

ch

6 1151Human Space Flight Development Summary (A.Priskos) Sch

7 1158 Fault Tolerance Requirements (J. Reuter) Sch

8 1156 Vehicle Controllability (J. Reuter) Perf

9 1159Inability to meet Earth Departure Stage (EDS)loiter time requirements (P. Sumrall) Perf

10 1152Ability of Heritage Hardware to Meet New CLVRequirements (R. Burt)

Cost/Sch

11 1114 Transition Between CLV and SSP (D. Dumbacher)Cost/S

ch

12 1116 Engineering Tools, Models and Processes (N.Otte) Perf

Example: ELP Top Project RisksAugust 10, 2006

Top Directorate Risk (TDR)

Proposed Top Director Risk (P-TDR)

Top Program Risk (TPR)

Proposed Top Program Risk (P-TPR)

Top Project Risk (TProjR)

Proposed Top Project Risk (P-TProjR)

Top Element Risk (TER)

Proposed Top Element Risk (P-TER)

1116

1114

1152

112811131151

11541118

1158

1155

Performance Cost

Schedule Safety

11561159

Likelihood

Consequences

5

4

3

2

1

1 2 3 4 5

812

11

10

4

3,5, 6

1,2

9

7

Daniel GuggenheimSchool of

E l CX IRMA Ri k 1118 S R t


52/127


Of Technology

School ofAerospace Engineering Example: CX IRMA Risk: 1118 Summary Report

Open Date: 7/10/2006 Status as of 10/30/2006 ECD: 1/1/2007

Risk Title: Ability for CLV to Meet Performance Requirements

Escalation Level: TPR

Risk Rank: 2

Owning WBS Element:ARES_I_VEH_INT

Risk Owner: James Reuter

Risk Statement:

Given the history of vehicle and payload growth; there is a possibility that the inability tomaintain the performance and margins needed to meet performance requirements.Children - ARM # 1596, 1563, 1358, 1099, 1606

5 - Likelihood

Consequence(s)

0 - Safety

4 - Performance

0 - Schedule

3 - Cost

Context:

(Imported from ARM Risk 1006) The CLV may not be able to meet mass and

performance requirements. These requirements are not yet well defined, other technicalrequirements may impact this further.

Flights Affected:

Status:

10/4/2006 The Performance Enhancement Team (PET) has began a trades optionanalysis study in order to mitigate this risk. The study results will provide the bestmitigation strategy to buy down this risk.

WBS Element Affected:

CLV



53/127


Of Technology

School ofAerospace Engineering Continuous Risk Management Process


Daniel GuggenheimSchool of Risk Planning Decisions: How Do I


54/127


Of Technology


Examine approaches to reduce BOTH the probability and impact of

identified risk Determine potential impact on any FOMs if risk occurs (e.g. payload)

Systematically look at mitigation options for offsetting effects on FOMs

Systematically examine all other design variables/assumptions, requirements,

or other control parameters that affect FOM (e.g., influence diagram) Must also look at effects of employing options on OTHER FOMs

Select option(s) that offset impact on desired FOM with minimal impact on

other FOMs including risk

This is a multi-attribute decision problem!

Also examine options to reduce probability that risk will occur Systematically examine all events in schedule/mission timeline or other

assumptions that affect probability (e.g., PRA, IMS, influence diagram)

Again, must also look at effects of employing options on ALL FOMs

Scope level of effort/methods to resources and criticality

Risk Planning Decisions: How Do IDecide Best Approach to Mitigate Risks?

Daniel GuggenheimSchool of Risk Planning Decisions:


55/127


Of Technology


gELO Performance Risk Mitigation

ELO #2 Risk: Ares I Ability to Meet Performance Requirements

Given the history of vehicle and payload growth, the concern is the

inability to maintain the performance and margins needed to meetpayload requirements and affordability goals.

Ares I configuration evolved from 4-Segment, SSME to 5-SegmentBooster, J2-X (January 2006)

Received challenge by Cx to provide additional payload capability

Developed weight allocation challenges to elements to meetperformance requirements while retaining performance margins

LI

Likeliho

od

Consequences

L = Lunar

I = ISS

DAC-1 conducted for SRD requirements feasibility and design maturation Weight allocations not yet met

Payload performance met only by use of performance margin

Performance Enhancement Team (PET) established to reduce or mitigate Ares I

Performance Risk Tasked to identify and evaluate candidate design refinements geared to meet or

exceed payload requirements without significant impact to system safety, cost,schedule, operability

Provide recommendations to the Project to support the DAC-2 configuration decision

Identify threats of successfully meeting technical and programmatic requirements withthe reference configuration

Daniel GuggenheimSchool of Risk Planning Decisions:


56/127


Of Technology

Aerospace Engineering

gELO Performance Risk Mitigation



57/127


Of Technology

Aerospace Engineering Continuous Risk Management Process


Daniel GuggenheimSchool of Risk Tracking and Control Decisions:


58/127


Of Technology


Continue risk identification, assessment/prioritization, and mitigation

planning process Continuous Risk Management Keep and update prioritized list and assess progress at reducing top

risks on regular basis in database (e.g., ARM)

Decide to reduce or increase risk exposure score as necessary using

processes discussed above Continually track progress at meeting requirements through allocated/

decomposed TPMs and integrated/validated systems analysis tools

Continually track readiness of critical technologies through TPMs

Use systems engineering database (e.g. Cradle) to link risks to TPMs

and requirements

Use logic-linked integrated master schedule

Use earned value management system to evaluate cost vs. budget

Use expert elicitation approaches to evaluate progress (see below)

Risk Tracking and Control Decisions:How Do I Track and Control Risks?



59/127


Of Technology

Aerospace EngineeringRisk Tracking and Control Decisions:

Sample Technical Performance Measures



60/127


Of Technology

Aerospace EngineeringRisk Tracking and Control Decisions:

Sample TPM Tracking Approaches

Daniel GuggenheimSchool ofA E i i

Risk Tracking and Control Decisions:


61/127


Of Technology


Technical Approach Overview for Technology Project

Tech AssessmentGuidelinesFrom theProgram

TechnologyProject DataReleased toExpert Team

TechnologyProject Personnel

Input toTeam Discussions

ReportResults

DefineRisk Assessment

Process andProvide SW Tool

Form IndependentExpert RiskAssessment

Teams

Establish RiskAssessmentCriteria andCollect Data

Expert TeamReaffirmsTPMs &

Reviews/DiscussesAvailable Data

Expert Team ProvidesCollaborative

Risk AssessmentUsing ITRACS

Process Expert Input Expert Team Reviews

DataITRACSInternet AccessibleSoftware

TeleconferencingSystem

Expert Elicitation Process (UAH/SAIC)

Daniel GuggenheimSchool ofA E i i



62/127


Of Technology


Mechanics of the Process

Select Technology to be assessed Select Technical or Programmatic Risk Metric to be assessed

Assessing the selected TPM or Programmatic Risk Metric:Given the available information and data on the Technology Development, and considering allthe risk assessment criteria, what numerical interval is most likely to contain the outcome tobe achieved for this metric? And what is the relative likelihood of the other potential outcome

intervals compared to the most likely interval?

Metric Interval Most Likely Relative Likelihood

20 to 25 units 5% as likely as 35 to 40

25 to 30 25% as likely as 35 to 40


35 to 40 100% (most likely interval)






63/127


Of Technology


Mechanics of the Process

Metric Interval Probability Distribution

20 to 25 units .02425 to 30 .09530 to 35 .357

35 to 40 .47640 to 45 .0481.000

ITRACS combines all the individual evaluators inputs to produce anormalized collaborative probability distribution:

The collaborative probability distribution coupled with the metric goal, isused to calculate the estimated risk of not achieving the development goal:

.6

.4

.2

020 25 30 35 40 45

Metric Value

Metric Goal

Risk Area (24%)

< 38





64/127


Of Technology



Perform initial assessment at project start (or during prioritization)

Use process to perform project audits at scheduled review periods




65/127


Of Technology


Probability of Success

Expected Value Mean oraverage value of the

estimated probability

distribution. It is the value

of the metric expected by

the evaluators

Expected Value Deviation

Deviation of the EV from the

goal, calculated as follows:

Absolute Value: EV Goal

Goal

A minus sign in front of the

calculated value indicates that

the EV is worse than the goal.

Assumption: The Low to High range contains

100% of the possible values of the metric.

SAICITRACS



Steps in Expert Elicitation Processes


66/127


Of Technology


(General)

EPA Expert Elicitation Task Force White Paper January 2009


Expert Elicitation Processes


67/127


Of Technology


(Cooke Approach)



68/127


Of Technology


Uncertainty and Margin in

Design

Daniel GuggenheimSchool ofAerospace Engineering Margin and Risk


69/127


Of Technology


Judicious use of MARGIN is the most important risk mitigation strategy!

Includes cost, schedule, and performance/weight margins

Use of margin is necessary but not sufficient for risk management

Still need to identify and mitigate root causes of risk

Increasing margin decreases risk, but at the expense of other FOMs

May decrease performance/payload or increase cost at some point Eventually reaches diminishing returns in customer value proposition

Finding the right balance between margin/risk and other FOMs is, once

again, a multi-attribute decision problem

How do I know how much increasing margin decreases risk?

Through probabilistic analysis using historical data regression or expertelicitation coupled with Monte Carlo simulation

How do I know how much increasing margin affects other FOMs?

Through integrated systems analysis capability or expert elicitation

Margin and Risk



70/127


Of Technology

p g g

Weight/Performance Margins and

Uncertainty

Daniel GuggenheimSchool ofAerospace Engineering Uncertainty Risk and Weight


71/127


Of Technology

p g g Uncertainty, Risk, and Weight

Weight is a key control variable during system design/development

Key performance driver (e.g., payload, range)

Can be traded for other FOMs (e.g., higher factors of safety, moreredundancy for reliability, cheaper but heavier materials for cost)

Finding the right balance between weight margin/risk and other FOMsis, once again, a multi-attribute decision problem

Why dont things weigh what I predicted? Inadequate model fidelity and human errors (I forgots)

Weight growth due to integration effects during development

> Brackets, welds, joints, integrated acoustics/vibration/thermal loads

Manufacturing tolerances and constraints on manufacturability

Ground operations requirements not modeled (access panels, space) Uncertainty/inadequate modeling of operational environment

Modifications to balance other FOMs (cost, safety, risk)

Changing requirements

Daniel GuggenheimSchool ofAerospace Engineering Determining Weight Margins


72/127


Of Technology

Determining Weight Margins

How do I mitigate risk of weight growth?

Gather as much relevant historical data as possible to improve models Use high-fidelity models to capture integrated weights and loads

Include manufacturing and operations personnel as an integral part of thedesign team and LISTEN to them

Gather as much data as possible on operational environment

Find the right balance between weight margin/risk and other FOMs upfront through integrated systems analysis

Spend adequate time up front defining requirements and DONT change

THEN, provide adequate weight margins!

How do I decide on adequate weight margins?

Depends on level of analysis/modeling used to derive weight prediction Gather as much relevant historical weight and growth data as possible

Include margin allocations for contingency for non-modeled items, weightgrowth during development, uncertainty in operating environments/loads,manufacturing/operations tolerances, life and factors of safety

Use probabilistic methods to capture historic knowledge


Spacecraft Weight DefinitionSPEC MIL M 38310A


73/127


Of Technology

MAXIMUM

Limit

Nominal

Target

Verified Uncertainty Manufacturing Variation

Allowance for adverseconditions

Criteria Changes

Growth

Contingency

Estimates Parametric StudiesBased on AssumedDesign Criteria

WeightIncrement

SPEC MIL-M-38310A

Daniel GuggenheimSchool ofAerospace Engineering Mass Margin Definition (JPL)


74/127


Of Technology

g ( )

Dry massCurrent Best Estimatetaking into accounteverything known

Spacecraft Dry MassCurrent Best Estimate

Spacecraft Mass Margin

SpacecraftDry Mass Allocation

Propellant(s) Sized for SpacecraftDry mass allocation

SpacecraftWet (Gross) Massallocation

Payload allocationAvailable fro Launch Vehicle

Launch Vehicle Margin (may be zero)

SpacecraftMass(normalized)


Mass Properties Control (JPL)*


75/127


Of Technology*Design, Verification/Validation and Operations Principles for Flight Systems (D-17868), Rev. 2Neil Yarnell, Mar 03, 2003

Ample margins enable risk management- balanced risk management is necessary to enable success.- prudent to have ample mass and power resources to account for and accommodate

uncertainties and expected growth.- ample mass and power resources in conjunction with ample funding resources provide flexibility

to resolve developmental and operational issues, and enable timely,

balanced risk management decisions without having to perform time-consuming trade studiesto micro-manage every kg. and watt.

Preliminary Missions

and Systems Review

Preliminary

DesignReview

Critical

DesignReview

Mass

(kg)

Launch

Mass GrowthMass Allocation

98% Mass Allocation

Mass Current Best Estimate

Mass Margin Requirement- Accommodates

mass growth forknowns and unknowns

30%

20%

10%

2%

Daniel GuggenheimSchool ofAerospace Engineering Mass Properties Control (AIAA)*


76/127


Of Technology

Mass Properties Control (AIAA)

*

Recommended Practice for Mass Properties Control for Satellites, Missiles,and Launch Vehicles, AIAA/ANSI R-020A-1999

Design Maturity

Structure

ThermalControl

Propulsion

Batteries

WireHarness

Mechanisms

Instrumentation

ElectricalCompo

nents

Estimated

(preliminary sketches)18 18 18 20 50 18 50 15

Layout(or major modification of

existing hardware)12 12 12 15 30 12 30 15

Pre-Release Drawings

(or minor modification of

existing hardware)8 8 8 10 25 8 25 10

Released Drawings

(calculated value)4 4 4 5 5 4 5 5

Existing Hardware(acutal mass from another

program)2 2 2 3 3 2 3 3

Actual Mass

(measured flight hardware)0 0 0 0 0 0 0 0

Customer Supplied

Equipment0 0 0 0 0 0 0 0

Percent Mass Growth Allowance


(AIAA Mass Properties Control for Space Systems, 2006)


77/127


Of Technology

G i I i


NASA SpacecraftWeight Growth History


78/127


Of TechnologyWeight Growth History

Pre-Phase A 25-35%

Phase A 25-35%

Phase B 20-30%Phase C 15-25%

NASA Design Margins

for Spacecraft

0 1 2 3 4 5 60.9

1.0

1.1

1.2

1.3

1.4

1.5

Mercury

X-20

Apollo LMX-15

Apollo CSM

Gemini

Time, years

S

tatusWeight/OriginalWeight

0 1 2 3 4 5 60.9

1.0

1.1

1.2

1.3

1.4

1.5

Mercury

X-20

Apollo LMX-15

Apollo CSM

Gemini

Time, years

S

tatusWeight/OriginalWeight

G i I t i t t


Aerospace VehicleWeight Growth History


79/127



1) Apollo CM 20) MGS

2) Apollo LM 21) Peacekeeper3) ASSET 22) PILOT4) Atlas III 23) PRIME5) Atlas V 24) Shuttle ET6) B-9U 25) Shuttle Orbiter7) Classified Program A 26) Skylab8) Classified Program B 27) Titan I9) Classified Program C 28) Titan II SLV10) Classified Program D 29) Titan III B

11) Classified Program E 30) Titan III C12) Classified Program F 31) Titan III D/E13) Classified Program G 32) Titan 34 D14) Classified Program H 33) Titan IV15) Clemintine 34) Viking16) Gemini 35) X-15A-217) H-33 36) X-3318) L-1011 37) X-34

19) Mercury 38) XB-70A

Lowest Weight Growth = 7%Average Weight Growth = 27.5%

Highest Weight Growth = 55%

G i I t i t t


Aerospace VehicleWeight Growth History By Category


80/127


Of TechnologyWeight Growth History By Category

0

10

20

30

40

50

60

CommercialAircraft

Fighters LaunchVehicles

HumanIn-Space

X-Vehicles

C-131F-106DC-10

DC-8DC-9

Concorde F-111

F-102

F-101

Saturn I S-I

Saturn V S-IISaturn V S-IV

STS OrbiterX-37

X-20

XB-70X-33

Apollo LM

Gemini

Apollo CSMMercury

Skylab



Aerospace Vehicle Mass GrowthCumulative Probability Distribution


81/127


Of TechnologyCumulative Probability Distribution

Mass Growth, Percent

0

0.2

0.4

0.6

0.8

1

0 10 20 30 40 50 60

Probability

50% Probability or less -> 28%




< = . 5

95.0%


82/127





Space Shuttle GrowthPhase C/D (1972-1983)


83/127


Of Technology

Wing 0.27Tail 0.14

LH Tank 0.13

LOX Tank 0.13

Body 0.03Gear 0.06

TPS 0.01

Propulsion 0.12Subsystems 0.50

Isp, sec -2.5

Phase C/D (1972-1983)

Percent



Historical Weight EstimatingRelationship (Wing)


84/127


Of Technology

y = 3079.7x0.5544

10000

100000

1 10 100 1000

Weight,

lbs

Wing Weight =30790.554

(1+.20)

Shuttle

H-33, Phase B Shuttle

NAR, Phase B Shuttle

747

C-5

L-1011

737

727-200

707

DC-8-17%

+20%

-

.17

Design Weight*Maneuver Load*Safety Factor*Structural Span

Root Chord

( )

Relationship (Wing)


Daniel GuggenheimSchool ofAerospace Engineering Weight Uncertainty Models


85/127


Of Technology

Normal(0, .113) Trunc(-.4,+inf) Shift=+.015X


86/127


Of TechnologyTriangular Distribution

Triang(-.17, 0, .2)X


87/127


Of TechnologyMonte Carlo Simulation

Triang(-.17, 0, .2)X


88/127

g

Of Technology

Dry Weight = 339Klbs 25% with 90% Confidence


Daniel GuggenheimSchool ofAerospace Engineering Weight Uncertainty Impacts


89/127

g

Of Technology

0%

20%

40%

60%

80%

100%

250000 300000 350000 400000 450000

Dry Weight, lbs

CumulativeProbability

Mean = 340Klbs

95% = 426Klbs



Probabilistic Weight Tracking High Speed Research Program


90/127

g

Of TechnologyHigh Speed Research Program

96 97 98 99 00 01

R

elativeMTOW

Weight

Year

0.8

1.0

1.2

1.4

Assessment Baseline

5-95th PercentileUncertainty Band

Benchmark


Daniel GuggenheimSchool ofAerospace Engineering Performance Margins Other Than Weight


91/127

Of Technology

Flight Performance Reserves

Specific Impulse Margin Mixture Ratio Bias

Maximum Temperature Margin

Acoustic/Vibration Margins

Maximum Pressure (Yield and Burst) Margin

Maximum Loiter Time

Launch Window/Availability Margin

Flight Control Margins

Power Margins

Delta-V Margins Payload Margin

Tank Ullage Margin

Boundary Layer Transition Margins

Etc.




92/127

Of Technology

Cost Margins and Uncertainty


Daniel GuggenheimSchool ofAerospace Engineering Methods of Developing Cost Estimates


93/127

Of Technology

Bottoms-Up Detailed Engineering Build-up

The separate elements are identified in great detail and summedinto the total cost.

Very complex for new systems since costs of development andproduction are unknown

Top-Down Parametric or Statistical Regression analysis is used to establish relations between cost

and initial parameters of the system, e.g. weight, size, speed,power, SLOC, etc.

where xi are the parameters, c and the exponents are determinedby regression of historical data.

Used in conceptual design Analogy

Future costs of a new project are based on costs of old projectswith allowances for cost escalation and complexity differencesbased on simple multiplication factors.

Cost, new = (x * Cost, old)

1 2Exponent Exponent

component 1 2Cost = c x x


Of T h l


NASA/Air Force Cost Model (NAFCOM)


94/127

Of Technology

Parametric cost model based on122 NASA and Air Force space

flight hardware projects

Launch Vehicles

Robotic Satellites

Human-Rated Spacecraft

Space Shuttle

Recent updates based onbenchmarking activity withcontractors, internal assessment

NAFCOM customers

MSFC, NASA HQ, IPAO, other NASAcenters

NAFCOM is used by over 800 civilservants and government contractors

NASA/Air Force Cost Model (NAFCOM)


Of T h l

Daniel GuggenheimSchool ofAerospace Engineering NAFCOM CER Complexity Modeling


95/127

Of Technology

Complexity Generator CERs Multi-variable equations based on sophisticated statistical

analysis of the NAFCOM data base

Identified 73 key technical and programmatic cost drivers,such as

> Funding availability

> Risk management> Integration complexity

> Pre-Development study

> New design

> Weight

> Structural efficiency

> Output Power

> Number of Transmitters

> Stabilization type

> Etc.


Of T h l


Modeling Cost Risk With CERs


96/127

Of Technologyg

$

Cost Driver (Weight)

Cost = a + bXc

Inputvariable

CostEstimate

Historical data point

Cost estimating relationship

Standard percent error boundsTechnical Uncertainty

Combined CostModeling and Technical

Uncertainty

Cost ModelingUncertainty


Of Technology

Daniel GuggenheimSchool ofAerospace Engineering Cost Cumulative Distribution

Function (CDF)


97/127

Of Technology

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

55 65 75 85 95 105 115 125 135 145 155 165 175 185 195 205 215 225

$M

Cumulativ

eProbability

Function (CDF)

80th

percentile$146M

95th percentile$184M

50% probability of cost coming in at or below $115M45% probability of cost coming in between $115M and $184M

20% probability of cost exceeding $146M5% probability of cost exceeding $184M

Mean

$120M

50th percentile$115M


Of Technology


Operations Cost Risk Hidden Costs


98/127

Of Technology

Direct (Visible) WorkTip of the Iceberg

Indirect (Hidden)

Support (Hidden)

+

+

Recurr ing Ops $$s

Direct (Most Visible) Work Drives Massive(and Least Visible) Technical &

Administrative Support Infrastructure Example: Direct Unplanned Repair Activity

Drives Ops Support Infra, Logistics,Sustaining Engineering, SR&QA and FlightCertification


99/127

Of Technology

Life Cycle Cost Gets Locked In Earlyusing only Systems Engineering Decomposition


Of Technology


Requirements Cost Risk


100/127

Of Technology

Requirements Cost/Program Cost, percent

0 5 10 15 20

200

160

80

40

0

120

GRO78OMV

TDRSSIRAS

Gali

TETH

EDO (recent start)

ERB77

HST

LAND76

COBESTSLAND78

GRO82ERB80

VOYAGER HEAO

GOES I-MCEN

MARSACTS

CHA.REC

SEASAT UARS

DE

SMM PIONVEN

Ulysses

IUE

ISEE

EUVE/EP

PAY NOWOR

PAY LATER

Targ

etCostOverrun,

Percent


Of Technology

Daniel GuggenheimSchool ofAerospace Engineering Requirements


101/127

Of Technology


Of Technology



102/127

Of Technology

Schedule Margins and Uncertainty


Of Technology

Daniel GuggenheimSchool ofAerospace Engineering Uncertainty, Risk, and Schedule


103/127

Of Technology

Why is it never on time?

I forgots Unaccounted-for interdependencies and temporal linkages

Test failures

Hardware/software integration

Requirements changes

Programmatic, organizational, and funding issues How do I reduce schedule uncertainty and risk?

Gather as much relevant historical relevant schedule data as possibleand use to anchor bottoms-up predictions

Include integration, test, manufacturing and operations personnel in

schedule development and LISTEN to them Use logic-linked, integrated master schedule software (e.g., Primavera)

Focus on critical path and top events that could get on critical path

Perform probabilistic analysis using historical data or expert elicitation

Provide adequate schedule margin based on probabilistic data


Of Technology


NASA Program Schedule DurationsFrom Red Star Database


104/127

gy

0 20 40 60 80 100 120 140

Mars Exploration Rover

Gemini - Manned

Skylab Workshop - Manned

Centaur-G' - Launch Vehicle

Voyager - Unmanned

Viking Lander - Planetary

Magellan - Planetary

Viking Orbiter - Unmanned

Apollo LM - MannedS-IVB - Launch Vehicle

Apollo CSM - Manned

Mars Observer - Unmanned

Skylab Airlock - Manned

S-II - Launch Vehicle

External TankShuttle Orbiter - Manned

Spacelab - Manned

Months

PDRCDR

DDTE

PDRCDRFirst Flight


Of Technology


NASA Programs/Projects DurationProbability Distribution


105/127

gy

InvGauss(5.4174, 18.7886) Shift=+0.88261

0.00

0.05

0.10

0.15

0.20

0.25

2 4 6 810

12

14

< >5.0%90.0%

2.98 11.91

All ProgramsMean = 6.3 years


Of Technology


NASA Programs/Projects DurationProbability Distributions


106/127

gy

ExtValue(8.1224, 1.7630)

0.00

0.05

0.10

0.15

0.20

0.25

0.30

0.35

5 6 7 8 910

11

12

13

< >5.0% 90.0%6.188 13.359

Gamma(1.9244, 1.5663) Shift=+2.3128

0.00

0.05

0.10

0.15

0.20

0.25

0.30

2 4 6 810

12

14

>5.0%90.0%2.82 9.55

Manned ProgramsMean = 9.1 years

Unmanned ProgramsMean = 5.3 years


Of Technology


Perform Probabilistic Critical PathAnalysis on Logic-Linked IMS


107/127


Of Technology

Daniel GuggenheimSchool ofAerospace Engineering Cost and Schedule Interactions


108/127

Program decision makers need understanding of how uncertainties in

costs and schedule interact Might choose a high risk schedule to meet a hard cost target

Might be willing to have higher costs to ensure meeting a launch date


Of Technology


Capturing Cost andSchedule Uncertainties


109/127

Difference between conditional median cost of ($107.8M) given a

schedule of 53 months and conditional median cost ($87.4M) given ahigh-risk schedule of 43 months is over $20M

This could be very significant to a decision maker who wished totrade cost, schedule, and risk

Use joint probability models to analyze cost-schedule interactions


Of Technology



110/127

Technology Risk Mitigation


Of Technology

Daniel GuggenheimSchool ofAerospace Engineering Technology Risk Mitigation Approach


111/127

Select Technologies that are Evolutionary -- Not Revolutionary

No Major Breakthroughs Required

Define Low-Risk Back-Ups for Each Technology Project

Includes Fall-Back and Fall-Up Positions (e.g., RLV Composite Tanks)

Mature Key Technologies to TRL Levels 6 or 7 Before ATP Decision

Define TPMs for Each Technology Task and Use to Track Progress

Conduct Risk/Progress Evaluations at Major Technology DevelopmentMilestone Reviews

Track Technology Development Progress Through Changes to TPMs

Evaluate Impact of Technology Progress on System Requirements

Evaluate Technology Development Risk and Take Corrective Actions

> Develop Detailed Risk Mitigation Plans> Introduce Back-up Technologies/Approaches as Needed

> Reallocate Funding as Required

Tools Exist to Facilitate the Process (e.g., Active Risk Manager)


Of Technology



112/127

Risk Leveling


Of Technology

Daniel GuggenheimSchool ofAerospace Engineering Risk Leveling


113/127

How much should we try to reduce a given risk? How safe is safeenough? Is a human life priceless?

In design, one requirement/constraint should not be a significantlylarger driver than others

Question requirementshow cost/benefit of relaxing requirement toDecision Maker

Good design has multiple simultaneous driving requirements/constraints

Process of requirements leveling

Same is true in risk analysismust perform risk leveling

Dont let one or two risk sources dominate or go unaddressed

Dont spend scarce resources trying to reduce one risk to a lower level ororder-of-magnitude than others

Make it safe enough and no safer

How do we know when the risks are leveled

Must have integrated systems analysis capability to model and asses risks

Can perform probabilistic risk analysis (PRA)


Of Technology


ESAS Mission ModeLoss of Crew FOM Results


114/127


Of Technology


Sources of Loss of Crew Risk forESAS Lunar Mission


115/127


Of Technology


Radiation Shielding Design ApproachPrior to ESAS


116/127

0

0.25

0.5

0.75

1

0 5 10 15Added Poly Shield Amount, g/cm^2

Gy-Eq(BFO)

4 X Aug 72 4 X Sep 89 Current LEO Limit

0

0.25

0.5

0.75

1

0 5 10 15Added Poly Shield Amount, g/cm^2

Gy-Eq(BFO)

4 X Aug 72 4 X Sep 89 Current LEO Limit

Based onGraphite (60%)-Epoxy (40%)Vehicle of samemass

Organ Dose 4Xs 1972-SPE Aluminum LSAM LSAM + Poly 5g/cm2

Skin (Gy-Eq) 5.49 5.78 4.05 0.86 0.91 0.67

Eye (Gy-Eq) 4.79 5.05 3.56 0.83 0.88 0.65

BFO (Gy-Eq) 0.86 0.91 0.67 0.24 0.26 0.20

Effective Dose (Sv) 1.08 1.14 0.84 0.28 0.29 0.23

Organ Dose 4Xs 1989-SPE Aluminum LSAM LSAM + Poly 5g/cm2

Skin (Gy-Eq) 0.91 0.96 0.72 0.29 0.31 0.26

Eye (Gy-Eq) 0.82 0.86 0.66 0.29 0.31 0.26

BFO (Gy-Eq) 0.29 0.30 0.26 0.17 0.18 0.16


Organ Dose 4Xs 1972-SPE Composite LSAM LSAM + Poly 5g/cm2

Skin (Gy-Eq) 4.23 4.45 3.09 0.74 0.78 0.56

Eye (Gy-Eq) 3.81 4.01 2.80 0.72 0.76 0.55

BFO (Gy-Eq) 0.73 0.77 0.56 0.21 0.23 0.17


Organ Dose 4Xs 1989-SPE Composite LSAM LSAM + Poly 5 g/cm2

Skin (Gy-Eq) 0.73 0.77 0.58 0.27 0.28 0.24

Eye (Gy-Eq) 0.68 0.72 0.55 0.27 0.28 0.24

BFO (Gy-Eq) 0.27 0.28 0.23 0.16 0.17 0.15


Based onAluminumVehicle


Of Technology


ESAS Analysis Cycle 2Radiation Risk Assessment


117/127

Solar particle event design environmentsbasis and considerations: 99% event for mortality risks (acute or

chronic risks) The August 72 event is generallyaccepted as the benchmark solarparticle event in measurable history.

Ones confidence of not exceeding the72 event fluence level above 30 MeVon a one year mission near the solarmaximum is about 97%.

To achieve 99.5% confidence levelabove 30 MeV one must assume afluence of 4 times the August 72event.

Radiation limits outside LEO do not currentlyexist - being developed by NCRP and theCHMO

LEO Career Limit Probability of 3% additional risk of lifetimelethal cancer within a 95% confidence interval

LEO Blood-Forming Organs (BFO) Short-term Limits 30-day limit - 25 cGy-Eq Annual limit 50 cGy-Eq

N x 1972 Event

2 4

%RiskofFatalCancer

0

4

8

12

16

20

EX CEV baseline

CEV with 5 g/cm2

poly shield

Risk Limit

Females 45-yr (no prior missions)

Polyethylene Augmentation Shield, g/cm2

0 2 4 6 8 10

%RiskofFatalCanc

er

0

4

8

12

16

20

Female 45-yr

Risk Limit

4x1972 Event for EX-CEV Design


Of Technology


ESAS Analysis Cycle 2CEV Acute and Late Risks


118/127

Estimated probability of an SPE that could cause debilitation+ (1.5X Aug 1972event)

Estimated probability of catastrophic event (4X Aug 1972 event)

Recommend maximum of 2 g/cm2 CEV shielding based upon risk leveling for a 16day maximum mission (0.04 year), 0.005 P (exceeding 72 levels), and riskprobabilities given in table below

HDPE Depth (g/cm2) % Acute Death* % Sickness % REID**

0 9.5 54.0 9.1 [3.2,17.3]

2 (0.02) (2.9) 3.8 [1.3,10.5]

5 0 0 1.5 [0.5,4.3]

HDPE Depth (g/cm2) % Acute Death* % Sickness % REID**

0 3.0 34.4 7.6 [2.7,16.7]

2 (0.01) (1.9) 3.4 [1.2,9.6]

5 0 0 1.4 [0.4,3.9]

Aluminum Vehicle, 4X 1972 SPE

Graphite-Epoxy Vehicle, 4X 1972 SPE

Death at 60-days with minimal medical treatment** Risk of Cancer death for 45-yr Females

+Debilitating event identified as dose that would cause vomiting within 2 days in 50% of total population

(99.5% confidence of not exceeding the 72 event fluence levelabove 30 MeV on a one year mission near the solar maximum)

(99.5% confidence of not exceeding the 72 event fluence levelabove 30 MeV on a one year mission near the solar maximum)


Of Technology


ESAS Analysis Cycle 3 SPE Risks versusProbability of SPE Occurrence in a 9-day Mission

2


119/127

Nx1972

Event F(>30 MeV)

%Probability for

9 day mission Acute Death Acute Sickness

Career Limit

Violation

30-Day Limit

Violation

4X 2x10

10

0.02


120/127

70.2

66.5

23.4

21.0

60

62

64

66

68

72

0 1 2 3 4 5

CEV Supplemental Radiation Protection (g/cm2)

TLIInjectedMass(t)

20

21

22

23

24

25

26

27

28

29

30

CEVMass(t)

CEV Mas s

I n jec ted Mass

RECOMMENDATION:

Eliminate supplementalradiation shielding

70 Injected Mass Sensitivity:~740 kg per g/cm2


Of Technology



121/127

Probabilistic Risk

Assessment


Of Technology

Daniel GuggenheimSchool ofAerospace Engineering Probabilistic Risk Assessment


122/127

The human mind cannot grasp the causes of phenomena in

the aggregate. But the need to find these causes is inherent inmans soul. And the human intellect, without investigating the

multiplicity and complexity of the conditions of phenomena,

any one of which taken separately may seem to be the cause,

snatches at the first, the most intelligible approximation to acause, and says: This is the cause!

Leo Tolstoy,

War and Peace

Scenario Development is used in risk analysis to facilitate thesystematic search for the causes of risk.


Of Technology

Daniel GuggenheimSchool ofAerospace Engineering Probabilistic Risk Assessment (PRA)


123/127

PRA provides thorough, quantitative scenario-based approach toassessing the probability that a risk will occur and its consequences

Used on Space Shuttle (Fragola) and ISS (Futron) Flow between process steps

Master Logic Diagram> Identifies how hazards are controlled

Functional Event Sequence Diagram

> Shows how the system responds to off normal events Event Trees

> Inductively models that represent the way pivotal events can combine inresponse to specific initiating events

Fault Trees> Deductive models that generate logical combinations of failures that can

cause a specific high level pivotal event Each technique addresses a part of the risk assessment problem

In combination, allow analyst to accurately and completely representsmajority of risk

Joseph Fragola NIA Risk-Based Design Short Course


Of Technology



124/127



Of Technology



125/127

High Mixture RatioNot Detected

Failure in Channel A

Erroneous Signal inChannel A

Harness Failure inChannel A

Logic Control Failurein Channel A

Failure in Channel B

Erroneous Signal inChannel B

Harness Failure inChannel B

Logic Control Failurein Channel B

Loss-of-Vehicle

LOV due toOrbiter Failure

LOV due toSolid Rocket

Booster Failure

LOV due toMain Engine

Failure

Loss ofContainment

Loss ofPropulsion

Loss of

Hydrogen Flow

High MixtureRatio in the

Fuel Preburner

Loss ofPressure in the

MCC

Loss of GrossHydrogen

Flow

Lower flowratetriggers active

computer controlsequence

Controller IncreasesOxidizer Flow to Fuel

PreburnerYes Yes

High Mixture RatioDetected

High Mixture Ratio inthe Fuel Preburner

No

High Mixture Ratio inthe Both Preburners

Yes S/D

No

LOV

Master Logic Diagram

Functional Event Sequence Diagram

Event Tree

Fault TreeJoseph Fragola NIA Risk-Based Design Short Course


Of Technology



126/127

Steps in PRA Process



Of Technology



127/127

Where does data come from?

Date post:	05-Apr-2018
Category:	Documents
Upload:	winacore306
View:	217 times
Download:	0 times

Uncertainty Course Stanley Wilhite Part 1.Aspx

Documents