Post on 06-May-2015
transcript
1
Jairus Hihn
Leigh Rosenberg
November 01, 2011
Jairus Hihn
Leigh Rosenberg
November 01, 2011
Team X Risk Tool Team X Risk Tool AndAnd
Team X Cost Risk ToolTeam X Cost Risk ToolOverviewOverview
2
Methodology
Team X focuses on risk identification and initial assessment In the early life cycle it is difficult to crisply distinguish between a risk, an issue, or a concern Items on the Team X Risk List are those items that the team feels are significant enough that the
customers’ or reviewers’ attention is required Many of these items can be addressed by adding detail or specificity to the proposal
Risk process: Prior to the study the Risk Chair reviews the Team X Risk Checklist and identifies potential risk
items Subsystems initiate/revise/reject proposed risks Subsystems score the risk on their subsystem list using the rating scale described in 8.1.2 Risk Chair reviews risks and talks to each subsystems engineer to clarify risk descriptions and
their scores as needed, enters system level scores (what you primarily see) After the study Risk Chair with key subsystems and the facilitator scrubs risks for consistency in
wording and scoring Construct Risk Adjusted Probabilistic Cost Estimate (Also known as S-Curve)
Background: Team X RAP Tool
3
Publications
• “Identification And Classification Of Common Risks In Space Science Missions”, Jairus Hihn, Debarati Chattopadhyay, Robert Hanna, Daniel Port, Proceedings AIAA Space 2010 Conference and Exposition, 1-3 September, Anaheim, CA.
• “Risk Identification and Visualization in a Concurrent Engineering Team Environment”, Jairus Hihn, Debarati Chattopadhyay, Robert Shishko, Proceedings of the ISPA/SCEA 2010 Joint International Conference, June 8-11, 2010, San Diego, CA.
• “Risk Mental Models in Concurrent Engineering Teams”, USC Systems and Software Cost Modeling Workshop, October 2010.
4
Example Risk Checklist: Propulsion
5
• Checklist of common risks developed for each subsystem, through review of a subset of prior Team X studies
• Checklists validated during interviews with Team X subsystem chairs
• Use of checklists during Team X studies revealed: Lists were useful to Risk chair Subsystem chairs felt the general
lists were long, should be tailored to the specific study
Would be easier to use if built into tool
Translation of impact and likelihood ratings into Red-Yellow-Green for NASA 5x5 risk matrix
6
New Risk Scoring Guidance
Risk Chair Master Sheet
7
Step 1
8
Risk Chair Goes through Template and identifies likely risksSelect copy risk tabSelect/import risks from
a template or studyBoth general risk and
subsystem risks
Risk Chair opens up Study Risks Window Initially blank In end summarizes all risks
once populated
Example S-Curve from NASA CEH
9
Go back to Add/Edit TabSelect Add hit edit to add additional
risks not in imported list
Select Add hit edit to add additional
risks not in imported list
Select risk and hit edit to revise, add details and score
Select risk and hit edit to revise, add details and score
All updates done here
10
Risk-Adjusted Probabilistic Cost Estimate Methodology
Estimate/Model Uncertainty
Estimated schedule risk based on inputs from Mission Design
Technical risks based on key risks based on risks identified by Team X
Risk-Adjusted Probabilistic Cost Estimate
Convolve Yields
11
Risk-Adjusted Probabilistic Cost Estimate Methodology
Each WBS level 2 item has its own distributional CER
Each WBS line item has an underlying algorithm such as
Where model error is captured through a normal distribution on the standard errors of the algorithm’s coefficients (the βs) and input uncertainty is captured by ranges on the inputs.
A Monte Carlo is run on these coefficients and variables
Each WBS’s distribution is then convolved using full correlation in a Monte Carlo run to produce a single total cost distribution
This provides a CDF that represents the model and cost driver uncertainty
nnWBS xxxCost
i
21210
12
Risk-Adjusted Probabilistic Cost Estimate MethodologySchedule Risk
Schedule Schedule distribution is derived from analysis and historical data Likelihood of slip is based on analysis of 19 historical JPL in-house and contracted missions Impact is based on Team X effort profiles and mission design determination of months
between launch opportunities Launch opportunities identifed by Mission Design
13
Risk Idenitification and Scoring
EHM Flyby is a relatively low risk mission compared to other similar space science missionsSC has relatively high heritageModerate number of instruments
There is one significant risks that need to be addressed ASRG performance and delivery
date of flight ready is still highly uncertain
Specific mitigations are not identified but the impact is based on a best estimate for the cost impact should the risk manifest.
14
Implementation Risks
L
i
k
e
l
i
h
o
o
d
5
4
3
2 R:1 R:1
1
1 2 3 4 5
Impact
R:1
R:2
Example EHM Orbiter
Estimate uses parametric cost model based on the Team X 50th-percentile estimate
Cost risk analysis indicates that proposed mission has a high likelihood of success Estimated cost with
reserves is 70% to 76%.. Typical NASA goal is 70%.
Identified risks consumes less than 1/3rd of planned reserves leaving sufficient reserves to cover ‘unknown-unknowns’
The 50th percentile team X estimate becomes 36% when the identified risks are taken into account
Risk-Adjusted Probabilistic Cost Distribution (S-Curve)
15