Date post: | 02-Jan-2016 |
Category: |
Documents |
Upload: | gregory-wilson |
View: | 214 times |
Download: | 1 times |
AMERICA’S ARMY:THE STRENGTH OF THE NATION
Mort [email protected]
1
Cost Risk and Uncertainty Analysis
MORS Special Meeting | 19-22 September 2011 Sheraton Premiere at Tysons Corner, Vienna, VA
AMERICA’S ARMY:THE STRENGTH OF THE NATION
AS OF: 04/20/23 02:54
The Need For: Without risk analysis, a cost estimate will usually be a point estimate, which may not account for all
of the uncertainties inherent in the effort Not accounting for potential uncertainties may lead to underfunding, cost overruns, and the
potential for a program to be reduced in scope in the future
Definitions: Cost risk and uncertainty analysis identifies the cost, in terms of dollars, time, and materials that
should be added to a point estimate to increase the probability of meeting the desired outcome Risk is the occurrence of an outcome subject to a known pattern of random variation (i.e., Known-
Unknown). Changes in those technical parameters that are captured in the estimating methodology Uncertainty is the occurrence of an outcome subject to an unknown random fluctuations (i.e.,
Unknown-Unknown). Changes in those parameters that are not part of the estimating methodology
Analysis Methodology:1. Identify Areas of Uncertainty2. Address the Uncertainty within the Cost Estimate3. Quantify the Risk within the Cost Estimate4. Presentation of Results
2
Cost Risk and Uncertainty Analysis
AMERICA’S ARMY:THE STRENGTH OF THE NATION
Technical Programmatic Cost Schedule
Physical Properties Material Availability Sensitivity to Assumptions
Degree of Concurrency
Material properties Skill Requirement Sensitivity to Technical Risk
Sensitivity to Technical Risk
Software Complexity
Environmental Impact
Sensitivity to Programmatic Risk
Sensitivity to Programmatic Risk
Integration Interface Contractors Stability Sensitivity to Schedule Risk
Sensitivity to Schedule Risk
Requirement Changes
Funding Profile Estimating Errors Number of Critical Path
Operational Environment
Political Advocacy Estimating Errors
Reasons for RiskReasons for RiskReasons for RiskReasons for Risk
3
Cost Risk and Uncertainty Analysis
AMERICA’S ARMY:THE STRENGTH OF THE NATION
Risk ConceptsRisk ConceptsRisk ConceptsRisk Concepts
Severity of Consequence
Pro
babi
lity
of O
ccur
renc
e High RiskLow Risk
Low RiskIncreasing
Moderate Risk
4
Cost Risk and Uncertainty Analysis
AMERICA’S ARMY:THE STRENGTH OF THE NATION
Risk Analysis MethodsRisk Analysis MethodsRisk Analysis MethodsRisk Analysis Methods
Qualitative Methods (e.g., subjective assessments of Low, Medium, or High Risk) are of most use when there is little or no historical data available or firm requirements have not yet been established
Quantitative Methods are considered where probability distribution on cost elements or drivers can be estimated from historical data or deduced from expert opinion
Analytical Method involves the mathematical determination of a total cost distribution from its components cost distributions
Simulation Method involves the computer generation of random costs from component distributions and aggregation into a total cost distribution
5
Cost Risk and Uncertainty Analysis
AMERICA’S ARMY:THE STRENGTH OF THE NATION
AS OF: 04/20/23 02:54
2. Address the Uncertainty within the Cost Estimate:
Need to determine which identified uncertainties to model (i.e., which are cost-sensitive) Cost methodology highly dependant, reliant upon data available Application of uncertainty dependant upon cost methodology
Uncertainty assessment is best done at the lowest-level variables possibleImportant to identify uncertainty associated with cost drivers at a minimum
Selection of appropriate probability distribution for each risk element is critical
7
Figure 1: Common Risk Distributions
Cost Risk and Uncertainty Analysis
AMERICA’S ARMY:THE STRENGTH OF THE NATION
AS OF: 04/20/23 02:54
3. Quantify the Risk within the Cost Estimate:
ODASA-CE utilizes the Automated Cost Estimating Integrated Tools (ACEIT)Integrated within ACEIT is a simulation-based risk analysis capability (RI$K) that allows
the analyst to perform cost, schedule, and technical risk and uncertainty analysisProvides the capability to calculate risk results for various confidence levels using Latin-
Hypercube sampling Input: Specification of risk within ACEIT
Output: ACEIT risk statistics reportDisplays statistical results in Base Year (BY) dollars for each WBS element at the
specified confidence level
8
Figure 2: Specifying a Risk Distribution within ACEIT
Figure 3: Example Statistics Report within ACEIT
Cost Risk and Uncertainty Analysis
AMERICA’S ARMY:THE STRENGTH OF THE NATION
AS OF: 04/20/23 02:54
4. Presentation of Results:
ACEIT can also be used to formulate a Cumulative Density Function (CDF) or S-CurveEach point on the S-Curve identifies the cumulative probability that the associated cost on
the x-axis will not be exceeded. This is referred to as the level of confidence in a particular estimate.
AoA cost estimates are presented as a point estimate at the 50% confidence level accompanied by a risk-informed cost range (±σ)
9
Figure 4: Example S-Curve
0.0%
0.5%
1.0%
1.5%
2.0%
2.5%
3.0%
$40.0 $60.0 $80.0 $100.0 $120.0 $140.0 $160.0 $180.0 $200.0
Prob
abili
ty D
ensi
tyDollars
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
$40.0 $60.0 $80.0 $100.0 $120.0 $140.0 $160.0 $180.0 $200.0
Cum
ulati
ve P
roba
bilit
y
Dollars
Point Estimate 50% Confidence Level 80% Confidence Level
Risk Dollars to 80% “Confidence Level” = $127.4 - $90.0 = $37.4)
Probability Density Function (pdf)
Cumulative Distribution Function (cdf)
“S-Curve”
*Cumulative Probability is often referred to as “Confidence Level”**P.E. = Point Estimate
Allocating Risk Dollars: If the decision is made to add Risk Dollars to an estimate, then those dollars may be allocated across lower level elements if it makes sense to report them as such. Risk Dollars can be allocated to lower level elements in a variety of ways. Confidence for lower level elements often will be different than for higher levels. Risk Dollars must add up to the total Risk Dollars needed to reach the desired Confidence at the higher level .
Phasing Risk Dollars: After Risk Dollars are allocated according to the desired approach, it may be necessary to phase the dollars over time. Risk Dollars can be spread over time using a variety of techniques, including pro-rating dollars according to the profile of the point estimate, defining a curve to match a particular profile, front-loading dollars to early years, back-loading dollars to later years, or manually inputting dollars to hedge against risks in specific years.
Cost Risk and Uncertainty Analysis
AMERICA’S ARMY:THE STRENGTH OF THE NATION
Risk Analysis ApproachesRisk Analysis ApproachesRisk Analysis ApproachesRisk Analysis Approaches
Detail
Effo
rt
• Detailed Network & Stochastic
• Detailed Monte Carlo Simulation (each WBS)
• Bottom Line Monte Carlo Simulation
• Add a Risk Factor/Percentage
• Discrete Technical, Schedule, and Estimating Risks
10
Cost Risk and Uncertainty Analysis
AMERICA’S ARMY:THE STRENGTH OF THE NATION
Probability DistributionsProbability DistributionsProbability DistributionsProbability Distributions
WBS Cost Distribution Total Cost
P.E.
P.E.
P.E.
+
+
=
Sum PE MLC (Mode)
Fre
quen
cy
CDF
Con
fide
nce
50%
20%
Sum PE MLC
11
Cost Risk and Uncertainty Analysis
Many More+
AMERICA’S ARMY:THE STRENGTH OF THE NATION
Probability Density FunctionProbability Density FunctionProbability Density FunctionProbability Density Function
Cell B28 Frequency Chart
Mean = $2,198
4,978 Trials Shown
.000
.006
.012
.018
.024
0
29.2
58.5
87.7
117
$1,250 $1,750 $2,250 $2,750 $3,250
Forecast: Results=
12
Cost Risk and Uncertainty Analysis
AMERICA’S ARMY:THE STRENGTH OF THE NATION
Cumulative Distribution FunctionsCumulative Distribution FunctionsCumulative Distribution FunctionsCumulative Distribution Functions
Cell B28 Cumulative Chart 4,977 Trials Shown
.000
.249
.498
.747
.995
0
4977
$1,250 $1,750 $2,250 $2,750 $3,250
Forecast: Results=
13
Cost Risk and Uncertainty Analysis
AMERICA’S ARMY:THE STRENGTH OF THE NATION
AS OF: 04/20/23 02:54
Cost Schedule Curve Cost Schedule Curve Cost Schedule Curve Cost Schedule Curve
Development Schedule
Lif
e C
ycle
Cos
t
Min Optimal
Typ
ical
Parallel EffortMore ECPLess Mature Design
Fixed CostTechnology Outdate
14
Cost Risk and Uncertainty Analysis
AMERICA’S ARMY:THE STRENGTH OF THE NATION
Schedule GoalsSchedule GoalsSchedule GoalsSchedule Goals
Defense
Development Schedule
Cos
t
Commercial Drivers•Technology Drives Schedule•Constraint Schedule•Goal is ROI Maximization
DoD Drivers•Funding Drives Schedule•Unconstrained Schedule•Goal is Cost Reduction
Commercial
15
Cost Risk and Uncertainty Analysis
AMERICA’S ARMY:THE STRENGTH OF THE NATION
AS OF: 04/20/23 02:54
ConceptConceptRefinementRefinement
System Development System Development & Demonstration& Demonstration
Production & DeploymentProduction & Deployment Operations & SupportOperations & SupportTechnology Technology DevelopmentDevelopment
Concept Decision
Design Readiness Review
FRP Decision Review
A B C FOCIOC
MDAP Process
Rang
e of
Unc
erta
inty
Estimating Accuracy Trumpet
Technical and Program Maturity
RO
M -
30%
to
+75
%
Par
amet
ric
-10%
to
+20
%
Ana
logy
-15
% t
o +
30%
Eng
inee
ring
-5%
to
+15
%
Act
ual
-3%
to +
10%
Most Likely Cost
Target Cost
Point estimates are the most likely within the range estimate
Point estimates are the most likely within the range estimate
Upper bound
Lower bound
+75%
-30%
Range estimates provide risk and uncertainty as well as most likely(more information)
Range estimates provide risk and uncertainty as well as most likely(more information)
16
Cost Risk and Uncertainty Analysis
Range Cost EstimatesRange Cost EstimatesRange Cost EstimatesRange Cost Estimates
AMERICA’S ARMY:THE STRENGTH OF THE NATION
COA-1 highest benefit COA-2 best value COA-3 lowest cost
Cost = $ quantifiable cost – $ quantifiable benefit or saving
Benefit = $ non-quantifiable benefit and $ non-quantifiable risk
CBA provide framework for making resource informed decisions
CBA StepsCBA Steps
Cost Risk and Uncertainty Analysis
17
AMERICA’S ARMY:THE STRENGTH OF THE NATION
Risk analysis reduces the uncertainty between requirements and funding
Cost Risk and Uncertainty Analysis
Risk ReductionRisk Reduction Cost Risk Analysis will not reduce the risk inherent in a program; however
it helps PM to understand the nature of the risk involved, and the uncertainty associated with the cost estimates
More frequent Cost Risk Analysis result in a more realistic assessment of the cost and schedule and the development of effective risk mitigation plan