PMI College of Scheduling‘’PMI’ is a registered trade and service
mark of the Project Management institute, Inc.
May 9, 2006
Quantitative Cost and Schedule
Risk Assessment and Management
for Large Infrastructure Projects
RT-05
Bill Roberds, Sc.D.
Travis McGrath, Ph.D., P.E.
Golder Associates, Inc.
Redmond, WA
PMI College of Scheduling‘’PMI’ is a registered trade and service mark of the Project Management institute, Inc.
2May 9, 2006
Introduction
Cost and Schedule overruns are common on large infrastructure projects. Historically1:
•90% of estimates for transportation infrastructure projects have been low
•On average, cost estimates for road projects have been 20% short of final costs
•Estimates for some projects have been off by 100% or more (e.g., the “Big Dig”)
1. Flyvbjerg et.al., Journal of the American Planning Association, Summer, 2002.
PMI College of Scheduling‘’PMI’ is a registered trade and service mark of the Project Management institute, Inc.
3May 9, 2006
Project Cost and Schedule Uncertainty
Poor cost and schedule estimates lead to:
•Cost and schedule over-runs or under-runs
•Poor decisions (i.e., among alternatives)
•Resource competition among projects
•Media attention
•Public mistrust (and therefore funding problems)
PMI College of Scheduling‘’PMI’ is a registered trade and service mark of the Project Management institute, Inc.
4May 9, 2006
Probabilistic Cost and Schedule Risk Assessment and Risk Management
(RA/RM)
Mitigate these problems by:
•Quantifying project cost and schedule uncertainty
• Identifying and prioritizing critical risks and key opportunities
•Quantifying potential benefits of risk-management strategies
• Improving project understanding and communication
• Increasing confidence in estimates and budgets
PMI College of Scheduling‘’PMI’ is a registered trade and service mark of the Project Management institute, Inc.
5May 9, 2006
Outline
•Key Concepts
•Process and Examples
•Performance to Date
•Challenges for Better Implementation
PMI College of Scheduling‘’PMI’ is a registered trade and service mark of the Project Management institute, Inc.
6May 9, 2006
Key Concepts
•Comprehensive look at the project at appropriate level of detail
•Quantify uncertainty in key project assumptions
•Utilize independent perspective (for validation)
•Employ a collaborative, team approach
•Achieve consensus (when possible)
•Focus attention on the key issues
•Update as the project changes significantly
PMI College of Scheduling‘’PMI’ is a registered trade and service mark of the Project Management institute, Inc.
7May 9, 2006
Key Concepts, cont’d
•Quantify uncertainty in project cost and schedule, which is a function of:
– Sequence of all significant project activities
– “Base” activity costs and durations
– Risks, Opportunities, and other uncertainties
•Prioritize critical activities, risks, and opportunities
• Identify and evaluate strategies to improve project performance
PMI College of Scheduling‘’PMI’ is a registered trade and service mark of the Project Management institute, Inc.
8May 9, 2006
Describing Uncertainty
0%
25%
50%
75%
100%
Cumulative Distribution Function (CDF)
Cu
mu
lati
ve
Pro
bab
ilit
yValue
Range in values
Rel
ativ
e L
ikel
ihood
Probability Density Function (PDF)
Standard
Deviation
Mean
(Expected)
Value
Percentiles
10th 90th
80% confidence interval
PMI College of Scheduling‘’PMI’ is a registered trade and service mark of the Project Management institute, Inc.
9May 9, 2006
“Base + Risk” Approach versus Traditional Single-Valued Approach
Cost0
Rel
ati
ve
Lik
elih
oo
d
Aggregate Risk and Opportunity
Base (with uncertainties)
“Base + Risk”
Traditional
Contingency
“below the line”
Traditional
Conservative
“above the line”
Traditional
Single-valued
(Deterministic) Estimate
+ =
PMI College of Scheduling‘’PMI’ is a registered trade and service mark of the Project Management institute, Inc.
10May 9, 2006
Outline
•Key Concepts
•Process and Examples
•Performance to Date
•Challenges for Better Implementation
PMI College of Scheduling‘’PMI’ is a registered trade and service mark of the Project Management institute, Inc.
11May 9, 2006
Steps in Quantitative Cost and Schedule Risk Assessment and Risk Management
Structuring: Review Project Plan, Develop “Flow Chart,” and Document Assumptions
Review Cost and Schedule Estimates
Develop Risk Register
Assess Risk Factors
Quantify Uncertainty and Sensitivity in Cost,
Schedule, and Other Measures
Develop Cost and Schedule
Uncertainty Model
Update Periodically or at Key Milestones
Identify and Evaluate Risk-Management Strategies
Develop Base Factors
Project team
implements
mitigation
strategy and
monitors
performance
Workshop setting
Collaborative effort
PMI College of Scheduling‘’PMI’ is a registered trade and service mark of the Project Management institute, Inc.
12May 9, 2006
Steps: Structuring the RA/RM
Structuring: Review Project Plan, Develop “Flow Chart,” and Document Assumptions
• Project team presents current plan
• Clarify project scope and delivery strategy
• List all significant project assumptions
• Develop project “flow chart” for each alternative:
– Sequence of major project activities (all precedence requirements)
– Based on team’s strategy and schedule
– Reasonableness / accuracy check on schedule logic
– Appropriate level of detail
– Basis for integrated cost and schedule model
PMI College of Scheduling‘’PMI’ is a registered trade and service mark of the Project Management institute, Inc.
13May 9, 2006
Example Project “Flow Chart”
14
Widen Existing
Embankment
15
Construct New
Embankment
16
Remaining Bridge
Construction
19
Demo Old
Bridge
22
Remove Old
Alignment
23
Close-out
Items
24
Complete
25
Open to
Traffic
21
Bridge Approaches
17
In-Water
Bridge
Construction
18
S + 1 month
Paving
20
Develop
Alternatives
2Design
and ROW
Funding
1Env Resource
Inventory
3
Design to 10%
4
Prepare
Resource
Reports
5
Environmental
Doc + Approval
6
Design to 60%
8
Finalize Design
and PS&E
9
Acquire
ROW
11
Permitting
12
Ad / Bid /
Award / NTP
14
Construction
Funding
13
14
F-1 month
Relocate
Utilities
10Design to 30% +
Design Approval
7
S+50%
S+50%Programmed:
1/1/2005
Programmed:
1/1/2007
Pre-Construction Activities
Construction Activities
14
Widen Existing
Embankment
15
Construct New
Embankment
16
Remaining Bridge
Construction
19
Demo Old
Bridge
22
Remove Old
Alignment
23
Close-out
Items
24
Complete
25
Open to
Traffic
21
Bridge Approaches
17
In-Water
Bridge
Construction
18
S + 1 month
Paving
20
Develop
Alternatives
2Design
and ROW
Funding
1Env Resource
Inventory
3
Design to 10%
4
Prepare
Resource
Reports
5
Environmental
Doc + Approval
6
Design to 60%
8
Finalize Design
and PS&E
9
Acquire
ROW
11
Permitting
12
Ad / Bid /
Award / NTP
14
Construction
Funding
13
14
F-1 month
Relocate
Utilities
10Design to 30% +
Design Approval
7
S+50%
S+50%Programmed:
1/1/2005
Programmed:
1/1/2007
Pre-Construction Activities
Develop
Alternatives
2Design
and ROW
Funding
1Env Resource
Inventory
3
Design to 10%
4
Prepare
Resource
Reports
5
Environmental
Doc + Approval
6
Design to 60%
8
Finalize Design
and PS&E
9
Acquire
ROW
11
Permitting
12
Ad / Bid /
Award / NTP
14
Construction
Funding
13
14
F-1 month
Relocate
Utilities
10Design to 30% +
Design Approval
7
S+50%
S+50%Programmed:
1/1/2005
Programmed:
1/1/2007
Pre-Construction Activities
Construction Activities
Critical path
PMI College of Scheduling‘’PMI’ is a registered trade and service mark of the Project Management institute, Inc.
14May 9, 2006
Steps: Reviewing Project Estimates
Structuring: Review Project Plan, Develop “Flow Chart,” and Document Assumptions
Review Cost and Schedule Estimates
• Collaborative effort
• Confirm that cost matches scope
• Confirm that schedule matches scope
• Check reasonableness of design and strategy
• Identify and remove contingency and conservatism
• Note: Estimate is for the ultimate cost and schedule
PMI College of Scheduling‘’PMI’ is a registered trade and service mark of the Project Management institute, Inc.
15May 9, 2006
Steps: Defining the “Base”
Structuring: Review Project Plan, Develop “Flow Chart,” and Document Assumptions
Review Cost and Schedule Estimates
Develop Base Factors
• The “Base”
– The planned scope and strategy, with no problems
– Unbiased, excluding contingency and/or float
• “Map” the “base” estimates to flow chart activities
• For each flow chart activity, develop:
– Base cost (current $)
– Base duration (calendar time)
– Cost-escalation rate
• Include significant uncertainties and correlations
PMI College of Scheduling‘’PMI’ is a registered trade and service mark of the Project Management institute, Inc.
16May 9, 2006
Example Base Factors (excerpt)
Flowchart Number
Project Activity Base Cost 1
(2005 $M)
Base Duration 1
(months)
Average Escalation
Rate 1 (%/yr) 0 Costs to Date 0.11 -
1 Design and ROW Funding milestone -
2 Develop Alternatives 0.02 1.0 3.0
3 Env. Resource Inventory 0.02 0.5 3.0 4 Design to 10% 0.30 6.0 3.0
5 Prepare Resource Reports 0.10 3.0 3.0 6 Environmental Doc + Approval 0.05 2.0 3.0
7 Design to 30% + Approval 0.30 4.0 3.0
… … … … … 14 Ad / Bid / Award / NTP 0.05 2.5 3.0
15 Widen Existing Embankment 1.30 2.0 3.0 16 Construct New Embankment 1.85 3.5 3.0
17 Bridge Approaches 1.75 3.0 3.0 18 In-Water Bridge Construction
0.37 0.75 3.0
19 Remaining Bridge Construction
2.13 3.0 3.0
… … … … … 24 Close-out Items 0.57 1.0 3.0 25 Complete milestone -
Note 1. May be uncertain and described with probability distributions; only the expected values are shown here.
PMI College of Scheduling‘’PMI’ is a registered trade and service mark of the Project Management institute, Inc.
17May 9, 2006
Steps: Developing the Risk Register
Structuring: Review Project Plan, Develop “Flow Chart,” and Document Assumptions
Review Cost and Schedule Estimates
Develop Risk Register
Develop Base Factors
• Identify all potential events - risks and opportunities -outside the “base” assumptions
• Comprehensive and non-overlapping
• Appropriate level of detail
• Include significant relationships among risks
• Consider possible causes and effects
PMI College of Scheduling‘’PMI’ is a registered trade and service mark of the Project Management institute, Inc.
18May 9, 2006
Steps: Quantifying the Risks
Structuring: Review Project Plan, Develop “Flow Chart,” and Document Assumptions
Review Cost and Schedule Estimates
Develop Risk Register
Assess Risk FactorsDevelop Base Factors
For each risk and opportunity, quantify:
• Consequences if event occurs
– Changes in costs and durations to affected activities
– Other measures?
• Likelihood of occurrence, consistent with consequences
• Include significant uncertainties and correlations
• Ensure consistent with “base”
PMI College of Scheduling‘’PMI’ is a registered trade and service mark of the Project Management institute, Inc.
19May 9, 2006
Steps: Quantifying the Risks, cont’d
Structuring: Review Project Plan, Develop “Flow Chart,” and Document Assumptions
Review Cost and Schedule Estimates
Develop Risk Register
Assess Risk FactorsDevelop Base Factors
Risk-factor assessments:
• Based on objective analysis when data are available
• Based on subjective assessments (from the experience and judgment of SMEs) if data are limited
•When properly elicited, subjective assessments have been shown to be accurate, defensible, and ‘the best we can do’
PMI College of Scheduling‘’PMI’ is a registered trade and service mark of the Project Management institute, Inc.
20May 9, 2006
Example Event Tree for Structuring Risks and Opportunities
Vertical
Concept &
Location
1. Tunnel
P1
3. Bridge /
Aerial
P3 = 1-P1-P2
2. At-grade
P2
Structure
Type A. Cut-and-Cover
PA|1
B. Bored / Mined
PB|1 = 1- PA|1
C. Concrete Box
PC|3
D. Steel Plate Girder
PD|3 = 1 - PC|3
Note: Probability of alternative concepts may consider factors other than cost or duration
Probability ∆∆∆∆$ ($M) ∆∆∆∆Months
PA|1P1 0 (“base”) 0
PB|1P1 +5 +2
P2 -4 -3
PC|3P3 -2 -1
PD|3P3 -3 -1
PMI College of Scheduling‘’PMI’ is a registered trade and service mark of the Project Management institute, Inc.
21May 9, 2006
Example Risk Register and Risk Factors (excerpt)
Item Risk or Opportunity Affected Project
Activities
Probability of
Occurrence
Cost Change (2005 $M)
Duration Change
(months)
D8
Project alignment different than assumed Includes related design, ROW, and construction impacts Potential mutually-exclusive outcomes include:
A. EIS alignment (“base”), which passes through town on existing roadway alignment
B. Alternative alignment #1, which bypasses town to north on new alignment
C. Alternative alignment #2, which passes through town on new alignment to east of EIS alignment
60% Design and
ROW Acquisition
Discrete distribution: A. 40%
(Base) B. 40% C. 20% Cost change perfectly correlated to duration change
A. 0 (Base) B. 1.0 to
Design; -5 to ROW C. 1.0 to
Design; Uniform
(2, 10) to ROW
A. 0 (Base) B. 6 to
Design; -3 to ROW C. 6 to
Design; Uniform
(3, 9) to ROW
P4
Delays in required transportation agency approvals Several agencies have review and approval authority over the design. Delays in approval are possible from delays in review cycle.
60% Design
Discrete distribution: A. 10% (Base) B. 60% C. 30%
Additional escalation costs are simulated as a function of duration change
Discrete distribution: A. 0 (base) B. 3 C. 6
PMI College of Scheduling‘’PMI’ is a registered trade and service mark of the Project Management institute, Inc.
22May 9, 2006
Steps: Developing the Integrated Cost and Schedule Model
Structuring: Review Project Plan, Develop “Flow Chart,” and Document Assumptions
Review Cost and Schedule Estimates
Develop Risk Register
Assess Risk FactorsDevelop Base Factors
Develop Cost and Schedule
Uncertainty Model
• Probabilistic computer model
• Can include
– Shut-down “windows”
– Decisions and other “triggered”actions
– Funding or other resource constraints
– Uncertainty in critical path
– Delay-related costs
PMI College of Scheduling‘’PMI’ is a registered trade and service mark of the Project Management institute, Inc.
23May 9, 2006
Steps: Quantifying Uncertainty in Project Cost and Schedule
Structuring: Review Project Plan, Develop “Flow Chart,” and Document Assumptions
Review Cost and Schedule Estimates
Develop Risk Register
Assess Risk FactorsDevelop Base Factors
Quantify Uncertainty and Sensitivity in Cost,
Schedule, and Other Measures
Develop Cost and Schedule
Uncertainty Model
• Combine base and risk factors in the model
• Simulate results of interest
PMI College of Scheduling‘’PMI’ is a registered trade and service mark of the Project Management institute, Inc.
24May 9, 2006
Integrated Cost and Schedule Model
Activity “Base” Costs (and Uncertainties)
Activity “Base” Durations (and Uncertainties)
Activity A
Activity C
Activity B
Start End
Uncertainty in
Total Escalated
Project Cost
and Duration, plus
Sensitivities
$
T
7 56
12111 0
8 4
21
9 3
_
7 56
1 21 11 0
8 4
21
9 3
_
Risk Events (likelihood of occurrence, and uncertainty
in activity cost and duration changes if the event occurs)
Event Y
Event X
time
escalation
cash flow
PMI College of Scheduling‘’PMI’ is a registered trade and service mark of the Project Management institute, Inc.
25May 9, 2006
Example Monte Carlo Sample Set Showing Cost and Schedule Correlation
Dec-11
Dec-12
Dec-13
Dec-14
Dec-15
Dec-16
Dec-17
Dec-18
Dec-19
Dec-20
Dec-21
0 200 400 600 800
Total Project Cost ($M)
4th
Bo
re O
pen
ing
Da
te
unescalated
escalated
Dec 2021
Dec 2020
Dec 2019
Dec 2018
Dec 2017
Dec 2016
Dec 2015
Dec 2014
Dec 2013
Dec 2012
Dec 2011
PMI College of Scheduling‘’PMI’ is a registered trade and service mark of the Project Management institute, Inc.
26May 9, 2006
Example CDF for Project Cost Showing Contributing Factors
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
2,4
00
2,5
00
2,6
00
2,7
00
2,8
00
2,9
00
3,0
00
3,1
00
3,2
00
3,3
00
3,4
00
3,5
00
3,6
00
3,7
00
3,8
00
3,9
00
4,0
00
4,1
00
4,2
00
4,3
00
4,4
00
4,5
00
Total Project Cost (2005 $Million or YOE $Million)
Cu
mu
lati
ve
Pro
ba
bil
ity
(Per
cen
tile
)Base (with no risk or opportunity; in 2005 $)
With Risk and Opportunity, but Without Escalation (2005 $)
With Risk and Opportunity and Escalation (Year-of-Expenditure $)
With Risk and Opportunity and $400M/year Funding Constraint (YOE $)
Mean risk and
opportunity
Mean cost
escalation
Mean
funding
constraint
PMI College of Scheduling‘’PMI’ is a registered trade and service mark of the Project Management institute, Inc.
27May 9, 2006
Example CDF for Project Schedule Showing Contributing Factors
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%Ja
n 2
01
5
Jan
20
16
Jan
20
17
Jan
20
18
Jan
20
19
Jan
20
20
Jan
20
21
Jan
20
22
Jan
20
23
Jan
20
24
Overall Project Completion Date
Cu
mu
lati
ve
Pro
ba
bil
ity
(Per
cen
tile
)Base (with no risk or opportunity)
With Risk and Opportunity but Without Funding Constraint
With Risk and Opportunity and $400M/year Funding Constraint
Mean risk and
opportunity
Mean impact of funding
constraint
PMI College of Scheduling‘’PMI’ is a registered trade and service mark of the Project Management institute, Inc.
28May 9, 2006
Example Probability of Meeting Specified Target
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
2,4
00
2,5
00
2,6
00
2,7
00
2,8
00
2,9
00
3,0
00
3,1
00
3,2
00
3,3
00
3,4
00
3,5
00
3,6
00
3,7
00
3,8
00
3,9
00
4,0
00
4,1
00
4,2
00
4,3
00
4,4
00
4,5
00
Total Project Cost (2005 $Million or YOE $Million)
Cu
mu
lati
ve
Pro
ba
bil
ity
(Per
cen
tile
)Base (with no risk or opportunity; in 2005 $)
With Risk and Opportunity, but Without Escalation (2005 $)
With Risk and Opportunity and Escalation (Year-of-Expenditure $)
With Risk and Opportunity and $400M/year Funding Constraint (YOE $)
Probability of
meeting $3.7B
budget = 85%
Probability of
meeting $3.7B
budget = 37%
Alternatively,
could identify required
budget for desired
confidence level
(e.g., 80th percentile)
PMI College of Scheduling‘’PMI’ is a registered trade and service mark of the Project Management institute, Inc.
29May 9, 2006
0
500
1,000
1,500
2,000
2,500
3,000
3,500
4,000
2005 2007 2009 2011 2013 2015 2017 2019 2021
Year
Pro
ject
Ca
sh F
low
Wit
ho
ut
$4
00
M/y
ear
Fu
nd
ing
Co
nst
rain
t (Y
OE
$M
)
Cumulative - 90th Percentile
Cumulative - Mean
Cumulative - 10th Percentile
Annual - 90th Percentile
Annual - Mean
Annual - 10th Percentile
Example Uncertainty in Cash Flow of Expenditures
(Unconstrained Funding Scenario)
PMI College of Scheduling‘’PMI’ is a registered trade and service mark of the Project Management institute, Inc.
30May 9, 2006
Example Impact of Scenario Assumptions on Uncertainty in Cash Flow
0
100
200
300
400
500
600
700
800
2005 2007 2009 2011 2013 2015 2017 2019 2021 2023 2025
Year
An
nu
al
Ca
shfl
ow
(Y
OE
$M
)
No Funding Constraint - 90th Percentile
No Funding Constraint - Mean
No Funding Constraint - 10th Percentile
Funding Constraint - 90th Percentile
Funding Constraint - Mean
Funding Constraint - 10th Percentile
Constraint delays
peak expenditure
by 5 years
Constraint
reduces peak
expenditure
by $270M
at the mean However, constraint
increases total cost
significantly
(increased schedule
and thus escalation)
PMI College of Scheduling‘’PMI’ is a registered trade and service mark of the Project Management institute, Inc.
31May 9, 2006
Steps: Quantifying Sensitivity in Project Cost and Schedule
Structuring: Review Project Plan, Develop “Flow Chart,” and Document Assumptions
Review Cost and Schedule Estimates
Develop Risk Register
Assess Risk FactorsDevelop Base Factors
Quantify Uncertainty and Sensitivity in Cost,
Schedule, and Other Measures
Develop Cost and Schedule
Uncertainty Model
• Evaluate sensitivity of cost and schedule to the uncertain “base”and “risk” factors:
– Mean
– Standard Deviation
– Percentile (combination)
• Prioritize critical activities and risks in costs and time
PMI College of Scheduling‘’PMI’ is a registered trade and service mark of the Project Management institute, Inc.
32May 9, 2006
Example Sensitivity of 80th Percentile of Project Cost to Uncertain Inputs
34.7
23.2
16.3
11.8
11.7
7.8
6.7
6.5
6
5.2
4.9
3.9
3.8
3.6
2.9
2.4
2
1.9
1.8
1.7
1.6
1.6
1.3
1.2
1.1
1.1
1
1
0 5 10 15 20 25 30 35 40
B.3. Limited Number of Qualified Bidders
D.26. Extended Pre-Construction Admin Costs
C.35. Extended Construction Admin Costs
Identified Minor Risks (aggregate)
Unidentified Risks (aggregate)
E.1. Challenge to ROD/FONSI due to public opposition
C.25. Unanticipated Mitigation of Construction Traffic
B.11. Non-Responsive Low Bid
D.7. Unanticipated Mitigation for Impacts Outside of ...
C.14. Valid Differing Site Condit ions (DSC) (e.g., ...
C.7. Inexperienced Contractor (Separate from C.5 ...
D.5. More Fire Protection / Ventilat ion Required
D.14. Conversion from Metric to English Units
C.5. Contractor Unfamiliarity with CalTrans Admin...
W Tunnel
C.15. Tunnel Face Instability / Excessive Inflow
E Tunnel
C.29. Unanticipated Work Stoppage due to Adverse...
E.5. Unexpected / Late Number of Public Comments ...
B.6. Poor Material Prices and Availability
C.20. Slope Instability / Retaining Wall Failure
B.5. Opposition from Local Contractors / Business...
D.16. Problems in Design Coordination (Including ...
D.17. Design Changes / Breakage (Separate from ...
E13. Additional Impact Analysis
August 2005$ / non-ROW escalat ion rate
C.16. Tunnel Collapse During Construction
D.4. Change in Design Standards (e.g., Seismic) ...
Uncertain Factors (base
uncertainty or
risk/opportunity)
Contribution ($M) to 80th Percentile
of Escalated Project Cost
PMI College of Scheduling‘’PMI’ is a registered trade and service mark of the Project Management institute, Inc.
33May 9, 2006
Steps: Managing Risk
Structuring: Review Project Plan, Develop “Flow Chart,” and Document Assumptions
Review Cost and Schedule Estimates
Develop Risk Register
Assess Risk Factors
Quantify Uncertainty and Sensitivity in Cost,
Schedule, and Other Measures
Develop Cost and Schedule
Uncertainty Model
Identify and Evaluate Risk-Management Strategies
Develop Base Factors
Project team
implements
mitigation
strategy and
monitors
performance
• Identify potential risk-management actions (e.g., prevention; mitigation)
• Quantify likely implementation costs
• Quantify likely benefits
• Simulate net effect
• Ties in to Value Engineering
PMI College of Scheduling‘’PMI’ is a registered trade and service mark of the Project Management institute, Inc.
34May 9, 2006
Risk Management
Performance (e.g., cost)
Relative
Likelihood Total with
Risk Management
Risks and Opportunities with
Risk Management
Note: Base costs include mitigation implementation costs
Base (with uncertainties; now
including RM costs)
PMI College of Scheduling‘’PMI’ is a registered trade and service mark of the Project Management institute, Inc.
35May 9, 2006
Example Risk Management: Mitigated Cost Uncertainty (if Mitigation Succeeds)
0
0.05
0.1
0.15
0.2
32
0
33
0
34
0
35
0
36
0
37
0
38
0
39
0
Total Project Cost (Future $M)
Pro
ba
bil
ity
Mitigated (Workshop Plan)
Unmitigated (July 19 2003)
The mitigation planwould save$10M at the owner’s desired confidence level (80th percentile)
PMI College of Scheduling‘’PMI’ is a registered trade and service mark of the Project Management institute, Inc.
36May 9, 2006
Example Risk Management: Mitigated Schedule Uncertainty
0%10%20%
30%40%50%60%70%
80%90%
100%
Jun
20
08
Sep
20
08
Dec
20
08
Mar
20
09
Jul
20
09
Oct
20
09
Jan
20
10
May
20
10
Revenue Service Date
Cu
mu
lati
ve
Pro
ba
bil
ity
(Per
cen
tile
)
July 2004 Mitigated
July 2004 Unmitigated
The mitigation planwould save7-8 months at the owner’s desired confidence level (80th percentile)
PMI College of Scheduling‘’PMI’ is a registered trade and service mark of the Project Management institute, Inc.
37May 9, 2006
Outline
•Key Concepts
•Process and Examples
•Performance to Date
•Challenges for Better Implementation
PMI College of Scheduling‘’PMI’ is a registered trade and service mark of the Project Management institute, Inc.
38May 9, 2006
Performance
• Not many projects evaluated using this approach have been completed
• Can’t fully evaluate performance (yet)
• As an interim measure, compared results from risk assessments to published costs for completed projects
• Favorable reaction from public (i.e., public “ok”with ranges expressing uncertainty)
• Improved project team understanding
• Focused risk management
• Improved confidence in cost and schedule estimates, however…
PMI College of Scheduling‘’PMI’ is a registered trade and service mark of the Project Management institute, Inc.
39May 9, 2006
Performance: Bias Comparisons
•Flyvbjerg et. al. (2002) calculated inaccuracy (bias) in cost estimates for completed projects as the percent difference between actual (ultimate) project cost and estimated cost at the
time of decision to build
•As an interim measure, the authors calculated the “apparent bias” in project team estimates relative to the mean RA/RM results for projects still in design as follows:
[mean unescalated cost from risk assessment – team cost estimate (unescalated)]
team cost estimate (unescalated)
PMI College of Scheduling‘’PMI’ is a registered trade and service mark of the Project Management institute, Inc.
40May 9, 2006
Performance: Results from Risk Assessments vs. Published Results for Completed Projects
0.0
0.1
0.2
0.3
0.4
0.5
0.6
-40 -20 0 20 40 60 80 100 120 140 160 180 200 220 240
Bias in Cost Estimate (%) 1
(numbers are upper bounds for bins)
Fre
qu
ency
From 62 Risk Assessments (2002 to 2005) on
Northwestern US Road Projects
After Flyvbjerg et. al. (2002) for 167
Completed Road Projects (Worldwide)
Mean = 20%
Standard deviation = 30%
Mean = 14%
Standard deviation = 24%
0.0
0.1
0.2
0.3
0.4
0.5
0.6
-40 -20 0 20 40 60 80 100 120 140 160 180 200 220 240
Bias in Cost Estimate (%) 1
(numbers are upper bounds for bins)
Fre
qu
ency
From 62 Risk Assessments (2002 to 2005) on
Northwestern US Road Projects
After Flyvbjerg et. al. (2002) for 167
Completed Road Projects (Worldwide)
Mean = 20%
Standard deviation = 30%
Mean = 14%
Standard deviation = 24%
Note: Larger variability in completed projects likely reflects larger data set and variability among agencies
Similar partial validation of approach
PMI College of Scheduling‘’PMI’ is a registered trade and service mark of the Project Management institute, Inc.
41May 9, 2006
Performance: Apparent Bias in Cost Estimate versus Design Level
-40
-20
0
20
40
60
80
100
Design Level at Time of Risk Assessment
Bia
s in
Co
st E
stim
ate
(%
) 1
From 62 Risk Assessments on
Northwestern US Road Projects
Conceptual Preliminary Final
Mean
Mean plus one standard deviation
Mean minus one standard deviation-40
-20
0
20
40
60
80
100
Design Level at Time of Risk Assessment
Bia
s in
Co
st E
stim
ate
(%
) 1
From 62 Risk Assessments on
Northwestern US Road Projects
Conceptual Preliminary Final
Mean
Mean plus one standard deviation
Mean minus one standard deviation
PMI College of Scheduling‘’PMI’ is a registered trade and service mark of the Project Management institute, Inc.
42May 9, 2006
Outline
•Key Concepts
•Process and Examples
•Performance to Date
•Challenges for Better Implementation
PMI College of Scheduling‘’PMI’ is a registered trade and service mark of the Project Management institute, Inc.
43May 9, 2006
Challenges for Better Implementation
• If properly implemented and averaged over many projects, risk-based approaches appear to “get closer” to the ultimate project cost than does deterministic estimating,
•However, RA/RM is not perfect:
– Bids for some recent projects evaluated using probabilistic RA/RM (by the authors and by others) have fallen outside the RA/RM ranges
– Effort for RA/RM for some projects has been excessive
•This implies that probabilistic RA/RM still needs improvement
PMI College of Scheduling‘’PMI’ is a registered trade and service mark of the Project Management institute, Inc.
44May 9, 2006
Challenges for Better Implementation
1. Convincing the owner that the RA should include all significant uncertainties
• Excluding major uncertainties makes the results:
– Conditional on key assumptions (often forgotten)
– Underestimate the true range of uncertainty, and perhaps inaccurate as well
• Examples of major uncertainties that are often excluded for political or other reasons:
– Uncertainty in the amount or timing of funding
– Potential delays due to process and management inefficiencies
– Uncertainty in key policies within stakeholder orgs
– Uncertainty in project scope (i.e., evaluating a particular scope instead of all possible project scopes)
PMI College of Scheduling‘’PMI’ is a registered trade and service mark of the Project Management institute, Inc.
45May 9, 2006
Challenges for Better Implementation
2. Removing optimism and/or “management bias” (e.g., hidden agendas) from assessments
• Consensus is desirable, but should not be achieved at the expense of accuracy
• If necessary, risk assessors should infuse their experience from other projects to de-bias and/or expand ranges of critical assessments to ensure reasonable results
PMI College of Scheduling‘’PMI’ is a registered trade and service mark of the Project Management institute, Inc.
46May 9, 2006
Challenges for Better Implementation
3. Making results of RA/RM understandable and useful to the owner
• Make sure the RA addresses the owner’s specific needs
• Educate the owner on how to interpret and use the results
• Don’t let the owner assign more certainty or precision to the results than is justifiable
• Qualify any limitations to avoid misunderstanding
PMI College of Scheduling‘’PMI’ is a registered trade and service mark of the Project Management institute, Inc.
47May 9, 2006
Summary
•Probabilistic risk assessment and risk management for large infrastructure projects can cost-effectively provide
– Better understanding and communication of project uncertainties
– More confidence in estimates
– Better project management and decision making
•However, careful (and better) implementation is required to ensure accurate and defensible results
PMI College of Scheduling‘’PMI’ is a registered trade and service mark of the Project Management institute, Inc.
48May 9, 2006
Dr. Travis McGrath, P.E.
Golder Associates, Inc.
18300 NE Union Hill Rd, Suite 200
Redmond, WA 98052
(425) 883-0777
Contact Information