ALON, Inc.
6800 Versar Center, Suite 303
Springfield, VA 22151
T: (703) 256.9503 F: (703) 256.9508
W W W . AL O N I N C . C O M
Copyright 2011 ALON, Inc., All Rights Reserved
COST ESTIMATING –
WHY HAVEN’T WE
GOTTEN BETTER?
SCOTT PATTON
PMIWDC
FAA LUNCHEON SERIES
DECEMBER 21, 2011
1 COMMITTED TO EXCELLENCE
W W W . AL O N I N C . C O M
Copyright 2011 ALON, Inc., All Rights Reserved
20+ years working in “project management”
– Project Manager. EHS projects in the nuclear weapons complex
– Consultant. Helping clients build project management capabilities
and competencies
– Business Owner. Benchmark assessments and knowledge sharing
– Adjunct Professor. Project management certificate program
10+ years supporting the profession
– PMP, PMI-RMP and OPM3 Professional certifications
– Supported development of the OPM3 ProductSuite assessment methodology
– Knowledge sharing through different venues
SCOTT PATTON
INTRODUCTIONS
2 COMMITTED TO EXCELLENCE
W W W . AL O N I N C . C O M
Copyright 2011 ALON, Inc., All Rights Reserved
Key principles of cost estimating
Challenges in developing estimates
Strategies to consider in developing
reliable estimates
AGENDA
3 COMMITTED TO EXCELLENCE
W W W . AL O N I N C . C O M
Copyright 2011 ALON, Inc., All Rights Reserved
Definitions
Estimating methodologies
Assumption validation
Reserve Analysis
References
COST ESTIMATING PRINCIPLES
4 COMMITTED TO EXCELLENCE
W W W . AL O N I N C . C O M
Copyright 2011 ALON, Inc., All Rights Reserved
To calculate approximately (the amount, extent, magnitude, position
or value of something)
To judge and form an opinion of value from imperfect data
To judge tentatively or approximately the value, worth or significance
DEFINITIONS
COST ESTIMATING PRINCIPLES
In other words . . .
estimating implies an effort to deal with uncertainty
by developing an educated guess that substitutes
for an actual measurement
5 COMMITTED TO EXCELLENCE
W W W . AL O N I N C . C O M
Copyright 2011 ALON, Inc., All Rights Reserved
Estimates define the effort needed to meet project requirements
Estimates set stakeholder expectations as to the size of the effort
Estimates provide a baseline for monitoring and controlling the project
Estimates provide the foundation for trade-off decisions
COST ESTIMATING PRINCIPLES
Performance
Schedule Cost
WHY ESTIMATE?
6 COMMITTED TO EXCELLENCE
W W W . AL O N I N C . C O M
Copyright 2011 ALON, Inc., All Rights Reserved
Commonly used cost estimating techniques
– Analogous
– Extrapolation from actual costs
– Parametric
– Engineering
– Grass Roots
– Delphi
– Round Table
– Expert Opinion
Select cost estimating methodologies and approaches based on availability
of “good” data
– Methodologies selected should be specific to each type of cost
ESTIMATING METHODOLOGIES
COST ESTIMATING PRINCIPLES
7 COMMITTED TO EXCELLENCE
W W W . AL O N I N C . C O M
Copyright 2011 ALON, Inc., All Rights Reserved
Assumption – a way of dealing with an uncertain future when there are
a number of possible options
Document assumptions
Validate assumptions with key stakeholders
Challenge assumptions to expose uncertainties/risks
“IF this assumption proved to be false, THEN the effect on the project
would be …”
DOCUMENT AND VALIDATE ASSUMPTIONS
COST ESTIMATING PRINCIPLES
Source: Society of Cost Estimating and Analysis
8 COMMITTED TO EXCELLENCE
W W W . AL O N I N C . C O M
Copyright 2011 ALON, Inc., All Rights Reserved
Contingency Reserves
– Used for future situations that may be planned for only in part (known unknowns)
– Based on cost risk assessment
– Determined by the quantitative (expected monetary value) assessment of impact and probability
Management Reserves
– Used for future situations that are impossible to predict (unknown unknowns)
– Based on assumption analysis
– May be specifically identified in the budget as “Cost Management Reserve”
– Based on the complexity, size, strategic importance of the project
– Reflective of the risk tolerance of sponsor/organization
RESERVE ANALYSIS
COST ESTIMATING PRINCIPLES
9 COMMITTED TO EXCELLENCE
W W W . AL O N I N C . C O M
Copyright 2011 ALON, Inc., All Rights Reserved
GAO Cost Estimating and Assessment Guide – Best Practices for
Developing and Managing Capital Program Costs; March 2009;
GAO-09-3SP; http://www.gao.gov/new.items/d093sp.pdf
System Engineering for Intelligent Transportation Systems; January 2007; http://ops.fhwa.dot.gov/publications/seitsguide/seguide.pdf
Practice Standard for Project Estimating; 2011; PMI
Identifying and Managing Project Risk by Tom Kendrick; 2009;
American Management Association
Society for Cost Estimating and Analysis (https://www.sceaonline.org/index.cfm)
Others???
REFERENCES
COST ESTIMATING PRINCIPLES
10 COMMITTED TO EXCELLENCE
W W W . AL O N I N C . C O M
Copyright 2011 ALON, Inc., All Rights Reserved
Project Failure
Factors
Caused By Inadequate
Estimates
Causes Estimation
Failures
Rebaselining
Customer Dissatisfaction
Lack of Resources
Unrealistic Expectations
Lack of Executive Support
ACTIVITY: PROJECT FAILURE FACTORS – WHICH
CAUSE OR ARE CAUSED BY ESTIMATION FAILURES?
Place an “X” in the appropriate column(s) for each row
x
x
x
x
x
x
x
x
x
x
11 COMMITTED TO EXCELLENCE
W W W . AL O N I N C . C O M
Copyright 2011 ALON, Inc., All Rights Reserved
UNCERTAINTY
WHAT IS THE CHALLENGE?
THE ESTIMATING CHALLENGE
TECHNICAL
PSYCHOLOGICAL
POLITICAL
MANAGEMENT
12 COMMITTED TO EXCELLENCE
W W W . AL O N I N C . C O M
Copyright 2011 ALON, Inc., All Rights Reserved
Timing of estimates
Data sensitivity
Changes in technology
Understanding complexity
UNCERTAINTY
THE ESTIMATING CHALLENGE
Cone of Uncertainty
Source: ops.fhwa.dot.gov
PMI – Accuracy of Estimates
ROM: +/- 50%
Budget Estimate: -10 to +25%
Definitive Estimate: -5 to +10%
13 COMMITTED TO EXCELLENCE
W W W . AL O N I N C . C O M
Copyright 2011 ALON, Inc., All Rights Reserved
THE ESTIMATING CHALLENGE
Source: Is it really true that these cartoons are public domain or is there somewhere a copyright owner on them?
14 COMMITTED TO EXCELLENCE
W W W . AL O N I N C . C O M
Copyright 2011 ALON, Inc., All Rights Reserved
Scope
– Incomplete requirements
– Misinterpretation of requirements/statement of work
– Improperly defined scope/incomplete work breakdown structure
Cost
– Lack of timely, relevant data
– Applying improper skill levels to tasks
– Failure to understand or account for cost escalation or inflation
– Failure to properly use the selected cost estimation method
– Failure to adequately consider overhead/indirect costs
– Failure to account for total lifecycle costs
TECHNICAL
THE ESTIMATING CHALLENGE
15 COMMITTED TO EXCELLENCE
W W W . AL O N I N C . C O M
Copyright 2011 ALON, Inc., All Rights Reserved
Being overly optimistic – natural tendency
Individuals tend to follow predictable
estimation patterns based on their position in
the organization
– Lower level managers tend to add reserves
and upper levels to delete them
– Acquisition managers tend to delete reserves
and PMs to add them
PSYCHOLOGICAL
THE ESTIMATING CHALLENGE
16 COMMITTED TO EXCELLENCE
W W W . AL O N I N C . C O M
Copyright 2011 ALON, Inc., All Rights Reserved
Decision making and planning are multi-actor
processes with conflicting interests
Budget and acquisition cycles take time
Project scope or ambition level will change
Changes in organizational priorities
Influenced by agendas
POLITICAL
THE ESTIMATING CHALLENGE
17 COMMITTED TO EXCELLENCE
W W W . AL O N I N C . C O M
Copyright 2011 ALON, Inc., All Rights Reserved
MANAGEMENT
THE ESTIMATING CHALLENGE
Planning has not improved in 15 years
0
1
2
3
4
5
1993 1996 1999 2002 2005 2008 (Est)
Risk
Value & Benefits
Planning
© Cooke-Davies and Patton, 2009
Lack of a systematic approach to
estimating
Inappropriate use of tools and
techniques
Varying estimating skill levels
Mangers do their own estimating
regardless of skill level
Decisions are made based on these
estimates
18 COMMITTED TO EXCELLENCE
W W W . AL O N I N C . C O M
Copyright 2011 ALON, Inc., All Rights Reserved
Understand the challenges
Manage bias
Manage uncertainty
Understand political nature of organization
Use sound practices
Build cost estimating capabilities
WHAT ARE THE STRATEGIES?
STRATEGIES FOR RELIABLE ESTIMATES
19 COMMITTED TO EXCELLENCE
W W W . AL O N I N C . C O M
Copyright 2011 ALON, Inc., All Rights Reserved
MANAGE BIAS
STRATEGIES FOR RELIABLE ESTIMATES
Three point estimate
Source: visionarytools.com
20 COMMITTED TO EXCELLENCE
W W W . AL O N I N C . C O M
Copyright 2011 ALON, Inc., All Rights Reserved
Program Evaluation Review Technique (PERT)
MANAGE UNCERTAINTY
STRATEGIES FOR RELIABLE ESTIMATES
CE = (O + 4M + P) ÷ 6
CE = estimated costs
O = most optimistic estimate
M = most likely estimate
P = most pessimistic estimate
21 COMMITTED TO EXCELLENCE
W W W . AL O N I N C . C O M
Copyright 2011 ALON, Inc., All Rights Reserved
Monte Carlo analysis – encompasses any technique of statistical sampling
employed to approximate solutions to quantitative problems
MANAGE UNCERTAINTY
STRATEGIES FOR RELIABLE ESTIMATES
$190,000,000 $200,000,000
Distribution (start of interval)
0
20
40
60
80
100
120
Hit
s
0% $181,419,283
5% $185,618,909
10% $186,690,787
15% $187,509,265
20% $188,176,720
25% $188,863,949
30% $189,432,362
35% $189,850,284
40% $190,317,773
45% $190,818,409
50% $191,418,280
55% $191,893,524
60% $192,332,421
65% $192,824,491
70% $193,380,512
75% $193,767,813
80% $194,274,786
85% $194,896,984
90% $195,972,844
95% $197,146,209
100% $203,887,262
Cu
mu
lati
ve F
req
uen
cy
New Attack SubmarineEntire Plan : Cost Makes extensive use of random
numbers to control the decision making
when a physical event has a number of
possible results
Translates the uncertainties specified at
a detailed level of the project into their
potential impact on project objectives
Source: Pertmaster, LLC
22 COMMITTED TO EXCELLENCE
W W W . AL O N I N C . C O M
Copyright 2011 ALON, Inc., All Rights Reserved
Base estimates on past results, reliable data, and validated assumptions
Have estimates developed by “experts” – use more than one person
Develop estimates based on sound requirements and understanding of
uncertainties
– Iterative development
– Refine estimates as more information becomes available
– Use quantitative analysis
– Use more than one approach
Present estimates as a range – defend/explain estimate range probabilities
Review, revisit, remove head from sand, repeat
USE SOUND PRACTICES
STRATEGIES FOR RELIABLE ESTIMATES
23 COMMITTED TO EXCELLENCE
W W W . AL O N I N C . C O M
Copyright 2011 ALON, Inc., All Rights Reserved
Stakeholder management – interpreting and influencing both the external
and internal environments and by creating positive relationships
UNDERSTAND POLITICAL NATURE OF ORGANIZATION
STRATEGIES FOR RELIABLE ESTIMATES
Adapted from www.business.utah.edu
Play with diligence and a full
understanding of the
landscape, players and rules
Be prepared – provide data
derived information, foster
alliances, build trust and
continually adjust your
approach
Develop negotiation and
conflict management skills
24 COMMITTED TO EXCELLENCE
W W W . AL O N I N C . C O M
Copyright 2011 ALON, Inc., All Rights Reserved
People
– Skilled and experienced cost estimators
– Knowledgeable and empowered program/project managers
Process
– Key component of acquisition/program management planning and execution
– Continual refinement of “adaptable” processes and techniques
Tools
– Integrated capacity to develop and maintain estimates
– Supports “what if” analysis
Organization
– Supportive and knowledgeable senior management
– Internal resources to build/improve estimating capabilities
BUILD COST ESTIMATING CAPABILITIES
STRATEGIES FOR RELIABLE ESTIMATES
24
25 COMMITTED TO EXCELLENCE
W W W . AL O N I N C . C O M
Copyright 2011 ALON, Inc., All Rights Reserved
Key principles of cost estimating
Challenges in developing estimates
Strategies to consider in developing
reliable estimates
IN CLOSING
26 COMMITTED TO EXCELLENCE
W W W . AL O N I N C . C O M
Copyright 2011 ALON, Inc., All Rights Reserved
Full service information technology and program management solutions company
with five core services:
Program Management
IT and Engineering Solutions
Information Assurance
Virginia corporation with headquarters
in Springfield and an office in DC
Certified ISO 9001:2008
Top 100 DHS Contractor
(Government Security News)
Top 40 Acquisition Support Contractor
(Federal Times)
CORPORATE PROFILE
Capital Planning and Investment Control
Financial Management
Department of Homeland Security
National Archives and Records
Administration
Federal Aviation Administration
Environmental Protection Agency
National Oceanic and Atmospheric
Administration
Department of Treasury
U.S. Census Bureau
Drug Enforcement Administration
Department of Defense
ALON IS CURRENTLY SUPPORTING:
27 COMMITTED TO EXCELLENCE
W W W . AL O N I N C . C O M
Copyright 2011 ALON, Inc., All Rights Reserved
CONTACT INFORMATION
Scott Patton Director, Program Support Services ALON, Inc.
[email protected] (703) 256-9503
28 COMMITTED TO EXCELLENCE
W W W . AL O N I N C . C O M
Copyright 2011 ALON, Inc., All Rights Reserved
QUESTIONS AND COMMENTS?
THANK YOU!