Post on 26-Dec-2015
transcript
1
Prof. Morteza Anvari
2
Cost Analysis Requirement
BA ConceptExploration
Concept & Technology Development
System Development& Demonstration
Production & Deployment
Pre-SystemsAcquisition
Systems Acquisition(Engineering and Manufacturing
Development, Demonstration, LRIP &Production)
Support
C
Sustainment
ReviewReviewReview
ComponentAdvanced
Development
SystemIntegration
SystemDemo
ProductionReadiness & LRIP
Rate Production &DeploymentN
EW
OLD
ConceptExploration
Program Definition &
Risk Reduction
Engineering & Manufacturing Development
(EMD)
Production, Fielding/
Deployment
MS 0 MS I MS II MS III
Cost Estimates Needed
3
Cost Estimating Process
Final Document
Data CollectNormalize
EstimateFormulation
Review/ Presentation
DefinitionPlanning
DOCUMENTATION
4
Develop Cost EstimateStructure and WBS
Data/CERsLCC EstimatesCost Drivers
EstablishGround
Rules andAssumptions
Prepare CostEstimatesfor EachElement
Test TotalSystem
Estimate
Prepare Documentation
EngineeringAnalogy
ParametricsExpert Opinion
ReasonablenessSensitivityAnalysisCost-Risk
Assessment
CompileData Base/
CERs/Models
Cost Analysis Model
Start
5
Situation
• You have just been tasked to develop a cost estimate, that is, a professional opinion about the cost of an item, a service or a thing.
• Let’s discuss a process for organizing and developing this estimate.
6
Definition and Planning
• Influences the success of the estimate
• Understanding the requirements and how you approach the process will establish the guidelines and procedures for the estimate.
• Ask lots of questions…They help you understand the requirement.
7
Questions
• Why is this cost estimate needed?• What decisions are pending on the results of this
estimate?• Will the estimate be briefed and to whom?• Will the results be incorporated into some
document?• What does the recipient expect to have included or
excluded?• What excursions or variations from the baseline
are anticipated?
8
Questions Continued
• What are the program and funding constraints especially if the program is a Joint Program?
• What are the time constraints for this estimate?• What is the acquisition phase of the program?• Is the program definition mature?• Does technology exist today to design, develop, test
and manufacture the system?• What is the interrelationship with other systems?• Are there previous contracts? How many? What type?• How have the contractors performed to date?
9
Definition and PlanningKnow Purpose of the Estimate
• Main purposes of estimates:
– Budget Formulation
– Comparative Studies
– Source Selection
10
Purpose of EstimateBudget Formulation Estimates
• Program Office Estimate (POE)• Component Cost Analysis (CCA)• Independent Cost Estimate (ICE)• What-if exercises• Rough Order of Magnitude (ROM)• Should Cost Estimates• CAIV
11
Purpose of EstimateComparative Studies Estimates
• Making cost & benefit comparisons between alternatives– Economic Analysis (EA)– Analysis of Alternatives (AoA)– Force Structure– Trade-off Studies– Source Selection– Prioritization
12
Definition and PlanningDefining the System
• Adequate description of the technical and program characteristics of the system
• What are the physical and performance characteristics?
• What are the development, production, and deployment schedules?
• How many systems are to be produced?• How will the systems be supported: contract, in-
house, two or three levels of maintenance?
13
ACTIVITY J F M A M J J A S O N D J F M A M J J A S O N D J F M A M J J A S O N D J F M A M J J A S O N D J F M A M J J A S O N D J F M A M J J A S O N D J F M A M J J A S O N D
1993 1994 1995 1996 1997 1998 1999 2000J A S O N D
PROGRAM REVIEWS
ENGINEERING & MANUFACTURING DEVELOPMENT
TEST & EVALUATION
LIVE FIRE TEST
LOG DEMO II
PROD VER TEST
ENHANCED PRODUCIBILITY PROGRAMLOW RATE INITIAL PRODUCTION
FY94
FY95
FY96
MULTIYEAR I & II (3 YR CONTRACT)
FY97
FY98
FY99
FIELDING
LLTI PEO IPR LRIP DECISION MILESTONE III
IOTE
TECH SUPPORT
QLFTT
QUAL
UE
FUE FUEUSMC
MSLS - 3316 741 CLUs - 423 133
MSLS - 1080 194 CLUs - 270 140
MSLS - 1020 141 CLUs - 206 48
MSLS - 1015 CLUs - 108
MSLS - 872 CLUs - 97
MSLS - 698 CLUs - 55
CA CA
CA
CA
CA
LEAD TIME PRODUCTION
PRODUCTION
LEAD TIMELLTI
LD II
LFT (A)(26)
(3)LUT
CA
LEGEND
QL - QUICK LOOK FTT - FIELD TACTICAL TRAINER UE - USER EVALUATION LUT - LIMITED USER TEST CA - CONTRACT AWARD MRRB - MATERIEL RELEASE
REVIEW BOARD
(6)
TODAY
MRRB
PRODUCTION
LFT (B,C) (8,4)
USA
LEAD TIME
LEAD TIME
AWEHANDOFF NTC
PRODUCTION
PVT(25 MSL + 10 CLU)
FT. HOOD
RANGER COMPL
82nd FIELDING BEGINS
82nd COMPLETE
ARMY USMC
LETHALITY ENHANCEMENT
LITE I / WARHEAD / LOW COST BST
DEVELOPMENT
CA MYII
(9)CLU
(28)MSL
II, III, FLIGHTI
Defining the SystemIntegrated System Schedule
14
Defining the System Work Breakdown Structure (WBS)
• WBS definition – product-oriented breakdown of hardware,
software, services, data and facilities that define the system.
• WBS breaks a total job down into manageable pieces & portrays the way work is to be done.
• WBS displays a company’s reporting structure.• Program managers may cite MIL-HDBK-881 “for
guidance only” in contract solicitations.
15
WBS
16
Defining the System Cost Element Structure (CES)
1.0 Research Development Test & Evaluation (RDT&E)
2.0 Production
3.0 Construction (CON)
4.0 Pay and Allowances
5.0 Operating and Maintenance Army (O&M)
17
Defining the System COST ELEMENT STRUCTURE – 1.0 RDT&E
CES# ELEMENT: FY00C$M TY$M1.01 DEV. ENG.
1.02 PEP
1.03 DEV. TOOL.
1.04 PROTO MFG.
1.05 SEPM
1.06 SYS T&E
1.07 TRAINING
1.08 DATA
1.09 SUPP EQUIP.
1.10 DEV. FACILITIES
1.11 OTHER
$ 39.039M $38.260M
0.408 0.386
0.457 0.450
110.421 107.724
78.266 76.363
11.112 10.927
1.989 1.954
3.439 3.413
4.897 4.758
0.0 0.0
0.968 0.928
18
CES# ELEMENT: FY00C$M TY$M2.01 NON REC PROD. 2.02 REC. PROD 2.03 ENG. CHG2.04 SEPM2.05 SYS T&E2.06 TRAINING2.07 DATA 2.08 SUPP. EQUIP. 2.09 OPER./SITE/ACT. 2.10 FIELDING 2.11 TRAIN. AMMO/MSLS 2.12 WAR RESV. 2.13 MODS 2.14 OTHER
$ 16.110M $ 16.583M 1,169.348 1,312.377 0.0 0.0 116.637 132.258 12.564 14.437 28.880 31.499 2.073 2.300 146.460 158.304 0.0 0.0 89.525 101.635 59.001 79.482 0.0 0.0 236.619 280.729 51.739 64.129
Defining the System COST ELEMENT STRUCTURE – 2.0 PROCUREMENT
19
CES# ELEMENT: FY00C$M TY$M3.01 DEVELOP. CONSTRUCTION 3.02 PRODUCT. CONSTRUCTION 3.03 OPERATION/SITE ACTIVATION3.04 OTHER N
Defining the System COST ELEMENT STRUCTURE – 3.0 CON
20
CES# ELEMENT: FY00C$M TY$M4.01 CREW 4.02 MAINTENANCE 4.03 SYSTEM SPECIFIC SUPPORT4.04 SEPM4.05 REPLACEMENT PERSONNEL4.06 OTHER
Defining the System COST ELEMENT STRUCTURE – 4.0
Pay & Allowances
21
CES# ELEMENT: FY00C$M TY$M5.01 FIELD MAINT., CIV LABOR5.02 SYS. SPECIFIC BASE OPS 5.03 REPLENISHMENT DLRs5.04 REPLEN. CONSUMMABLES5.05 POL5.06 END ITEM MAINTENANCE5.07 TRANSPORTATION 5.08 SOFTWARE5.09 SEPM 5.10 TRAINING 5.11 OTHER
Defining the System COST ELEMENT STRUCTURE – 5.0
Operating & Maintenance (O&M)
22
Defining the System Cost Analysis Requirements Description
(CARD)
• Source of a system’s description• Describes important features• Is provided to other groups preparing cost
estimates• Helps ensure all groups are costing out the same
“program.”• Prepared by program office; approved by
DoD Component Program Executive Officer
23
Defining the System Cost Analysis Requirements Description
(CARD) Continued1.0 System Overview1.1 System Characterization
1.1.1 System Description1.1.2 System Funcitonal Relationships1.1.3 System Configuration1.1.4 Government Furnished Equipment/Information
1.2 System Characteristics1.2.1 Technical/Physical Description 1.2.1.1 Subsystem Description 1.2.1.2 Functional and Performance Description 1.2.2 Software Description
1.3 System Quality Factors1.3.1 Reliability1.3.2 Maintainability1.3.3 Availability1.3.4 Portability and Transportability1.3.5 Additional Quality Factors
1.4 Embedded Security1.5 Predecessor/Reference System2.0 Risk
24
Defining the SystemCARD Continued
3.0 System Operational Concept3.1 Organizational Structure3.2 Basing and Deployment Description3.3 Security3.4 Logistics
4.0 Quantity Requirements5.0 System Manpower Requirements6.0 System Activity Rates7.0 System Milestone Schedule8.0 Acquisition Plan or Strategy9.0 System Development Plan
10.0 Element Facilities Requirements11.0 Track to Prior Card12.0 Contractor Cost Data Reporting Plan
25
Definition and PlanningGround Rules & Assumptions
• State the conditions which must take place in order for the estimate to be valid
• Ground rules and assumptions must be documented since changes in these areas provide an audit trail for changes in the cost estimate.
26
Data Collection and Analysis
• Collection and analysis represent a significant amount of the overall estimating task in terms of time. The analysis will include decisions on what programs to include in the data set to whether to truncate lot data on a program for which you are calculating a learning curve.
• Document data in your analysis, and any assumptions you make
27
Data Collection and Analysis
• The direction we take in collecting historical data will be determined by our choice of estimating methodologies.
• This step may also dictate a change in estimating approach due to the availability or non-availability of certain data.
• Data collection is not limited to cost data. We must also collect technical and program data if we want the total picture of the historical systems. This will help us ensure the comparability of the systems that we are collecting data on with the system we are estimating.
28
Data Collection and Analysis Most Difficult Task in Cost Estimating
• Data Sources– Data Types: Cost/Resource, Technical, Program– Categories: Primary, Secondary
• Data Problems– Wrong Format – Matching up – Definition– Temporal Factors - comparability
• Normalization• Data Location
29
Tools for Total Ownership Cost Estimating
Life Cycle Cost Management Tools
APPN ACEIT PRICE SEER ACDB AMCOS OSMIS SBC/ISR CO$TAT
RDT&E X X X X X X
PROC X X X X X X
MILCON X X X X X
MILPERS X X X X X
O&M X X X X X X X
30
Definition and PlanningSelect the Estimating Approach
• Techniques available– Analogy– Parametric– Engineering– Extrapolation– Expert OpinionSelect the technique that is most applicable
to a specific WBS element
31
Definition and PlanningEstimating Methods
• Analogy– Basic Comparison– Factors
• Parametric– Regression Analysis
• Engineering– Detailed
• Expert Opinion– Committee– Delphi
32
ConceptRefinement
TechnologyDevelopment
System Development& Demonstration
Production &Deployment
Operations& Support
Life Cycle Cost Estimates
• Cost estimates based on confidence intervals• Parametric analysis based on similar systems and similar attributes (regression)• Engineering data based on reliability projections used for bottoms-up estimate• Actual system costs used to extrapolate future system costs• Cost estimate revised every two years after production• Weights associated with non-actuals decrease as system matures• MS B is a true hard stop for systems
A B C D
Cost Estimating Model
Disposal
ParametricEngineeringActuals
xy0
ParametricEngineering
xy0
ParametricEngineering
xy0
ParametricEngineeringActuals
xy0
ParametricEngineering
xy0
ParametricEngineering
xy0
ParametricEngineering
xyz
ParametricEngineering
xyz
ParametricEngineeringActuals
xyz
ParametricEngineering
xyz
ParametricEngineering
xyzActuals
ParametricEngineeringActuals
0yz
ParametricEngineering
33
Cost Estimating MethodsAnalogy Method
• Based on direct comparison with historical information of similar existing activities, systems, or components.
• Compares new system with one or more existing similar systems where there is accurate cost and technical data.
• Analyst must show validity of comparison.
34
Cost Estimating MethodsAnalogy Method
• Based on known costs of a similar program• Adjustments for complexity, technical, physical• Strengths
– Based on representative experience– Less time consuming than others– Can be used as a check on other techniques
• Weaknesses– Small sample size– Heavy reliance on judgment– Sometimes difficult to identify analogy and associated
costs
35
Analogy Estimating with Factors
Element Old Sys1 Old Sys2 Old Sys3 New Sys
Airframe $500/lb $250/lb $750/lb 1.25*S1
Engine 2M/Unit 3M/Unit 5M/Unit .8*S3
Avionics $3K/lb $2K/lb $4K/lb 1.0*S2
Payload 6M/Unit 8M/Unit 7M/Unit .65*S1
Cost(New) = Cost(Old) x Adjustment Factor
36
Cost Estimating MethodsParametric Method
• Known as Statistical Method or Top Down Method
• Relates cost to physical attributes or performance characteristics
• Uses database of elements from similar systems• Uses multiple systems• Most beneficial in earlier stages of the system or
project life cycle
37
Cost Estimating MethodsParametric Method
• Statistical relationships between cost and physical or performance parameters of past systems.
• Strengths– Captures major portion of cost– Quick what if type estimates
• Weaknesses– Less detailed– Getting accurate data
38
Cost Estimating MethodsParametric Method
(Extrapolation)
• Use historical values to establish a trend for the future.
• Example problem: Given the actual productivity and labor rates in the given table. How much will it take to complete a 3-year software development project of 10K lines of code, if 50% is completed in the second year and 25% is completed in first and third years?
39
Cost Estimating MethodsEngineering Method
• Known as bottom up method• Requires extensive knowledge of system
characteristics• Divide into segments; estimate costs for each
segment• Combine segments plus integration cost• Uses a combination of cost estimating methods• Detailed knowledge of new technologies may not
be available.
40
Cost Estimating MethodsEngineering Method
• Strengths– Detailed
• Best when long stable production process
• Weaknesses– Requires a lot of time– Cost– Cannot be used until system well defined
41
Cost Estimating MethodsExpert Opinion Method
• Subjective judgment of an experienced individual or group
• Use if time does not permit a more thorough analysis
• Document source(s) of opinion of experts• List attributes of the source(s) Example: Delphi Technique
42
Cost Estimating MethodsExpert Opinion Method
• Consulting with one or more experts who use their knowledge and experience to arrive at an estimate
• Group techniques include– Consensus (Committee)– Delphi
• Strengths and weaknesses
43
Expert Opinion MethodDelphi Technique
• Query expert opinion from group• Seek information from each expert• Summarize the results• Send report to each expert• Gather second opinion after each individual
reviews report• Summarize results• Iterative process continues until the experts reach
a consensus, or near‑consensus.
44
Expert Opinion Method Example
Labor typeHours Needed
Hourly Rate% of Total Hrs
%*rate
Senior Engineer 1000 $13 10.5 $1.37
Design Engineer 3000 $11 31.6 $3.48
Tool & Die 500 $11 5.3 $.58
Machinist5000 $9 52.6 $4.73
Totals 9500 $10.16
45
As the quantity of a product produced doubles, the man-hours- per-unit expended to produce the product decreases at a fixed rate or constant percentage (usually 10% to 15%).
Learning Curve Theory
46
• Job familiarization by both production workers and supervisory personnel.• Changes in product design which do not materially affect the product, but result in increased ease and speed of production.• Changes in tooling, machinery, and equipment which simplify or speed up the production process.• Improved production planning and scheduling, and improvements in production techniques and operational methods.• Improvements in shop organization, engineering coordination and liaison.• Improvements in the handling and flow of materials, and in the materials and parts supply systems
Learning Curve TheoryFactors Contributing to Efficiency
47
TABLE FOR FIGURESF-1-1 and F-1-2
Unit No. Unit Person-hours
1 100.002 80.004 64.008 51.2016 40.9632 32.7764 26.21
The table is based on the assumption that the first unit required 100 person-hours to produce. The table indicates a constant rate of reduction of 20% for each doubling of the unit number; the value of the second and each succeeding item in the table is 80% of the value of the preceding item.
Learning Curve Theory
48
80% Unit Curve on Arithmetic Paper
Learning Curve Theory
49
Learning Curve Theory
1
10
100
1000
1 10 100 1000 10000
90%
85%
80%
70%
Production Unit Number (LOG)
UnitCost
(LOG)
50
Learning Curve TheoryUses
– Evaluating contract production costs.– Assessing impact of production interruptions, product
changes and production rate change. – Rate of improvement experienced by a particular contractor
on a prior product may be indicative of rate of improvement expected on new product of similar size, complexity, and construction.
– Improvement curve pattern experienced in the production of past item can be extended to calculate costs of future items.
51
Estimate Formulation
• We have defined our tasks, planned the estimate, assigned cost responsibilities, and performed data collection and analysis.
• Here we apply our estimating methodologies and tools: develop the factors, analogies, CERs, and learning curves.
• We will aggregate the various cost elements into development, production, and O&S estimates, fiscally spread the costs, and apply inflation.
52
Cost Estimating MethodsLaying Out the Estimating Approach
Aircraft System – sum of level II elements (cross-check with analogy)Air Vehicle – sum of level III elements
Airframe – CERsAir Vehicle Software – Expert OpinionPropulsion – CERsAvionics – AnalogyArmament – Catalog Price
System Eng/Program Mgt – Factor of Air VehicleSystem Test & Evaluation – Factor of Air VehicleTraining – Factor of Air VehicleData – Factor of Air VehiclePeculiar Support Equipment – Factor of Air VehicleInitial Spares – Factor of Air Vehicle
53
Review and Presentation
• We want to ensure that the estimate is reasonable, realistic, and complete.
• Reasonableness addresses areas such as: using appropriate and acceptable methodologies; presenting methodologies systematically; the use of relevant data; and ensuring that assumptions are valid and clearly stated.
54
Review and Presentation (Continued)
• The realism test checks to see if the assumptions and ground rules are consistent with the statement of work and if the costs are in line with historical data.
• We evaluate completeness by determining whether the estimate includes all the work stated in the request for proposal and whether the costs are traceable and reconcilable.
Risk and Uncertainty AnalysisRisk and Uncertainty Analysis
Risk Analysis Approaches
Detail
Eff
ort
• Detailed Network & Stochastic
• Detailed Monte Carlo Simulation (each WBS)
• Bottom Line Monte Carlo Simulation
• Add a Risk Factor/Percentage
• Discrete Technical, Schedule, and Estimating Risks
Risk and Uncertainty AnalysisRisk and Uncertainty Analysis
Probability DistributionsWBS Cost Distribution Total Cost
P.E.
P.E.
P.E.
+
+
+ Many More
=
Sum PE MLC (Mode)
CDF
Con
fide
nce
50%
20%
Fre
quen
cy
Sum PE MLC
Risk and Uncertainty AnalysisRisk and Uncertainty Analysis
Probability Density Function
Cell B28 Frequency Chart
Mean = $2,198
4,978 Trials Shown
.000
.006
.012
.018
.024
0
29.2
58.5
87.7
117
$1,250 $1,750 $2,250 $2,750 $3,250
Forecast: Results=
Risk and Uncertainty AnalysisRisk and Uncertainty Analysis
Cumulative Distribution Functions
Cell B28 Cumulative Chart 4,977 Trials Shown
.000
.249
.498
.747
.995
0
4977
$1,250 $1,750 $2,250 $2,750 $3,250
Forecast: Results=
59
Reasons for RiskReasons for Risk
Technical Programmatic Cost Schedule
PhysicalProperties
MaterialAvailability
Sensitivity toAssumptions
Degree ofConcurrency
Materialproperties
SkillRequirement
Sensitivity toTechnical Risk
Sensitivity toTechnical Risk
SoftwareComplexity
EnvironmentalImpact
Sensitivity toProgrammaticRisk
Sensitivity toProgrammaticRisk
IntegrationInterface
ContractorsStability
Sensitivity toSchedule Risk
Number ofCritical Path
RequirementChanges
Funding Profile EstimatingErrors
EstimatingErrors
OperationalEnvironment
PoliticalAdvocacy
60
Development Schedule
Lif
e C
ycle
Cos
t
Min Optimal
Typ
ical
Cost Schedule Curve Cost Schedule Curve
Parallel EffortMore ECPLess Mature Design
Fixed CostTechnology Outdate
61
Development ScheduleDevelopment Schedule
DoD
Pre-1992 Starts 132 months (11 years)
Post-1992 Starts 89 months (7.4 years) on-going
F-22 216 months (18 years) IOC 2004
Comanche 264 months (22 years) IOC 2006
Commercial
Boeing 777 54 months (4.5 Years)
Mean Cycle Time to IOC
62
Defense
Schedule GoalsSchedule Goals
Development Schedule
Cos
t
Commercial Drivers• Technology Drives Schedule• Constraint Schedule• Goal is ROI Maximization
DoD Drivers• Funding Drives Schedule• Unconstrained Schedule• Goal is Cost Reduction
Commercial
63
DoD Program Schedule DriversDoD Program Schedule Drivers
Development Schedule
Cos
t
Cycle Time Reduction Goal
Funding Allocation
Acquisition Process
Program Execution
64
Final Documentation
• Provide the means for other analysts to get the same results that we have in our cost estimate.
• Providing good directions and a clear trail to follow are essential in having an estimate that can be replicated.
• Provide step-by-step documentation of the methodologies, supporting data, ground rules, and assumptions, equations, examples, etc.,
• Ability to interpret or evaluate someone else’s cost documentation is as important as ability to prepare good cost documentation.
65
CAIV Process
• Set realistic but aggressive cost objectivesearly in acquisition program
• Manage risks
• Track progress using appropriate metrics
• Motivate Government and industrymanagers to achieve program objectives
• Incorporate incentives to reduce O&S costsfor fielded systems
66
Cost As An Independent Variable (CAIV)
• Best time to reduce cost is early in the process.
• Involves the stakeholders in the process.• Cost tradeoffs must be addressed early in
the acquisition process and embedded in program requirement documents, Request For Proposals (RFPs), contract provisions, and source selection process.
67
Cost Analysis ProcessDatabases, Tools, & Models
Requirements Acquisition Assumptions
Design Parameters Risk Assessment
ASARCCAIG, DABAPBPPBESAOA
Inputs (Descriptions) Process (Models) Outputs (Costs)
Types of Cost StudiesStudies Should be Known Unknown Class
Cost Estimating Descriptions, Models Costs Analysis
CER DevelopmentDescription, Historical
CostsModels Synthesis
Needed Capability CAIV Cost Goals, Models
Descriptions ( Design and Performance Parameters)
Control
Current Capability
Performance Based and Design Based Cost Models Currently do not Exist
Cost Analysis Domain
68
Analysis of Alternatives
• Set realistic, aggressive cost objectives earlyin development
• Manage risks
• Track progress with appropriate metrics
• Motivate government/industry managersto achieve program objectives
• Incorporate incentives to reduce O&S costsfor fielded systems.
69
Cost Analysis
Art or Science?