1 June 18, 2008
Data for Costing & Cost Uncertainty
Mort AnvariDirector, Acquisition Costing
(SAFM-CEA)
June 18, 2008 Cost Management Panel
PRICE Customer Symposium
2 June 18, 2008
Key Points
We have Several Data Quality Issues and Potential for Type I (α) and II (β) Errors in our Cost Estimating.
Opportunity for Cost Analysis Enterprise Resource Planning (CA-ERP) and Data for Costing Architecture.
Need for Range Cost Estimating and Cost Confidence Analysis
Need Data for Costing:Future Technologies; Processes; Performances; Products; Services; Personnel; and Schedule.
3 June 18, 2008
Army Cost Data Sources
DCARC CSDR Repository Automated Cost Databases (ACDB) Cost Analysis Requirements Description (CARD) Capability Knowledge Base (CKB) Operation and Support Management Information Systems (OSMIS) Contractor Logistic Support (CLS) Data Joint Integrated Analysis Tool (JIAT) Software Data Collection Initiatives
Army Cost Positions (ACP) Database ACE-IT Sessions, Periodical SAR, DAES, WSR, APB, P-Form, Contracts
Summaries
+
4 June 18, 2008
Data for Cost Estimating
Cost Models Historical Data: Cost and (Physical, Technical, Performance, Capability, Schedule).
Subject System Data: Physical, Technical, Performance, Capability, Schedule.
Information KnowledgeData
Performance
Cost Schedule
Estimates are only as good as the data that they are based on.
5 June 18, 2008
Data Issues
Current Data Reporting and Usage Issues (CSDR CCDR, SRDR, and CPR).
Family or System of Systems (Commonality Issue) LSI, JV, Prime, Sub (Contracting Issue) CPIF, FFP,. . Reporting Requirements (Contract Type Issue) Manufacturing, Services, SW, and ERP (Activity Type Issue) Mod, ECP, and Incremental Contracts (Completeness Issue) Competition and Business Base Issues (Quantity issue) Physical, Technical, TRL, Performance (Technology Issue)
KPP and Capability Data Issue
Allocation Issue: Mapping to Standard ReportsLabor – Material
Direct – Indirect
Recurring – Non-Recurring
6 June 18, 2008
Data Quality Challenges
Lack of data. Physical, technical, performance, capability. S/W ERP FFP Contract. ACAT II and below programs.
Policies and procedures. DoDI 5000-2 dated May 2003.
Requires SRDR (w/o Cost) but not CCDR for some programs. Vague in some areas; CSDR “not required for procurement of commercial systems, or for non-commercial systems bought under competitively awarded, firm fixed-price contracts, as long as competitive conditions continue to exist”.
Army regulations do not require additional cost reporting.
Data collection process. Existing data is not readily available to the Cost Analysts; sometimes they are not even aware that data exists. (Data in Jail) Contractors required to map CSDR reports IAW DoD MIL-HDBK-881.
7 June 18, 2008
Data Collection Process
Contractor
1
Contractor
1
Contractor 2
Contractor 2
Contractor3
Contractor3
Contractor Accounting Systems:
Contractor Accounting Systems:
S1
S3
S4
S2S5S6
S7
S8
S10
S9
Contractor Data / Reports:
Contractor Data / Reports:
R1
R3
R4
R2
R5
R6
R7R8
R10
R9
R11
R12
R13
R14
R15
R16
R17
R18
Labor vs. MaterialDirect vs. IndirectRec. vs. Non Rec.
Unique vs. CommonContractor allocation
Contracting
Labor vs. MaterialDirect vs. IndirectRec. vs. Non Rec.
Unique vs. CommonContractor allocation
ContractingG1
G2
G3
Co
ntr
ac
tor
All
oc
ati
on
Data Quality?
Process affects data quality. Increases probability for data errors. Results in mapping / allocation inconsistencies.
Process vs. Quality.
COST AND SOFTWARE DATA REPORTING PLAN Form Approved OMB No. 0704-0188
1a. PROGRAM 2a. WEAPON SYSTEM TYPE 3. SUBMISSION TYPE 4. DATE AS OF (MM/DD/YY) 5. REPORT DATE (MM/DD/YY)
X INITIAL SUBMISSION
1b. MILESTONE A B X C: LRIP C: PROD CHANGE
6. POINT OF CONTACT (POC) INFORMATION 7. W BS 8. PREPARING ORGANIZATIONa. POC AND ADDRESS (Include ZIP Code) X PROGRAM
CONTRACT 9. REVIEW AND REFERENCE NUMBER
12. 13.CONTRACTOR CONTRACT d. DD 2630
a. PROGRAM b. CONTRACT (DUNS Code) NUMBER REQUIRED
DD FORM 2794, Oct 2003 PREVIOUS EDITION IS OBSOLETE
6d. E-MAIL ADDRESS:
REQUIREDWBS
REPORTING ELEMENTS
14. REPORT FREQUENCYa. DD 1921 b. DD 1921-1 (Part 1) c. DD 1921-1 (Part 2)
Public reporting burden for this collection of information is estimated to average 15 hours per response, including the time for reviewing instructions, searching existing data sources, gathering and maintaining the data needed, and completing and reviewing the collection of information. Send comments regarding this burden estimate or any other aspect of this collection of information, including suggestions for reducing the burden to Department of Defense, Washington Headquarters Service, Directorate for Information Operations and Reports (0704-0188), 1215 Jefferson Davis Highway, Suite 1204 Arlington, VA 22202-4302. Respondents should be aware that notwithstanding any other provisions of law, no person shall be subject to any penalty for failing to comply with a collection of information if it does not display a currently valid OMB control number. PLEASE DO NOT RETURN YOUR FORM TO THE ABOVE ADDRESS.
REQUIRED
6b. TELEPHONE NUMBER
6c. FAX NUMBER
10.WBS ELEMENT CODE
11.
REQUIRED
CSDR
8 June 18, 2008
Cost Data Flow
Need for Data Warehousing. Repository of electronically stored data.
Several databases typically supply data to the warehouse.
Improved ease of data retrieval and data quality.
Contractor Cost Accounting Systems
Government InformationalSystems / Databases
Contractor
Sys1
Contractor
Sys2
Contractor
Sys3
Contractor
Sys4
ACDB OSMIS CLS
Current Information Access Flow.
Cost Analysts
9 June 18, 2008
Data Warehousing
Raw Data loaded directly from contractor accounting system into data warehouse.
Metadata: ‘Data about Data’.
Data visualization / data warehousing / intelligent data mining.
Global definitions that can be referenced by many different databases.
InformationInformationDelivery SystemDelivery System
Government InformationalSystems / Databases
DataWarehouse
ExternalData
ExternalUsers
Proposed Information Access Flow.
Contractor Cost Accounting Systems
Data Marts
10 June 18, 2008
Cost Analysis ERP
Potential for Enterprise Resource Planning (ERP) of Cost Analysis Data and Models across the DoD.
Enable data and information sharing. Allow automation and integration of business processes.
Cost Analysts need to be more involved in the process.
Data for Costing – Vision: Collect raw data with more flexibility and focus on metadata development – XML. Data warehousing of raw and current data. Acquiring data mining and search tools to assist data analysis.
Data ERP for Costing? Check out http://www.anvari.net/E_Proc_Model.htm
11 June 18, 2008
Cost Risk Analysis
Risk analysis reduces the uncertainty between requirements and funding
12 June 18, 2008
Con
fiden
ce
100%
75%
50%
0%
25%
Cost $
Causes of Cost Variations
Cost Confidence Curve or “S” Curve
Technical & Schedule Risk (Included)
Cost Estimating Methodology Uncertainty (to-be Included)
– Engineering– Analogy– Parametric
• Physical Parameters• Technical Parameters• Performance Parameters• Capability Parameters
– Expert Opinion
Programmatic Changes (Mod) (Excluded)– Restructuring– Reprogramming – Rescheduling– Resourcing
13 June 18, 2008
Current:• Point Estimate Costing and Reporting, with
Additional Cost Range Modeling Option at Technical & Schedule Risk Items.
• Narrow Band Confidence. • Utilities are Risk Management and Execution.
Proposed:• Range Estimate Costing and Reporting, with Point
Estimate Reporting Option at Different Confidence Level.
• Wide Band Confidence. • Utilities are Planning and Programming.
Current & Proposed Estimating Methods
14 June 18, 2008
Cumulative Distribution Functions
Increasing cost confidence from 50% to 75%
Con
fiden
ce L
evel
100%
75%
50%
0%
25%
Cost $
Tec
hnic
al R
isk
All Risks and Uncertainties
Path Forward: Increase cost estimating confidence and implement rigorous risk analysis
15 June 18, 2008
Backups
Backups
16 June 18, 2008
Current Cost Data Collection Efforts
Automated Cost Databases (ACDB) Primary source of data is CSDRs, CPRs, etc. Includes programmatic and technical information.
Cost Analysis Requirements Description (CARD) Redesigned to reduce variation (IAW Lean Six Sigma process).
On-line living document. Standardized format for documents. Controlled author access to CARD sections. Milestone feeder documents (draft) provided early.
New process allows variation to be controlled (man-hour avoidance).
Capability Knowledge Base (CKB) Performance/capability-based relationships. Uses capabilities and costs of current systems to provide rough order of magnitude estimates for capability-filling solutions. Cost support for earlier acquisition decisions.
Missiles / MunitionsAircraft
Com / Electronics
Surface Vehicles
17 June 18, 2008
Current Cost Data Collection Efforts
Operating and Support Management Information System (OSMIS) Used by over 1,400 DoD employees (government and contract). Over 25,000 queries generated annually. Data used to generate OPTEMPO Weapon System Factors; direct link to PPBE process. Historical archive of O&S data for period covering 1993 to 2007. Prior year data is available by 1 April of the following year; Monthly activity reports by serial number level are available by 25th of the following month.
Contractor Logistic Support (CLS) Data CLS accounting for more of the total OPTEMPO requirement. Established a standardized process for CLS data collection. Requirement place on contract in conjunction with CSDR requirements. Cost data will enable the Army to forecast more reliable cost estimates for supply support.
18 June 18, 2008
Current Cost Data Collection Efforts
Joint Integrated Analysis Tool (JIAT) Cost estimating, engineering design, requirements, capability and performance analysis linked. Ready access to cost and technical data in each commodity area: Missiles, Aircraft, Vehicles and Communications and Electronics Systems. Supports cost as an independent variable (CAIV) analysis, capabilities costing and trade-off analysis. Near real-time cost estimating capability to the acquisition and requirements communities.
Software (SW) Data Collection Initiatives Collecting SW development actuals on past programs. Emphasis on improving SW maintenance estimates . . . Working with Army SW Engineering Centers collecting SW maintenance actuals. Partnering with Air Force & Navy . . . Collect & share data on Enterprise Resource Planning (ERP) Projects and SW intensive programs.