Date post: | 06-May-2018 |
Category: |
Documents |
Upload: | trinhquynh |
View: | 222 times |
Download: | 6 times |
How to Pass an Estimating System
Review
February 27, 2013
ANNOUNCEMENTS
Mr. James J. GallagherPartner, McKenna Long & Aldridge LLP
Ms. Jami LevyPricing Manager, Northrop Grumman Corporation
Mr. David E. TrottaEstimating Manager, ViaSat Inc.
The DOD Business Systems Rule
February 27, 2013
Jay GallagherMcKenna Long & Aldridge LLP
• Applicable through clauses in DOD, CAS-covered contracts awarded after May 2011
• Imposes numerous system requirements, including SOX like internal controls procedures
• Approval by ACO/DACO/CACO based upon DCAA opinion• Applies to the “contractor,” meaning the legal entity• Rule citations
– Interim rule (76 Fed. Reg. 28,856 (May 11, 2011))– Final rule (77 Fed. Reg. 11,355 (Feb. 24, 2012))
Overview of Final Rule
• Requires existence of six approved systems – Accounting (includes internal controls and billing)– Purchasing– Estimating– Materials Management and Accounting– Government Property– Earned Value Management
Overview of Final Rule (cont.)
Overview of System Approval Fundamentals
• Approval requires that no “significant deficiency” exists– Affirmative finding– What if no finding?
• Initial assessment by DCAA• Each system is to be approved by ACO/DACO/CACO
– Must rely on DCAA input unless basis exists to reject– Rejection requires board of review– Potential for “rubber stamping” DCAA assessment
System Approval Fundamentals:No Significant Deficiency
• The DFARS Rule defines a significant deficiency as:– “A shortcoming in the system that materially affects the ability of
officials of [DOD] to rely upon information produced by the system that is needed for management purposes”
• The DFARS Rule does not define “material” – Focus is on DOD’s perspective– SOX/GAAP standards unlikely determinative
• Comments to Final Rule suggest any failure to comply with a system criterion is material and a significant deficiency
No Significant Deficiency (cont.)
• DCAA guidance– Significant deficiency results from a “material weakness”– Material weakness means
• A “deficiency or combination of deficiencies, in internal control over compliance such that there is a reasonable possibility that a material noncompliance with a compliance requirement . . . will not be prevented, or detected and corrected on a timely basis”
– Material noncompliance means a material noncompliance with DFARS criteria
No Significant Deficiency (cont.)
• DCAA guidance (cont.)– Material
• A “reasonable possibility that noncompliance with DFARS criteria will result in material noncompliance with . . . applicable government contract laws and regulations, thus materially affecting reliability of the data produced by the system”
• Is not financial reporting standard• Monetary impact not required
No Significant Deficiency (cont.)
• DCMA guidance– Consistent with DCAA guidance– Material weaknesses include impacts on any of the following
• Information the government relies upon to fund contracts• Information the government relies upon to price contracts
• Summary – Judgmental– While circular in concept, the fundamentals are
• A noncompliance with a DFARS criterion• That is a material noncompliance with law, regulation or contract• That impacts the reliability of contractor data
No Significant Deficiency (cont.)
• DCAA’s low materiality threshold, combined with a vague deficiency standard, means that the burden will be on the Contractor to show a deficiency is not significant
• Based on the facts and circumstances; e.g.,– System design issue
• Final control point vs.• Initial entry subject to final control
– Compliance issue• Number of occurrences (e.g., bad statistical sample)• Amount involved
• Based on system criteria in the relevant system clauses in covered contracts
No Significant Deficiency (cont.)
• Electronic systems that have – Strong internal controls– Function with minimum errors– Are integrated, reconcilable and auditable
• Robust written policies and procedures that clearly describe the relevant systems and compliance requirements
• Robust practices for ensuring compliance– Training– Internal compliance reviews– Internal controls (e.g., approval levels)
• Appropriate and timely resolution of deviations
Approval Fundamentals: Key System Criteria
• Includes cost accounting, cost allowability, billing and ethics• Selected areas of focus
– CAS compliance for contract element cost allocations (among CLINs, sub-CLINS or units, etc.); relates to EVMS Requirements
– Indirect cost allocations• Consistent practices • Reflective of beneficial/causal relationship
– Cost allowability, including costs of subcontracts, bid and proposal, professional and legal services, travel and insurance
Accounting System
• Selected areas (cont.)– Internal controls
• Automated and integrated systems with built-in segregation of employee responsibilities
• Internal audit activity– Functionally and organizationally independent– Has requisite specialized knowledge to review government contract
compliance– Periodically performed
• System output can be and is reconciled– Financial ledgers vs. contract ledgers– Sub-ledgers vs. general ledger
Accounting System (cont.)
• Selected areas (cont.)– Internal controls (cont.)
• Error correction– Timely – Accurate
• Ethics practices– Timely investigation of issues– Documented resolution and corrective action– Appropriate disclosures and follow-up
– Billings• Errors minimized• Reconciled to ledgers
Accounting System (cont.)
• Selected areas of focus– Integration of estimating and purchasing systems– Enforcing TINA requirements – Timely and adequate cost and price analysis– Contemporaneous and written justification for lack of competition– Authorized personnel make purchases based on authorized
requisitions– Subcontracts executed before any work is performed– Update policies and procedure routinely and immediately
following new requirements added to FAR/DFARS– Training
Purchasing System
• New requirement (DFARS § 252.215-7002(d)(1), d(3), (g))– Disclose system in writing to ACO– Comply with the system as disclosed– “Timely” disclosure to ACO of “significant changes”– Significant deficiencies can prompt withholds in accordance with
DFARS § 252.242-7005, when included in the contract• Estimating system integration with
– Accounting system– Purchasing system– Budgeting process
Estimating System
• Selected areas of focus– Internal audits of estimating system
• Too infrequent• Lack adequate scope• Inadequately documented corrective action
– Independent management review• Insufficient management oversight concerning high-risk proposals• Unclear lines of authority for proposal approval• Lack of documentation of review/approval process
Estimating System (cont.)
• Selected areas (cont.)– Continuous system maintenance/improvement
• Use internal audits to identify weaknesses and implement corrective action(s); document improvements
• Comprehensive training plan• Timely policy revisions/distribution of updates
– Sharp focus on proposal format and content• Comply with FAR § 15.408, Table 15-2 (when TINA applies)
Estimating System (cont.)
• Selected areas of focus– Failure to properly tag and track government property
• Failure to appreciate the breadth of what is government property under cost-type and T&M contracts
• Use of government property for commercial operations without Government approvals
• Comingling of contractor and government property; capital type projects that integrate government funding susceptible to challenge
Government Property System
• Selected areas (cont.)– Government property terms and conditions are not flowed down
to subcontractors– Lack of audits/due diligence to ensure government property
systems are functioning• Assumption/reliance upon accounting/purchasing systems to track
government property• Can result in difficult process to identify government property when
questions arise during audits or closeout
Government Property System (cont.)
• Additional risks of noncompliance– Cost disallowances because unable to substantiate that property
was actually purchased in the quantities for which reimbursement is sought
– Contractor bears the risk of loss resulting from willful misconduct or lack of good faith by a contractor’s managerial personnel
Government Property System (cont.)
• Selected areas of focus– Lack of controls to timely identify unauthorized transfers/loans of
materials– Inadequate inventory controls to track removal of materials from
inventory and use on contract– Failure to ensure that loaned materials are timely repaid to
lending contract – Transfers/loans from cost-type contracts to commercial or fixed
price work
MMAS
• Selected areas of focus– Lack of necessary integration between systems (accounting,
purchasing, inventory, program management) to support EVMS– Failure to update budgets and schedules as a result of changes,
modifications, government delays– Inadequate accounting for level of effort functions in program
budgets– Adjustments of incurred costs lack reconciliation to accounting
records
EVMS
• Resist temptation to accept imposition of what the government says is a “better” practice when the current practice is better for your business
• Monitor government procurement decisions – Timely file appropriate bid protests
• If excluded from a procurement, must file pre-proposal
• If not awarded, obtain a debriefing and ascertain the impact of unapproved system(s)
Potential Issues
• Established at the DCMA HQ Level• Chaired by the Director, Contract Policy
– Members include: HQs Legal Counsel, System PolicyOwner, CBS Policy Owner
– Purpose: To ensure regulations/policy are followed,consistently applied, action is sufficiently documented, and proper coordination has occurred
● 18 Reviews as of October 2012:– 8 Accounting, 3 Estimating, 2 EV, 3 Property, 2 Purchasing– 12 disapproved systems– 3 pending further fact finding– 3 panel recommended alternative course of action
DCMA Implementation,Business System Review Panels
Jami Levy
Pricing Manager, Northrop Grumman Corporation
30
“data driven” Estimating – Key to Estimating System Health
Jami LevyNorthrop Grumman Aerospace Systems
Pricing and Estimating Compliance Systems, Training and Tools
31
“data driven” Estimating
Why Basis of Estimate (BOE) Focus?
BOE - Description
“data driven” Estimating - Definition
Fundamentals of “data driven” Estimating
“data driven” Estimate Examples – We are the Customer
Non-Proprietary BOE Training Material Available to All
32
Background/Objectives
Background: There is a significant industry effort to increase the quality of contractor estimating processes. The industry has applied the term “data driven” estimating to this new method of developing and documenting our estimates/BOEs. “data driven” Estimating methods have certain characteristics that are starting to emerge as standards within our industry.
Objectives:— What is driving our industry toward the use of data driven estimating
methods and why is this a business imperative
— Be able to identify the characteristics of good data driven estimating processes and how they are represented in our BOEs
— Provide examples of “data driven” versus non-data driven BOEs
— Raise the bar by providing access to non-proprietary BOE training
33
Responsibilities
Mapping of DFARS 252.215.7002 Cost Estimating System Requirements to Major Estimating System Components
Roles
Training
Estimate Data Sources
Estimate Preparation
Estimate Review
Estimate Approval
Internal Controls
Well Supported Data Driven Estimating in Key to Estimating System Health
1. Establish clear responsibility for preparation, review, and approval of cost estimates and budgets.
2. Provide a written description of the organization and duties of the personnel responsible for preparing, reviewing, and approving cost estimates and budgets.
3. Ensure that relevant personnel have sufficient training, experience, and guidance to perform estimating and budgeting tasks in accordance with the established procedures.
4. Identify and document the sources of data and the estimating methods and rationale used in developing cost estimates and budgets.
9. Provide for the use of historical experience, including historical vendor pricing information, where appropriate.
11. Integrate information available from other management systems.
15. Provide procedures that ensure subcontract prices are reasonable based on a documented review and analysis provided with the prime proposal, when practicable.
5. Provide for adequate supervision throughout the estimating and budgeting process.
6. Provide for consistent application of estimating and budgeting techniques.
7. Provide for detection and timely correction of errors.
8. Protect against cost duplication and omissions.
10. Require use of appropriate analytical methods.
14. Provide procedures to update cost estimates and notify the Contracting Officer in a timely manner throughout the negotiation process.
12. Require management review, including verification of the estimating and budgeting policies, procedures, and practices .
13. Provide for internal review of, and accountability for, the acceptability of the estimating system, including the budgetary data supporting indirect cost estimates and comparisons of projected results to actual results, and an analysis of any differences.
16. Provide estimating and budgeting practices that consistently generate sound proposals that are compliant with the provisions of the solicitation and are adequate to serve as basis to reach a fair and reasonable price.
17. Have an adequate system description, including policies, procedures, and estimating and budgeting practices, that comply with the Federal Acquisition Regulation and Defense Federal Acquisition Regulation Supplement.
34
DFARS 252.215.7002 Cost Estimating System Requirements4. Identify and document the sources of data and the estimating methods
and rationale used in developing cost estimates and budgets.
9. Provide for the use of historical experience, including historical vendor pricing information, where appropriate.
11. Integrate information available from other management systems.
6. Provide for consistent application of estimating and budgeting techniques.
10. Require use of appropriate analytical methods.
15. Provide procedures that ensure subcontract prices are reasonable based on a documented review and analysis provided with the prime proposal, when practicable. Provide for adequate supervision throughout the estimating and budgeting process.
Estimate Review
Estimate Data Sources
Estimate Preparation
35
Purpose of a BOE
Documents and substantiates effort required to perform tasks — Documents basis and assumptions— Details understanding of requirements— Conveys credibility by clearly defining tasks and assumptions— Demonstrates that estimates are credible, defendable and realistic
(historical data)
Provides evidence that estimate is consistent with other proposal volumes
Time-phased resources consistent with IMP/IMS Allocates resources such as skill mix appropriate for task Aligns with Work Breakdown Structure (WBS), SOW, and other
product structures Meet customer funding, timing, and/or scheduling constraints
36
BOEs – The Components A BOE consists of the following main elements:
1. Work Element ID – WBS, SOW, Location, CLIN, NRE/Rec…2. Description of the products or services delivered and the tasks
to be performed3. Estimating method, rationale, assumptions, and computations
used to develop the estimate— Method of quoting
4. Relevant historical data and comparability of the historical data to the current program— Explanation of any adjustments made to historical cost or
programmatic data (i.e., use of modifiers)
5. Estimated labor and ODC resources to perform the defined tasks
6. Expenditure profile (time-phasing) of resources7. Other elements as required by the cost proposal instructions
37
BOE Estimating Methods Estimates are developed using one or more of these typical
estimating methods— Judgment – based on subject matter expert (SME) experience with similar
efforts. — Level-of-Effort/Support - uses resource-based estimates that are often
based on a predetermined level of support for a given period of time — Analogy/Comparison Method – estimate based on historical frame of
reference from a similar effort with adjustments for programmatic or technical differences
— Standard Time Estimating - Labor Standards and Realization/ Performance Factors - time necessary to complete a defined element of work following a prescribed (recognized) technique or method.
— Unit Method – generally a simple ratio (not always statistically validated)— Follow-on Based On Actuals – extrapolation of actuals (CEAC, production)— Parametric Cost Models – use of calibrated, internally developed, or
commercially available cost models — Cost Estimating Relationships (CERs) - equations developed based on
historical, statistically correlated relationships(See full training package for more details)
38
Customer Expectations
Customers/DCAA/DCMA expect contractors to begin adopting improved estimating methods to support TINA-compliant proposals and other estimating products
Least Preferred
Judgment
Most Preferred
“Fact Based”Estimating Methods
Move To Fact Based ASAP
39
BOE Methodologies – Credibility Scale
Historical resource based estimates are easier to support than judgmental type estimates.
Customer Credibility LaborHigh Follow-on Projections
based on ActualsAnalogy/Comparison
CER’sParametric Estimates /
Cost ModelsStandards
Unit MethodsLevel Of Effort
Low Judgment
40
Customer Consequences – Poor Estimating
Failure to be responsive to customer estimating processes could be grounds for— Non-compliance with RFP or government regulations— Rejection of proposal and possibly defective pricing— Holdbacks
Penalties for bad BOEs include:— Questioned costs— DCAA/Government Flash Report— Negotiation losses— Negative factor in competitive evaluations— Defective pricing and fraud prosecution— Suspension of bidding and debarment
41
“data driven” BOEs – Basic Definition Fact Based Estimating: Estimating processes/BOEs that
provide the supporting information and rationale necessary for accurate, credible and traceable estimates
Information includes:— Use of defined estimating methods
— Historical data from past programs
— Use of objective scope measures and complexity factors where appropriate
Rationale addresses:— Complete description of the estimating approach
— Explanation of the applicability of the historical information
Estimating process must be documented, consistent and repeatable
42
Fact Based BOEs – The Fundamentals
1. Estimate developed using a defined and documentedestimating processes
2. Methods rely on scope measures (attributes) and complexity factors (parametrics) – when applicable
3. Complete estimate development process is explained and documented in the BOE — Professional judgments, modifiers/adjustment factors, and historical
context is adequately explained— Process is consistent and repeatable
4. Estimate relies on relevant historical data and context that is accessible and verifiable — Shows use of latest and most current relevant data— Supports all resource estimates with verifiable data
5. Estimating processes are monitored for learning and improvement as required to support future estimates
43
Objective Work-Scope Measures (OSM)
Scope Measures = estimating predictive measures – technical measures relating to the size of the effort (attributes)— Generally physical and technical measures— Sized at different product levels - system, subsystem, box, etc. — Example:
— Work Products: # drawings by type, sheet counts, # documents & sheet count, SLOC by category, # requirements, # reviews, # test & type, weight, quantities, etc.
Traits/Complexity Factors: characteristics that can improve the accuracy of relationships developed for the OSMs — Example:
— Scope Measures = Drawings— Traits: (numeric and descriptive): # Drawings, Type of Drawing; Drawing
Heritage; Item Next Higher Assembly/Element; Number of Sheets; # of Revisions
— Example: — Scope Measures = Source Lines of Code (SLOC)— Traits: Software Language; New SLOC; Modified SLOC; Reuse SLOC; Actual
Total Delivered SLOC, Deleted SLOC, Auto Generated Code, etc
44
Why Use Scope Measures and Traits in BOEs?
Scope Measures - help to scope the size of the effort to be estimated — Objective quantitative measures of scope to be performed — Directly correlate to (predictors) resource estimates for work products and
tasks — Understanding the size (scope) of the proposed effort is critical to
developing an accurate estimate (BOE) — The BOE author should explain the derivation, source, and application of
the scope measures in the BOE
Traits/Complexity Modifiers – address unique characteristics of the scope measures that can affect required effort to produce— Capture the effects of technical and programmatic differences between
historical programs and the current proposed effort.— The “produceability” can vary widely for any scope measure - traits can
be used to explain the differences— Sometimes requires judgment
45
Explaining Adjustment to a BOE
Provided below are some examples of adjustments that may be required to improve the accuracy of the estimate. The list is not meant to be totally inclusive of adjustments an estimator has available in their tool box.— Technical Complexity ~ Labor Estimate
— Improvement Curves ~ Labor Estimate
— Schedule adjustments ~ Labor Estimate
— Quote Decrement ~ Material Estimate
— Quantity Adjustment ~ Material Estimate
— Escalation ~ Material Estimate
— Exchange Rate ~ Material Estimate
In order to use an objective or subjective adjustment, an estimator must explain it thoroughly.
46
Using Historical Data in a BOE
Use of Historical Data Must Address:— Source of data – generally from the organization’s certified
historical data repositories— Complete identification of the nature, source and details of
relevant historical data
Ensure compliance with A.C.T.T.R. by documenting the following: — Actual historical hours— Charge numbers included— Technical data used— Time frame of historical data— Repository or source of historical data
47
“data driven” Estimating - Examples
We are all confronted in our daily lives with estimates for products and services— E.g. Home remodel, auto repair
We evaluate the credibility of these estimates based upon the content of the estimate
We need to apply this customer mentality when we review our own proposal cost volumes and BOEs
Example: Everyone has dealt with an automobile repair estimate that we struggle to feel comfortable with. Evaluate
the following BOEs for automobile repair service – how do we as consumers know that we are getting a fair price for these
services
48
D&G Auto – Poor, Fair or Good BOECriteria
• Description of the products or services delivered and the tasks to be performed
• Estimated labor resources to perform the defined tasks
• Time-phasing of labor resources
• The estimating method, rationale, assumptions, and computations used to develop the estimate
• Identification of the relevant historical data and comparability of the historical program to the current program
• Explanation of any adjustments made to the historical data or estimate (i.e., use of modifiers)
49
D&G Auto – Poor, Fair or Good BOECriteria
• Description of the products or services delivered and the tasks to be performed
• Estimated labor resources to perform the defined tasks
• Time-phasing of labor resources
• The estimating method, rationale, assumptions, and computations used to develop the estimate
• Identification of the relevant historical data and comparability of the historical program to the current program
• Explanation of any adjustments made to the historical data or estimate (i.e., use of modifiers)
50
D&G Auto – Poor, Fair or Good BOECriteria
• Description of the products or services delivered and the tasks to be performed
• Estimated labor resources to perform the defined tasks
• Time-phasing of labor resources
• The estimating method, rationale, assumptions, and computations used to develop the estimate
• Identification of the relevant historical data and comparability of the historical program to the current program
• Explanation of any adjustments made to the historical data or estimate (i.e., use of modifiers)
51
“data driven” Estimating – The Challenges Ahead
Focus – Improve estimating products across all estimating stake-holders (e.g. BOEs, EV, CAPE, ICE, Business Case Analysis)
The debate as we evolve to our future –— At what level will we be generating estimates (e.g. subsystem,
assembly)— How will we document our estimating processes
Historical databases, processes and strategies need to be flexible as we evolve toward the future “improved” estimating products— Estimating, Cost Modeling and Engineering teams working
together— As the desired characteristics of these high quality estimating
methods evolve – our methods must evolve also.
Immediate needs: — Documented estimating process — Good historical data — Libraries of BOEs demonstrating data driven estimating
approaches
52
Back Up
53
Estimated Cost (New Design) = Historical Cost (Similar Design) + Complexity or Scaling Factor
Estimating Method – Analogy/Comparison
Analogy/Comparison: This method is based on historical context or frame of reference from a similar effort. Compares items being estimated to items of similar configuration and known cost to produce estimate— Includes use of historical data— Addresses differences between historical data and proposed solution
being estimated— May use relevant estimating attributes and complexity or adjustment
factors (modifiers) to reflect programmatic, technology, requirements or technical differences
— Clearly document in BOE why and how factors are used and validated
54
Judgment: Least preferred, least credible estimating method –very subject to bias— Generally not based upon historical data, scope measures or other sound
estimating approaches
Relies on SME estimate’s of small work packages aggregated to complete hardware, software, etc tasks— State author’s experience from which to make an estimate
— Programs worked; Years in industry
— Decompose effort into well defined tasks and subtasks
Disclose if historical methods were investigated and not used, and why they were not used
Estimating Method - Judgment
Rarely are we asked to quote something that does not have some similar historical likeness !
55
Estimating Method - LOE Level of Effort - Use when customer specifies effort to be
provide over a stated period of time on work that can be stated only in general terms— Requirement for specific skills/talents that tend to be headcount driven— Used when product cannot be well defined
Contract is for delivery of hours not product. Payment based on effort expended not results achieved.— Restricted LOE Contracts — SOW that specifically calls out LOE by customer
What is NOT LOE— Level loading of labor not specifically directed by customer— Staffing a position (i.e. Program Manager) as 155 hrs/month during
period of performance on a completion type contract
David E. Trotta
Estimating Manager, ViaSat, Inc.
57
Estimating Systems Overview and Audit ScopeFebruary 27, 2013
Presented by David E. Trotta
Agenda
• Estimating System Overview and Requirements• What to Expect in an Audit• Steps to Compliance• Questions
Estimating Overview• Estimating System▫ Need to disclose system in writing▫ Comply with the system as disclosed▫ You must make “timely” disclosure to ACO of
“significant changes”▫ Significant deficiencies can prompt withholds in
accordance with DFARS 252.242-7005▫ Integrate with other systems (e.g., Accounting
System)▫ Meet requirements of law and regulation
Cost Estimating System Requirements –DFARS 252.215-7002 “Estimating system” means the Contractor's policies, procedures, and practices for budgeting and planning controls, and generating estimates of costs and other data included in proposals submitted to customers in the expectation of receiving contract awards. Estimating system includes the Contractor's—
(1) Organizational structure;(2) Established lines of authority, duties, and responsibilities;(3) Internal controls and managerial reviews;(4) Flow of work, coordination, and communication; and(5) Budgeting, planning, estimating methods, techniques, accumulation of historical costs, and other analyses used to generate cost estimates.
DFARS Requirement 215.407-5-70
• Requires all DoD contractors to have acceptable estimating systems in writing, provides guidelines concerning the characteristics of an adequate estimating system
• Creates the requirement of the contracting officer to determine adequacy of the cost estimating system
“Acceptable estimating system” means... system that complies with the system criteria… and provides for a system that—
(1) Is maintained, reliable, and consistently applied;(2) Produces verifiable, supportable, documented, and timely cost estimates that are an acceptable basis for negotiation of fair and reasonable prices;(3) Is consistent with and integrated with the Contractor’s related
management systems; and(4) Is subject to applicable financial control systems.
“Significant Estimating System Deficiency” means
• “Significant deficiency” means a shortcoming in the system that materially affects the ability of officials of the Department of Defense to rely upon data and information produced by the system that is needed for management purposes.(252.215-7002 Cost Estimating System Requirements)
Estimating System• Is different from other system because of it’s unique
nature• Comprised of policies and procedures as well as
input from other systems• A fair amount of judgment can be involved on any
particular estimate; what data from what systems is relevant▫ Often times changes to the solution being proposed
does not easily tie back to prior programs or organization changes complicates how past data is relevant
Changes in the Business Systems Rules - From ‘Should’ to ‘Shall’Cost Estimating System Requirements – DFARS 252.215-
7002 (4) The Contractor’s estimating system shall provide for the use of appropriate source data, utilize
sound estimating techniques and good judgment, maintain a consistent approach, and adhere to established policies and procedures. An acceptable estimating system shall accomplish the following functions:
(i) Establish clear responsibility for preparation, review, and approval of cost estimates and budgets;
(ii) Provide a written description of the organization and duties of the personnel responsible for preparing, reviewing, and approving cost estimates and budgets;
(iii) Ensure that relevant personnel have sufficient training, experience, and guidance to perform estimating and budgeting tasks in accordance with the Contractor's established procedures;
(iv) Identify and document the sources of data and the estimating methods and rationale used in developing cost estimates and budgets;
(v) Provide for adequate supervision throughout the estimating and budgeting process;(vi) Provide for consistent application of estimating and budgeting techniques; (vii) Provide for detection and timely correction of errors;(viii) Protect against cost duplication and omissions;(ix) Provide for the use of historical experience, including historical vendor pricing information,
where appropriate;
From ‘Should’ to ‘Shall’ (continued)
Cost Estimating System Requirements – DFARS 252.215-7002 (continued)
(x) Require use of appropriate analytical methods;(xi) Integrate information available from other management systems;(xii) Require management review, including verification of the company's estimating and
budgeting policies, procedures, and practices;(xiii) Provide for internal review of, and accountability for, the acceptability of the
estimating system, including the budgetary data supporting indirect cost estimates and comparisons of projected results to actual results, and an analysis of any differences;
(xiv) Provide procedures to update cost estimates and notify the Contracting Officer in a timely manner throughout the negotiation process;
(xv) Provide procedures that ensure subcontract prices are reasonable based on a documented review and analysis provided with the prime proposal, when practicable;
(xvi) Provide estimating and budgeting practices that consistently generate sound proposals that are compliant with the provisions of the solicitation and are adequate to serve as a basis to reach a fair and reasonable price; and
(xvii) Have an adequate system description, including policies, procedures, and estimating and budgeting practices, that comply with the Federal Acquisition Regulation and Defense Federal Acquisition Regulation Supplement.
Penalties associated with Significant Estimating System Deficiencies• Contracts subject to withholds are contracts
subject to CAS coverage that contain DFARS 252.242-7005 and the applicable clause▫ Subject to limitation, ACO has discretion to identify
contracts from which to withhold payments• Amount▫ 5% for one disapproved system or up to 10% for two or
more disapproved systems, of all billings under covered contracts
▫ Pre-existing withholds should preclude or reduce further withholds for system disapproval
Payments subject to withhold• Interim payments under▫ Cost-reimbursement contracts; Incentive type
contracts; Time and Material contracts; Labor-hour contracts; Progress payments; Performance based payments
• Once contracting officer is notified of deficiencies correction CO will discontinue withhold if corrections are sufficient▫ Has 90 days to make this determination or future
withhold percentage is reduced by at least 50%
Pre Audit Process (Auditor’s Perspective)• Auditor will notify contractor of planned audit of
the Estimating System• Auditor will review prior work done by auditors
and any flash estimating system reports• Auditor will contact DCMA about any known
concerns• Auditor will request a listing of proposals for the
last 12 months▫ Summarized by customer, contract type and dollar
amount
Entrance Conference will be Conducted• Contractor will be asked▫ To provide a system orientation or briefing▫ Outline any changes to the Estimating System
since the last examination▫ Discuss the contractor’s risk assessment process▫ Discuss the contractor’s monitoring process▫ Discuss any prior weaknesses and actions or
improvements steps taken to address these weaknesses
• Auditor will prepare an internal risk assessment
Control Environment/Information & Communications• Auditor will evaluate the internal controls this
includes looking at ICAPS, integrity and ethical values, competence, Board/Audit committee participation, management philosophy, authority and structure, HR policies and Financial capability
• Access overall accounting controls, IT systems, files, any recent related changes and query contractor staff on estimating system and process
Contractor Monitoring Process / IA /System Description• Audit will then look at the contractor’s ongoing
monitoring process, including internal audit, any external reviews and controls in place
• Including recommendation and follow up as well as review frequency
• Review of Estimating System Description▫ Organization charts, written policy, procedures
and directives, flowcharts, lines of authority, review and approval, written instructions and integration with other Management Systems
Training• Determine if regular training is required for
those responsible for preparing estimates• Includes training of the use of statistical aids and
advanced estimating techniques• Training covers estimating system policies and
procedures• Auditor will ask to examine records of training
Cost Estimating Development
• Are estimates developed using appropriate methods, compliance with CAS 401 and 402
• Use of historical data, escalation methods, use of Forward Pricing factors, formula pricing methods, catalog pricing , format of proposal (Table 15-2),Certification of Cost or Pricing Data and Subcontractor policies
Cost Estimating Development (continued)• Selective evaluation of recent pricing proposals
for adherence to procedures, • Determine if the following characteristics are in
place▫ Adequate supervision, management reviews both
interim and final reviews that cover soundness of estimates and adherence to policy
Audit Conclusion• Auditor does an assessment of the contractor• Makes determination of adequacy• Prepares a draft audit report• Conducts an exit conference• Finalize the audit report incorporating the
contactor’s response
Steps to Compliance• Start now, don’t wait until you are being notified of
an Estimating System Audit• Evaluate your Estimating System to ensure required
changes are included▫ Consider 3rd party / consultant
• Ensure you have your management’s support• Review Guidance Provided▫ DCAA Audit Program is posted on DCAA website
under Standard Audit Programs▫ Additional guidance is provided in DCAA’s Contract
Audit Manual (CAM)
Steps to Compliance (continued)• Document your Estimating Policies and
Procedures▫ Update or create an Estimating System Manual
• Ensure the controls are in place but make sure you have the flexibility to respond to all proposals in a compliant manner
• When a deviation is required to comply with solicitation requirements document them in your proposal and review process
Steps to Compliance (continued)
• Training – this will greatly reduce the risk of problems and is an audit hot button
• Build your controls and system around your business, there is no one size fits all▫ Services are very different from products▫ Larger businesses may have different policies and
procedures for different business segments• Auditor may not be familiar with your business,
don’t assume DCAA staff currently reviewing your business will conduct the audit▫ Spend the time necessary to brief and explain your
unique issues and approaches
Steps to Compliance (continued)• During the Audit engage with your auditors▫ Be responsive to issue▫ Be proactive, if there is problem start mitigating it
now▫ Demonstrate mitigation steps before the final
audit report is issued, request briefing and draft report before report is released
▫ If issues can’t be resolved with auditor, contractor should provide the auditor with contractors written position to be included in the report
Conclusion• Be proactive and get your management’s support• The proposal you are doing now will likely be
looked at during the Estimating System Audit so start now
• Establish a Training and Communication Plan• Any single failure can result in a inadequacy
determination so don’t create a hurdle you can’t clear
For more information about our organizations visit us on our websites
www.sdaga.org
www.ncmasd.org
www.ispa-cost.org