How to Pass an Estimating System Review

How to Pass an Estimating System

Review

February 27, 2013

ANNOUNCEMENTS

Mr. James J. GallagherPartner, McKenna Long & Aldridge LLP

Ms. Jami LevyPricing Manager, Northrop Grumman Corporation

Mr. David E. TrottaEstimating Manager, ViaSat Inc.

The DOD Business Systems Rule

February 27, 2013

Jay GallagherMcKenna Long & Aldridge LLP

• Applicable through clauses in DOD, CAS-covered contracts awarded after May 2011

• Imposes numerous system requirements, including SOX like internal controls procedures

• Approval by ACO/DACO/CACO based upon DCAA opinion• Applies to the “contractor,” meaning the legal entity• Rule citations

– Interim rule (76 Fed. Reg. 28,856 (May 11, 2011))– Final rule (77 Fed. Reg. 11,355 (Feb. 24, 2012))

Overview of Final Rule

• Requires existence of six approved systems – Accounting (includes internal controls and billing)– Purchasing– Estimating– Materials Management and Accounting– Government Property– Earned Value Management

Overview of Final Rule (cont.)

Overview of System Approval Fundamentals

• Approval requires that no “significant deficiency” exists– Affirmative finding– What if no finding?

• Initial assessment by DCAA• Each system is to be approved by ACO/DACO/CACO

– Must rely on DCAA input unless basis exists to reject– Rejection requires board of review– Potential for “rubber stamping” DCAA assessment

System Approval Fundamentals:No Significant Deficiency

• The DFARS Rule defines a significant deficiency as:– “A shortcoming in the system that materially affects the ability of

officials of [DOD] to rely upon information produced by the system that is needed for management purposes”

• The DFARS Rule does not define “material” – Focus is on DOD’s perspective– SOX/GAAP standards unlikely determinative

• Comments to Final Rule suggest any failure to comply with a system criterion is material and a significant deficiency

No Significant Deficiency (cont.)

• DCAA guidance– Significant deficiency results from a “material weakness”– Material weakness means

• A “deficiency or combination of deficiencies, in internal control over compliance such that there is a reasonable possibility that a material noncompliance with a compliance requirement . . . will not be prevented, or detected and corrected on a timely basis”

– Material noncompliance means a material noncompliance with DFARS criteria


• DCAA guidance (cont.)– Material

• A “reasonable possibility that noncompliance with DFARS criteria will result in material noncompliance with . . . applicable government contract laws and regulations, thus materially affecting reliability of the data produced by the system”

• Is not financial reporting standard• Monetary impact not required


• DCMA guidance– Consistent with DCAA guidance– Material weaknesses include impacts on any of the following

• Information the government relies upon to fund contracts• Information the government relies upon to price contracts

• Summary – Judgmental– While circular in concept, the fundamentals are

• A noncompliance with a DFARS criterion• That is a material noncompliance with law, regulation or contract• That impacts the reliability of contractor data


• DCAA’s low materiality threshold, combined with a vague deficiency standard, means that the burden will be on the Contractor to show a deficiency is not significant

• Based on the facts and circumstances; e.g.,– System design issue

• Final control point vs.• Initial entry subject to final control

– Compliance issue• Number of occurrences (e.g., bad statistical sample)• Amount involved

• Based on system criteria in the relevant system clauses in covered contracts


• Electronic systems that have – Strong internal controls– Function with minimum errors– Are integrated, reconcilable and auditable

• Robust written policies and procedures that clearly describe the relevant systems and compliance requirements

• Robust practices for ensuring compliance– Training– Internal compliance reviews– Internal controls (e.g., approval levels)

• Appropriate and timely resolution of deviations

Approval Fundamentals: Key System Criteria

• Includes cost accounting, cost allowability, billing and ethics• Selected areas of focus

– CAS compliance for contract element cost allocations (among CLINs, sub-CLINS or units, etc.); relates to EVMS Requirements

– Indirect cost allocations• Consistent practices • Reflective of beneficial/causal relationship

– Cost allowability, including costs of subcontracts, bid and proposal, professional and legal services, travel and insurance

Accounting System

• Selected areas (cont.)– Internal controls

• Automated and integrated systems with built-in segregation of employee responsibilities

• Internal audit activity– Functionally and organizationally independent– Has requisite specialized knowledge to review government contract

compliance– Periodically performed

• System output can be and is reconciled– Financial ledgers vs. contract ledgers– Sub-ledgers vs. general ledger

Accounting System (cont.)

• Selected areas (cont.)– Internal controls (cont.)

• Error correction– Timely – Accurate

• Ethics practices– Timely investigation of issues– Documented resolution and corrective action– Appropriate disclosures and follow-up

– Billings• Errors minimized• Reconciled to ledgers

Accounting System (cont.)

• Selected areas of focus– Integration of estimating and purchasing systems– Enforcing TINA requirements – Timely and adequate cost and price analysis– Contemporaneous and written justification for lack of competition– Authorized personnel make purchases based on authorized

requisitions– Subcontracts executed before any work is performed– Update policies and procedure routinely and immediately

following new requirements added to FAR/DFARS– Training

Purchasing System

• New requirement (DFARS § 252.215-7002(d)(1), d(3), (g))– Disclose system in writing to ACO– Comply with the system as disclosed– “Timely” disclosure to ACO of “significant changes”– Significant deficiencies can prompt withholds in accordance with

DFARS § 252.242-7005, when included in the contract• Estimating system integration with

– Accounting system– Purchasing system– Budgeting process

Estimating System

• Selected areas of focus– Internal audits of estimating system

• Too infrequent• Lack adequate scope• Inadequately documented corrective action

– Independent management review• Insufficient management oversight concerning high-risk proposals• Unclear lines of authority for proposal approval• Lack of documentation of review/approval process

Estimating System (cont.)

• Selected areas (cont.)– Continuous system maintenance/improvement

• Use internal audits to identify weaknesses and implement corrective action(s); document improvements

• Comprehensive training plan• Timely policy revisions/distribution of updates

– Sharp focus on proposal format and content• Comply with FAR § 15.408, Table 15-2 (when TINA applies)

Estimating System (cont.)

• Selected areas of focus– Failure to properly tag and track government property

• Failure to appreciate the breadth of what is government property under cost-type and T&M contracts

• Use of government property for commercial operations without Government approvals

• Comingling of contractor and government property; capital type projects that integrate government funding susceptible to challenge

Government Property System

• Selected areas (cont.)– Government property terms and conditions are not flowed down

to subcontractors– Lack of audits/due diligence to ensure government property

systems are functioning• Assumption/reliance upon accounting/purchasing systems to track

government property• Can result in difficult process to identify government property when

questions arise during audits or closeout

Government Property System (cont.)

• Additional risks of noncompliance– Cost disallowances because unable to substantiate that property

was actually purchased in the quantities for which reimbursement is sought

– Contractor bears the risk of loss resulting from willful misconduct or lack of good faith by a contractor’s managerial personnel

Government Property System (cont.)

• Selected areas of focus– Lack of controls to timely identify unauthorized transfers/loans of

materials– Inadequate inventory controls to track removal of materials from

inventory and use on contract– Failure to ensure that loaned materials are timely repaid to

lending contract – Transfers/loans from cost-type contracts to commercial or fixed

price work

MMAS

• Selected areas of focus– Lack of necessary integration between systems (accounting,

purchasing, inventory, program management) to support EVMS– Failure to update budgets and schedules as a result of changes,

modifications, government delays– Inadequate accounting for level of effort functions in program

budgets– Adjustments of incurred costs lack reconciliation to accounting

records

EVMS

• Resist temptation to accept imposition of what the government says is a “better” practice when the current practice is better for your business

• Monitor government procurement decisions – Timely file appropriate bid protests

• If excluded from a procurement, must file pre-proposal

• If not awarded, obtain a debriefing and ascertain the impact of unapproved system(s)

Potential Issues

• Established at the DCMA HQ Level• Chaired by the Director, Contract Policy

– Members include: HQs Legal Counsel, System PolicyOwner, CBS Policy Owner

– Purpose: To ensure regulations/policy are followed,consistently applied, action is sufficiently documented, and proper coordination has occurred

● 18 Reviews as of October 2012:– 8 Accounting, 3 Estimating, 2 EV, 3 Property, 2 Purchasing– 12 disapproved systems– 3 pending further fact finding– 3 panel recommended alternative course of action

DCMA Implementation,Business System Review Panels

Jami Levy

Pricing Manager, Northrop Grumman Corporation

30

“data driven” Estimating – Key to Estimating System Health

Jami LevyNorthrop Grumman Aerospace Systems

Pricing and Estimating Compliance Systems, Training and Tools

[email protected]

31

“data driven” Estimating

Why Basis of Estimate (BOE) Focus?

BOE - Description

“data driven” Estimating - Definition

Fundamentals of “data driven” Estimating

“data driven” Estimate Examples – We are the Customer

Non-Proprietary BOE Training Material Available to All

32

Background/Objectives

Background: There is a significant industry effort to increase the quality of contractor estimating processes. The industry has applied the term “data driven” estimating to this new method of developing and documenting our estimates/BOEs. “data driven” Estimating methods have certain characteristics that are starting to emerge as standards within our industry.

Objectives:— What is driving our industry toward the use of data driven estimating

methods and why is this a business imperative

— Be able to identify the characteristics of good data driven estimating processes and how they are represented in our BOEs

— Provide examples of “data driven” versus non-data driven BOEs

— Raise the bar by providing access to non-proprietary BOE training

33

Responsibilities

Mapping of DFARS 252.215.7002 Cost Estimating System Requirements to Major Estimating System Components

Roles

Training

Estimate Data Sources

Estimate Preparation

Estimate Review

Estimate Approval

Internal Controls

Well Supported Data Driven Estimating in Key to Estimating System Health

1. Establish clear responsibility for preparation, review, and approval of cost estimates and budgets.

2. Provide a written description of the organization and duties of the personnel responsible for preparing, reviewing, and approving cost estimates and budgets.

3. Ensure that relevant personnel have sufficient training, experience, and guidance to perform estimating and budgeting tasks in accordance with the established procedures.

4. Identify and document the sources of data and the estimating methods and rationale used in developing cost estimates and budgets.

9. Provide for the use of historical experience, including historical vendor pricing information, where appropriate.

11. Integrate information available from other management systems.

15. Provide procedures that ensure subcontract prices are reasonable based on a documented review and analysis provided with the prime proposal, when practicable.

5. Provide for adequate supervision throughout the estimating and budgeting process.

6. Provide for consistent application of estimating and budgeting techniques.

7. Provide for detection and timely correction of errors.

8. Protect against cost duplication and omissions.

10. Require use of appropriate analytical methods.

14. Provide procedures to update cost estimates and notify the Contracting Officer in a timely manner throughout the negotiation process.

12. Require management review, including verification of the estimating and budgeting policies, procedures, and practices .

13. Provide for internal review of, and accountability for, the acceptability of the estimating system, including the budgetary data supporting indirect cost estimates and comparisons of projected results to actual results, and an analysis of any differences.

16. Provide estimating and budgeting practices that consistently generate sound proposals that are compliant with the provisions of the solicitation and are adequate to serve as basis to reach a fair and reasonable price.

17. Have an adequate system description, including policies, procedures, and estimating and budgeting practices, that comply with the Federal Acquisition Regulation and Defense Federal Acquisition Regulation Supplement.

34

DFARS 252.215.7002 Cost Estimating System Requirements4. Identify and document the sources of data and the estimating methods

and rationale used in developing cost estimates and budgets.

9. Provide for the use of historical experience, including historical vendor pricing information, where appropriate.

11. Integrate information available from other management systems.

6. Provide for consistent application of estimating and budgeting techniques.

10. Require use of appropriate analytical methods.

15. Provide procedures that ensure subcontract prices are reasonable based on a documented review and analysis provided with the prime proposal, when practicable. Provide for adequate supervision throughout the estimating and budgeting process.

Estimate Review

Estimate Data Sources

Estimate Preparation

35

Purpose of a BOE

Documents and substantiates effort required to perform tasks — Documents basis and assumptions— Details understanding of requirements— Conveys credibility by clearly defining tasks and assumptions— Demonstrates that estimates are credible, defendable and realistic

(historical data)

Provides evidence that estimate is consistent with other proposal volumes

Time-phased resources consistent with IMP/IMS Allocates resources such as skill mix appropriate for task Aligns with Work Breakdown Structure (WBS), SOW, and other

product structures Meet customer funding, timing, and/or scheduling constraints

36

BOEs – The Components A BOE consists of the following main elements:

1. Work Element ID – WBS, SOW, Location, CLIN, NRE/Rec…2. Description of the products or services delivered and the tasks

to be performed3. Estimating method, rationale, assumptions, and computations

used to develop the estimate— Method of quoting

4. Relevant historical data and comparability of the historical data to the current program— Explanation of any adjustments made to historical cost or

programmatic data (i.e., use of modifiers)

5. Estimated labor and ODC resources to perform the defined tasks

6. Expenditure profile (time-phasing) of resources7. Other elements as required by the cost proposal instructions

37

BOE Estimating Methods Estimates are developed using one or more of these typical

estimating methods— Judgment – based on subject matter expert (SME) experience with similar

efforts. — Level-of-Effort/Support - uses resource-based estimates that are often

based on a predetermined level of support for a given period of time — Analogy/Comparison Method – estimate based on historical frame of

reference from a similar effort with adjustments for programmatic or technical differences

— Standard Time Estimating - Labor Standards and Realization/ Performance Factors - time necessary to complete a defined element of work following a prescribed (recognized) technique or method.

— Unit Method – generally a simple ratio (not always statistically validated)— Follow-on Based On Actuals – extrapolation of actuals (CEAC, production)— Parametric Cost Models – use of calibrated, internally developed, or

commercially available cost models — Cost Estimating Relationships (CERs) - equations developed based on

historical, statistically correlated relationships(See full training package for more details)

38

Customer Expectations

Customers/DCAA/DCMA expect contractors to begin adopting improved estimating methods to support TINA-compliant proposals and other estimating products

Least Preferred

Judgment

Most Preferred

“Fact Based”Estimating Methods

Move To Fact Based ASAP

39

BOE Methodologies – Credibility Scale

Historical resource based estimates are easier to support than judgmental type estimates.

Customer Credibility LaborHigh Follow-on Projections

based on ActualsAnalogy/Comparison

CER’sParametric Estimates /

Cost ModelsStandards

Unit MethodsLevel Of Effort

Low Judgment

40

Customer Consequences – Poor Estimating

Failure to be responsive to customer estimating processes could be grounds for— Non-compliance with RFP or government regulations— Rejection of proposal and possibly defective pricing— Holdbacks

Penalties for bad BOEs include:— Questioned costs— DCAA/Government Flash Report— Negotiation losses— Negative factor in competitive evaluations— Defective pricing and fraud prosecution— Suspension of bidding and debarment

41

“data driven” BOEs – Basic Definition Fact Based Estimating: Estimating processes/BOEs that

provide the supporting information and rationale necessary for accurate, credible and traceable estimates

Information includes:— Use of defined estimating methods

— Historical data from past programs

— Use of objective scope measures and complexity factors where appropriate

Rationale addresses:— Complete description of the estimating approach

— Explanation of the applicability of the historical information

Estimating process must be documented, consistent and repeatable

42

Fact Based BOEs – The Fundamentals

1. Estimate developed using a defined and documentedestimating processes

2. Methods rely on scope measures (attributes) and complexity factors (parametrics) – when applicable

3. Complete estimate development process is explained and documented in the BOE — Professional judgments, modifiers/adjustment factors, and historical

context is adequately explained— Process is consistent and repeatable

4. Estimate relies on relevant historical data and context that is accessible and verifiable — Shows use of latest and most current relevant data— Supports all resource estimates with verifiable data

5. Estimating processes are monitored for learning and improvement as required to support future estimates

43

Objective Work-Scope Measures (OSM)

Scope Measures = estimating predictive measures – technical measures relating to the size of the effort (attributes)— Generally physical and technical measures— Sized at different product levels - system, subsystem, box, etc. — Example:

— Work Products: # drawings by type, sheet counts, # documents & sheet count, SLOC by category, # requirements, # reviews, # test & type, weight, quantities, etc.

Traits/Complexity Factors: characteristics that can improve the accuracy of relationships developed for the OSMs — Example:

— Scope Measures = Drawings— Traits: (numeric and descriptive): # Drawings, Type of Drawing; Drawing

Heritage; Item Next Higher Assembly/Element; Number of Sheets; # of Revisions

— Example: — Scope Measures = Source Lines of Code (SLOC)— Traits: Software Language; New SLOC; Modified SLOC; Reuse SLOC; Actual

Total Delivered SLOC, Deleted SLOC, Auto Generated Code, etc

44

Why Use Scope Measures and Traits in BOEs?

Scope Measures - help to scope the size of the effort to be estimated — Objective quantitative measures of scope to be performed — Directly correlate to (predictors) resource estimates for work products and

tasks — Understanding the size (scope) of the proposed effort is critical to

developing an accurate estimate (BOE) — The BOE author should explain the derivation, source, and application of

the scope measures in the BOE

Traits/Complexity Modifiers – address unique characteristics of the scope measures that can affect required effort to produce— Capture the effects of technical and programmatic differences between

historical programs and the current proposed effort.— The “produceability” can vary widely for any scope measure - traits can

be used to explain the differences— Sometimes requires judgment

45

Explaining Adjustment to a BOE

Provided below are some examples of adjustments that may be required to improve the accuracy of the estimate. The list is not meant to be totally inclusive of adjustments an estimator has available in their tool box.— Technical Complexity ~ Labor Estimate

— Improvement Curves ~ Labor Estimate

— Schedule adjustments ~ Labor Estimate

— Quote Decrement ~ Material Estimate

— Quantity Adjustment ~ Material Estimate

— Escalation ~ Material Estimate

— Exchange Rate ~ Material Estimate

In order to use an objective or subjective adjustment, an estimator must explain it thoroughly.

46

Using Historical Data in a BOE

Use of Historical Data Must Address:— Source of data – generally from the organization’s certified

historical data repositories— Complete identification of the nature, source and details of

relevant historical data

Ensure compliance with A.C.T.T.R. by documenting the following: — Actual historical hours— Charge numbers included— Technical data used— Time frame of historical data— Repository or source of historical data

47

“data driven” Estimating - Examples

We are all confronted in our daily lives with estimates for products and services— E.g. Home remodel, auto repair

We evaluate the credibility of these estimates based upon the content of the estimate

We need to apply this customer mentality when we review our own proposal cost volumes and BOEs

Example: Everyone has dealt with an automobile repair estimate that we struggle to feel comfortable with. Evaluate

the following BOEs for automobile repair service – how do we as consumers know that we are getting a fair price for these

services

48

D&G Auto – Poor, Fair or Good BOECriteria

• Description of the products or services delivered and the tasks to be performed

• Estimated labor resources to perform the defined tasks

• Time-phasing of labor resources

• The estimating method, rationale, assumptions, and computations used to develop the estimate

• Identification of the relevant historical data and comparability of the historical program to the current program

• Explanation of any adjustments made to the historical data or estimate (i.e., use of modifiers)

49








50








51

“data driven” Estimating – The Challenges Ahead

Focus – Improve estimating products across all estimating stake-holders (e.g. BOEs, EV, CAPE, ICE, Business Case Analysis)

The debate as we evolve to our future –— At what level will we be generating estimates (e.g. subsystem,

assembly)— How will we document our estimating processes

Historical databases, processes and strategies need to be flexible as we evolve toward the future “improved” estimating products— Estimating, Cost Modeling and Engineering teams working

together— As the desired characteristics of these high quality estimating

methods evolve – our methods must evolve also.

Immediate needs: — Documented estimating process — Good historical data — Libraries of BOEs demonstrating data driven estimating

approaches

52

Back Up

53

Estimated Cost (New Design) = Historical Cost (Similar Design) + Complexity or Scaling Factor

Estimating Method – Analogy/Comparison

Analogy/Comparison: This method is based on historical context or frame of reference from a similar effort. Compares items being estimated to items of similar configuration and known cost to produce estimate— Includes use of historical data— Addresses differences between historical data and proposed solution

being estimated— May use relevant estimating attributes and complexity or adjustment

factors (modifiers) to reflect programmatic, technology, requirements or technical differences

— Clearly document in BOE why and how factors are used and validated

54

Judgment: Least preferred, least credible estimating method –very subject to bias— Generally not based upon historical data, scope measures or other sound

estimating approaches

Relies on SME estimate’s of small work packages aggregated to complete hardware, software, etc tasks— State author’s experience from which to make an estimate

— Programs worked; Years in industry

— Decompose effort into well defined tasks and subtasks

Disclose if historical methods were investigated and not used, and why they were not used

Estimating Method - Judgment

Rarely are we asked to quote something that does not have some similar historical likeness !

55

Estimating Method - LOE Level of Effort - Use when customer specifies effort to be

provide over a stated period of time on work that can be stated only in general terms— Requirement for specific skills/talents that tend to be headcount driven— Used when product cannot be well defined

Contract is for delivery of hours not product. Payment based on effort expended not results achieved.— Restricted LOE Contracts — SOW that specifically calls out LOE by customer

What is NOT LOE— Level loading of labor not specifically directed by customer— Staffing a position (i.e. Program Manager) as 155 hrs/month during

period of performance on a completion type contract

David E. Trotta

Estimating Manager, ViaSat, Inc.

57

Estimating Systems Overview and Audit ScopeFebruary 27, 2013

Presented by David E. Trotta

Agenda

• Estimating System Overview and Requirements• What to Expect in an Audit• Steps to Compliance• Questions

Estimating Overview• Estimating System▫ Need to disclose system in writing▫ Comply with the system as disclosed▫ You must make “timely” disclosure to ACO of

“significant changes”▫ Significant deficiencies can prompt withholds in

accordance with DFARS 252.242-7005▫ Integrate with other systems (e.g., Accounting

System)▫ Meet requirements of law and regulation

Cost Estimating System Requirements –DFARS 252.215-7002 “Estimating system” means the Contractor's policies, procedures, and practices for budgeting and planning controls, and generating estimates of costs and other data included in proposals submitted to customers in the expectation of receiving contract awards. Estimating system includes the Contractor's—

(1) Organizational structure;(2) Established lines of authority, duties, and responsibilities;(3) Internal controls and managerial reviews;(4) Flow of work, coordination, and communication; and(5) Budgeting, planning, estimating methods, techniques, accumulation of historical costs, and other analyses used to generate cost estimates.

DFARS Requirement 215.407-5-70

• Requires all DoD contractors to have acceptable estimating systems in writing, provides guidelines concerning the characteristics of an adequate estimating system

• Creates the requirement of the contracting officer to determine adequacy of the cost estimating system

“Acceptable estimating system” means... system that complies with the system criteria… and provides for a system that—

(1) Is maintained, reliable, and consistently applied;(2) Produces verifiable, supportable, documented, and timely cost estimates that are an acceptable basis for negotiation of fair and reasonable prices;(3) Is consistent with and integrated with the Contractor’s related

management systems; and(4) Is subject to applicable financial control systems.

“Significant Estimating System Deficiency” means

• “Significant deficiency” means a shortcoming in the system that materially affects the ability of officials of the Department of Defense to rely upon data and information produced by the system that is needed for management purposes.(252.215-7002 Cost Estimating System Requirements)

Estimating System• Is different from other system because of it’s unique

nature• Comprised of policies and procedures as well as

input from other systems• A fair amount of judgment can be involved on any

particular estimate; what data from what systems is relevant▫ Often times changes to the solution being proposed

does not easily tie back to prior programs or organization changes complicates how past data is relevant

Changes in the Business Systems Rules - From ‘Should’ to ‘Shall’Cost Estimating System Requirements – DFARS 252.215-

7002 (4) The Contractor’s estimating system shall provide for the use of appropriate source data, utilize

sound estimating techniques and good judgment, maintain a consistent approach, and adhere to established policies and procedures. An acceptable estimating system shall accomplish the following functions:

(i) Establish clear responsibility for preparation, review, and approval of cost estimates and budgets;

(ii) Provide a written description of the organization and duties of the personnel responsible for preparing, reviewing, and approving cost estimates and budgets;

(iii) Ensure that relevant personnel have sufficient training, experience, and guidance to perform estimating and budgeting tasks in accordance with the Contractor's established procedures;

(iv) Identify and document the sources of data and the estimating methods and rationale used in developing cost estimates and budgets;

(v) Provide for adequate supervision throughout the estimating and budgeting process;(vi) Provide for consistent application of estimating and budgeting techniques; (vii) Provide for detection and timely correction of errors;(viii) Protect against cost duplication and omissions;(ix) Provide for the use of historical experience, including historical vendor pricing information,

where appropriate;

From ‘Should’ to ‘Shall’ (continued)

Cost Estimating System Requirements – DFARS 252.215-7002 (continued)

(x) Require use of appropriate analytical methods;(xi) Integrate information available from other management systems;(xii) Require management review, including verification of the company's estimating and

budgeting policies, procedures, and practices;(xiii) Provide for internal review of, and accountability for, the acceptability of the

estimating system, including the budgetary data supporting indirect cost estimates and comparisons of projected results to actual results, and an analysis of any differences;

(xiv) Provide procedures to update cost estimates and notify the Contracting Officer in a timely manner throughout the negotiation process;

(xv) Provide procedures that ensure subcontract prices are reasonable based on a documented review and analysis provided with the prime proposal, when practicable;

(xvi) Provide estimating and budgeting practices that consistently generate sound proposals that are compliant with the provisions of the solicitation and are adequate to serve as a basis to reach a fair and reasonable price; and

(xvii) Have an adequate system description, including policies, procedures, and estimating and budgeting practices, that comply with the Federal Acquisition Regulation and Defense Federal Acquisition Regulation Supplement.

Penalties associated with Significant Estimating System Deficiencies• Contracts subject to withholds are contracts

subject to CAS coverage that contain DFARS 252.242-7005 and the applicable clause▫ Subject to limitation, ACO has discretion to identify

contracts from which to withhold payments• Amount▫ 5% for one disapproved system or up to 10% for two or

more disapproved systems, of all billings under covered contracts

▫ Pre-existing withholds should preclude or reduce further withholds for system disapproval

Payments subject to withhold• Interim payments under▫ Cost-reimbursement contracts; Incentive type

contracts; Time and Material contracts; Labor-hour contracts; Progress payments; Performance based payments

• Once contracting officer is notified of deficiencies correction CO will discontinue withhold if corrections are sufficient▫ Has 90 days to make this determination or future

withhold percentage is reduced by at least 50%

Pre Audit Process (Auditor’s Perspective)• Auditor will notify contractor of planned audit of

the Estimating System• Auditor will review prior work done by auditors

and any flash estimating system reports• Auditor will contact DCMA about any known

concerns• Auditor will request a listing of proposals for the

last 12 months▫ Summarized by customer, contract type and dollar

amount

Entrance Conference will be Conducted• Contractor will be asked▫ To provide a system orientation or briefing▫ Outline any changes to the Estimating System

since the last examination▫ Discuss the contractor’s risk assessment process▫ Discuss the contractor’s monitoring process▫ Discuss any prior weaknesses and actions or

improvements steps taken to address these weaknesses

• Auditor will prepare an internal risk assessment

Control Environment/Information & Communications• Auditor will evaluate the internal controls this

includes looking at ICAPS, integrity and ethical values, competence, Board/Audit committee participation, management philosophy, authority and structure, HR policies and Financial capability

• Access overall accounting controls, IT systems, files, any recent related changes and query contractor staff on estimating system and process

Contractor Monitoring Process / IA /System Description• Audit will then look at the contractor’s ongoing

monitoring process, including internal audit, any external reviews and controls in place

• Including recommendation and follow up as well as review frequency

• Review of Estimating System Description▫ Organization charts, written policy, procedures

and directives, flowcharts, lines of authority, review and approval, written instructions and integration with other Management Systems

Training• Determine if regular training is required for

those responsible for preparing estimates• Includes training of the use of statistical aids and

advanced estimating techniques• Training covers estimating system policies and

procedures• Auditor will ask to examine records of training

Cost Estimating Development

• Are estimates developed using appropriate methods, compliance with CAS 401 and 402

• Use of historical data, escalation methods, use of Forward Pricing factors, formula pricing methods, catalog pricing , format of proposal (Table 15-2),Certification of Cost or Pricing Data and Subcontractor policies

Cost Estimating Development (continued)• Selective evaluation of recent pricing proposals

for adherence to procedures, • Determine if the following characteristics are in

place▫ Adequate supervision, management reviews both

interim and final reviews that cover soundness of estimates and adherence to policy

Audit Conclusion• Auditor does an assessment of the contractor• Makes determination of adequacy• Prepares a draft audit report• Conducts an exit conference• Finalize the audit report incorporating the

contactor’s response

Steps to Compliance• Start now, don’t wait until you are being notified of

an Estimating System Audit• Evaluate your Estimating System to ensure required

changes are included▫ Consider 3rd party / consultant

• Ensure you have your management’s support• Review Guidance Provided▫ DCAA Audit Program is posted on DCAA website

under Standard Audit Programs▫ Additional guidance is provided in DCAA’s Contract

Audit Manual (CAM)

Steps to Compliance (continued)• Document your Estimating Policies and

Procedures▫ Update or create an Estimating System Manual

• Ensure the controls are in place but make sure you have the flexibility to respond to all proposals in a compliant manner

• When a deviation is required to comply with solicitation requirements document them in your proposal and review process

Steps to Compliance (continued)

• Training – this will greatly reduce the risk of problems and is an audit hot button

• Build your controls and system around your business, there is no one size fits all▫ Services are very different from products▫ Larger businesses may have different policies and

procedures for different business segments• Auditor may not be familiar with your business,

don’t assume DCAA staff currently reviewing your business will conduct the audit▫ Spend the time necessary to brief and explain your

unique issues and approaches

Steps to Compliance (continued)• During the Audit engage with your auditors▫ Be responsive to issue▫ Be proactive, if there is problem start mitigating it

now▫ Demonstrate mitigation steps before the final

audit report is issued, request briefing and draft report before report is released

▫ If issues can’t be resolved with auditor, contractor should provide the auditor with contractors written position to be included in the report

Conclusion• Be proactive and get your management’s support• The proposal you are doing now will likely be

looked at during the Estimating System Audit so start now

• Establish a Training and Communication Plan• Any single failure can result in a inadequacy

determination so don’t create a hurdle you can’t clear

For more information about our organizations visit us on our websites

www.sdaga.org

www.ncmasd.org

www.ispa-cost.org

http://www.sdaga.org/�




























http://www.ncmasd.org/�

http://www.ispa-cost.org/�

Date post:	19-Oct-2021
Category:	Documents
Upload:	others
View:	8 times
Download:	0 times