Criterion Systems, Inc. ITES-3S Ordering Guidelines January 4, 2019 1 · Criterion Systems, Inc....

Criterion Systems, Inc, 8330 Boone Blvd., Ste. 400 Vienna, VA 22182

Criterion Systems, Inc. ITES-3S Ordering Guidelines January 4, 2019 1


Criterion Systems, Inc. ITES-3S Ordering Guidelines January 4, 2019 i

This Ordering Guide contains the information needed to use the Criterion Systems, Inc.

(Criterion) Information Technology Enterprise Solutions-3 Services (ITES-3S) contract to

obtain Information Technology (IT) services worldwide. The Criterion ITES-3S contract is

structured as Indefinite Delivery/ Indefinite Quantity (ID/IQ) contract, using task orders for

acquisition of specified services. This contract is available to the Army, Department of

Defense (DOD), and other Federal agencies.

Questions regarding this ordering guide and services available from Criterion through the

ITES-3S contract should be directed to the Criterion ITES-3S Program Management

Office (PMO).

Questions regarding guidelines and procedures for placing orders against the contracts

should be directed to Computer Hardware, Enterprise Software and Solutions (CHESS).

Questions of a contractual nature should be directed to the Procuring Contracting Office

(PCO), Army Contracting Command - Rock Island (ACC-RI). These guidelines will be

revised, as needed, to improve the process of awarding and managing orders under the

ITES-3S contracts.

Criterion Systems, Inc.

ITES-3S Program Management Office

ATTN: Mark Batis

8330 Boone Blvd., Ste. 400

Vienna, VA 22182

[email protected]

301-368-4060

CHESS

ATTN: SFAE-PS-CH

9351 Hall Road, Bldg. 1456 Fort Belvoir, VA 22060-5526

Toll Free Customer Line 1-888-232-4405 [email protected]

ACC-RI

ATTN: CCRC-TA

3055 Rodman Avenue Rock Island, IL 61299-8000 309-782-4886

[email protected]

mailto:[email protected]



Criterion Systems, Inc. ITES-3S Ordering Guidelines January 4, 2019 ii

Table of Contents

ITES-3S General Information ................................................................................... 1

1.1 Background .......................................................................................... 1

1.2 Scope ................................................................................................... 1

1.3 The Criterion ITES-3S Team ................................................................ 2

1.4 Contract Terms / Approach .................................................................. 3

1.5 Performance-Based Service Acquisition .............................................. 3

1.6 Fair Opportunity to be Considered........................................................ 3

1.7 Situations Requiring Hardware or Software Acquisition ....................... 4

1.7.1 Software ...................................................................................................... 4

1.7.2 Related incidental Commercial off-the-shelf (COTS) Hardware and Software .................................................................................................................... 4

Roles and Responsibilities ....................................................................................... 5

2.1 Army Contracting Command – Rock Island (ACC-RI) .......................... 5

2.2 Computer Hardware, Enterprise Software AND Solutions (CHESS) .... 5

2.3 Requiring Activity (RA) ......................................................................... 5

2.4 Ordering Contracting Officer (OCO) ..................................................... 6

2.5 Ordering Contracting Officer’s Representative (OCOR) ....................... 6

2.6 Contractors ........................................................................................... 7

2.7 Ombudsman ......................................................................................... 7

ITES-3S Ordering Guidance .................................................................................... 8

3.1 General ................................................................................................. 8

3.2 Pricing .................................................................................................. 8

3.2.1 Small Business Set Aside ............................................................................ 9

3.3 Order Forms and Numbering .............................................................. 10


Criterion Systems, Inc. ITES-3S Ordering Guidelines January 4, 2019 iii

3.4 Delivery Requirements ....................................................................... 10

3.5 Security Considerations ...................................................................... 10

3.6 Fair Opportunity to be Considered...................................................... 10

3.6.1 Exceptions to Fair Opportunity .................................................................. 11

3.7 Ordering Procedures .......................................................................... 11

3.7.1 Task Order Request .................................................................................. 11

3.7.2 Task Order Request Preparation ............................................................... 13

3.7.3 Evaluation Criteria ..................................................................................... 13

3.7.4 Evaluation .................................................................................................. 15

3.7.5 Award ........................................................................................................ 15

3.7.6 Post Award Debriefing ............................................................................... 17

3.7.7 Evaluation of Contractor’s TO Performance .............................................. 17

Attachment 1: Subcontractors ....................................................................................... 19

Attachment 2: ITES-3S Task Order Request Checklist and Instructions ....................... 20

Attachment 3: Performance Based Service Acquisition ................................................ 23

General .................................................................................................................. 23

1 Policy ..................................................................................................................... 23

2 Contract Type......................................................................................................... 23

3 Performance Work Statements .............................................................................. 23

3.1 Performance Standards/Metrics ......................................................... 24

3.2 Performance Incentives ...................................................................... 24

3.3 QASP ................................................................................................. 24

4 SOO ....................................................................................................................... 25

Attachment 4: Format for Statement of Work ................................................................ 26


Criterion Systems, Inc. ITES-3S Ordering Guidelines January 4, 2019 iv

Attachment 5: Format for Performance Work Statement ............................................... 27

Attachment 5A: Quality Assurance Surveillance Plan ................................................... 29

QASP Revision History ................................................................................................. 30

QASP Signatures....................................................................................................... 30

1 QASP General Information .................................................................................... 31

1.1 Introduction and Purpose of QASP .................................................... 31

1.2 Project A Program Quality Surveillance Roles ................................... 32

1.2.1 Federal Agency Representatives ............................................................... 32

1.2.2 Project A Team Representatives ............................................................... 33

2 Performance .......................................................................................................... 34

2.1 Methods of Surveillance ..................................................................... 34

2.2 Performance Standards ...................................................................... 35

2.3 Service Level Objectives .................................................................... 35

2.3.1 Performance Objectives ............................................................................ 36

2.4 Performance Evaluation Methodology ................................................ 46

3 Availability .............................................................................................................. 47

3.1 Work Requirements ............................................................................ 47

4 Quality .................................................................................................................... 48

4.1 QASP Implementation ........................................................................ 48

4.1.1 Frequency of Evaluations .......................................................................... 48

4.1.2 Base Year Considerations ......................................................................... 48

4.2 Documentation ................................................................................... 48

4.3 Evaluation Process ............................................................................. 49

4.3.1 Acceptable Performance ........................................................................... 50


Criterion Systems, Inc. ITES-3S Ordering Guidelines January 4, 2019 v

4.3.2 Unacceptable Performance ....................................................................... 51

QASP Appendix A – Continuous Improvement Opportunity Form ................................ 52

QASP Appendix B – Contract Discrepancy Report (CDR) ............................................ 53

Attachment 6: Statement of Objectives (SOO) .............................................................. 54

Attachment 7: ITES-3S Evaluation Plan ........................................................................ 56

Attachment 8: Letter Request for Task Order Proposals ............................................... 58

Attachment 9: Proposal Submission Instructions and Evaluation Criteria ..................... 61

Proposal Submission Instructions .......................................................................... 61

4.4 Tab 1: Technical Proposal .................................................................. 61

4.5 Tab 2: Cost / Price Proposal ............................................................... 61

4.6 Evaluation Criteria .............................................................................. 61

Attachment 10: ITES-3S Selection Recommendation Document .................................. 63

List of Figures

Figure 1: ITES-3S TO Award Process ............................................................... 17

Figure 2: Project A Project Fee Allocation Weighting ........................................ 46

Figure 3: Federal Agency Project A Performance Timeline ............................... 48

Figure 4: Rating Scheme ................................................................................... 50



ITES-3S General Information

1.1 Background

The objective of the ITES-3S contracts is to meet the Army’s enterprise infrastructure and

infostructure goals with a full range of innovative, world-class Information Technology (IT)

support services and solutions at a fair and reasonable price. ITES-3S is a multiple award,

ID/IQ contract vehicle. It is the Army’s primary source of IT-related services worldwide.

All DOD and other Federal agencies are authorized to use the contracts to satisfy their IT

requirements.

Working in partnership with the prime contractors, CHESS manages the contracts, in

coordination with the ACC-RI Contracting Center. Through the use of ITES-3S, users

have a flexible means of meeting IT service needs quickly, efficiently, and cost-effectively.

Orders may be placed by any contracting officer from the aforementioned agencies.

There is no fee to place orders against the ITES-3S contract.

Ordering under the CACI ITES-2S contract is decentralized and is authorized to meet the

needs of the Army, Department of Defense (DoD), and other Federal agencies. Orders

may be placed by any Army, DoD or Federal agency Contracting Officer.

1.2 Scope

The ITES-3S contracts encompass a full range of innovative, world-class information

technology support services and solutions at a reasonable price. Firm Fixed Price (FFP),

Time and Materials (T&M), and Cost Reimbursement (CR) Task Orders (TOs) are

authorized under this contract. Contract Line Item Numbers (CLINs) cover the following

services.

IT Solution Services

IT Subject-Matter Expert (SME)

IT Functional Area Expert (FAE)

Incidental Construction

Other Direct Costs

IT Solution Equipment

Travel and Per Diem

IT Solution Software

IT Solution – Other Direct Costs (ODCs)

The types of services and solutions offered by ITES-3S fall under the following Task

Areas: Cybersecurity Services; Information Technology Services; Enterprise Design,

Integration and Consolidation Services; Network/Systems Operation and Maintenance

Services; Telecommunications/Systems Operation and Maintenance Services; Business

Process Reengineering Services; IT Supply Chain Management Services; and IT

Education & Training Services. Copies of the ITES-3S contracts can be found on the



CHESS IT e-mart. The IT e-mart Web site is https://chess.army.mil. Services will be

acquired by issuing individual TOs.

Contract types will be determined IAW the FAR and Defense Federal Acquisition

Regulation Supplement (DFARS) based on the circumstances of each order.

1.3 The Criterion ITES-3S Team

Criterion was awarded a prime ITES-3S contract, W52P1J-18-D-A056. The Criterion

ITES-3S Team consists of many specialized vendors with a solid history of providing

outstanding IT solutions and support to the Army community and the Federal

Government:

Company Name Socio Economic Status

Absolute Business Solutions Corporation Small Disadvantaged Business (SDB)

Advanced Onion, Inc. Service-Disabled Veteran-Owned Small Business (SDVOSB)

Applied Technical Systems, Inc. Small Business (SB)

Bellator Group, Inc. HUBZone

Information Management Group, Inc. (IMG) SB

InTec, LLC SDVOSB

Phoenix Operations Group, LLC SDB

NextiraOne Federal, LLC d/b/a Black Box Network Large Business (LB)

CyberCore Technologies, LLC LB

Fulcrum IT Services, LLC LB

S&K Aerospace, LLC LB



1.4 Contract Terms / Approach

The Criterion ITES-3S contract has the following contract terms and provisions:

Contract Maximum $12,100,000,000 The contract maximum represents the total requirement for the life of the

contract (including options, if exercised)

Period of Performance 9 Years: One five-year base Ordering Period Four one-year Ordering Period options (if exercised)

Pricing Structure Firm Fixed Price (FFP) Time and Material (T&M) Cost Reimbursable (CR)

Performance-Based Contracting

Preferred method for acquiring services

Fair Opportunity to be Considered

Subject to FAR 16.505

Ordering Guidance and Process

See Chapter 3 Ordering Guide - Ordering Guidance

1.5 Performance-Based Service Acquisition

Performance-Based Service Acquisition (PBSA) is an acquisition strategy structured

around the results to be achieved as opposed to the manner by which the work is to be

performed. Orders placed under ITES-3S are not required to be performance-based

under all circumstances; however, policy promulgated by the NDAA for FY 2001 (PL 106-

398, section 821), FAR 37.102, and FAR 16.505(a), establishes PBSA as the preferred

method for acquiring services. In addition, for DOD agencies, DFARS 237.170-2 requires

higher-level approval for any acquisition of services that is not performance-based.

Accordingly, it is expected that most ITES-3S orders will be performance-based. A

Performance Work Statement (PWS) or Statement of Objectives (SOO) should be

prepared to accompany the Task Order Request (TOR) to the ITES-3S contractors.

Reference this Ordering Guide’s Attachments 2, 3, 4, and 5 for further information on

PBSA and specific details and resources for the preparation of a PWS or SOO.

1.6 Fair Opportunity to be Considered

IAW10 U.S. Code § 2304c(b) and FAR 16.505(b), the OCO must provide each ITES-3S

contractor a fair opportunity to be considered for each order exceeding $3,000 unless an

exception applies.

FAR 16.505, DFARS 216.5, and Chapter 3, Paragraph 6, below contain procedures on

exceptions to the fair opportunity process, as well as details on the applicability and

implementation of fair opportunity to be considered.



1.7 Situations Requiring Hardware or Software Acquisition

1.7.1 Software

In situations where it is necessary to purchase new commercial software, including

preloaded software, to satisfy the requirements of a particular TO, Criterion will first be

required to review and utilize available DOD Enterprise Software Initiative (ESI)

agreements. If software is not available to Criterion through a DOD ESI source, Criterion

shall be authorized to obtain the software through an alternate source. For Army users, a

Statement of Non-Availability (SoNA) is required from CHESS when acquiring non-ESI

software regardless of the dollar value. The customer shall access the SoNA process,

located on the IT e-mart at https://chess.army.mil/Content/Page/SONA. The SoNA should

be included in the TO file upon award. For DOD users, a Non-DOD contract certification

and approval is required for software buys, with the exception of the Microsoft Premier

software IAW DFARS 217.78. This Non-DOD documentation is required because the ESI

Blanket Purchase Agreements are established against General Services Administration

(GSA) ID/IQs.

1.7.2 Related incidental Commercial off-the-shelf (COTS) Hardware and Software

If related incidental hardware and software are required for a particular TO, the CHESS

hardware contracts are the preferred source of supply. For Army users, it is the mandatory

source for hardware and software IAW Army Federal Acquisition Regulation Supplement

(AFARS) 5139.101. CHESS also has a representative sample list on its web site of

Commercial IT Products and Services authorized for use by customers worldwide. A

request for quote may be submitted for products not found on the CHESS site. If the

hardware and related software required is not available from a CHESS contract or the

authorized list, Criterion shall be authorized to obtain the hardware through an alternate

source. For Army users, a SoNA is required for purchase of products from another source

regardless of dollar value. The listing of COTS hardware available from CHESS sources

can be viewed on the IT e-mart at https://chess.army.mil. The customer shall access the

SoNA process, located on the IT e-mart at https://chess.army.mil/Content/Page/SONA.

The SoNA should be included in the TO file upon award.

https://chess.army.mil/Content/Page/SONA

https://chess.army.mil/

https://chess.army.mil/Content/Page/SONA



Roles and Responsibilities

This section provides a summary of the roles and responsibilities for the primary

organizations in the ITES-3S contract process.

2.1 Army Contracting Command – Rock Island (ACC-RI)

The ACC-RI Procuring Contracting Officer’s (PCO) roles and responsibilities are as

follows:

Serves as the PCO for the ITES-3S contracts. The PCO has overall contractual

responsibility for the ITES-3S contracts. All orders issued are subject to the terms

and conditions of the contract. The contract takes precedence in the event of

conflict with any order or the Ordering Guide.

Provides advice and guidance to Requiring Activities’ (RA), OCOs, and contractors

regarding contract scope, acquisition regulation requirements, and contracting

policies.

Approves and issues base ITES-3S contract modifications.

Represents the Contracting Officer position at various contract-related meetings.

2.2 Computer Hardware, Enterprise Software AND Solutions (CHESS)

The CHESS organization’s roles and responsibilities are as follows:

Requiring Activity (RA)s / Administrative Contracting Officer Representative

(ACOR) for this acquisition

Maintains the IT e-mart, a no-fee flexible procurement strategy through which an

Army user may procure COTS IT hardware, software, and services.

The CHESS IT e-mart website is: https://chess.army.mil.

With support from the Information Systems Engineering Command, Technology

Integration Center, CHESS assists Army organizations in defining and analyzing

requirements for meeting the Army’s enterprise infrastructure and infostructure

goals.

Works with other RAs, including those outside of the Army, to help them

understand how ITES-3S can best be used to meet their enterprise requirements.

Conducts periodic meetings with the prime contractors, e.g., In-Process Review,

as needed to ensure requirements, such as approved DOD standards, are

understood.

2.3 Requiring Activity (RA)

RA is defined as any organizational element within the Army, DOD, or other Federal

Agencies. The RA’s roles and responsibilities are as follows:




Adheres to the requirements and procedures defined in the ITES-3S contracts and

these ordering guidelines.

Defines requirements.

Prepares TO requirements packages.

Funds the work to be performed under ITES-3S orders.

Provides personnel to evaluate proposals submitted.

Provides past performance assessments.

Monitors and evaluates contractor performance.

2.4 Ordering Contracting Officer (OCO)

The OCO’s roles and responsibilities are as follows:

OCOs within the Army, DOD, and other Federal agencies are authorized to place

orders within the terms of the contract and within the scope of their authority.

Not authorized to make changes to the contract terms and/or conditions. The

OCOs authority is limited to the individual orders.

Serves as the interface between the contractor and the Government for individual

orders issued under the ITES-3S contracts.

Responsible for determining if bundling of requirements (see FAR 2.101) is in

compliance with FAR 7.107.

Responsible for determining whether consolidation of requirements, compliance,

and approval are IAW DFARS 207.170.

Responsible for requesting, obtaining, and evaluating proposals/quotations and for

obligating funds for orders issued.

The OCO reserves the right to withdraw and cancel a task if issues pertaining to

the proposed task arise that cannot be satisfactorily resolved.

Responsible for identifying when Earned Value Management System is applicable

at the TO level IAW DFARS 252.234-7002.

2.5 Ordering Contracting Officer’s Representative (OCOR)

The Task Order OCOR’s roles and responsibilities are as follows:

Task Order CORs will be designated by letter of appointment from the OCO.

Serves as the focal point for all task activities, and primary Point of Contact (POC)

with the contractors.

Provides technical guidance in direction of the work; not authorized to change any

of the terms and conditions of the contract or order.

Shall use the measures and standards set forth in the Quality Assurance

Surveillance Plan (QASP) to assess contractor performance, thereby ensuring the

quality of services required by the TO are met.



Obtains required COR training. Note: The Army Contracting Command (ACC)

COR Guide provides a list of approved COR training courses:

https://www.us.army.mil/suite/doc/24452057&inline=true.

2.6 Contractors

The principal role of Criterion is to perform services and/or deliver related products that

meet requirements and/or achieve objectives/outcomes described in orders issued under

the ITES-3S contracts.

2.7 Ombudsman

IAW FAR 16.505(b), ITES-3S contractors that are not selected for award under a Task

Order competition may seek independent review by the designated ITES-3S ordering

agency's Ombudsman. The Ombudsman is responsible for reviewing complaints from

contractors and ensures that all contractors are afforded a fair opportunity to be

considered, consistent with the procedures set by this contract and regulation. The ACC-

RI Ombudsman will review complaints from contractors on all TOs issued by ACC-RI.

The Ombudsman for Task Orders not issued by ACC-RI will be the Ombudsman that

supports the OCO. The designated Ombudsman for ITES-3S Task Orders issued by

ACC-RI is:

Amy VanSickle

Army Contracting Command-Rock Island (ACC-RI) CCRC-OC

3055 Rock Island Arsenal Rock Island Arsenal

309-782-1002 / DSN 793-1002

[email protected]

Note: IAW FY08 Authorization Act, Section 843, the U.S. Government Accountability

Office (GAO) will entertain a protest filed on or after May 27, 2008, for delivery orders

valued at more than $10M. Procedures for protest are found on 4 Code of Federal

Regulations Part 21 (GAO Bid Protest Regulations).

https://www.us.army.mil/suite/doc/24452057%26inline%3Dtrue




ITES-3S Ordering Guidance

3.1 General

Ordering is decentralized for all ITES-3S requirements. Ordering under the contracts is

authorized to meet the needs of the Army, DOD, and other Federal agencies. There are

no approvals, coordination, or oversight imposed by the PCO on any OCO. OCOs are

empowered to place orders IAW the terms and conditions of the ITES-3S contracts, ITES-

3S ordering guidelines, the FAR, DFARS (as applicable), and the OCO’s agency

procedures.

The PCO will not make judgments or determinations regarding orders awarded under the

ITES-3S contracts by an OCO. All issues must be resolved consistent with individual

agency procedures and/or oversight.

Upon request, the PCO is available to provide guidance to OCOs executing orders under

the ITES-3S contracts.

The CHESS IT e-mart at https://chess.army.mil is available to make price comparisons

among all ITES-3S awardees and solicit competitive quotes. Only services and related

incidental hardware/software items are to be released on the ITES-3S IT e-mart.

Hardware / Software-only items are to be placed on ADMC-2 or ITES- 2H/3H. The OCO

will initiate the Task Order Request (TOR) process by issuing a TOR to all awardees via

the CHESS IT e-mart. OCOs MUST issue the Request for Proposal (RFP)/TORs via the

IT e-mart.

When posting an RFP/TOR, RAs are not to simply submit an ITES-3S contractor’s quote

as an RFP/TOR. This is considered to be contractor proprietary information.

When posting a TOR, include specific delivery instructions for proposal responses.

Contractors will indicate their interest via CHESS IT e-mart; however, proposal packages

shall be delivered by means identified in the TOR.

3.2 Pricing

All TOs awarded pursuant to this contract on a FFP or T&M basis must be priced IAW the

pricing set forth in the Labor Rate Table (reference contractors’ ITES-3S Price Matrix,

Section J, Attachment 3). The labor rates in the labor rate table reflect the fully burdened

composite rates for each labor category and will apply to all direct labor hours. The

composite rates include separate rates for work performed at the contractor site and at

the Government site for each labor category. An ITES-3S contractor may propose labor

rates that are lower than those specified in its Labor Rate Table, but shall not exceed the

labor rates in its Labor Rate Table.



CR TOs are allowable under ITES-3S. CR TOs are suitable for use only when

uncertainties involved in contract performance do not permit costs to be estimated with

sufficient accuracy to use any type of FP TO. A CR TO may be used only when the

contractor’s accounting system is adequate for determining costs applicable to the TO

and appropriate Government surveillance during performance will provide reasonable

assurance that efficient methods and effective cost controls are used.

The Government’s minimum requirements for each labor category are identified in Labor

Category Descriptions. Contractors may augment their labor categories and job

descriptions on a TO basis. If a contractor decides to augment a labor category; the labor

type and cost shall not change. Augmenting a labor category is not defined as adding a

new labor category. TO proposals shall be limited to only those labor categories contained

within the base contract. The contractor may propose to the Government, at its discretion,

additional labor categories and job descriptions within the scope of ITES-3S. The PCO is

the only official authorized to add a labor category to the base contract via contract

modification.

Unlike other labor categories, the IT subject-matter expert (SME), IT FAE, and incidental

construction categories may only be used if no other labor category can satisfy the

requirement. If the ITES-3S contractor proposes these categories when not directed by

the OCO, no fee or profit is allowed. OCOs are discouraged from directing the use of

FAEs and SMEs. However, if the OCO deems it necessary to direct the ITES-3S

contractor to propose these categories, a fixed fee of 3% is allowable. ITES-3S

contractors are required to seek and obtain approval from the OCO for the use of these

categories when proposed in a TO. There is no fixed labor rate associated with the SME,

FAE, and incidental construction categories.

3.2.1 Small Business Set Aside

The following clauses only apply at the order level when the requirement has been set-

aside for Small Business:

52.219-3 - Notice of HUBZone Set-Aside or Sole Source Award (Nov 2011)

52.219-6 - Notice of Total Small Business Set-Aside (Nov 2011) with Alternate I

52.219-13 - Notice of Set-Aside of Orders (Nov 2011)

52.219-14 - Limitations on Subcontracting (Nov 2011)

52.219-27 - Notice of Service-Disabled Veteran-Owned Small Business Set-Aside

(Nov 2011)

52.219-29 - Notice of Set-Aside for, or Sole Source Award to, Economically

Disadvantaged Women-Owned Small Business Concerns (Dec 2015)

52.219-30 - Notice of Set-Aside for, or Sole Source Award to, Women-Owned

Small Business Concern Eligible Under the Women-Owned Small Business

Program (Dec 2015)



Note that the Limitations on Subcontracting clause only applies at the Task Order level.

Small businesses may compete on unrestricted TOs without having to meet the

requirements of the Limitations on Subcontracting clause.

3.3 Order Forms and Numbering

An appropriate order form (Defense Department (DD) Form 1155, Order for Supplies or

Services, or Non-DOD Federal agencies equivalent) shall be issued for each TO. The

use of Government credit cards is also authorized IAW applicable rules and procedures.

TOs may be issued via telephone, fax, e-mail, postal mail or CHESS’s IT e-mart.

3.4 Delivery Requirements

Delivery of services shall be IAW individual orders.

3.5 Security Considerations

The level of classified access will be incorporated into individual TOs as necessary. If

determined necessary based on the level of classification, a DD Form 254, Contract

Security Classification Specification, should be prepared and included in the TO request

and resulting order.

3.6 Fair Opportunity to be Considered

IAW FAR 16.505(b)(2), for all orders exceeding $3,500, the OCO shall give every ITES-

3S contractor a fair opportunity to be considered for a TO unless one of the exceptions to

fair opportunity applies (see paragraph below for further discussion of exceptions). The

OCO must consider all ITES-3S contractors for the work though he/she is not necessarily

required to contact any of them. The OCO must document his/her rationale if applying

one of the exceptions to fair opportunity; however, no special format is required. All orders

exceeding $150,000 for DOD agencies must be placed on a competitive basis IAW FAR

16.505 unless a written waiver is obtained, using the limited sources justification and

approval format in FAR 16.505(b)(2)(ii)(B). OCO should refer to their agency’s approval

authorities for placing orders on an “other than a competitive” basis. This competitive

basis requirement applies to all orders by, or on behalf of, DOD. Non-DOD agencies shall

comply with their own agency’s procedures.

For orders by, or on behalf of, DOD exceeding $150,000, the requirement to place orders

on a competitive basis is met only if the OCO:

Provides a notice of intent to purchase to every ITES-3S contractor, including a

description of work to be performed and the basis upon which the selection will be

made; and

Affords all ITES-3S contractors responding to the notice a fair opportunity to submit

an offer and to be fairly considered.



3.6.1 Exceptions to Fair Opportunity

As provided in FAR 16.505(b)(2), the OCO may waive the requirement to place an order

on a competitive basis with a written limited sources justification and approval if one of

the following circumstances applies:

The agency’s need for the supplies or services is so urgent that providing a fair

opportunity would result in unacceptable delays. Use of this exception requires a

justification that includes reasons why the ITES-3S processing time for a fair opportunity

to be considered will result in an unacceptable delay to the agency. The justification

should identify when the effort must be completed and describe the harm to the agency

caused by such a delay.

Only one contractor is capable of providing the supplies or services required at the level

of quality required because the supplies or services ordered are unique or highly

specialized. Use of this exception should be rare. When using this exception, explain: (1)

what is unique or highly specialized about the supply or service, and (2) why only the

specified contractor can meet the requirement.

The order must be issued on a sole-source basis in the interest of economy and efficiency

because it is a logical follow-on to an order already issued under these contracts, provided

that all awardees were given a fair opportunity to be considered for the original order.

A statute expressly authorizes or requires that the purchase be made from a specified

source.

FAR 16.505(b)(1)(ii) provides that the OCO is not required to contact each of the

awardees if information is available that will ensure that each awardee is provided a fair

opportunity to be considered for each order.

The OCO must follow his/her agency’s procedures for documenting the process and

rationale for selection of the awardee for each TO. At a minimum, the OCO must

document the selection to include price consideration.

3.7 Ordering Procedures

3.7.1 Task Order Request

The RA prepares the TOR package and submits it to the OCO. Ordering Guide

Attachment 1 is an example of a TO checklist.

NOTE: When submitting requests ensure that the customer and/or site address is correct

and includes as much information as possible to allow for an accurate proposal. (i.e. serial

numbers, manufacturer/part numbers, quantities, whether the requirement is a renewal



or new requirement, customer ID number, contract numbers, renewal contract number or

other type of account identifier.)

At a minimum, the package should contain the following:

Statement of Work (SOW), PWS, or SOO: the RA may select from these work

statements, depending on their specific requirements; however, performance-

based orders must be used to the maximum extent possible for services as

required by FAR 37.102 and FAR 16.505(a) (see Ordering Guide Attachment 2).

Specific formats have been developed to streamline the processing time. See

examples of the SOW at Ordering Guide Attachment 3, the PWS at Ordering Guide

Attachment 4, and the SOO at Ordering Guide Attachment 5.

The PWS identifies the technical, functional, and performance characteristics of

the Government’s requirements. The PWS describes the work in terms of the

purpose of the work to be performed rather than either “how” the work is to be

accomplished or the number of hours to be provided.

The SOO is an alternative to the PWS. It is a very brief document (commonly 2-10

pages, depending upon complexity, although there is no maximum or minimum

required length) that summarizes key agency goals and outcomes to which

contractors respond. It is different from a PWS in that, when a SOO is used,

offerors are asked to develop and propose a PWS as part of their solution.

Typically, SOO responses would also propose a technical approach, performance

standards, incentives/disincentives, and a QASP based upon commercial

practices.

At a minimum, a SOO must contain the following information:

Purpose

Scope or mission

Period and place of performance

Background

Performance objectives (i.e., required results)

Any operating constraints

Upon award, the winning offeror’s solution to the SOO should be incorporated into the

resulting TO; the SOO itself is not part of the TO.

Funding Document: ITES-3S Orders are funded by the OCO’s RA. Individual

OCOs should provide specific instructions as to the format and content.

Independent Government Cost Estimate: the estimate will assist the OCO in

determining the reasonableness of the contractors’ cost and technical proposals.

The estimate is for Government use only and should not be made available to the

ITES-3S Contractors.

Basis for TO Award: the OCO, in conjunction with the RA, develops the evaluation

criteria that form the basis for TO award. Whether the award will be based on low

price, technical acceptability or best value, the criteria should be provided to the



contractor. If the award will be based on best value, evaluation factors and

significant sub factors that will affect contract award and their relative importance

should be shown. Ordering Guide Attachment 6, Proposal Evaluation Plan, has

been developed as a recommended format for documenting the basis for award.

3.7.2 Task Order Request Preparation

The OCO will issue a TOR to all ITES-3S contractors for orders exceeding $3,500.00.

The request will include a transmittal letter identifying the TO strategy, contract type,

proposal receipt date and time, estimated contract start date, period of performance, and

any other related information not contained elsewhere; the appropriate work statement;

instructions for submission of a technical and cost/price proposal and selection

criteria/basis for award, any special requirements (i.e., security clearances, travel, special

knowledge); and other information deemed appropriate for the respective order. Ordering

Guide Attachment 7contains a recommended memo requesting proposals and Ordering

Guide Attachment 8 contains sample instructions/basis for award.

Recommend a submission date of 10 calendar days after issuing a TO request for receipt

of proposals; however, the scope and complexity of the TO should be considered when

determining proposal due date.

If unable to perform a requirement, the contractor shall submit a “no bid” reply in response

to the proposal request. All “no bids” shall include a brief statement as to why the

contractor is unable to perform, e.g., conflict of interest.

In responding to proposal requests that include a requirement to provide products as part

of an overall IT services solution, ITES-3S contractors are expected to use CHESS

hardware contracts as preferred sources of supply. Other sources may be proposed, but

will require justification by the contractor and the approval of the OCO. In addition,

contractors are expected to facilitate maximum utilization of ESI source software.

3.7.3 Evaluation Criteria

All evaluation criteria must be identified and clearly explained in the TOR. The TOR must

also describe the relative importance of the evaluation criteria. The OCO, in conjunction

with the RA, may consider the following evaluation criteria (price or cost must be a factor

in the selection criteria) to evaluate contractors’ proposals:

Technical/management approach:

Understanding of the requirement

Technical and management approach

Staffing plan (e.g., skill mix, personnel experience or qualifications and availability

of personnel, performance location)

Areas of expertise



Past performance on prior TOs under this contract (e.g., approach, personnel,

responsiveness, timeliness, quality, and cost control) (NOTE: If practicable,

automated systems such as Past Performance Information Management System

or Past Performance Information Retrieval System should be utilized in lieu of

requesting past performance information from the contractors).

Current distribution of workload

Knowledge of the customer’s organization

Teaming arrangements (including subcontracting)

Security (including clearance level)

Performance-based approach

Other specific criteria as applicable to the individual TO

Cost / Price

This part of the proposal will vary depending upon the contract type planned for

the TO. It should include detailed cost/price amounts of all resources required to

accomplish the TO (labor hours, rates, travel, etc.). The contractor may not exceed

the labor rates specified in its respective contract’s ITES-3S Price Matrix, Section

J, Attachment 3. However, the contractor is permitted to propose labor rates that

are lower than those established in the Labor Rate Table. The contractor shall fully

explain the basis for proposing lower rates. The proposed reduced labor rates will

not be subject to audit; however, the rates will be reviewed to ensure the

Government will not be placed at risk of nonperformance. The reduced labor rates

will apply only to the respective TO and will not change the fixed rates in Labor

Rate Tables. The level of detail required shall be primarily based on the contract

type planned for use, as further discussed below.

FFP and T&M. The proposal shall identify labor categories IAW the ITES-3S Price

Matrices and the number of hours required for performance of the task. The proposal

must identify and justify use of all non-labor cost elements. It must also identify any

Government-Furnished Equipment (GFE) and/or Government Furnished-Information

(GFI) required for task performance. If travel is specified in the TOR, airfare and/or local

mileage, per diem rates by total days, number of trips, and number of contractor

employees traveling shall be included in the cost/price proposal. Other information shall

be provided as requested in the proposal request.

CR. Both “sanitized” and “unsanitized” cost/price proposals will be required for CR- type

TOs only. “Unsanitized” cost proposals are complete cost proposals that include all

required information. “Sanitized” cost proposals shall exclude all company proprietary or

sensitive data but must include a breakdown of the total labor hours proposed and a

breakout of the types and associated costs of all proposed ODCs. Unless otherwise

noted, unsanitized proposals will only be provided to the OCO, while sanitized proposals

may be provided to the evaluator(s) and other personnel involved in the procurement.

Cost/price proposals shall include, at a minimum unless otherwise indicated in the TOR,

a complete work breakdown structure that coincides with the detailed technical approach



and provides proposed labor categories, hours, wage rates, direct/indirect rates, ODCs,

and fees. CR proposals shall be submitted IAW FAR clause 52.215-20 “Requirements for

Cost or Pricing Data or Information Other Than Cost or Pricing Data.”

3.7.4 Evaluation

If a “mini-competition” is being conducted, a panel of evaluators should be appointed to

review the proposals submitted by ITES-3S contractors. For each non-price evaluation

factor, the evaluators should identify strengths and weaknesses in the proposals and

should assign an adjectival rating (e.g., outstanding, good, etc.) for each non-price factor.

The evaluators’ findings should be documented in a written evaluation report. The price

factor should be evaluated independently from the non-price factors. Individuals who are

evaluating non-price aspects of the proposal should not have access to pricing

information while performing their evaluations. Evaluations must be conducted fairly and

IAW the selection criteria in the solicitation. After an initial evaluation of proposals,

negotiations (discussions) may be held. Refer to FAR Part 15 for general guidance on the

proper conduct of discussions.

3.7.5 Award

Once evaluations are completed, an authorized selection official must make an award

decision and document the rationale for his/her decision. Prior to making a decision,

copies of all evaluations must be forwarded to the selection official for his/her review and

consideration. Ordering Guide Attachment 9 is an example of the Selection

Recommendation Document.

The Selection Recommendation Document is signed by the selection official and

forwarded to the OCO. This form can also be used to document an exception to the fair

opportunity requirements.

At a minimum, the following information shall be specified in each TO Award:

Date of order

POC (name), commercial telephone and fax numbers, and e-mail address

OCOs commercial telephone number and e-mail address

Description of the services to be provided, quantity unit price and extended price,

or estimated cost and/or fee (TO INCLUDE THE CLIN FROM PART B). The work

statement should be attached; the contractor’s proposal may be incorporated by

reference.

Delivery date for supplies

Address and place of performance

Packaging, packing, and shipping instructions, if any



Accounting and appropriation data and Contract Accounting Classification

Reference Number (ACRN) (Defense Finance and Accounting Service requires an

ACRN(s) on all orders.)

Specific instructions regarding how payments are to be assigned when an order

contains multiple ACRNs

Invoice and payment instructions

Any other pertinent information

IAW 10 U.S. Code § 2304c(d) and FAR 16.505(a)(10), the ordering agency’s award

decision on each order is generally not subject to protest under FAR Subpart 33.1 except

for a protest that an order increases the scope, period, or maximum value of the contract.

In lieu of pursuing a bid protest, ITES-3S contractors may seek independent review by

the designated Ombudsman. The Ombudsman will review complaints from the

contractors and ensure that all contractors are afforded a fair opportunity to be considered

for each order, consistent with the procedures in the contract. The designated

Ombudsman is identified in Chapter 2, paragraph 7, of these guidelines.

The executed order will be transmitted via fax, e-mail, or by verbal direction from the

OCO. If verbal direction is given, written confirmation will be provided within five working

days.

After award, timely notification shall be provided to the unsuccessful offerors and will

identify, at a minimum, the awardee and award amount.



Figure 1: ITES-3S TO Award Process

3.7.6 Post Award Debriefing

Under 10 USC § 2305(b)(5), unsuccessful offerors in competitions for TOs exceeding

$5,500,000 have the right to a post-award debriefing if they meet certain request deadline

requirements. The deadline requirements can be found in FAR 15.506(a)(1). Under FAR

15.506(a)(4)(i), untimely debriefing requests may be accommodated, and 15.506 is not

limited to unsuccessful offerors. Timely requests for a post-award debriefing for TOs

meeting the threshold above must be honored, and their debriefings must meet the

requirements of FAR 15.506. Also, contracting officers are encouraged to provide

debriefings to untimely offerors under competitions exceeding $5,500,000 and to offer a

debriefing to all other offerors under TO competitions, even those valued below the

mandatory threshold described above. Non- mandatory debriefings should follow all of

the requirements in FAR 15.506(d), (e), and (f). Debriefings may be done orally, in writing,

or by any method acceptable to the contracting officer.

3.7.7 Evaluation of Contractor’s TO Performance

At TO completion, the ITES-3S contractor submits a request for a performance evaluation

to the order’s COR or his/her designated representative. The order’s COR or his/her

designated representative shall complete these evaluations for each TO, regardless of



dollar value, within 30 days of completion. Performance evaluations shall also be

completed annually for orders that have a performance period in excess of one year.

Annual performance evaluations shall be completed within 30 days of TO renewals.

Performance evaluations may also be done, as otherwise considered necessary,

throughout the duration of the order (but generally no more than quarterly). The

performance evaluations will be located on the CHESS IT e-mart at

https://chess.army.mil/Static/SRV_ITS_SB_EVL_CON.

Contractor Performance Assessment Reports (CPARs) are required in the Information

Technology or Services sectors for actions valued at $1M or above. A final CPAR is

performed when all performance on the contract is completed. Interim CPARs must be

performed on deliveries/performance exceeding 18 months. A CPAR should contain past

performance information that is current and relevant information for future source

selection purposes. It includes the contractor’s record of conforming to contract

requirements, standards of good workmanship, forecasting and controlling costs,

adherence to contract schedules, administrative aspects of performance, reasonable and

cooperative behavior, commitment to customer satisfaction, and business-like concern

for the interest of the customer.

https://chess.army.mil/Static/SRV_ITS_SB_EVL_CON


Criterion Systems, Inc. ITES-3S Ordering Guidelines January 4, 2019 Attachment 1 19

Attachment 1: Subcontractors

Information Technology Enterprise Solutions – 3 Services (ITES-3S)

Contract Number: W52P1J-18-D-A056

Prime: Criterion Systems, Inc.

Subcontractors:

Company Name Socio Economic Status

Absolute Business Solutions Corporation Small Disadvantaged Business (SDB)

Advanced Onion, Inc. Service-Disabled Veteran-Owned Small Business (SDVOSB)

Applied Technical Systems, Inc. Small Business (SB)

Bellator Group, Inc. HUBZone

Information Management Group, Inc. (IMG) SB

InTec, LLC SDVOSB

Phoenix Operations Group, LLC SDB

NextiraOne Federal, LLC d/b/a Black Box Network Large Business (LB)

CyberCore Technologies, LLC LB

Fulcrum IT Services, LLC LB

S&K Aerospace, LLC LB



Attachment 2: ITES-3S Task Order Request Checklist and Instructions

This form constitutes a request for contract support under the ITES-3S contracts. The

requiring activity (RA) shall complete this form, together with the associated Ordering

Guide attachments, and forward the entire package to the appropriate ordering

contracting officer for processing.



ITES-3S Task Order Request Checklist and Instructions

1. Task Order (TO) Title.

2. RA Point of Contact. Include name, title, organization, commercial and DSN phone numbers for voice and fax, and e-mail address:

3. Designated Order Contracting Officer Representative (COR). Include name, title, organization, commercial and DSN phone numbers for voice and fax, and e- mail address (If same as block 2, type “same”):

Attachments Checklist. Complete package must include the following items. Send files electronically via e-mail or fax to the ordering contracting officer.

Work Statement (check one)

Statement of Work

Performance Work Statement includes QASP

Statement of Objectives

Funding Document(s) (scanned or other electronic version is preferable)

Independent Government Cost Estimate

Proposal Evaluation Plan Bundling Determination (if needed)

Consolidation Determination (if needed)

Justification for Work Statement that is not Performance-Based

TO unique Defense Department Form 254 (only if security requirements)

TO Information Contract Type (check one) Time and Materials (T&M) and Cost Reimbursement (CR) contract types require justification in accordance with (IAW) Federal Acquisition Regulation (FAR) (the ordering contracting officer makes the final determination of which order type is in the best interest of the government).

Firm Fixed Price (no justification required)

CR (provide justification in the box, below)

T&M (provide justification in the box, below)

Rationale: T&M and CR contract types require justification IAW FARs.



ITES-3S Task Order Request Checklist and Instructions

Federal Acquisition Streamlining Act (FASA) Exception. If you are citing a FASA exception to fair opportunity competition, designate which one below with a justification.

FASA Exception Justification:

The agency need for services is of such urgency that providing such opportunity would result in

unacceptable delays.

Only one such contractor is capable of providing services required at the level of quality required

because they are unique or highly specialized.

The order should be issued on a sole-source basis in the interest of economy and efficiency as a

logical follow-on to an order already issued under this contract, provided that all ITES-3S contractors were given a fair opportunity to be considered for the original order. A statue expressly authorizes or requires that the purchase be made from specified source.

6. Order COR Training Certification: Army Order CORs are required to have COR training prior to appointment IAW paragraph 1.7 of the Army Contracting Command (ACC) Acquisition Instruction. Appendix A of the ACC Acquisition Instruction contains a list of ACC-approved training courses. Refer to: https://arc.army.mil/COR/CORHandbooks_SelfServe.aspx

Order COR Training Certification Date:

https://arc.army.mil/COR/CORHandbooks_SelfServe.aspx



Attachment 3: Performance Based Service Acquisition

General

Performance-Based Service Acquisition (PBSA) is the preferred method of contracting

for services and supplies. PBSA means an acquisition structured around the results to be

achieved as opposed to the manner by which the work is to be performed. Essential

elements of PBSA include: (1) performance requirements, expressed in either a

performance work statement (PWS) or statement of objectives (SOO). Performance

requirements should be described in terms of what the required output is and should not

specify how the work is to be accomplished; (2) Performance standards or

measurements, which are criteria for determining whether the performance requirements

are met; (3) Appropriate performance incentives, either positive or negative; and (4) A

surveillance plan that documents the Government’s approach to monitoring the

contractor’s performance.

These elements are discussed further below.

1 Policy

FAR 37.102 has established the policy to use a PBSA approach, to the maximum extent

practicable, for all services. Services exempted from this policy are: architect-engineer,

construction, utility, and services that are incidental to supply purchases. Use of any other

approach has to be justified to the ordering contacting officer. For Defense agencies,

DFARS 237.170-2 requires higher-level approval for any acquisition of services that is

not performance-based.

2 Contract Type

The order of precedence set forth in FAR 37.102(a)(2) must be followed for all Task

Orders (TOs). It is:

A Firm Fixed Price (FFP), performance-based contract or TO.

A performance-based contract or TO that is not FFP.

A contract or TO that is not performance-based. Requiring activities should use

the contract type most likely to motivate contractors to perform at optimal levels.

FFP is the preferred contracting type for PBSA. Work statements should be

developed in sufficient detail to permit performance on a fixed-price basis.

3 Performance Work Statements

The PWS identifies the technical, functional, and performance characteristics of the

Government’s requirements. The PWS describes the work in terms of the purpose of the

work to be performed rather than either how the work is to be accomplished or the number



of hours to be provided. The format for the PWS is similar to the traditional statement of

work. In addition, the PWS will include performance standards, incentives, and a QASP:

3.1 Performance Standards/Metrics

Reflects level of service required by the Government to meet performance objectives.

Standards may be objective (e.g., response time) or subjective (e.g., customer

satisfaction). They must also:

Use commercial standards where practicable, e.g., ISO 9000

Ensure the standard is needed and not unduly burdensome

Must be measurable, easy to apply, and attainable

If performance standards are not available, the PWS may include a requirement for the

contractor to provide a performance matrix, as a deliverable, to assist in the development

of performance standards for future TOs.

3.2 Performance Incentives

Incentives may be positive or negative, monetary or non-monetary.

NOTE: if a financial incentive is promised, ensure that adequate funds are available at

time of TO award to pay incentives that may be earned.

Examples of monetary incentives include:

Incentive fees

Share-in-savings

A negative incentive can be included if the desired results are not achieved

(deduction should be equal to the value of the service lost).

Examples of non-monetary incentives include:

Revised schedule

Positive performance evaluation

Automatic extension of contract term or option exercise

Lengthened contract term (award term contracting) or purchase of extra items

(award purchase)

3.3 QASP

The QASP is a plan for assessing contractor performance to ensure compliance with the

Government’s performance objectives. It describes the surveillance schedule, methods,

performance measures, and incentives.



The level of surveillance should be commensurate with the dollar amount, risk, and

complexity of the requirement

Don’t inspect the process, just the outputs

QASP is included as part of the PWS

A PWS sample format, including a QASP, is provided as Ordering Guide Attachment 4.

4 SOO

The SOO is an alternative to the PWS. It is a very brief document (commonly two to 10

pages, depending upon complexity, although there is no maximum or minimum length)

that summarizes key agency goals and outcomes to which contractors respond. It is

different from a PWS in that, when a SOO is used, offerors are asked to develop and

propose a PWS as part of their solution. Typically, offerors would also propose a technical

approach, performance standards, incentives/disincentives, and a QASP based upon

commercial practices. At a minimum, a SOO must contain the following information:

Purpose

Scope or mission

Period and place of performance

Background

Performance objectives (i.e., required results)

Any operating constraints

Upon award, the winning offeror’s solution to the SOO should be incorporated into the

resulting TO. The SOO itself is not part of the TO.

A SOO sample format is provided as Ordering Guide Attachment 6.



Attachment 4: Format for Statement of Work

1. Project Title: Provide a short, descriptive title of the work to be performed.

2. Background: Describe the need for the services, the current environment, and

the office’s mission as it relates to this requirement. Provide a brief

description/summary of the services sought.

3. Scope: Indicate which ITES-3S contract task area(s) apply to the work to be

performed. Include a high- level view of the procurement, its objectives, size, and

projected outcomes. Do not include anything that won’t contribute to the expected

result. Do include impacts/implications.

4. Applicable Documents: List legal, regulatory, policy, security, etc. documents

that are relevant. Include publication number, title, version, date, where the

document can be obtained, etc. If only portions of documents apply, so state.

5. Specific Tasks: Provide a narrative of the specific tasks that make up the SOW.

Number the tasks sequentially, e.g., Task 1 - Title of Task and description, Task 2

- Title of Task and description, etc. Describe in clear terms, using active language,

what work will be performed. The requirement must be defined sufficiently for the

contractor to submit a realistic proposal and for the Government to negotiate a

meaningful price or estimated cost. SOWs must be “outcome-based,” i.e., they

must include the development and delivery of actual products (e.g., assessment

report, migration strategy, implementation plan, etc.).

6. Deliverables and Delivery Schedule: List all outputs/outcomes with specific due

dates or time frames. Include media type, quantity, and delivery point(s). State due

dates in terms of calendar days after task order award.

7. Government-furnished Equipment and Information (GFE/GFI): Identify the

Government-furnished equipment and information, if any, to be provided to the

contractor, and identify any limitations on use. Be as specific as possible.

8. Place of Performance: Specify whether the work will be performed at the

contractor’s site or at a Government site, with exact address if possible. Describe

any local or long distance travel the contractor will be required to perform.

9. Period of Performance: State in terms of total calendar days after TO award (e.g.,

365 calendar days after TO award), or in terms of start and end date (e.g.,

September 1, 20XX through August 30, 20XX.

10. Security: State whether the work will be UNCLASSIFIED, CONFIDENTIAL,

SECRET, TOP SECRET or TOP SECRET SCI. Contract Section H.8 requires that

the level of classified access be incorporated into individual TOs as necessary.

The Contract Security Classification Specification, DD Form 254 should be

included if required.



Attachment 5: Format for Performance Work Statement

1. Project Title: Provide a short, descriptive title of the work to be performed.

2. Background: Describe the need for the services, the current environment, and

the office’s mission as it relates to this requirement. Provide a brief

description/summary of the services sought.

3. Scope: Indicate which ITES-3S contract task area(s) apply to the work to be

performed. Include a high- level view of the procurement, its objectives, size, and

projected outcomes. Do not include anything that won’t contribute to the expected

result. Do include impacts/implications.

4. Applicable Documents: List legal, regulatory, policy, security, etc. documents

that are relevant. Include publication number, title, version, date, where the

document can be obtained, etc. If only portions of documents apply, so state.

5. Performance Requirements: Provide a narrative of the specific performance

requirements or tasks that make up the PWS. Describe the work in terms of the

required output, i.e., what is expected from the contractor, rather than how the

work is to be accomplished or the number of hours to be provided. Number the

tasks sequentially, e.g., Task 1 - Title of Task and description, Task 2 - Title of

Task and description, etc. The requirement must be defined sufficiently for the

contractor to submit a realistic proposal and for the Government to negotiate a

meaningful price or estimated cost.

6. Performance Standards: Performance standards establish the performance

levels required by the Government. Examples of performance standards:

7. Quality Standards: Condition, Error rates, Accuracy, Form/Function, Reliability,

Maintainability

8. Quantity Standards: Capacity, Output, Volume, Amount

9. Timeliness Standards: Response times, Delivery, Completion times, Milestones

10. Incentives: Incentives should be used when they will encourage better quality

performance. They may be either positive, negative or a combination of both.

Incentives may be monetary or non-monetary. Incentives do not need to be

present in every performance-based contract as an additional fee structure. In a

fixed price contract, the incentives would be embodied in the pricing and the

contractor could either maximize profit through effective performance or have

payments reduced because of failure to meet the performance standard.

11. Positive Incentives: Actions to take if the work exceeds the standards. Standards

should be challenging, yet reasonably attainable.

12. Negative Incentives: Actions to take if work does not meet standards.



13. Deliverables and Delivery Schedule: List all outputs/outcomes with specific due

dates or time frames. Include media type, quantity, and delivery point(s). State due

dates in terms of calendar days after task order award.

14. Government-furnished Equipment and Information (GFE/GFI): Identify the

Government-furnished equipment and information, if any, to be provided to the

contractor, and identify any limitations on use. Be as specific as possible.

15. Place of Performance: Specify whether the work will be performed at the

contractor’s site or at a Government site, with exact address if possible. Describe

any local or long distance travel the contractor will be required to perform.

16. Period of Performance: State in terms of total calendar days after TO award (e.g.,

365 calendar days after TO award), or in terms of start and end date (e.g.,

September 1, 20XX through August 30, 20XX).

17. Security: State whether the work will be UNCLASSIFIED, CONFIDENTIAL,

SECRET TOP SECRET or TOP SECRET SCI. and include Contract Security

Classification Specification, DD Form 254, as required in individual TOs. ITES-3S

Contract Section H.8 requires that the level of classified access be incorporated

into individual TOs as necessary.

18. Quality Assurance Surveillance Plan (QASP): This portion of the PWS explains

to the contractor what the Government’s expectations are, how (and how often)

deliverables or services will be monitored and evaluated, and incentives that

encourage the contractor to exceed the performance standards and that reduce

payment or impose other negative incentives when the outputs/outcomes are

below the performance standards. Attach the QASP to the PWS. An example is

provided on the next page.


Criterion Systems, Inc. ITES-3S Ordering Guidelines January 4, 2019 Attachment 5a 29

Attachment 5A: Quality Assurance Surveillance Plan Project Name

Quality Assurance Surveillance Plan (QASP)

Contract Number:

August 19, 2014 Version 2.0

Company Name Address Email Phone



QASP Revision History

QASP Signatures

The signatures below represent authorization for the Project A team to use COMPANY B

and project resources to perform tasks in association with the successful completion of

this Quality Assurance Plan and subsequent activities.

QASP Revision History

Date Version Description Modified By

08/13/2012 1.0 Baseline Version

____________________________________________________ ______________

XXXXXXXX, Project A Contracting Officer Representative Date

____________________________________________________ ______________

XXXXXX Project A Project Manager Date

____________________________________________________ ______________

XXXXX, Project A Government Task Leader Date

__________________________________________________ _____________

XXXXXX, Project Manager Date

____________________________________________________ ______________

XXXXX, QA Manager Date



1 QASP General Information

1.1 Introduction and Purpose of QASP

This Quality Assurance Surveillance Plan (QASP) is a Government applied document

used to facilitate systematic quality assurance methods in the administration of the

Federal Agency (Project A) performance based service contract. The intent is to ensure

that the Project A Team performs in accordance with the performance metrics set forth

herein and that the Government receives the quality of services called for in the contract

while maintaining the project scope.

This QASP contains the high-level requirements, procedures, and guidelines needed to

manage and provide the proper quality for Project A. The QASP serves as the plan for

surveillance of performance and identifies the methods to be used to track, monitor, and

report on performance plan compliance. This QASP:

Identifies the services and products that will be measured;

Establishes the specific standards of performance for each required output;

Establishes the responsibilities for performing the measurement;

Defines the Government role in overseeing the performance;

Establishes timeframes for communicating performance improvements; and

Outlines the proposed evaluation process for the end of each award fee period.

This QASP does not detail how the Project A Team will accomplish the work. A detailed

Quality Assurance Plan (QAP) will be developed by the Project A Team to establish the

goals, processes, activities and procedures used in support of Project A work products

and services delivery quality. The QAP will provide the framework necessary to ensure a

consistent approach to quality assurance throughout the project life cycle. In comparison,

the QASP is put in place to define Government surveillance oversight of the contractor’s

efforts and to assure that the Project A Team’s deliverables and outcomes are timely,

effective and are achieving the results specified in the contract.

This draft QASP is a “living document” and the Government may review and revise it on

a regular basis. However, the Government will coordinate changes with the Project A

Team. Updates will ensure that the QASP remains a valid, useful, and enforceable

document. The revisions process will include the ongoing identification and approval of

additional performance indicators, acceptable quality levels, methods of surveillance, and

performance-based incentives to achieve continuous improvement throughout the Project

A contract period of performance.

The following recommendations will facilitate QASP revisions:

Evaluate and assess adherence to initial QASP within 60 business days of contract

award;



Finalize Project A QASP within six (6) months of award of contract and prior to the

end of the award fee period;

Conduct subsequent reviews of QASP within 6 months of award prior to end of

next award fee period and annually thereafter if no other changes are needed; and

As required, apply revisions in response to program requirements, objectives, or

priority changes.

All review and recommended modifications require Federal Agency approval through the

COTR and CO before updates may be implemented.

1.2 Project A Program Quality Surveillance Roles

The following Government personnel shall oversee and coordinate surveillance activities

for the Federal Agency Project A Program.

1.2.1 Federal Agency Representatives

1. Contracting Officer Technical Representative (COTR): The COTR has

responsibility to receive and assess the preliminary award fee recommendation

and prepare any additions to the report, which includes any information, obtained

from his/her position as COTR. The COTR is responsible for technical

administration of the contract and shall assure proper Government surveillance of

the Project A Team’s performance.

Assigned COTR: Todd Northwood

Telephone: 410-786-0247

Email: Todd.Northwood@Federal Agency.hhs.gov

2. Federal Agency Project A Government Task Leader: The Project A

Government Task Leader will assess the preliminary award fee recommendation

in conjunction with the COTR and prepare any additions to the report, which

includes any information obtained from his/her position as Federal Agency Project

A Government Task Leader. The Federal Agency Project A Government Task

Leader oversees the Federal Agency Project A Project to monitor the contractor’s

performance and also supports the COTR’s authority to provide technical direction

to the contractor.

Assigned Project A Government Task Leader:

Sharlene Mansaray and Timothy Purcell

Telephone: 410-786-2103

410-786-0501

Email: Sharlene.Mansaray@Federal Agency

Timothy.Purcell@Federal Agency



3. Federal Agency Project A Team: The Federal Agency Project A Team and

Business Function Leads will monitor the contractor's performance and support

the COTR’s authority to provide technical direction to the contractor. Technical

monitors support project officers by assessing contractor performance and

reporting their findings to the project officers.

1.2.2 Project A Team Representatives

The following members of the contractor team are proposed to serve as key

representatives of the Project A Team and are responsible for management and quality

control actions to meet the terms of this contract.

1. Project Manager: XXXXXX – The Project Manager is ultimately responsible for

management of all project tasks, including managing the PMP, schedule, cost,

staffing, and risks; managing and submitting project and financial status

information; monitoring and reporting project metrics to Federal Agency and

COMPANY B senior management at periodic program reviews; coordinating

activities with GTL/COTR and other Federal Agency contractors; and ensuring

compliance with COMPANY B/ Federal Agency policies and procedures.

Telephone: XXXXXXX

Email: gshetty@COMPANY Bnc.com

2. Chief Architect: XXXXXX – The Chief Architect leads all Development,

Maintenance and Integration Management activities. He will coordinate and

communicate Project A integration requirements to the project teams, Federal

Agency, and other application stakeholders. He will also support IM/AM and SOA

Architect in developing overall integration strategy and approach based on

application design and architecture.

Telephone: 301.977.7884 x538

Email: ivinagradov@COMPANY Bnc.com

3. Chief Security Architect: XXXXXX – The Chief Security Architect leads security

analysis, development and maintenance of the SSP and ISRA documents; works

closely with COMPANY B/Federal Agency/Terremark Team’s technical and

functional staff to ensure compliance with FISMA/NIST/OMB guidelines and

standards, Federal Agency security policies, and IT security procedures; supports

security reviews and ST&E as required; and coordinates with Federal Agency on

receiving ATO and assist with all Information System Certification and

Accreditation process.

Telephone: 301.977.7884 x223

Email: mchien@COMPANY Bnc.com

4. Quality Assurance Manager: XXXXXX – The Quality Assurance Manager

ensures compliance with contract requirements, applicable quality, and CMMI



standards; organizes, participates in and conducts formal and informal reviews,

audits, walkthroughs, and peer reviews at predetermined points throughout the life

cycle; and ensures processes are repeatable and quality controls are embedded

in the process.

Telephone: 301.977.7884 x232

Email: ijohnson@COMPANY Bnc.com

2 Performance

2.1 Methods of Surveillance

Performance evaluation will be accomplished by monitoring compliance with the

performance requirements as described in Attachment J-1 of the Project A Statement of

Work Service Level Objectives (SLOs) and in the following section of this QASP. The

method of evaluation will depend on the service type, and each service type may be

assessed by more than one method.

This QASP is based on the premise that the government desires to maintain a quality

standard for Project A technical support services. The Project A Team is responsible for

management and quality control actions to meet the terms of the contract. The role of the

government is quality assurance to ensure that contract standards are achieved.

In this contract the quality control program is the basis for service quality. The contractor

is required to deliver technical support services that conform to the requirements of the

contract. Various methods exist to monitor performance. The COTR shall use the

surveillance methods listed below in the administration of this QASP.

Regardless of the surveillance method, the COTR shall contact the Project A Team's task

manager or on-site representative if and when a defect is identified and shall inform the

manager of the specifics of the problem. The COTR, with assistance from the Project A

Government Task Leader, shall be responsible for monitoring the Project A Team’s

performance in meeting a specific performance standard and maintaining acceptable

quality levels (AQLs).

The following are the specific evaluation methods to be used on the Project A contract.

Effective quality surveillance will involve a combinational use of the following methods:

100 Percent Inspection: This is recommended for mission critical issues and issues

related to actual system failures and incidents; otherwise, it is not cost-effective

and is too stringent. Key contract deliverables are inspected and evaluated upon

completion. This method is applicable to each contract deliverable and review and

inspection is provided by the COTR and/or Project A Government Task Leader. If

a major or critical service or product is judged as nonconforming or unacceptable,



Federal Agency will inspect the Project A Team's correction or replacement to

ensure that it is satisfactory.

Random Sampling: Appropriate for recurring tasks or production requirements.

Periodic Inspection: Periodic inspection entails review of deliverables that do not

require 100% Inspection or Random Sampling or where such inspection is not

practicable. Periodic inspection should be employed to ensure that multi-step tasks

are progressing as per the intent of SLOs. Periodic inspection should be used

when the number of services performed spans a wide range of activities.

Customer Feedback (User Survey): Suitable for service-oriented tasks; this

surveillance method uses a standard form to document customer satisfaction with

a defined set of services.

Operational Monitoring: Examination of reports generated by automated tools and

logs and websites. Appropriate for tasks like system maintenance where the

contractor can provide system records collected by tracking and monitoring tools

of documented performance; for development projects, monthly reports can detail

problems encountered.

Independent Verification and Validation (IV&V): This method is verification and

validation performed by an individual or organization that is technically,

managerially, and financially independent of the development organization and is

used to determine that the work product(s) delivered by the contractor matches the

content requirement in the contract. This can include various types of system tests,

analysis or other V&V techniques.

Surveillance results may be used as the basis for actions (e.g., to include payment

disincentives for unsatisfactory performance) against the contractor. The proposed

performance evaluation methodology and incentive structure for the Federal Agency

Project A Program are outlined in Section 4 of this QASP, respectively.

2.2 Performance Standards

Performance standards define desired services. The Government performs surveillance

to determine if the contractor exceeds, meets or does not meet these standards.

The Service Level Objectives (SLOs) Matrix, in the following section includes

performance standards against which performance will be measured. The Government

shall use these standards to determine contractor performance and shall compare

contractor performance to the AQL, as defined in the PRS.

2.3 Service Level Objectives

The primary purpose of Service Level Objectives (SLOs) is to identify those performance

requirements considered most critical to acceptable contract performance and the

corresponding standards of performance. Each of the 34 SLOs identify the Acceptable

Quality Level (AQL) for the required service outlined in the SOW. Each SLO is based on



the Federal Agency Project A SOW tasks detailed in this document and is tracked in the

Federal Agency Project A WBS and Project Schedule. The award fee is assessed based

on a technical performance (including deliverables as detailed in Table 1) split of 65%

and a management performance split of 35%.

2.3.1 Performance Objectives

The performance requirements for the 47 deliverables are detailed in a separate table in

the following section; however, they are still considered a technical performance

requirement.



SLO Num.

SOW Reference

Performance Objective / Desired Outcome / and / or Service Definition

Performance Standard

Acceptable Quality Level (ACL) * Monitoring Method(s)

01 C.2.1 New Application or Service Deployment

The Contractor’s performance for the successful deployment of a new application or service, including testing.

Deliverable Timeliness and Completeness

Meet or exceed deadlines as defined in the project plan or schedule 99% of the time. All missed deadlines reported to Federal Agency and approved by Project A GTL team not less than one (1) week prior to original milestone completion due date.

Contractor reporting (including the Status Report Published Monthly before the 10th of the month for Federal Agency) and Federal Agency inspection / monitoring

02 C.2.1 Periodic reassessment, recertification, and reauthorization of the system or other IT solution are supported

Timeliness and Quality

Reassessment and recertification sessions scheduled and conducted according to the project plan date, unless altered by agreement with Federal Agency; meeting notes and action items taken and tracked to completion

Contractor reporting and Federal Agency inspection

03 C.2.1, C.3.5.3, etc.

System documentation, test strategies, and operations manual are updated to reflect changes due to maintenance modifications.


100% of approved Modification Requests (MRs) are reflected within System documentation, test strategies, and operations manual.


04 C.2.1.3 Project A Integration Support Timeliness and Quality

All negotiated and approved application integrations are prioritized, scheduled, and tracked in the project plan; action plans created and managed for each approved application integration


05 C.2 (inclusive)

Measures activity associated with software maintenance and enhancements, release implementation and documentation, production support for Project A, and the investigation/resolution of problems. Activities include, but are not limited to, developing, testing, documenting and maintaining software programs and interfaces.

Release Timeliness and Completeness

99% of Releases are implemented on schedule and all Federal Agency-approved functionality is deployed with the Release.




SLO Num.

SOW Reference




06 C.2 (inclusive)

Release Quality: Measures activity associated with software maintenance and enhancements, release implementation and documentation, production support for Project A, and the investigation/resolution of problems. Activities include, but are not limited to, developing, testing, documenting and maintaining software programs and interfaces.

Release Quality Severity and Frequency of Release Software Defects is minimal: Zero (0) Critical, no more than one (1) High, and no more than thirty (30) Medium Trouble Tickets opened and associated with the Release during the warranty period.

Weekly Trouble Ticket Report; Monitoring, and Inspection.

07 C.2.1.3 Code libraries are developed and maintained


100% of Code library members are delivered according to the project plan date or as requested by Federal Agency, unless altered by agreement with Federal Agency


08 C.2.1.3 Effective training on the use of code libraries and web services is delivered


90% of student satisfaction surveys returned with ratings of 4 (good) or 5 (excellent)


09 C.2.1.3, C.2.4 Effective and well written Training materials are created


Training material is free of grammatical and spelling errors, contains all required content to effectively communicate the required information, and is delivered according to the project plan date (unless altered by agreement with Federal Agency)


10 C.2.2 Routine operations and maintenance are successfully performed. The solution supports measurement and reporting of reliability and availability. System uptime exclusive of planned downtime (releases, upgrades) and unplanned downtime due to external factors


Solution Uptime: 99.999% Based on calculations over rolling three-month period

Contractor reporting and Federal Agency inspection, including inspection of system generated reports



SLO Num.

SOW Reference




11 C.2.2 Maintain operational efficiency by adhering to production implementation schedule.

Operational efficiency and effectiveness.

Meet or exceed milestone completion dates as defined in the project plan.

Contractor reporting (including the Status Report Published Monthly before the 10th of the month for Federal Agency) and Federal Agency inspection.

12 C.2.2 Turn around on Estimates of Work Resulting from Change Requests: Contractor’s Responsiveness to new requirements or changes to the Project A System


99% of the time, the Contractor shall provide Level of Effort (LOE) estimates on CRs issued to the contractor No Later Than (NLT) 5 Business Days from issuance by Federal Agency Project A GTL Team

Contractor Response through CR Tracking System and Monthly Status Report.

13 C.2.2 Each Change Request (CR) and/or other component included in a release is received, classified, and analyzed - with prepared LOEs.


100% of CRs are evaluated; 98% are classified and prioritized correctly. For 95% of the CRs or other components included in a Release, the Actual Hours reported for each CR or component will be within 5% of the estimated hours for that CR or component; for 85% of the CRs or other components included in a Release, the Actual Hours reported for each CR or component will be within 10% of the estimated hours for that CR or component.

Release Follow-up and Variance Report



SLO Num.

SOW Reference




14 C.2.2 Contractor’s ability to keep required documentation of the Project A System current and accurate; required documentation updated as specified in SOW and/or Federal Agency XLC


99% of the time, the contractor shall deliver all relevant Release documentation at the time of the appropriate gate review (as specified in the SOW and/or XLC). The documentation delivered will be complete and accurate.

This information will be included in the Environment Readiness Review documentation for each gate review and presented to the Federal Agency Project A Team prior to promotion. Documentation not submitted prior to deployment will be noted in the next weekly status report.

15 C.2.2 Required documentation is complete and accurate when delivered to Federal Agency: All documentation will be delivered to Federal Agency complete, accurate, and fully reviewed; with sign-off by the Contractor’s QA/QC Representative


98% of documentation delivered to Federal Agency will be of acceptable quality for final acceptance after one (1) review and comment iteration by Federal Agency. 99% of documentation delivered to Federal Agency will be of acceptable quality for final Federal Agency acceptance after no more than two (2) review and comment iterations by Federal Agency. If no comments are received from Federal Agency after ten (10) business days, delivered documentation will be considered accepted by Federal Agency. Any comments received from Federal Agency after 10 business days will not count toward this metric.



SLO Num.

SOW Reference




16 C.2.2, C.2.2.2, C2.2.3

Provide O&M services and configuration of the Project A system, including custom, COTS, and GOTS software, using planned releases


All negotiated and approved application releases are prioritized, scheduled, and tracked in the project plan; project and action plans created and managed for each approved application release


17 C.2.4.1 Provide an efficient and fully staffed Helpdesk between the hours of 7:00 AM EST and 9:00 PM PST, Monday through Friday


Shifts are fully covered, and time logs reflect attendance


18 C.2.4.1 Support service scripts developed accurately and timely


Scripts are created timely, with professional and appropriate wording


19 C.2.4.1 Help Desk efficiency and effectiveness: Minimal elapsed time from receipt of an inbound Help Desk call to the caller being responded to and helped

Timeliness and Quality Help Desk efficiency and effectiveness

95% of calls answered within 60 seconds. 98% of calls answered within 120 seconds. 100% of calls answered within 150 seconds.


20 C.2.4.1 Help Desk

Help Desk efficiency and effectiveness: Timely response to emails


95% of emails responded to within 4 hours; 99% of emails responded to within 6 hours; 100% of emails responded to within 8 hours.


21 C.2.4.1 Help Desk efficiency and effectiveness: First Call Resolution – High Percentage of customer issues resolved on the user’s first contact to the Project A Helpdesk, except for services not considered for first call resolution performance. Ticket exclusions include: acquisition, system change requests, non-Project A problem ticket requests (e.g. hard drive failures), request for training, new login account requests, and tickets that go to other Helpdesk systems.


First Call Resolution – 99% of customer issues resolved on the user’s first contact to the Project A Helpdesk

Contractor reporting and Federal Agency inspection, including inspection of monthly incident summary report



SLO Num.

SOW Reference




22 C.2.4.1 Effective confirmation of problem resolution


Time to resolve 95% of all internally- caused incidents: Critical (Sev 1) < 24 hours High (Sev 2) = 24 hours Medium (Sev 3) < 3 business days


23 C.2.4.1 Effective ticket incident management Timeliness and Quality

98% of all resolved tickets confirmed as closed each month


24 C.2.4.1 Help Desk efficiency and effectiveness for Level 2 type Trouble Ticket support: Acceptable service to person or entity reporting Level 2 type Help request


99% of the time, the contractor shall respond either telephonically or by e- mail, to the source of the trouble ticket, within 1 hour of receiving the notice, to advise that the trouble ticket has been received and assigned. The Contractor shall log this entry into the trouble ticket system at the time of response.

Contractor reporting and Federal Agency inspection, including Trouble Ticket log reports provided with Monthly Status Reports.

25 C.2.5 Work collaboratively with other vendors and applications to ensure successful installation, integration, deployment, and operation of Project A components and services


Integration meetings and review sessions scheduled and conducted with vendors and application owners according to the project plan date, unless altered by agreement with Federal Agency; meeting notes and action items taken and tracked to completion


26 C.2.6 Maintenance performed when scheduled and does not negatively impact end user activities or system performance.


Routine maintenance activities will be completed on time with no unscheduled service impact to end users or degradation to system performance

Contractor reporting (including the Status Report Published Monthly before the 10th of the month for Federal Agency) and Federal Agency inspection.



SLO Num.

SOW Reference




27 C.2.7 Identification and Notification Time for Critical Severity Problems: Minimal elapsed time from detection of a critical issue to appropriate notification and/or escalation.

Timeliness and Quality; Contractor responsiveness

99% of Critical Service Impact notifications reported appropriately and accurately within 15 minutes or less from first detection.

Contractor reporting (including the Monthly Status Report) and Federal Agency inspection.

28 C.2.7.1 Accuracy of Severity Level assignment: Accurate assignment of Severity Level to each Problem Report (PR) [also known as a Remedy Ticket (RT) or Trouble Ticket (TT)]


98% of PRs will have a final Severity Level equal to the initial Severity Level assigned by the Contractor. PRs with an initial Severity Level assigned by Federal Agency, or another Federal Agency contractor, are excluded from this metric.

Contractor reporting and Federal Agency Inspection.

29 C.2.7.3 Production Support efficiency and effectiveness: A HIGH severity (Sev 1) Problem Report (PR) must be resolved within the timeframes specified in the Project A SOW


98% of HIGH severity PRs are resolved within 24 hours (i.e., the problem is fixed, or an acceptable work around has been approved by Federal Agency and is in place). If a work around is developed, the PR for the solution will be assigned to a release within 3 business days for the ticket to be considered resolved within the 24-hour timeframe.

Contractor reporting and Federal Agency Inspection.

30 C.3.1.11 Project Schedules are accurate and complete when finalized, and delivered timely (Master Project Schedule and the Project Schedule for each release, integration, transition, or project)


100% of Federal Agency changes incorporated in the final project schedule, and 100% of the required tasks and due dates are included. The draft and final schedules are delivered to Federal Agency when due. Any revision(s) to the Project Schedule are due to delays beyond the control or influence of the Contractor. The final decision regarding whether or not a schedule delay was outside the control of the Contractor shall be made by the COR and/or GTL.

Contractor reporting and Federal Agency Inspection



SLO Num.

SOW Reference




31 C.3.1.18 Effective meeting support Timeliness and Quality

100% of meetings and review sessions scheduled and conducted according to the project plan date or as requested by Federal Agency, unless altered by agreement with Federal Agency; meeting notes and action items taken and tracked to completion

Contractor reporting and Federal Agency Inspection

32 C.3.1.20 and C.3.5.3

Software Support / Root Cause Analysis (RCA): The Contractor’s ability to isolate and analyze a problem and provide RCA assuring the actual problem (not a symptom) is resolved and being monitored to provide assurance that the problem will not reoccur; and provide code that assures reliable and predictable functionality and performance


At Federal Agency request, the Contractor shall provide a fully documented RCA report on all instances requiring an Emergency Release or code promotion. 99% of RCAs delivered to Federal Agency during an Award Fee Period will be of acceptable quality for final acceptance after one (1) review and comment iteration by Federal Agency.

Contractor reporting and Federal Agency Inspection (Release management reports and Contractor provided Monthly Status Reports)

33 C.3.5 Coding Quality for Change Requests and code fixes: Quality code providing reliable systems and predictable customer support: Contractor’s ability to provide quality code and an effective, efficient, and reliable system


99% of the time, the Contractor shall provide code and releases that introduce no Critical (Sev 1) impacts resulting in unscheduled outages and emergency code promotion. The contractor will not be penalized for emergency code promotions whose RCA identifies late changes by the Business Owners (Federal Agency) or Requested Changes that, in themselves, cause the fault.

Trouble Ticket Reports and Release Management System reports in conjunction with Monthly Contractor Status Reports.



SLO Num.

SOW Reference




34 C.3.6 Requirements, testing, and defect management will be complete and accurately reported to Federal Agency: All requirements, test artifacts, and defects will be complete, accurate, and fully maintained in the Federal Agency approved application lifecycle management tool


98% of requirements, testing, and defects maintained in the Federal Agency approved application lifecycle management tool will be of acceptable quality for final Federal Agency review one (1) full business day before an environment readiness gate review meeting for an application integration or system release

Contractor reporting and Federal Agency Inspection, including Dashboard reporting from the Federal Agency approved application lifecycle management tool



2.4 Performance Evaluation Methodology

The performance evaluation will be conducted based on the method of surveillance

(defined in Section 3.1) and as identified in Attachment J-1 of the SOW, and the assigned

weightings for each performance standard, resulting in the fully weighted range for the

Management of the Contract (accounting for 35% of the total award fee) and Overall

Technical Performance (accounting for 65% of the total award fee for that period).

Performance standard weightings are indicated on the SLO Matrix. Weightings will be

applied according to the relative weighting assigned to specific measures within each

category of performance and as agreed upon with Federal Agency. Management and

technical measures collectively will still be weighted as defined (35% and 65%) even in

award fee periods where the total number of management metrics outnumbers the

technical measures.

Figure 2: Project A Project Fee Allocation Weighting



3 Availability

3.1 Work Requirements

The detailed work requirements of the Project A contract are set forth in the SOW and

summarized in Table 2. Specifically, the table specifies the Performance Objectives

(Desired Outcomes) for respective SOW sections.

Work Requirements

Maintenance Services

Integration and Migration Support Services

Operations and Maintenance Services

User Support Services

Project Management

The successful implementation of Project A will depend on Federal Agency’ ability to

provide four critical functions across the Agency’s diverse community base and programs:

Identity Services: Register users, vet their identity, and provision users with

system access credentials.

Authentication: Verify the credentials of a user when accessing Federal Agency

systems.

Authorization Assistance: Manages the rights and privileges of each user to

access specific Federal Agency information resources.

Auditing: Verify the proper compliance with access control policies and legitimate

access by users.



4 Quality

4.1 QASP Implementation

4.1.1 Frequency of Evaluations

The frequency of evaluations is established to ensure consistency in surveillance and

confidence in results. There is a balance between the frequency of evaluations, the

staffing available to perform the evaluations, and the need to monitor COMPANY B

contractor operations to ensure performance. Individual performance standards will be

measured on a variable basis – some metrics to be measured and reported monthly (e.g.,

EVM statistics) at the monthly status report. However, in line with the semi-annual award

fee Periods and the quarterly Project A releases, the QASP self-evaluation report will be

submitted to Federal Agency within 10 days of receipt.

4.1.2 Base Year Considerations

The Project A QASP will undergo modifications in the first three to six months following

award, during the Base Year. At the end of the Base Year, the average of the performance

scores of the four quarters of the year will be used for the first Award Term decision.

Award fees thereafter will be based upon the average of the two quarterly performance

scores for each six-month award fee period.

Figure 3: Federal Agency Project A Performance Timeline

4.2 Documentation

All surveillance activities must be documented to provide the required audit trail to justify

award fee determinations. The official language of the contract is English, and all

documentation will be prepared in English. The documented audit trail of the surveillance

activities is required by FAR 46.104(c), Contract Administration office responsibilities,

which states:



“Maintain, as part of the performance records of the contract, suitable records reflecting,

(1) The nature of Government contract quality assurance actions, including, when

appropriate, the number of observations made and the number and type of defects; and

(2) Decisions regarding the acceptability of the products, the processes, and the

requirements, as well as action to correct defects.

4.3 Evaluation Process

The Government shall use the Award Fee Pool tied to the Project A Team’s performance

for each period as incentive to reward desired outcomes and use disincentives for not

meeting performance objectives. Incentives shall be based on a five-tiered rating scheme

as represented in Table 3 below and in the Federal Agency Project A Award Fee Plan

document.

At the end of each award fee period within ten working days, the Project A Team shall

submit the Project A Self-Evaluation Report to the designated COTR. The report will

include a performance assessment of factors outlined in Statement of Work (SOW) and

the Project A QASP, supporting data, and a description of the applicability of specific

measures for that performance period. For example, if a release was originally scheduled

for the prior quarter but was delayed due to an application upgrade, the Project A Team

would report on any measures impacted for that period.

The COTR and Federal Agency Government Task Leader will review the work performed

during that period and review the Project A Self-Evaluation Report. The COTR and the

Federal Agency Government Task Leader will evaluate the Project A Team’s

performance in accordance with the performance evaluation categories, the fee allocation

weighting, and the Service Level Agreements set forth in the Performance Work

Statement.

The COTR, Federal Agency Government Task Leader and the Federal Agency Project A

Team will make a recommendation regarding the percentage of fee to be awarded. This

recommendation also involves the Contracting Officer’s review of the contractors’

performance on the contract. This recommendation is provided to management and,

finally, the Federal Agency Fee Determination Official. The Fee Determination Official has

the prerogative to change the recommendation. By the 40th workday after the end of the

performance period, the Fee Determination Official has made a determination and an

official letter will be issued by the Contracting Officer to the COMPANY B Project Manager

stating the earned award fee.

Using the Performance Evaluation Methodology, a Composite Annual Score is computed

for the Base Year by taking the average of the four quarterly scores and rated against the

possible full range of points for the combined Management of the Contract (0 to 100 points

weighted by 35%); and for the Overall Technical Performance (0 to 100 points weighted

by 65%). The Project A Team proposes for Federal Agency to base determination of all



subsequent Award Fees on the average of the quarterly scores for each of the specified

six (6) month performance periods.

The Award Fee will be determined in accordance with the following Rating Scheme:

Figure 4: Rating Scheme

Points Earned

Award-Fee Adjectival Rating

Award-Fee Pool Available To Be

Earned

Description

95-100 Excellent 91%-100% Contractor has exceeded almost all of the significant award-fee criteria and has met overall cost, schedule, and technical performance requirements of the contract in the aggregate as defined and measured against the criteria in the award-fee plan for the award-fee evaluation period.

88-94 Very Good 76%-90% Contractor has exceeded many of the significant award-fee criteria and has met overall cost, schedule, and technical performance requirements of the contract in the aggregate as defined and measured against the criteria in the award-fee plan for the award-fee evaluation period.

76-87 Good 51% - 75% Contractor has exceeded some of the significant award-fee criteria and has met overall cost, schedule, and technical performance requirements of the contract in the aggregate as defined and measured against the criteria in the award-fee plan for the award-fee evaluation period.

70-75 Satisfactory No Greater than 50% Contractor has met overall cost, schedule, and technical performance requirements of the contract in the aggregate as defined and measured against the criteria in the award-fee plan for the award-fee evaluation period.

0-69 Unsatisfactory 0% Contractor has failed to meet overall cost, schedule, and technical performance requirements of the contract in the aggregate as defined and measured against the criteria in the award-fee plan for the award-fee evaluation period.

The contractor will not be paid any award fee when the total award fee score is Marginal

or Unsatisfactory (less than 70).

4.3.1 Acceptable Performance

The Government shall document positive performance. Any report may become a part of

the supporting documentation for fixed fee payments, award fee payments, or other

actions. In an effort to continuously improve the quality of service, FEDERAL AGENCY

may also submit a Continuous Improvement Opportunity (CIO) form to COMPANY B as

needed or as part of the lessons learned following a performance evaluation. The CIO



represents a letter written by a Quality Assurance Evaluator through the CO to the

COMPANY B contractor recommending an improvement to the contractor's operations

and intended for use at the contractor's discretion. A sample CIO is presented in Appendix

A.

4.3.2 Unacceptable Performance

When unacceptable performance occurs, the COTR shall inform the contractor. This will

normally be in writing unless circumstances necessitate verbal communication. In any

case, the COTR shall document the discussion and place it in the COTR file.

When the COTR determines formal written communication is required, the COTR shall

prepare a Contract Discrepancy Report (CDR) and present it to the contractor's task

manager or on-site representative. The CDR is a letter written by a Contract Administrator

to the contractor identifying a deficiency in the contractor's performance and requesting

a response that identifies root cause and proposed corrective and prevention actions. A

CDR template is attached to this QASP as Appendix B.

The PROJECT A Team will acknowledge receipt of the CDR in writing. The CDR will

specify if COMPANY B is required to prepare a corrective action plan to document how

the contractor shall correct the unacceptable performance and avoid a recurrence. The

CDR will also state how long after receipt the contractor has to present this corrective

action plan to the COTR. The Government shall review the contractor's corrective action

plan to determine acceptability.

Any CDRs may become a part of the supporting documentation for contract payment

deductions, fixed fee deductions, award fee nonpayment, or other actions deemed

necessary by the CO.



QASP Appendix A – Continuous Improvement Opportunity Form

(Letter format with letterhead stationary)

MEMORANDUM FOR Project A Team

Attention: Project Manager

FROM: (Office Name/Symbol)

(Office Address)

SUBJECT: Continuous Improvement Opportunity (Assigned Identification Number)

1. The following opportunity for improvement is forwarded for your consideration.

2. (State recommendation along with appropriate benefits.)

3. This Continuous Improvement Opportunity is forwarded for use at your discretion and

is not to be considered as government direction. Please advise this office if this CIO is

accepted or rejected.

4. (Provide the name of the originator who prepared the CIO and phone number.

Example: For additional information, contact John Smith, ###-###.)

NAME

Quality Assurance Evaluator

Federal Agency Project A Technical Support Contract

cc: Kevin Newton, Contracting Officer



QASP Appendix B – Contract Discrepancy Report (CDR)

1. Contract Number: <insert number>

2. TO: (Contractor Task Manager or on-site representative) <insert name>

3. FROM: (Name of COTR) <insert name>

4. Date and time observed discrepancy:

5. Discrepancy OR Problem:

<Describe in detail. Identify any attachments.>

5. Corrective action plan:

A written corrective action plan < is / is not > required.

< If a written corrective action plan is required include the following. > The written

Corrective Action Plan will be provided to the undersigned not later than five (5) business

days after receipt of this CDR.

Prepared by: <Enter COTR’s name>

_____________________________ ________________

Signature – Contracting Officer’s Representative Date

Received by:

_____________________________ ________________

Signature – Project A Team Task Manager or on-site representative Date

< The COTR may initiate a CDR at any time, including whenever the number of monthly

recorded defects for a performance standard exceeds the allowable number of defects;

anytime unacceptable performance is determined critical in nature and requires formal

corrective action; and whenever an unfavorable trend is detected in contractor

performance.>



Attachment 6: Statement of Objectives (SOO)

The Statement of Objectives (SOO) provides basic, top-level objectives of a task order,

and is provided in lieu of a government-written statement of work (SOW) or Performance

Work Statement (PWS). It provides contractors the flexibility to develop cost-effective

solutions and the opportunity to propose innovative alternatives meeting the objectives.

Format

I. Purpose:

II. Scope or Mission:

III. Period and Place of Performance

IV. Background:

V. Performance Objectives, Goals and Outcomes

Examples include:

Overall Objectives:

1. Personnel: Provide a proper skill mix, experience, and required number of

qualified personnel

2. Materials: Provide all necessary supplies, spares, tools, and test equipment,

consumables, hardware, software, automatic data processing equipment,

documentation, and other applicable properties.

3. Facilities: Provide administrative and workspaces.

4. Organizational Processes: provide internal controls, management oversight, and

supply support.

Task Order Objectives:

Most objectives will already be identified within the contract document. You may include

specific task order objectives here. If you do include this type of objective, you may need

to include instructions for how you wish the ITES-3S contractors to address these

objectives within their proposals. Objectives identified within the SOO are addressed by

the ITES-3S contractors within a SOW, which they write. Therefore, consider how

objectives identified in this section could be addressed within a SOW.

Technical Objectives:

1. Make maximum use of commercial products.

2. Install the system with a minimum impact to other systems that may be located in

the designated facility.



3. Develop and document procedures for managing system engineering, software

and hardware development. Utilize commercial standards and procedures to the

maximum extent in achievement of this objective. The system engineering process

includes parts management, quality assurance, Electro-Static Discharge (ESD)

control, reliability, maintainability, system safety, etc.

Program Objectives:

1. Establish a program management that provides accurate and timely schedule and

performance information throughout the life cycle of the program.

2. Establish a sound risk management system, which mitigates program risks and

provides for special emphasis on software development efforts through integration

of metrics to monitor program status.

3. Establish a comprehensive configuration management system.

4. Obtain sufficient rights in technical data, both software and hardware, such that

the Government can maintain and modify the training system using Government

personnel and third-party contractors.

5. Use electronic technologies to reduce paper copies of program information

generated throughout the life of this contract.

6. Use electronic technologies to communicate and pass data between government

and contractor organizations.

Operating or Programmatic Constraints

The following specifications, standards, policies and procedures represent the constraints

placed on this Task Order:



Attachment 7: ITES-3S Evaluation Plan

(CHECK ONE):

Best Value Trade-Off

Lowest Price, Technically Acceptable

Non-Price Factors

Note: Describe the relative weight of each evaluation factor compared with the other

evaluation factors. For example, the evaluation factors may all be approximately equal in

importance, or one factor may be more important than others.

List the specific areas of your technical/management requirements to be evaluated.

These areas should correspond with, and relate to, specific requirements.

Technical/Management Approach

List the specific areas of your past performance requirements to be evaluated.

Past Performance

These areas should relate to specific work statement requirements.

Other Factors (if applicable).

List any other evaluation criteria important to you and the associated weights below.



Price Factors

Adjectival ratings (e.g., outstanding, good, etc.) are assigned to corporate experience,

technical/ management approach and any other non-price criteria for which you may want

to evaluate contractor proposals. Note that balancing price against non-price factors

facilitates a best value trade-off decision, and, as a result, a rating is not assigned to the

price factor. Indicate whether all non-price evaluation factors, when combined are:

Significantly more important than the price factor

More important than the price factor

Comparatively equal to the price factor

Less important than the price factor

Significantly less important than the price factor



Attachment 8: Letter Request for Task Order Proposals

LETTERHEAD

IN REPLY REFER TO: (DATE)

MEMORANDUM TO: (ITES-3S) Contractors

SUBJECT: Request for Task Order (TO) Proposals

1. The Network Enterprise Center for [insert command] has a requirement for

[insert, as appropriate]. The period of performance is [insert duration of order].

The anticipated contract type is [insert as appropriate]. This requirement has

been assigned tracking number [insert number].

2. It is requested that you submit written technical and price proposals in response to

the attached [insert, as appropriate, e.g., statement of work, performance

work statement, or statement of objectives] (Ordering Guide Attachment 1).

Specific proposal instructions and evaluation criteria are also attached (Ordering

Guide Attachment 2). Your proposal or “no-bid reply” shall be submitted no later

than [insert date/time]. Any “no-bid reply” must include a brief statement as to

why you are unable to perform. Please upload your proposal or no bid reply to the

Computer Hardware, Enterprise Software and Solutions IT e-mart at:

https://chess.army.mil

3. Virtual Reading Room. A Virtual Reading Room has been established to provide

access to information related to this acquisition [insert specific information as

appropriate].

4. Due Diligence. As part of the proposal preparation process, the Government will

offer the ITES-3S contractors the opportunity for due diligence. This will enhance

your understanding of the requirements and is in keeping with the principles

identified by Federal Acquisition Regulation Part 15.201, Exchanges with Industry

before Receipt of Proposals. The following arrangements have been made for

interested contractors to contact appropriate Government representatives to ask

questions that by their very nature they would not ask if the response would be

posted and provided to their competition: [insert information, as appropriate].

5. Resolution of Issues. The ordering contracting officer reserves the right to withdraw

and cancel the proposed task. In such event, the contractor shall be notified in

writing of the ordering contracting officer’s decision. This decision is final and

conclusive and shall not be subject to the “Disputes” clause or the “Contract

Disputes Act.”




6. Questions should be addressed to the ordering contracting officer at the following

e-mail address: [insert address]. Please provide any questions no later than

[insert date / time]. Questions received after this date may or may not be

answered. Contact [insert name / telephone number] if you have any questions

or require additional information.



Sincerely,

ITES-3S Ordering Contracting Officer

Attachments:

Work Statement

Proposal Submission Instructions and Evaluation Criteria



Attachment 9: Proposal Submission Instructions and Evaluation

Criteria

Proposal Submission Instructions

Technical and Price Proposals shall be separate documents and consist of the following

tabs: Note: While the Technical Proposal must not contain any reference to price,

resource information (such as data concerning labor hours and categories, materials,

subcontracts, etc.) must be provided so that a contractor’s understanding of the

requirements may be evaluated.

4.4 Tab 1: Technical Proposal

Technical proposal information will be streamlined. Page limits are specified below. As a

minimum, technical proposals shall address the following elements:

Technical/Management Approach

Key Personnel Assigned

Teaming Arrangements (including subcontractors)

Risks and Risk Mitigation Plan

Period of Performance

Government Furnished Equipment (GFE) / Government Furnished Information

(GFI)

Security (including clearance level)

Other Pertinent Data (10 pages)

Note: If instructions are for a performance-based task order, and if a performance

work statement (PWS) is not already included in the task order request, the

Technical Proposal shall also include the offeror’s proposed statement of work

(SOW) or PWS detailing the performance requirements resulting from the

statement of objectives. (No page limit)

4.5 Tab 2: Cost / Price Proposal

This part of the proposal shall include details for all resources required to accomplish the

requirements (e.g., labor hours, rates, travel, incidental equipment, etc.). The price

proposal shall identify labor categories in accordance with the Labor Rate Tables

contained in Section B. It must also identify any GFE and/or GFI required for task

performance. If travel is specified in the SOW or PWS, airfare and/or local mileage, per

diem rates by total days, number of trips, and number of contractor employees traveling

shall be included.

4.6 Evaluation Criteria

This is a best value award, and the evaluation criteria for this award will be based on the

following factors and weights assigned to each factor.



Insert criteria as appropriate; describe the relative weight of each evaluation factor

compare with the other evolution factors. For example, the evaluation factors may be

approximately equal in importance or one factor may be more important than others.

1. Technical/Management Approach:

(1)

(2)

(3)

2. Past Performance:

(1)

(2)

(3)

3. Other Factors:

(1)

(2)

(3)

4. Cost/Price: In performing the best value trade-off analysis, all non-price evaluation

factors, when combined, are APPROXIMATELY EQUAL IN IMPORTANCE TO

cost/price.



Attachment 10: ITES-3S Selection Recommendation Document

1. Task Order (TO) Title. Enter the title as shown in the work statement.

2. Recommended Prime Contractor. Check the name/number of the ITES-3S prime contractor for whom you are recommending an award.

Fill in Contractor Name and Contract Number. (Example: Contractor ABC – Contractor #123)

3. Justification. Note: The “Fair Opportunity to be Considered” evaluation and justification is mandatory unless the requirement meets one of the five Federal Acquisition Streamlining Act (FASA) – specified/Section 803 exceptions described in part 4 below. If one of the exceptions applies, leave section 3 blank and complete sections 4 and 5.

Attach a narrative summarizing the evaluation results, including the adjectival ratings for each non-price evaluation factor and the identified strengths and weaknesses of the proposals received. Describe the evaluation methodology and the best value analysis that led to the recommendation of the prime contractor that should be awarded the TO the ITES-3S Proposal Evaluation Plan. The justification should be streamlined while containing the following: Results of Non-Price Evaluations: Discuss the results of

the non-price evaluations for each vendor that submitted a proposal.

Results of Price Evaluations: Discuss the results of the price evaluations for each vendor that submitted a proposal.

Trade-off Analysis: Describe the analysis that led to the recommendation of the prime contractor that should be awarded the TO.

4. Exception: Note: Complete section 4 only if an exception to the “Fair Opportunity to be Considered” process is being claimed.

If the specific requirements meet the criteria for one of the five FASA-allowed (Section 803) exceptions to the Fair Opportunity and the TO is, therefore, exempt from the evaluation described in section 3 above, check the appropriate exception and provide justification for why this TO is exempt from Fair Opportunity. The agency has such urgent need for services that

providing such opportunity would result in unacceptable delays. (Attach Justification)

Only one contractor is capable of providing such services required at the level of quality required because the services ordered are unique or highly specialized. (Attach Justification)

The order should be issued on a sole-source bases in the interest of economy and efficiency as a logical follow-on to a TO already issued under the ITES-3S contract, provided that all contractors were given “Fair Opportunity to be Considered” for the original order. (Enter the contract and TO number of the original TO.)

Contract W91QUZ-06-D , TO It is necessary to place an order to satisfy a minimum

guarantee. A statute expressly authorizes or requires that the

purchase be made from a specified store.



1. Task Order (TO) Title. Enter the title as shown in the work statement.

5. Authorized Official: Selection Recommendation Document must be signed by the authorized selection official, e.g. ordering contracting officer. Electronic signature (//s//) is acceptable.

Name, Signature and Date:

Date post:	17-Jun-2020
Category:	Documents
Upload:	others
View:	0 times
Download:	0 times