DOC

Office of the Chief Information OfficerOffice of the Assistant Secretary for Budget, Technology and Finance

Department of Health and Human Services

CPIC Procedures Appendices

December 30, 2005

Project: CPIC Procedures Appendices

Document Number: HHS-OCIO-2005-0005P-A

HHS-OCIO-2005-0005P-A December 30, 2005

HHS IT CPIC PROCEDURES - APPENDICES

APPENDIX A: REQUIREMENTS BY INVESTMENT CATEGORIES.......1

APPENDIX B: CPIC PROCESS CHECKLIST.........................................4

B.1 Select Phase Screening Process.....................................................................................................................4

B.2 Select Phase Pre-Selection Process................................................................................................................4

B.3 Select Phase – Portfolio Prioritization Process............................................................................................6

B.4 Control Phase – Periodic and Milestone Review Process...........................................................................6

B.5 Evaluate Phase Post Implementation Review (PIR) for New Investments...............................................7

B.6 Evaluate Phase Annual Evaluation Process for Existing Investments......................................................7

B.7 Process Improvement.....................................................................................................................................7

B.8 IT Management & Budget Formulation Calendar.....................................................................................8

APPENDIX C: SAMPLE PROSPECTUS SUMMARY FORM..................10

C.1 Purpose..........................................................................................................................................................10

C.2 Prospectus Summary Template..................................................................................................................10

C 3. Sample Prospectus Summary Form – Major/Tactical..............................................................................11

APPENDIX D: RISK MANAGEMENT..................................................21

D.1 Purpose..........................................................................................................................................................21

D.2 Process...........................................................................................................................................................21

APPENDIX E: PERFORMANCE MEASUREMENT...............................24

E.1 Purpose..........................................................................................................................................................24

E.2 Process...........................................................................................................................................................25

APPENDIX F: PROJECT MANAGEMENT............................................33

F.1 Purpose..........................................................................................................................................................33

F.1 Relationship of Project Management to Investment Management.........................................................34

CPIC Procedures Appendices i


F.2 Components...................................................................................................................................................35

APPENDIX G: EARNED VALUE MANAGEMENT................................39

G.1 Purpose..........................................................................................................................................................39

G.2 Process...........................................................................................................................................................41

APPENDIX H: POST-IMPLEMENTATION REVIEWS..........................44

H.1 Purpose..........................................................................................................................................................44

H.2 Process...........................................................................................................................................................45

H.3 Sample Initiative Evaluation Sheet.............................................................................................................46

H.4 Investment Management Report.................................................................................................................47

H.5 IT Initiative Evaluation Data Sheet............................................................................................................48

APPENDIX I: OPERATIONAL ANALYSIS...........................................49

I.1 Purpose..........................................................................................................................................................49

I.2 Management Objectives...............................................................................................................................49

I.3 Roles and Responsibilities............................................................................................................................49

I.4 Process...........................................................................................................................................................50

APPENDIX J: IT INVESTMENT RATING AND RANKING CRITERIA- HHS PRIORITY AND QUALITY SCORING..........................................59

J.1 Purpose..........................................................................................................................................................59

J.2 Background...................................................................................................................................................59

J.3 Scoring Methodology....................................................................................................................................59

J.4 Prioritization Criteria..................................................................................................................................61

APPENDIX K: REFERENCES..............................................................66

CPIC Procedures Appendices ii


Appendix A: Requirements by Investment

CategoriesIT investment managers shall enter and maintain data in the HHS Portfolio Management Tool as necessary to allow adequate CPIC review of individual IT investments and management of the overall IT investment portfolio. The extent and level of detail of reporting required shall vary depending on the IT Investment Category and the particular nature of each IT investment.

Major IT investments shall be subject to the following requirements:

Designation within the IT portfolio management application as a “major” investment.

Documentation that the investment is appropriately aligned with HHS strategic goals and objectives, architectural and security principles and practices, and follows sound IT investment planning and management practices (including full implementation of earned value management).

Compliance with all capital planning, FISMA, A-11, A-130, IT investment management and EVM requirements identified for major IT investments. Examples of those requirements include:

o Preparation and submission of sufficient to allow generation of a Capital Asset Plan and Business Case (OMB Exhibit 300) and OMB Exhibit 53 line item.

o Inclusion of each major IT investment and related IT investments in the HHS FISMA review process, as appropriate.

o Application of the full ANSI-EIA 748 Earned Value Management Standards for DME > $10 million during the DME phases of the investment life cycle.

o Use of operational analysis during the steady-state phases of the investment to ensure that sound investment management practices are in place, to determine how close the investment is to meeting its operational cost, schedule and performance goals, and that appropriate decisions are made regarding investment maintenance, upgrade, modernization and enhancement.

o Assignment of a qualified investment manager, or the equivalent, as determined by the HHS CIO.

Tactical IT investments shall be subject to the following requirements:

Designation within the IT portfolio management application as a “tactical” investment.

Documentation that the investment is appropriately aligned with HHS strategic goals and objectives, architectural and security principles and practices, and follows sound IT investment planning and management practices (including appropriate implementation of earned value management).

Compliance with all appropriate capital planning, FISMA, A-11, A-130, IT investment management and EVM requirements. Examples of those requirements include:

o Preparation and submission of sufficient investment data to generate an HHS OMB Exhibit 53 line item.

o Inclusion of each tactical IT investment and related IT investment in the HHS FISMA review process, as appropriate.

CPIC Procedures Appendices - Appendix A Page 1 of 67


o Application of an abbreviated ANSI-EIA 748 Earned Value Management Standards (i.e., 10 of the 32 ANSI standard criteria) during the DME phases of the investment life cycle. The abbreviated EVM standards shall be identified in the HHS Earned Value Management Plan and in IT Capital Planning procedures as designated by the HHS CIO.

Use of operational analysis for the steady-state phases of the investment to ensure that sound IT investment management practices are in place, to determine how close the investment is to meeting its operational cost, schedule and performance goals, and that appropriate decisions are made regarding investment maintenance, upgrade, modernization and enhancement.

Assignment of at least a qualified investment manager, or the equivalent, as determined by the HHS CIO.

Supporting IT investments shall be subject to the following requirements:

Identification within the HHS Portfolio Management Tool as a “supporting” investment.

Documentation that the investment is appropriately aligned with HHS strategic goals and objectives, architectural and security principles and practices, and follows sound project planning and management practices (including appropriate implementation of earned value management).

Compliance with all appropriate capital planning, FISMA, A-11, A-130, project management and EVM requirements. Examples of those requirements include:

o Preparation and submission of sufficient investment data to generate an HHS OMB Exhibit 53 line item.

o Inclusion of each supporting IT investment and related IT investments in the HHS FISMA review process.

o Application of appropriate earned value management techniques during the DME phases of the investment to ensure that cost and schedule variances remain within 10 percent.

Each IT investment with steady state activities (includes mixed life-cycle investments) shall demonstrate, through operational analysis, how close it is to meeting original and current cost, schedule and performance goals, whether it continues to meet HHS and user requirements, and that the total investment costs for an IT investment covers the life cycle and includes all budgetary resources.

Assignment of at least a qualified investment manager, or the equivalent, as determined by the HHS CIO.

New IT Investments

IT investment managers seeking approval of new IT investments must provide Prospectus information for the appropriate investment category.

o General contact, organization, investment type

o Brief description of IT investment

o Alignment with mission

o Potential benefits

o Major milestones

o Estimated cost and source of funding

o Acquisition risks



o Assignment of a qualified investment manager, as determined by the HHS CIO.



Appendix B: CPIC Process Checklist

B.1 Select Phase Screening Process



What are the business needs for the investments?

The Business Sponsor, in coordination with OPDIV management, identifies an unmet need and assigns an Investment Manager to formally establish a business need and mission alignment.

The Investment Manager develops the investment concept and prepares the preliminary Prospectus Form in the HHS Portfolio Management Tool (PMT).

The Business Sponsor reviews the prospectus for quality, thoroughness, accuracy, etc. along with the Business Steering Committee, which reviews the prospectus for possible opportunities for integration or consolidation with existing cross-cutting IT/Business systems.

Investment Managers address any outstanding issues and update the Prospectus Summary Form, as necessary.

The CPIC Team reviews the Prospectus Form and provides any comments and or questions for the Investment Manager to address, if necessary.

The Critical Partners review the Prospectus Form and provide any comments and or questions to the Investment Manager to address, if necessary.

The Technical Review Board, CIO Council, or other technical review organization reviews these materials and makes recommendations regarding initiation of additional planning activities for the investment

The ITIRB reviews the Prospectus Form. If the ITIRB approves the recommendation, the investment moves forward into Select Phase Pre-Selection Process. If it is not approved, it could return to the Investment Manager for additional work or cancellation.



B.2 Select Phase Pre-Selection Process



How do you know you have selected the best investments?

The Business Sponsor along with the Investment Manager develops, or reviews and updates (in the case of existing investments) the Business Case and high-level Project Management Plan in the PMT.

The Business Steering Committee reviews the Business Case for possible opportunities for integration or consolidation with existing cross-cutting IT/Business systems

The CPIC Team reviews the Business Case and Project Management Plan and provides any comments and or questions for the Investment Manager to address, if necessary.

The Critical Partners review the Business Case to ensure the investment complies with standards for EA, Security, and Acquisition and provides any comments and or questions to the Investment Manager to address, if necessary.

The Technical Review Board, CIO Council, or other technical review organization, reviews the Business Case and Project Management Plan and makes recommendations regarding the technical feasibility of the investment. If approved, the Business Case and Project Management Plan are forwarded to the ITIRB for final review. If not, they are returned to the Business Sponsor and Investment Manager for update.

The Departmental HHS ITIRB makes the final investment decisions. If approved, a Project Manager Accountability Agreement is signed between the Investment Manager and the ITIRB. If not approved the Business Case and Project Management Plan are returned to the Business Sponsor and Investment Manager for update and revision.



B.3 Select Phase – Portfolio Prioritization Process



How do you know you have selected the best investments across the HHS entire IT portfolio?

The HHS/OPDIV CPIC Team, CIO Council/Technical Review Board, and HHS/OPDIV ITIRB develop, refine, and select scoring criteria to compare, score and rank investments across portfolios and establish an Investment Portfolio.

HHS/OPDIV CPIC Team develops the initial prioritized portfolio and presents it to the CIO Council/Technical Review Board and then the HHS/OPDIV ITIRB for review. The portfolio may be adjusted based on these reviews or changing HHS strategic direction, funding issues, quality scores, etc

The HHS/OPDIV ITIRB reviews and approves the prioritized Portfolio, adjusting prioritization, as required

The final prioritized HHS-wide IT portfolio as recommended and approved by the HHS ITIRB is sent to the Secretary’s Budget Council (SBC) for final funding decisions.

The SBC makes the final funding decisions on the IT investment portfolio. Those investments which are funded advance to the Control Phase.



B.4 Control Phase – Periodic and Milestone Review Process



What are you doing to ensure that the investments will deliver the benefits projected?

The Investment Manager maintains current initiative costs, security, schedule, and technical baselines.

The Investment Manager, IPT, and OPDIV Sponsor assess the initiative’s progress against performance measures.

The Investment Manager prepares the periodic (monthly or quarterly) or milestone control reviews, annual investment review submission package, and proposes a Corrective Action Plan or Re-Baseline, if necessary. The Business Sponsor evaluates periodic or milestone control review documents.

The CPIC Team and Critical Partners review the initiative and provide an opportunity for Investment Managers to raise issues concerning the IT developmental process, including security, telecommunications, enterprise architecture alignment, E-Government, GPEA compliance, Section 508 concerns, etc.

The Investment Manager updates the Periodic review documents in the PMT according to feed back by the CPIC Team and Critical Partners.

The OPDIV CIO, or Technical Review Board, reviews the initiative. If the initiative is within cost and schedule, the OPDIV ITIRB recommends an appropriate action to the HHS ITIRB. If the initiative has deviated from its baseline by more than 5%, a Corrective Action Plan is formed and implemented. However, if major business, technology, or strategic changes have occurred, a re-baseline may be suggested.

The Departmental HHS ITIRB makes final investment Control decisions to continue or discontinue operations.



B.5 Evaluate Phase Post Implementation Review (PIR) for New Investments



Based on your evaluation, did the investments deliver what you expected?

An Independent Auditor, along with the Business Sponsor, and Investment Managers, conduct a PIR.

Using the results of the independent audit, the Investment Manager updates the Business Case, if necessary, and presents results to the CPIC Team, Critical Partners, and Technical Review Board.

The CPIC Team and Critical Partners review the PIR results, comparing the actual costs, quality, schedule and technology to the original plan. The reviewers document lessons learned and revise the process, as appropriate.

The Technical Review Board reviews the PIR results and recommends the appropriate action to the ITIRB.

The ITIRB reviews the PIR results and makes the final investment decision.



B.6 Evaluate Phase Annual Evaluation Process for Existing Investments



Do the investments still cost-effectively support requirements?

The Business Sponsor and the Investment Manager determine if the investment is still effective and supports the mission requirements.

The Business Sponsor and Investment Manager assess user and customer satisfaction.

The Investment Manager conducts a technology assessment.

The Business Sponsor and Investment Manager review O&M costs.

The Business Sponsor and Investment Manager update the Business Case.

The CPIC Team and Critical Partners review and approve the updated Business Case.

The Technical Review Board reviews the initiative and recommends an appropriate action to the ITIRB.

The Departmental HHS ITIRB makes final investment decisions.



B.7 Process Improvement

Is the current CPIC process progressing through the stages of ITIM maturity?

The OPDIV CPIC Manager performs an annual ITIM self-assessment.

The OPDIV CPIC Manager provides the analysis to the HHS CPIC Team for inclusion in the Department’s process inventory and also forwards on any process improvements to the HHS CPIC Officer.

The HHS CPIC Officer provides recommendations and supports the OPDIV CPIC Manager to help the OPDIV achieve the next stage.

The OPDIV CIO reviews the changes and recommends an appropriate action to the OPDIV ITIRB. The OPDIV ITIRB sends their recommendation to the HHS ITIRB.

The Departmental HHS ITIRB makes final process change decisions.

In Addition:

At any time, the OPDIV CPIC Team, Investment Manager, or Business Sponsor can recommend a process improvement. These recommendations should be sent to the HHS CPIC Officer.

Quarterly, the HHS CPIC Officer monitors periodic/ quarterly reports and corrective action plans and provides any necessary guidance and direction.

Semi –annually, HHS CPIC Advisory Board meets to conduct process review workshops.

B.8 IT Management & Budget Formulation Calendar

The Select, Control, and Evaluate Phases of the CPIC process are tied to the IT Management and Budget Formulation Calendar, which defines the annual timing for key events needed to generate IT Management Plans and Budgets. The calendar defines the timing for key deliverables such as the IT Strategic Plan, Enterprise Architecture Plan, Security Plan, and IT Budget, among others.Key Activities on the Timeline:

January: HHS submits its IT budget Exhibits 53 and 300 to OMB for inclusion in the President’s Budget (CY+1). HHS CIO completes IT Strategic Planning to inform HHS OPDIVs of Departmental IT priorities.

February: HHS submits President’s Budget to Congress (CY+1). March: Each HHS OPDIV presents a prioritized HHS OPDIV IT investment

portfolio to the HHS CIO for HHS CPIC review and consolidation into an HHS IT investment portfolio (CY+2).

April: HHS issues the Secretary’s budget guidance and HHS OPDIVs begin development of HHS Component budgets.

June: HHS OPDIVs present HHS OPDIV budgets and final OPDIV IT investment portfolios to the DAS for Budget and the HHS CIO, respectively.



June: The HHS CIO Council and ITIRB review HHS OPDIV and HHS Enterprise-wide portfolios and establish a prioritized HHS IT investment portfolio.



July: The Secretary’s Budget Council (SBC) reviews HHS OPDIV budget requests individually, along with the OPDIV’s prioritized IT investment portfolio.

August: HHS OPDIVs revise budget requests and IT investment portfolios in response to SBC decisions.

September: HHS submits the Secretary’s Budget to OMB with Exhibit 53 and 300 forms

October: HHS begins the annual update of IT, EA and Security strategic plans for the next budget year.

November: OMB sends budget passback guidance to HHS and HHS provides its response. The HHS CIO considers passback results in conducting strategic planning for the following budget year.


Appendix C: Sample Prospectus Summary Form

C.1 Purpose

The PMT Prospectus Summary Form is completed during the Select Phase, Pre-Selection Process. The Prospectus Form is a summary document that describes the operational problem and presents the major decision factors that an ITIRB should evaluate in considering the need and proposed investment. The Prospectus Form summarizes the results of the analytic work completed, provides additional cost information, and places it within the PMT for further review.

The following section provides a template for preparing the Prospectus Summary Form located within the HHS PMT. Detailed quantitative and analytical information should be included as attachments as appropriate to the level and complexity of the investment.

C.2 Prospectus Summary Template

General Instructions for Completing the Prospectus Summary Form

The Prospectus Summary Form is created during the Pre-Selection Process and must analytically justify:

(1) the need for action to resolve a shortfall in the Department’s ability to provide the services needed by its users or customers, or

(2) the need to explore a technological opportunity for performing Department missions more effectively.

The mission need must be derived from rigorous mission analysis (e.g., continuous analysis of current and forecasted mission capabilities in relationship to projected demand for services) and must contain sufficient quantitative information to establish and justify the need and decision.

C 3. Sample Prospectus Summary Form – Major/Tactical

The Prospectus Summary Form is shown on the next few pages. It is a blank form as found within the HHS PMT.

Prospectus Summary - Major/Tactical (Form)

INVESTMENT PROSPECTUS - MAJOR/TACTICAL

1 Investment Prospectus

A. Investment Name @

B. What organization will be directly responsible for the planning and execution of the investment?

C. Who is the Project Manager? @

D. Investment Type (Major, Non-Major, Moving an existing investment to a PMA e-Gov or LOB common solution, or Partner Agency contributions to PMA e-Gov or LOB initiative?) @

E. If the investment is a non-major, is it a Tactical or Supporting program investment?

F. Should this investment be reviewed at the OPDIV or Department level? @

G. What does this investment do? Describe the functions this investment will perform. Please limit your response to no more than 100 words.

H. Identify the executive sponsor/process owner who sets the requirements for the investment (i.e., the entity that is the “customer” for the investment). Provide the name, title and contact information. @

I. How does this investment meet the business needs of the executive sponsor/process owner? Discuss in terms of the sponsor’s mission, not the investment’s functionality. Please limit your response to no more than 100 words.

J. What is the proposed life-cycle cost in $M (DME and SS for current year and beyond)

K. What are the benefits to be achieved and/or mandates to be satisfied through this investment. Discuss return on investment.

L. Major Milestones (Description, Start and End Dates, Duration (in Days), Planned Cost (in millions), Funding Agency)

L.1 Proposed Projects

Description of Milestone

Start Date

End Date

Duration (in days)

Planned Cost (millions)

Funding Agency

L.2 Existing Projects


Start Date

End Date

Duration (in days)

Planned Cost (millions)

Funding Agency

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

26

27

28

29

30

2 Cost Summary

Cost Elements FY04

FY05

FY06

FY07

FY08

FY09

FY10

FY11

Total

Planning Costs

1 Program Staff

2 Training & Other Admin Support

3 Equipment

4 Contract Services

5 Software

6 Hardware

7 Facilities

8 Security

9 Other

Sub-Total

Acquisition Costs

1 Program Staff


3 Equipment

4 Contract Services

5 Software

6 Hardware

7 Facilities

8 Security

9 Other

Sub-Total

Maintenance Costs

1 Program Staff


3 Equipment

4 Contract Services

5 Software

6 Hardware

7 Facilities

8 Security

9 Other

Sub-Total

Total

3 Funding Source

A. Funding History (Funding by source by year)

Funding Source FY04 FY05 FY06 FY07 FY08 FY09 FY10 FY11 Total

Enterprise Information Technology Fund

HHS Service and Supply Fund

Other

Total Funding

B. Describe the proposed strategy for funding the costs identified in Section 2 (i.e., the justification for the sources proposed, any commitments requested or received from those sources, and any expected shifts in funding source over the investment’s life cycle). Identify other funding sources listed in the table above.

C. Current Funding Status (funding level and date approved, and obligation status by funding source and Fiscal Year)

Funding Source

Fiscal Year

Approved Funding Level

Approval Date

Obligated

Obligated - Not Invoiced

Invoiced

D. If currently funded, when will this investment run out of approved funding at planned expenditure rates?

E. What funding alternatives are available for this investment should proposed funding sources not be available or provide partial funding?

F. What Government FTE are required to support this investment (in millions) @ (by fiscal year)

FY04 FY05 FY06 FY07 FY08 FY09 FY10 FY11 Total

G. Comments:

4 Budget Impact

A. What effect would approving this investment have on the budget (increase base, base neutral, decrease base through increased efficiencies)? @

B. What would be the impact on the sponsor’s program of not fully funding this investment? @

5 Acquisition Strategy

A. How risky is this investment overall and what are the most significant risk elements? How do you plan to mitigate those risks?

B. Discuss the contracts awarded or to be awarded (date, vendor, contract type, period of performance (base and option periods), current year funding (along with amount of current year funding obligated and invoiced/paid). Discuss how the contracts relate to one another and to the investments cost, schedule and performance goals. Discuss how the acquisition strategy relates to identified investment risks. Identify contract incentives, awards and other provisions designed to allow the contractor to provide innovative, transformational solutions.

C. For other than firm fixed price, performance-based contracts, explain why the Government must assume the risk of contract achievement of cost, schedule and performance goals. Explain the amount of risk the Government will assume.

6 Performance

A. What is the current schedule variance for the investment?

B. What is the current cost variance for the investment?

7 Dependencies

A. Identify existing and proposed investments (Type, name, manager, start and end date, current phase) that this investment will depend upon.

# Type Name Manager Start Date End Date Current Phase

B. Identify existing and proposed investments (type, name, manager, start and end date, current phase) that depend on this investment.

# Type Name Manager Start Date End Date Current Phase

C. Identify non-HHS investments related to this investment (e.g., E-gov initiatives)

D. Has this investment adequately planned and budgeted to meet security requirements? @

8 Supplemental Documentation

A. Attach supplemental information to support the business case for this investment, if applicable.

Name Link Size Owner Uploaded

9 Comments/Recommendations

A. Investment manager comments.

B. CPIC staff comments/recommendations regarding this investment.

PMT INVESTMENT PROSPECTUS FORM GUIDANCEManagers of new investments will provide the data required via the Portfolio Management Tool. Data for existing investments will populate many of the categories on the Prospectus Form. Managers of existing investments should verify the accuracy of the existing data. Note that the Cost information is derived from the PMT Alternative Analysis Form not the PMT Summary of Spending Table. All dollars should be shown in millions using the following format: $2,225,000 = 2.225.

SECTIONS COMMENTSSECTION 1: GENERAL INFORMATIONA. Investment NameB. What organization will be directly responsible for the planning and execution of the investment?

Answered if an organization other than the sponsoring organization will be managing day-to-day project execution.

C. Who is the Project Manager? Provide name, title, telephone number and e-mail address.

D. Investment Type (Major, Non-Major, Moving an existing investment to a PMA e-Gov or LOB common solution, or Partner Agency contributions to PMA e-Gov or LOB initiative?)E. If the investment is a non-major, is it a Tactical or Supporting program investment?F. Should this investment be reviewed at the OPDIV or Department level?

The primary level of ITIRB review for the investment.

G. What does this investment do? Describe the functions this investment will perform. Please limit your response to no more than 100 words.

Explains what the investment does in specific functional terms. Not looking for broad program-level requirements.

H. Identify the executive sponsor/process owner who sets the requirements for the investment, (i.e., the entity that is the “customer” for the investment). Provide the name, title and contact information.

The organization ultimately approving the cost, schedule and performance goals for the investment (essentially certifying business need).

I. How does this investment meet the business needs of the executive sponsor/process owner? Discuss in terms of the sponsor’s mission, not the investment’s functionality. Please limit your response to no more than 100 words.

Refer to alignment with strategic goals and objectives, President’s Management Agenda or Secretarial Initiatives. State the drivers and mandates in program (not technology) terms.

J. What is the proposed life-cycle cost in $M (DME and SS for current year and beyond)

Gives a sense of what HHS is buying into with the current year’s funding.

SECTIONS COMMENTSK. What are the benefits to be achieved and/or mandates to be satisfied through this investment. Discuss return on investment.

Looking here for a more quantifiable discussion of benefits and at least rudimentary cost/benefit analysis. Is this a good financial investment or is it just something we can’t afford not to do for other reasons?

L. Major Milestones (Description, Start and End Dates, Duration (in Days), Planned Cost (in millions), Funding Agency)

Should provide a sense that the sponsor has thought through cost and schedule issues and can present a feasible high level project plan.Two tables are provided. - Proposed investments will use table L.1. - Existing investments will pull data from the Exhibit 300 Section I.H.4 to populate table L.2.

SECTION 2: COST SUMMARYA. Cost breakout by year by cost element for planning, acquisition (DME) and Maintenance (SS)

Shows where money will be spent on the investment. Can be compared with goals and objectives, milestones.

SECTION 3: FUNDING SOURCEA. Funding History (Funding by source by year)

Captures from current year minus two to budget year plus 4 to show: Historical expenditures (even “new” investments may have a history) Current year budget requests Future year funding requirements that will follow from funding the current year.The “Other” row refers to any funds received from sources other than the Enterprise IT Fund or the Service and Supply Fund.

B. Describe the proposed strategy for funding the costs identified in Section 2 (i.e., the justification for the sources proposed, any commitments requested or received from those sources, and any expected shifts in funding source over the investment’s life cycle). Identify other funding sources listed in the table above.

Narrative discussion of how the sponsor plans to fund the costs identified in Section 2 over the life of the investment. State why this is an important project for funding.

C. Current Funding Status (funding level and date approved, and obligation status by funding source and Fiscal Year)

Indicates approved funding from all sources and obligation/commitment rates. Why ask for more money if there is problem spending the money already provided?

D. If currently funded, when will this investment run out of approved funding at planned expenditure rates?

Provide the date. Shows adequacy of project planning and indicates the urgency of funding decisions.

E. What funding alternatives are available for this investment should proposed funding sources not be available or provide partial funding?

SECTIONS COMMENTSF. What Government FTE are required to support this investment ($ in millions) (by fiscal year)

Dollars in millions, not the number of FTE.

G. Comments: Any explanatory text regarding the FTE required.

SECTION 4: BUDGET SUPPORTA. What effect would approving this investment have on the budget (increase base, base neutral, decrease base through increased efficiencies)?

Increases Budget, Base Neutral or Reduce Base.Plus a text box for explanation. The effect of the investment on the base is important, especially for infrastructure and other overhead investments. They won’t necessarily align with high-priority strategic goals, but may reduce overhead cost structures significantly, freeing funds for other uses.

B. What would be the impact on the sponsor’s program of not fully funding this investment?

Low, Medium or High.Plus a text box for explanation. Decision makers need to understand in graphic terms what will happen if funds aren’t provided, including direct effects on programs, ripple effect on other investments/programs, etc.

SECTION 5: ACQUISITION STRATEGYA. How risky is this investment overall and what are the most significant risk elements? How do you plan to mitigate those risks?

Is the investment of High, Medium or Low risk? Describe the risks and the mitigation strategies.

B. Discuss the contracts awarded or to be awarded (date, vendor, contract type, period of performance (base and option periods), current year funding (along with amount of current year funding obligated and invoiced/paid). Discuss how the contracts relate to one another and to the investments cost, schedule and performance goals. Discuss how the acquisition strategy relates to identified investment risks. Identify contract incentives, awards and other provisions designed to allow the contractor to provide innovative, transformational solutions.

Investment manager needs to show a coherent strategy for deploying contracts for the investment. Is the acquisition strategy consistent with the risk portrayed?

C. For other than firm fixed price, performance-based contracts, explain why the Government must assume the risk of contract achievement of cost, schedule and performance goals. Explain the amount of risk the Government will assume.SECTION 6: PERFORMANCEA. What is the current schedule variance for the investment?

Applies to existing investments only. Calculated by the PMT.

SECTIONS COMMENTSB. What is the current cost variance for the investment?

Applies to existing investments only. Calculated by the PMT.

SECTION 7: DEPENDENCIESA. Identify existing and proposed investments (Type, name, manager, start and end date, current phase) that this investment will depend upon.B. Identify existing and proposed investments (type, name, manager, start and end date, current phase) that depend on this investment.C. Identify non-HHS investments related to this investment (e.g., e-gov initiatives)D. Has this investment adequately planned and budgeted to meet security requirements?

A “checklist item” to ensure that security has been properly considered for this investment.

SECTION 8: SUPPLEMENTAL DOCUMENTATIONA. Attach supplemental information to support the business case for this investment, if applicable.

An opportunity to provide PowerPoint slides, copies of mandating documents, etc.

SECTION 9: COMMENTS/RECOMMENDATIONSA. Investment manager comments.B. CPIC staff comments/recommendations regarding this investment.

Provides an opportunity for OCIO staff to provide an assessment of the information provided from a Departmental or OPDIV perspective.

Appendix D: Risk Management

D.1 Purpose

A risk is an uncertain event or condition that, if it occurs, has a positive or negative affect on a project objective. Risk, is one of those words that immediately conjure up an image of something bad, but it is important to remember that risk can also provide positive benefits as well as negative ones.

Risk management is the systematic process of identifying, analyzing and responding to project risk. We want to maximize the probability and impact of any positive risk factors and minimize the probability and impact of those that might negatively affect the project.

The need to manage risk increases with the complexity of the investment. It is an ongoing process that requires continuous risk identification, assessment, planning, and monitoring.

D.2 Process

The Risk Management process includes two phases:

Risk assessment involves identifying, analyzing and prioritizing risks; and

Risk response involves developing and planning risk response strategies, executing those plans, evaluating the results of the responses and documenting the results.

There are several ways that an Investment Manager may choose to manage or respond to a specific risk. These options can be categorized into three broad areas:

Avoid the specific threat, usually by eliminating the cause. (e.g.; conduct a study or develop a prototype)

Mitigate the specific threat by reducing the expected monetary or schedule impact of the risk, or by reducing the probability of its occurrence.

Manage (accept) the consequences of the risk.

Risk management activities need to be balanced. The magnitude of the effort required to identify, assess, manage, and monitor risks, must be commensurate with the magnitude of the potential impact to the project. Making informed decisions by consciously assessing what could go wrong, as well as the likelihood and the severity of the impact is at the heart of risk management.

CPIC Procedures Appendices - Appendix D Page 30 of 67

1. Risk Assessment

It is the responsibility of everyone associated with an investment to identify and document risks. A risk identification process should be identified, communicated and supported.

Table D-1 provides a means by which risk identification can be easily captured, documented, and analyzed.

RiskPriority

Risk Category

Date Identified

RiskDescription

Overall Risk Rating(h-m-l)

Risk Response Strategy

Status

Table D-1 Example of Risk Management Table

Each risk must be:

Described as completely as possible

Identified by phase or stage, along with who identified the risk, the date it was identified, and who was assigned as the primary point of contact

Analyzed for its probability of occurrence (high, medium, low)

Analyzed in terms of impact to the project schedule and budget

Given an overall risk (severity) rating (high, medium, low);

Categorized within the mandatory and optional areas of risk as identified by OMB

Prioritized among all identified risks.

2. Risk Response Development and Control

After all risks have been identified, rated and categorized, each risk is then prioritized. Not all risks identified will be carried into the risk plan for mitigation and management. Project managers should establish a pragmatic cut-off that is consistent with the scope of the project. Each significant risk must then include a description of the risk response strategy and activities. The risks must then be categorized by strategy - eliminate, mitigate, or manage.

The risk management plan provides a means by which risks can be easily tracked and managed. It identifies the priority, area of risk, description, overall rating, risk response strategy category, and status (new, increasing, static, decreasing, or eliminated). The risk management plan will be used to track and communicate risk response activities, their status and their potential impact on the schedule and budget.


3. Common Areas of Risk

The following common areas of risk are consistent with OMB Circular A-11 risk requirements for IT investments. Below are some examples of risks areas that could be addressed in the Risk Management Table for each category.

MANDATORY IT RISK AREAS — at least one risk must be identified, rated and prioritized, and include a risk response strategy in each of the following risk areas.

Technology — Lack of expertise, software and hardware maturity or immaturity, installation requirements, customization, O&M requirements, component delivery schedules availability, uncertain and or changing requirements, design errors and or omissions, technical obsolescence.

Project Schedule and Resources — Scope creep, requirements changes, insufficient or unavailable resources, overly optimistic task durations, unnecessary activities within the schedule, critical deliverables or reviews not planned into the schedule.

Business —Poorly written contracts, market or industry changes, new competitive products become available, creating a monopoly for future procurements.

Organizational and Change Management — Business process re-engineering acceptance by users and management, time and commitment managers will need to spend overseeing the change, lack of participation of business owners in the re-engineering process, necessary change in manuals and handbooks, personnel management issues, labor unions.

Strategic — Project does not tie to Department’s mission or strategic goals, project is not part of the Department’s IT Capital Planning and Investment Control (CPIC) process.

Security — Project does not conform to the requirements of OMB Circular A-130.

Privacy — Project does not conform to the requirements of OMB Circular A-130.

Data — Data standards not defined, data acquisition and or conversion cost are unknown.


Appendix E: Performance Measurement

E.1 Purpose

Performance measurement is the process whereby an organization establishes the parameters to measure how programs, investments, and acquisitions are reaching the desired results in support of mission goals. Performance measures are set during the Select Phase and assessed during subsequent phases. The focus of performance measurement is on outcomes, or how well the IT investment enables the program or Department to accomplish its primary mission. Consequently, performance measurement should look beyond measures of input (resource consumption), activities (milestones), and output (production numbers), which are more directly related to operational performance. This focus, however, does not imply that input, activity, and output measures are not useful. Indeed, internal measures are used to track resources and activities and make necessary adjustments since investments are only successful if hardware, software, and capabilities are delivered on time and meet specifications.

To be useful, performance measure must evaluate the proper activities. It is therefore vital that performance measures be aligned to the goals of any investment to the outcomes specified in the Department’s Strategic Plan, and to the HHS Enterprise Architecture.

Performance is evaluated using two criteria—effectiveness and efficiency. Effectiveness demonstrates that an organization is doing the correct things, while efficiency demonstrates that an organization is doing things optimally. New acquisitions and upgrades should indicate in their Investment Prospectus how the investment will result in effectiveness or efficiency improvements. For example, a new computer network might result in enhanced efficiency because work is processed faster, digital images are transferred among remote sites, or messages are transmitted more securely.

Some questions that facilitate performance measure development include:

What product will be produced, shared, or exchanged?

Who will use the results?

What decisions or actions will result from delivery of products from this system?

Answers to these questions will help Investment Managers develop effective performance measures with the following characteristics:

Strategically relevant

Directed to factors that matter and make a difference

Promote continuous and perpetual improvement

CPIC Procedures Appendices - Appendix E Page 33 of 67

Focus on the customer

Agreed to by stakeholders.

Short, clear, and understandable

Meaningful.

Realistic, appropriate to the organizational level, and capable of being measured.

Valid

o Link to activity and provide a clear relationship between cause and effect

o Focus on managing resources and inputs, not simply costs

o Discarded when utility is lost or when new, more relevant measures are discovered.

OMB has developed a Performance Reference Model (PRM) as part of the Federal Enterprise Architecture. The PRM contains a set of performance measures that can be tailored to fit the specific Departmental needs and a framework for using them. These performance measures are categorized into four broad areas: mission & business results, customer results, processes & activities, and technology. This model should be used to develop specific investment performance measures. The following section describes the business processes for linking together the various elements of performance management described here.

E.2 Process

Developing performance measures is the first step in a broader 6-step performance management process that defines activities through a project or investment’s lifecycle. This process is depicted in Figure 1.


Define Measures

Collect Data

Analyze Results

Take Action

Report Findings

InstitutionalizeSystem


Define Measures

Collect Data

Analyze Results

Take Action

Report Findings



Figure E-1: Strategic Performance Management System

It is important to understand that defining measures is only one part of a more comprehensive process. After measures are developed, baseline information is gathered if it does not already exist, and performance information is collected, analyzed, interpreted, and used throughout the investment’s life. Steps one and two are completed during the Select Phase. Steps two, three, and four are repeated activities during the Control Phase. In the Evaluate Phase findings are reviewed and corrective action, if necessary, is taken (Steps four/five). For the system to be effectively institutionalized, each step requires a commitment of management attention and resources.

Each of these process steps is defined in the following sections.

1. Define Measures

Step 1, Define Measures, can be accomplished using two activities: Analyze How the Investment Supports the Mission and Reduces Performance Gaps; and Develop IT Performance Measures that Characterize Success.

Analyze How the Investment Supports the Mission and Reduces Performance Gaps

Effective outcome-based performance measures are derived from the relationship between the new investment and how users will apply investment outputs. Specifically, the users’ mission and critical success factors (those activities and outputs that must be accomplished if users are to achieve their mission) must be clearly understood. The critical element of this step is linking proposed and in-process IT investments and activities to the user mission and critical success factors.

This concept is often described as a method of strategically aligning programs and support functions with the Department’s mission and strategic priorities. The first step in effectively developing outcome-based IT performance measures is to identify the organization’s mission, the critical tasks necessary to achieve the mission, and the strategies that will be implemented to complete those tasks. One structured method of accomplishing this step is to develop a Logic Model linking the mission to IT performance measures.

Answers to the following questions will aid logic model development:

Identify the system.


What will the system do?

What are major functions or features that the system will provide (for example, what functionality or information)?

Is this system a stand-alone system or is it used or integrated with another system?

What is the purpose of that system?

How is it used?

What aspects of the system, service, and information quality are needed for the system to perform optimally or acceptably?

Identify who will use the system. What is the principal business task they perform? How will using the system help them with that task?

How does completion of that task contribute to a business function?

How does completion of the business function contribute to achievement of program goals?

How does completion of program goals contribute to organizational goals?

How does completion of organizational goals contribute to Departmental goals?

Determine whether there are related IT investments that impact the mission area and goal(s) selected. Understand the relationships between various IT investments that address the same or similar needs. This will help identify potential areas for consolidation.

Once the mission is clearly defined, a gap analysis is performed to understand how IT can improve mission performance. The analysis begins with the premise that IT will improve effectiveness, efficiency, or both. To accomplish this, requirements are defined and the following questions are answered:

Why is this application needed?

How will the added functionality help users accomplish the mission?

How will the added functionality improve day-to-day operations and resource use?

Once this information is defined, it can be used to prepare the Investment Prospectus, specifically section 1, questions G, H, I, K justifying the investment and section 7 describing project dependencies.

Develop IT Performance Measures that Characterize Success

Well-designed performance measures define success parameters for the IT initiative. The following questions should be asked for each performance measure and answered affirmatively before deploying the measure:


Is it useful for monitoring progress and evaluating the degree of success?

Is it focused on outcomes that stakeholders will clearly understand and appreciate?

Is it practical? Does it help build a reliable baseline and cost-effectively collect performance data at periodic intervals?

Can the performance measure be used to determine the level of investment risk and whether the investment will meet performance targets?

Answering these questions affirmatively results in an agreement that the IT investment, by supporting improvements identified earlier, will support organizational goals and objectives. Additionally, it will help limit the number of performance measures and focus management attention on the requirements that have the greatest priority or impact. After three to five major requirements have been identified, the following questions are asked:

What are the performance indicators for each major requirement?

How well will those outputs satisfy the major requirements?

What additional steps must be taken to ensure outputs produce intended outcomes?

How does this IT investment improve capabilities over the current method?

Once requirements to be measured are identified, determine when each requirement is met. Some requirements may need to be changed if they are too difficult to measure. Or, if the requirement has indirect rather than direct outcomes, it may be necessary to use “surrogate” performance measures that mirror actual outcomes. For example, it is difficult to measure the direct benefit of computer-based training (CBT) systems. In this case, a surrogate measure might be the percentage of staff achieving certifications through the CBT with implications that certified staff are more desirable than non-certified staff because they have demonstrated initiative and are more proficient.

Of the possible performance indicators, select one or more to report performance against each requirement. One performance indicator may provide information about more than one requirement. The objective is to select the fewest number of performance indicators that will provide adequate and complete information about progress.

Selecting the fewest performance indicators necessary is important because data collection and analysis can be costly. The cost is acceptable if the benefit of the information received is greater than the cost of performance measurement, and if the data collection does not hinder accomplishment of primary missions. Costs are calculated by adding the dollars and staff time and effort required to collect and analyze data. When calculating costs, you must consider whether they are largely confined to initial or up-front costs, or will occur throughout the IT lifecycle. For example,


the cost of developing and populating a database may have a large initial cost impact but diminish significantly for later maintenance. Answers to the following questions will help to determine the cost of tracking a specific performance indicator:

What data are required to calculate the performance measure?

Who collects the data and when?

What is the verification and validation strategy for the data collection?

What is the method to ensure the quality of the information reported?

In addition to determining costs, it is also necessary to determine the baseline performance, target performance, and expected time to reach the target. The baseline value is the start point for future change. If performance measures are currently in use, the data collected can provide the baseline. Otherwise the manager must determine the baseline by a reasonable analysis method including the following:

Benchmarks from other agencies and private organizations

Initial requirements

Internal historical data from existing systems

Imposed standards and requirements.

To determine the target value, obtain stakeholder agreement regarding the quantifiable benefits of the new system. These targets may be plotted as a function over time, especially for IT investments that are being installed or upgraded or as environmental factors change.

2. Develop Collection Plan and Collect Data

To ensure performance data is collected in a consistent, efficient, and effective manner, it is useful to develop and publish a collection plan so all participants know their responsibilities and can see their contributions. The collection plan details the following items:

Activities to be performed

Resources to be consumed

Target completion and report presentation dates

Decision authorities

Individuals responsible for data collection.

In addition, the collection plan answers the following questions for each performance measure:

How is the measurement taken?

What constraints apply?

Who will measure the performance?


When and how often are the measurements taken?

Where are the results sent and stored, and who maintains results?

What is the cost of data collection?

While costs should have been considered during the previous step, the actual cost will be more evident at this stage. Excessively costly performance measures may require investment managers to find a different, less costly mix of performance measures for the IT investment. Or it may be necessary to creatively collect the measures to reduce collection cost. For example, a sampling may produce sufficiently accurate results at significantly less cost than counting every occurrence, and some results can be automatically generated by the system and accessed through a standard report.

To ensure data is being collected in a cost-effective and efficient manner, it is important to ensure the data collectors are involved in developing performance measures. The collectors will do a much better job if they believe the performance measures are valid and useful, and they will have insight regarding the best way to collect the data.

4. Analyze Results

Performance measures are useful in monitoring the investment against expected benefits and costs. To evaluate performance, data is compiled and reported according to the collection plan that was previously constructed. The data is then evaluated and the following questions are answered regarding the collected data and the investment’s performance:

Did the investment exceed or fall short of expectations? By how much and why?

If the data indicates targets are successfully reached or exceeded, does that match other situational perceptions?

What were the unexpected benefits or negative impacts to the mission?

What adjustments can and should be made to the measures, data, or baseline?

What actions or changes would improve performance?

This evaluation reveals any needed adjustments to the IT investment or performance measures. It also helps surface any lessons learned that could be fed back to the investment management process.

A key part of the activity in this step is to analyze the continuing relevance of the measures themselves.

Performance measures provide feedback to managers and help them make informed decisions on future actions. To ensure that performance measures are still relevant and useful, answer the following questions:

Are the measures still valid?


Have higher-level mission or IT investment goals, objectives, and critical success factors changed

Are threshold and target levels appropriate in light of recent performance and changes in technology and requirements?

Can success be defined by these performance measures?

Can improvements in mission or operations efficiency be defined by the measures?

Have more relevant measures been discovered?

Are the measures addressing the right things?

Are improvements in performance of mission, goals, and objectives addressed?

Are all objectives covered by at least one measure?

Do the measures address value-added contributions made by overall investment in IT and or individual programs or applications?

Do the measures capture non-IT benefits and customer requirements?

Are costs, benefits, savings, risks, or ROI addressed?

Do the measures emphasize the critical aspects of the business?

Are measures linked to a specific and critical organizational process?

Are the measures the right ones to use?

Are measures targeted to a clear outcome (results rather than inputs or outputs)?

Are measures understood at all levels that must evaluate and use them?

Do the measures support effective management decisions and communicate achievements to internal and external stakeholders

Are measures accurate, reliable, valid, and verifiable?

Are measures built on available data at reasonable costs and in an appropriate and timely manner for the purpose?

Are measures able to show interim progress?

Are measures used in the right way?

Are measures used in strategic planning (e.g., to identify baselines, gaps, goals, and strategic priorities) or to guide prioritization of program initiatives?

Are measures used in resource allocation decisions and task, cost, and personnel management?


Are measures used to communicate results to stakeholders?

5. Report Findings

In the Report Findings step the main goal is to communicate progress on the strategic goals and objectives (i.e. IT outcomes) and status of IT projects (i.e. IT outputs). Managers close to the day-to-day administration of a project are typically interested in both outputs and outcomes. Those farther removed from the project are interested in the project’s outcomes; for “major” IT projects oversight bodies are interested in both the outcomes and the impacts of projects, and whether these projects are on schedule and within budget.

Reports on progress are made via Earned Value analysis for projects in Acquisition or Mixed Lifecycle phases and Operational Analysis and Post Implementation Reviews are used for those projects transitioning to or in Steady State.

6. Take Action

For the performance measures embedded in a performance management system, the intrinsic purpose is to guide management decisions and actions. The take action step leverages the preceding steps by putting the performance information into action. During this step, management will rely on reports to gain insight into the investment project’s performance. This information is used to make sound decisions about performance issues and to re-align strategies for the future.

If target goals are not being met, management must take corrective actions to improve performance. The approach should be as follows:

Determine range of corrective actions

Prioritize corrective actions based on overall risk mitigation strategies and goals

Select most appropriate corrective action(s)

Analyze the root cause of performance deviations to determine the corrective actions that might be applied. Corrective actions might include: providing training for staff; establishing new processes and procedures; changing or realigning business strategies; transitioning funding between IT initiatives; or establishing new IT initiatives.

There may be several corrective actions applicable to a single performance issue. However, some may be inappropriate to the scale of the problem, or be too costly, or not be feasible for other resource reasons. Applicable corrective actions should be prioritized in the ascending order of cost and impact.


Corrective action selection is dependent upon the magnitude of the problem. In the case of initiative-specific progress or performance issues, the program manager must interface with HHS management to develop a solution. In the event that broader gaps are identified in achieving the IT strategies, management should realign its business focus through the CPIC process to ensure that the IT and other asset portfolios are comprehensive, complete and aligned to Department mission and goals.


Appendix F: Project Management

F.1 Purpose

Project Management is a crucial element for IT investment success. It involves executing the necessary skills and management practices to ensure successful investment development and implementation. This integrated skill set addresses such areas as project planning, scope management, cost, schedule, performance, risk, and organizational management. Project Management is carried out by a Project Manager, who is the single accountable individual provided sufficient authority and responsibility to accomplish project objectives for development, production, and sustainment. The Project Manager (PM) manages the assigned project consistent with applicable statutes, regulations, and directives. Accountability is ensured through cost, schedule, and performance reporting.

A Project Manager responsible for the management of an investment is also referred to as an Investment Manager; however, Project managers may also manage projects that are sub-components of an investment. For the purpose of this section, project management refers to the management of an IT investment or a sub-component of an IT investment that requires the use of project management principles.

The Project Manager is ultimately responsible for the investment, or project’s, success and ensuring the investment, or project, delivers the functionality and capabilities expected by stakeholders (e.g., users, customers, and senior leaders.) Perhaps the greatest project management challenge is the development of a realistic risk-adjusted plan that can be executed successfully in scope, on schedule and within budget.

HHS’s project management (PM) practice and competency policies are based on the ANSI/PMI/99-001-2004 standard Project Management body of Knowledge (PMBOK®) and the Office of Management and Budget (OMB)-Federal Chief Information Officers Councils (OCIO) guidance on information technology (IT) project management. The PMBOK describes and recognizes nine management areas that are generally accepted as PM professional practices. Listed below are the nine management (knowledge) areas:

1. Integration Management

2. Scope Management

3. Time Management

4. Cost Management

5. Quality Management

6. Human Resource Management

CPIC Procedures Appendices - Appendix F Page 43 of 67

7. Communication Management

8. Risk Management

9. Procurement Management

When optimized, the CPIC and Project Management processes come together as described in the following section titled; Relationship of Project Management to Investment Management.

F.1 Relationship of Project Management to Investment Management

Purpose: Establish Business Need and Mission Alignment

• Identify Alternatives• Calculate Financial Metrics• Develop Performance Goals

ScreeningPurpose: Establish Business Need and Mission Alignment

• Identify Alternatives• Calculate Financial Metrics• Develop Performance Goals

Screening

Project Management Artifacts and HHS’s Investment Life-Cycle Process

Initiate• Prospectus

Summary Form(PMT)

Plan

• PMT Investment Data• Project Plan• PM Accountability

Agreement

HH

S C

PIC

Pro

cess

es

Pro

ject

Sys

tem

Dev

elo

pm

ent

Man

agem

ent

Lif

ecyc

le S

tep

s

Step 0ConceptDefinition

Step 1Concept

Development

Step 2System Designand Prototype

Step 5System

Operation

Execute

• Updated Project Plans• Change Requests• Status Reports

Close• PIR• Project Closeout

Report

Control

• Updated Project Plans• Change Requests• Status Reports• Milestone Review

Reports • PIR Schedule

Step 4System

Deployment

Step 3DevelopmentAnd Testing

Select Control Evaluate

Purpose: Monitor Cost, Schedule, Performance

• Review Earned Value Information• Develop Corrective Action Plan, if

necessary

Periodic ReviewsPurpose: Monitor Cost, Schedule, Performance

• Review Earned Value Information• Develop Corrective Action Plan, if

necessary

Periodic ReviewsPurpose: Ensure alignment with mission and IT portfolio priorities

• Perform strategic evaluation• Re-baseline, if necessary

Milestone ReviewsPurpose: Ensure alignment with mission and IT portfolio priorities

• Perform strategic evaluation• Re-baseline, if necessary

Milestone Reviews

Purpose: Justify Investment

• Develop Business Case• Develop Project Management Plan

Pre-SelectionPurpose: Justify Investment

• Develop Business Case• Develop Project Management Plan

Pre-Selection

Purpose: Select and Fund Best Investments

• Compare investments across portfolios

• Create a portfolio of IT investments

Portfolio PrioritizationPurpose: Select and Fund Best Investments

• Compare investments across portfolios

• Create a portfolio of IT investments

Portfolio Prioritization

Purpose: Identify Lessons Learned to improve development and CPIC process

• Perform Independent Audit

Post Implementation Review

Purpose: Identify Lessons Learned to improve development and CPIC process

• Perform Independent Audit

Post Implementation Review

Purpose: Identify emerging gaps in functionality, performance, and opportunities for collaboration.

• Perform Operational Analysis• Perform E-gov Strategy Review• Modify Investment, if necessary

Annual EvaluationPurpose: Identify emerging gaps in functionality, performance, and opportunities for collaboration.

• Perform Operational Analysis• Perform E-gov Strategy Review• Modify Investment, if necessary

Annual Evaluation

Diagram F. 1 Project Management & CPIC

F.2 Components

Management (business sponsors and investment managers) should complete the following project management components to help ensure the investment’s successful completion:


Project Plan:

A Project Plan is a document that describes the technical and management approach to carrying out a defined scope of work, including the project organization, resources, methods, and procedures and the project schedule. This document, and event, should follow PMBOK Guidance and should include the nine project management areas sub-plans that include their respective execution controls and change control management (CCM) procedures. The nine sub-plans include:

1. Integration Management Plan, including integrated change control

2. Scope Management Plan, including the scope definition

3. Time Management Plan, including the project schedule baseline

4. Cost Management Plan, including the cost baseline

5. Quality Management Plan, including the quality monitoring and control

6. Human Resource Management Plan, including responsibility matrix

7. Communication Management Plan, including performance reporting

8. Risk Management Plan, including the risk adjusted plan

9. Procurement Management Plan, including solicitation, selection and contract administration

The Investment Manager is responsible for developing a project plan. The project plan should meet the following requirements:

The planning process should be a defined part of the CPIC/PM process sequence, (see diagram E.1 in previous section)

The project plan’s acceptance should be under the OPDIV or HHS ITIRB’s authority and OPDIV or HHS ITIRB responsibility, as appropriate.

The project plan’s review of adherence to standards should be by the project review process within the CPIC process, organized and supported by the HHS and OPDIV OCIOs.

The project plan should be an acceptance prerequisite to the OMB’s developmental phase start (see diagram E.1 in previous section)

Project plan changes, corrective action plans, and re-baselining, if applicable, should conform to HHS’s CPIC policies and guidance.

Project plan changes, corrective action plans, and re-baselining, if applicable, should result in changes (updates) to the Business Case (see diagram E.1 in previous section.)


Project Plan - Work Breakdown Structure (WBS)

Project planning may occur at the Investment level or for the sub-components of investments, which may, of themselves, comprise projects. For purposes of this section, project refers to an investment or a sub-component of an investment that requires the discipline of project management.

Investments typically involve multiple components that may be complex or interface with other proposed or existing systems or data. Integrating these components is very challenging. To support improved integration and management, it is useful to develop a Work Breakdown Structure for project planning. A Work Breakdown Structure (WBS) is an organized method to break down a project into logical subdivisions or subprojects at lower and lower levels of details. It is very useful in organizing a project.

A WBS provides a management framework by separating the investment lifecycle into distinct, manageable components related to various phases or stages of activities and interfaces. Each component is defined with appropriate sub-components and activities, such that one individual or team can implement each component. This enables the Project Manager to more effectively estimate the cost and schedule for completing individual components. It also supports sequencing activities and identifying interdependencies, and it provides a basis to identify milestones and tasks as part of the investment’s or project’s specific WBS.

Table F-1—WBS Example

Plan Project

100 Define Project

10 Determine Project Objectives

20 Define Project Scope

30 List Project Products

40 Determine Project Constraints

50 Select Project Approach

60 Determine Project Standards

70 Assess Project Risks

200 Make Project Plan

10 Define Work Breakdown Structure for the investment or project

20 Determine Activity Dependencies

30 Define Project Milestones

40 Determine Project Organization

50 Estimate Effort

60 Allocate Resources

70 Schedule Activities

80 Develop Budget


90 Assess Project Risks

300 Obtain Project Approval

10 Assemble Project Plan

20 Present Project Plan

30 Agree to Project Plan

Table F-1 Example of a Project Planning WBS Activities during the Select Phase

Scope Management—The scope frames what is expected of the investment’s ultimate capability and functionality, it directly impacts functional and system requirements development. The Investment Manager should obtain the Business Sponsor’s concurrence on the investment’s scope, and then effectively manage that scope and mitigate scope creep. This can be accomplished by maintaining requirements traceability throughout the project lifecycle and implementing configuration management procedures. It is important for the Business Sponsor to determine whether existing requirements have been redefined, new requirements have been identified, or existing requirements eliminated based upon events.

The project scope should be based on the business requirements identified during the Pre-Select Phase and traced throughout the project lifecycle. All system features, functions, and capabilities should be linked to original customer requirements throughout the entire planning, acquisition, design and implementation phases to ensure accurate system or network design.

Risk—Risk is inherent in every investment. To aid in effectively identifying, analyzing, developing responses, and managing risk, Investment Managers should develop a risk management plan early in the planning stages, ideally during the Select Phase. Investment Managers should employ subject matter experts (SMEs) among the various functional areas of the investment to identify risk and provide mitigation strategy. Key risk areas may include technology, cost, schedule, and performance or quality. The risk management plan is continually updated throughout the investment’s lifecycle and is part of periodic reviews.

Appendix D: Risk Management provides additional guidance on risk assessment and management.).

Cost and Schedule Management—Effective investment management entails establishing cost and schedule baselines. Actual information is collected, analyzed, and compared to original projections and the current baseline. Variances are identified, and appropriate actions are taken to inform senior management and mitigate the impacts of increased costs and schedule slippages. The WBS, milestones, activities, and project plan assist the development and tracking of cost and schedule. Earned value techniques provide a means to more completely evaluate costs and schedule, and assist in early risk identification.


Performance—An investment’s ultimate objective is to meet or exceed stakeholder performance expectations by ensuring the investment satisfies the mission need and business requirements. In the Pre-Select and Select Phases, performance planning includes defining performance measures and identifying activities required to ensure performance objectives will be met (see Appendix E: Performance Measurement). This may include benchmarking to establish a baseline and to further refine the investment’s performance objectives. The Control Phase includes a continual monitoring of the performance baseline to potentially include quality reviews, tests, or pilot tests. In the Evaluate Phase, a PIR helps compare actual investment performance with expectations, performance measures are analyzed to determine whether investments are continuing to meet mission needs and performance expectations.

Organizational Management—Organizational management skills needed to manage an investment include project staffing, communications, and organizational understanding. Investment Managers should be able to identify the needed skill sets and assign appropriate personnel to accomplish a given set of activities. Investment Managers should also have the requisite interpersonal and leadership skills to communicate with the project team, Business Sponsor, and stakeholders. This includes possessing a vision for the investment and how to best meet stakeholder expectations, as well as ensuring the project team is able to focus on assigned tasks or activities. Additionally, Investment Managers should be able to communicate and build consensus with key stakeholders, since this ultimately impacts the investment’s success or failure.


Appendix G: Earned Value Management

G.1 Purpose

Earned Value Management (EVM) is a program management technique developed at the Department of Defense that uses an investment’s past performance and work to evaluate and forecast the investment’s future performance. This enables the Investment Manager to make changes that keep the investment at or bring the investment closer to planned expectations.

HHS requires EVM for all IT investments. The requirement is established in the HHS Earned Value Management Policy. Earned value analysis is part of a performance based management system required by OMB for all IT investments. Earned Value analysis is built into the business case template. Earned value management may be applied at the Investment level or applied to sub-components of the investment that require the application of project management principles, included earned value management principles. For purposes of this section, IT investment refers to both a single IT investment and to sub-components of investments.

The Investment Manager plans work breakdown structure (WBS) tasks and builds budget estimates for each task in the project plan. As the plan is executed, the Investment Manager tracks actual progress and expenditures at the completion of each WBS against planned figures to obtain cost and schedule variances. These variances can then be used to identify schedule and cost-over or under-runs so they can be resolved as quickly as possible. For the purpose of this section, Investment Manager applies to the project manager responsible for managing an investment and to project managers of sub-component of investments.

The earned value methodology requires an investment to be fully defined at the outset. The information that is required to complete an earned value analysis includes:

List of all WBS tasks and critical milestones

Planned cost of each WBS task

Planned WBS start and completion dates

Total budget for the investment

Any project reserve.

As the project plan is executed, the Investment Manager tracks:

Work (WBS tasks) completed

Value of the completed work

The actual cost of the work performed.

Earned Value analysis is based on the budgeted cost of work performed (BCWP), the budgeted cost work scheduled (BCWS) and the actual cost of work performed (ACWP). These three

CPIC Procedures Appendices - Appendix G Page 49 of 67

parameters provide the Investment Manager, Business Sponsor, and others with all the input data required to assess project cost, scope, schedule, and performance.

The approach can provide accurate and reliable assessments from as early as 15 percent into the investment’s lifecycle. It provides early indications of cost and schedule variances, which help investment managers, take appropriate risk mitigation steps. Typically, investments that are over-budget and cost variance percentage, when 15 percent of the investment is finished, will result in cost overruns. Once a cost overrun is identified, it can generally be reduced by only 10 percent, which indicates the need to support early awareness of potential cost and schedule risks. Early investment assessment and identification of cost and schedule variances is critical for the overall success of the investment, and supports improved cost and schedule control.

When calculating Earned Value, the following perspective should be considered:

Government Activities

Second and Third Party Agreements; contacts, inter-organization agreements.

Components of Government expenditures include:

Labor (like staff time)

Equipment

Materials

Facilities (like office space)

Activities (like travel)

Second and Third Party agreements are based on (contract) terms including:

A “firm-fixed” contract, tracking specified deliverables expenditures by planned verse actual delivery schedules (dates).

A “time and materials” contract, tracking expenditures (as with the government):

o Labor

o Equipment

o Materials

o Facilities

o Activities

G.2 Process

Before completing earned value analysis, the Investment Manager needs to complete the following project management tasks:


Define investment activities

Develop a project plan for the activities

Develop a WBS, including costs and schedule estimates, for each activity

Chart and evaluate the investment’s status.

The Investment Manager will then have the basis for periodically assessing the investment’s performance and completing the following four steps in the earned value analysis process:

1. Update the Schedule

The scheduled activities are reported as started, completed, or with a remaining duration as appropriate. For unfinished activities, the percent complete is reported. For work that results in discrete or concrete deliverable products (e.g., reports, studies, briefings, etc.), it generally is easy to determine the percent complete. For efforts that are not so easily measured, special “earning rules” may be employed. A common “earning rule” is to report percent complete according to completed milestones within an activity. There also many other Earned Value Management techniques for accessing completed work (e.g., 50-50 rule)

2. Record Actual Costs

After updating the schedule, actual costs from the investment’s or organization’s accounting system are recorded. In situations where the accounting system does not provide the level of detail required to obtain actual accounting costs, the Investment Manager may need to estimate what percentage of actual costs should be assigned to the investment, but this should only be done when actual costs are not readily available.

3. Calculate Earned Value Measures

After recording actual costs for the reporting period, earned value measures are calculated and reports generated. This can be done, in part, by creating an earned value chart as shown below.

Figure H-1—Sample Earned Value Analysis Chart (This can be accomplished using a standard project management or spreadsheet software’s charting functionality or through an EVM tool.)


Figure G-1. Sample Earned Value Analysis Chart

The sample chart includes the following earned value measures:

Actual Cost of Work Performed (ACWP)—The sum of costs actually incurred and recorded in accomplishing the work performed through the data date.

Budget at Completion (BAC)—The sum of all planned budgets established for the investment.

Budgeted Cost for Work Performed (BCWP)—The sum of the budgets for completed work packages and completed portions of open work packages, plus the applicable portion, usually a percentage, of the budgets for level of effort and apportioned effort as of the data date, this is also called the “earned value.”

Budgeted Cost of Work Scheduled (BCWS)—The sum of all WBS element budgets that were planned or scheduled for completion as of the data date.

Contract Budget Base (CBB)—The total cost of all budgeted activities necessary to complete a task.

Cost Performance Index (CPI)—Earned value divided by the actual cost (BCWP divided by ACWP).

Cost Variance (CV)—Earned value minus the actual cost of work performed (BCWP minus ACWP).

Cost Variance Percentage (CV percentage)—Cost variance divided by earned value (CV divided by BCWP).


Estimate at Completion (EAC)—The actual costs incurred, plus the estimated costs for completing the remaining work (BAC divided by CPI).

Estimate to Complete (ETC)—The budget necessary to complete all tasks from the ACWP end date through the investment’s conclusion (EAC minus ACWP).

Management Reserve (MR)—The amount of the total allocated budget withheld for management control purposes rather than designated for the accomplishment of a specific task or set of tasks; not part of the performance measurement.

Performance Measurement Baseline (PMB)—The time-phased budget plan against which investment performance is measured.

Schedule Performance Index (SPI)—Earned value divided by the planned budget for the completed work (BCWP divided by BCWS).

Schedule Variance (SV)—Earned value minus the planned budget for the completed work (BCWP minus BCWS).

Schedule Variance Percentage (SV percentage)—Scheduled variance divided by the planned budget for the completed work (CV divided by BCWS).

Variance at Completion (VAC)—The difference between the total budget assigned to a contract, WBS element, organizational entity, or cost account and the estimate at completion; represents the amount of expected overrun or under run.

4. Analyze the Data and Report Results

The critical path milestones used to complete the earned value analysis are directly derived from the project plan. These are the milestones that require completion before a successive milestone can begin. The data is collected and monitored for each milestone throughout the project to achieve maximum effectiveness.


Appendix H: Post-Implementation Reviews

H.1 Purpose

Post-Implementation Reviews (PIRs) support the Evaluation Phase of the process (see Chapter 7—Evaluate Phase Post Implementation Reviews). PIRs help determine whether investments have achieved expected benefits, such as lowered cost, reduced cycle time, increased quality, or increased speed of service delivery.

The PIR has a dual focus:

It provides an assessment of the implemented investment, including an evaluation of the development process.

It indicates the extent to which the HHS’s decision-making processes are sustaining or improving the success rate of IT investments.

The PIR usually occurs either after a system has been in operation for about six months or immediately following investment termination.

In order to ensure independent validation and verification, a team of independent reviewers or other selected individuals (as appropriate) should conduct the PIR. The team should not include members from the investment under review. The PIR team should review the following investment elements:

Mission alignment

IT architecture including security and internal controls

Performance measures

Project management

Customer acceptance

Business process support

Cost versus anticipated savings.

As a minimum, the PIR team will evaluate stakeholder and customer or user satisfaction with the end product, mission or program impact, and technical capability, as well as provide decision-makers with lessons learned so they can improve investment decision-making processes.

The review will provide a baseline to decide whether to continue the system without adjustment, to modify the system to improve performance or, if necessary, to consider alternatives to the implemented system. Even with the best system development process, it is quite possible that a new system will have problems or even major flaws that must be rectified to obtain full investment benefits. The PIR should provide decision-makers with useful information on how best to modify a system, or to work around the

CPIC Procedures Appendices - Appendix H Page 54 of 67

flaws in a system, to improve performance and bring the system further in alignment with the identified business needs.

H.2 Process

There are seven major steps to conducting a PIR:

1. Initiate PIR

The OCIO and the investment sponsor select the PIR review team. The review team initiates a PIR by preparing and sending a memorandum to the Investment Manager stating the review has begun. The memorandum should include a schedule for the planned review and indicate any areas that may receive special review emphasis.

2. Analyze Documentation

The review team reviews existing investment documentation and analyzes the information to understand the investment scope, generate interview and survey questions, prepare for system overview briefings, and plan the PIR. The review team also reviews any existing reports and memoranda from the Pre-Select, Select, and Control Phases to uncover any findings or outstanding issues.

3. Interview Key Players

The review team interviews all key IT and business process players. The interviews should help the team develop an understanding of the system’s goals, objectives, benefits, and costs as described in the Exhibit 300 or 300-1 business case submitted during the Select Phase. Additionally, the interviews will help the team determine how efficiently and effectively the system’s objectives, goals, performance measures, and benefits are being achieved, as well as identify system deficiencies and enhancement needs.

4. Measure Performance

The review team assesses the investment performance measures established during the Select Phase. These performance measures are compared to actual data generated during the operations and or production stage. In the absence of certain statistics, the review team may perform onsite observations to measure specific criteria.

5. Perform User Surveys

The review team conducts qualitative surveys with users to determine user satisfaction with the system. Executing the survey may include designing questionnaires, distributing survey questionnaires to remote users’ locations, receiving responses, analyzing results, and generating a survey results memorandum. The survey measures the system’s efficiency and effectiveness in achieving its stated goals and benefits and in satisfying user needs.


If users are external to HHS we need approval from OMB prior to sending them a questionnaire.

6. Perform Analysis

The review team analyzes all documentation, survey results, and performance measurements to determine if the system efficiently and effectively achieved its objectives.

7. Findings and Recommendation Report

After comments are received from the Business Sponsor, the review team prepares the Final Report and submits it for the OPDIV CIO and OPDIV ITIRB review. Report findings and recommendations must be clear and concise to avoid any misunderstandings.

8. Final Decision

The CIO, investment manager and Department sponsor determine the appropriate course of action to resolve any outstanding issues. Decisions will also be made whether to continue the system without adjustment, modify, or terminate, based on the PIR recommendations.

H.3 Sample Initiative Evaluation Sheet

SAMPLE INITIATIVE EVALUATION SHEET

General information

Title:

Description:

Business Sponsor:

UPI:

PIR Conducted By:

Date of PIR:

Performance Measures

Item Baseline Actual Variance Comments

Quantitative

Financial

Non-Financial

Baseline Status

Item Baseline Actual Variance Comments

Lifecycle Cost

Lifecycle Return

Schedule

Enterprise Architectural Analysis

Enterprise Architectural Assessment

IT Accessibility Analysis

IT Accessibility Assessment

Telecommunications Analysis

Telecommunications Assessment


Risk Analysis

Risk Management

Security Analysis

System security risk management or mitigation review. Additional mitigation strategies and counter measures (if needed).

Stakeholder Assessment

General Comments

Lessons Learned

Project Management Assessment

Technical Assessment

Table H-1 IT Initiative Evaluation Data Sheet

H.4 Investment Management Report

INVESTMENT MANAGEMENT REPORT

Name of Investment:

Business Sponsor:

Date of PIR:

Background (Description of Project)

Management Approach

Organizational Structure

Resources

Acquisition Strategy

Contracting Strategy

Security Strategy

Documentation

Technical Approach

Architecture (description, adherence to ISTA, and IT accessibility requirements, security, telecommunications, and architecture standards)

Development (if applicable)

Testing

Lessons Learned

List of lessons learned

Recommended best practices

Table H-2 IT Investment Management Report Data Sheet


H.5 IT Initiative Evaluation Data Sheet

Initiative Development

Screen ScorePre-

SelectSelect Control Evaluate

Steady State

Was each phase conducted at the appropriate time in the process?

Was the data content sufficient to move forward to the next phase in the process?

Were there enough resources (e.g., people) allocated for each phase in the process?

Were the right types of people and expertise involved?

Was there an acceptable level of information flow?

Was HHS PMT able to support the activity in each phase in the process?

List suggested corrective actions for any phase in the process.

Comments:

Table H-3 IT Initiative Evaluation Data Sheet


Appendix I: Operational Analysis

I.1 Purpose

This document provides guidance for conducting an Operational Analysis review for Information Technology (IT) investments. OMB requires that all Steady State projects must be reviewed at least annually to document the continued effectiveness in supporting mission requirements and minimize the cost of asset ownership. ( The cost of asset ownership is defined as the total of all costs incurred by the owners and users to obtain the benefits of a given project or investment.) The intent, in part, is to reduce the number of stove-piped legacy systems that are expensive to maintain. Operational Analysis results are reported to OMB each year in the Exhibit 300's Project (Investment) and Funding Plan section. A investment manager may choose to perform an Operational Analysis more frequently.

The annual Operational Analysis is a key practice within the Government Accountability Office's (GAO) Information Technology Investment Management (ITIM) Stage 2 maturity model.

Using verifiable data, each investment board must regularly review the performance of IT projects and systems against stated expectations. Investment boards use of the Steady State project's Operational Analysis support ITIM Stage 2.

I.2 Management Objectives

Ownership costs such as: operations, maintenance, service contracts, and disposition, can easily consume as much as 80% of the total life-cycle costs. Operations are a critical area where improved effectiveness and productivity can have the greatest net measurable benefit in cost, performance, and mission accomplishment.

The Operational Analysis formally assesses how well an investment is meeting program objectives, customer needs, and is performing within baseline cost, schedule, and performance goals. The results may signal to management the need to redesign an asset if undetected faults in the design, construction, or installation are discovered during the course of operations. Two examples may be: if Operation and Maintenance (O&M) costs are higher than anticipated; or, if the asset fails to meet program requirements.

I.3 Roles and Responsibilities

Business Sponsor or System Owner: Coordinates with the Investment Manager to schedule the Operational Analysis and provides guidance to the Investment Manager.

CPIC Procedures Appendices - Appendix I Page 59 of 67

Investment Manager: Prepares the Operational Analysis.

OCIO: Receives and analyzes the Operational Analysis report.

IT Investment Review Board: Reviews the Operational Analysis report and makes recommendations for disposition of the investment.

I.4 Process

The Business Sponsor must establish a schedule (annually, semi-annual or quarterly) to conduct an operational analysis. The Business Sponsor must establish a strategy to solicit user or customer input. This strategy can be a survey, focus groups or regular user group meetings. The Business Sponsor must document the schedule and strategy, and notify all users or customers of this formal and regular schedule.

Based on projected project or investment costs and benefits (e.g., cost, schedule and performance), the survey, focus group or regular user group results will determine whether the Steady State project is meeting its original or revised objectives. The results are documented in the Operational Analysis template.

Enhancements outside of the existing project scope are considered a new investment. To fund an enhancement, the Business Sponsor must enter the Pre-Select CPIC processes where alternatives to close the gap are identified, and their costs and benefits estimated.

The general OA methodology is summarized below and the actions documented in the Operational Analysis Template.


PERFORM CUSTOMER ASSESSMENT

Establish Customer Data Collection StrategyEstablish Data Collection Frequency

Collect and Summarize Data

PERFORM MISSION ANALYSIS

Link to OPDIV and Dept’s Missionand Strategic Goals

Perform E-Gov Review

PERFORM OPERATIONAL ASSESSMENT

EA Compliant TechnologyProcess Reengineering

Collaboration

PERFORM GAP ANALYSIS

Document and Assess Performance Goals/Measures Reported in PMTDocument and Assess Earned Value Variances

Compare Actual to Customer Required PerformanceIdentify New Functionality or Performance Requirements

Step 1

Step 2

Step 3

Step 4

Step 1: Perform Mission Analysis. Describe how the investment supports the OPDIV 's and the Departments Mission, Goals, and Objectives. Establish the level of functionality and performance provided by the existing investment.

Describe also how the investment supports the HHS E-Government Implementation Plan.

Step 2: Perform Customer Assessment. Establish a strategy to document customer or user requirements. Periodic surveys, focus groups, or user group meetings are often assessed. Also examine usage trends, system reports, and change order requests – these can give insight into emerging requirements. Summarize and categorize the information into either performance needs or new functional requirements.

Step 3: Perform Gap Analysis. Report Performance and Earned Value variances based on information provided in the HHS PMT. Based on the Customer and User Requirements, Performance and Earned Value Variance analyses, discuss the root cause of any gap. Identify what, if any additional functionality or performance is required.

The following table summarizes example topics for consideration:


Cause of Gap or ProblemRequired Functionality or

Performance

Limited interoperability within OPDIV , Department or Federal Government

Scalable platform to support EA compliant IT infrastructure

Non-compliance with EA System consistent with EA

Poor data sharing and data integrity Enterprise-based interoperable systems with shared data standards, descriptions, and relationships

Poor reliability Modernized workstations and frequent technology refresh to maximize system reliability

Cannot meet growing demand or transaction volume

Increased capacity to meet processing, service, and mission demands

Inadequate information and computer security

Enterprise-based security authentication and or control, and strengthened IT and information security

Poor customer service Electronic application submission and processing to improve customer service

Technical architecture not scalable Fewer operational disruptions, reduced O&M costs

Limited legislative and regulatory compliance

Meets Congressional mandates and GAO oversight concerns

Step 4: Perform Operational Assessment. Identify solutions that can provide the needed functionality or performance. This may include designing new processes, implementing technologies compliant with the Department's Enterprise Architecture, or collaborating with other initiatives within the federal government. The system may have been targeted for replacement by our modernization blueprinting efforts.

Recommend if the existing system should be a) continued with no additional investment, b) enhanced, or c) terminated d) migrated to a similar system and retired.


OPERATIONAL REVIEW TEMPLATE

Example

1. Administrative Information

Investment Title

Office

Date of Operational Analysis

System Manager

System Owner

Submission Date

Revision Number

Revision Date

System Owner Signature & Date

2. Project Description

Provide a brief summary describing the asset and a description of the business processes that the investment supports.

3. Mission Analysis

3. a. For each OPDIV and Departmental mission or strategic goal that the investment supports, explain how the investment is continuing to meet OPDIV and Department mission or strategic goals.


3. b. Describe how the project supports the Department’s E-Government Strategy.

4. User or Customer Assessment

Briefly describe the investment's users and the process (e.g., surveys, user group meetings, customer focus groups, etc.) used to assess user or customer satisfaction. Summarize the results of surveys or other user or customer inputs, and usage trends. Is the existing system providing customers the needed functionality and performance?


5. Gap Analysis

Identify the need for additional functionality and performance. Examine gaps in supporting the OPDIV and Department’s Mission and Strategic Goals, Technical Performance as measured by Earned Value, and results from the User or Customer Survey.

a. Performance Variance

For the prior and current fiscal year, provide the Performance Goals and Measures table(s) with prior year actual results and current year interim results, if known. Complete Tables 1 and or Table 2 below. If the project collects, manages or reports to other performance goals and measures, add rows to record those goals and measures.

Table 1

Fiscal Year

Strategic Goal(s) Supported

Existing Baseline

Planned Performance Improvement Goal

Actual Performance Improvement Results

Planned Performance Metric

Actual Performance Metric Results

2004

2004

2005

2005

Table 2

Fiscal Year

Measurement Area

Measurement Category

Measurement Indicator

BaselinePlanned Improvements to the Baseline

Actual Results

2004

2004

2004

2004

2005

2005

2005

2005

b. User or Customer Analysis

Based on your user or customer inputs, is actual performance consistent with user or customer expectations, or do the current performance goals reflect current user or customer functional or performance requirements? Has the investment exceeded expectations, and the performance measures need to be re-baselined? Discuss how your project addresses the following operational indicators:

1) efficiency

2) effectiveness

3) maintainability

4) productivity

5) security

6) availability

7) reliability

8) energy usage


c. Earned Value Analysis

OMB-Approved Baseline Actual Outcome


SchedulePlanned

CostFunding Agency

SchedulePercent

CompleteActual CostStart

DateEnd Date

Duration (in days)

Start Date

End Date

Completion date: OMB-approved baseline: Estimated completion date:

Total cost: OMB-approved baseline: Estimate at completion:

Earned Value Variance

Provide the following cumulative earned value data.

Cost Variance = (BCWP-ACWP) =

Cost Variance % = (CV/BCWP) x 100% =

Schedule Variance = (BCWP-BCWS) =

Schedule Variance % = (SV/BCWS) x 100% =

c.1. Earned Value Variance AnalysisIf cost or schedule variances are a negative 5% or more, explain the reason for the variance. Discuss the actions that will be taken to correct the variances, the risk associated with the actions, and how close the planned actions will bring the investment to the original baseline.

c.2. Technical Performance Variance Analysis

If the performance variances are a negative 5% or more, explain the reason for the variance. Discuss the actions that will be taken to correct the variances, the risk associated with the actions, and how close the planned actions will bring the investment to the original planned improvement.

d. Gap Analyses

Based on the Customer and User Requirements, Performance Analysis, and Earned Value Variance analyses,


discuss the root cause of a gap, and what, if any, additional functionality or performance is required.

6. Operational Assessment

a. Opportunities

Based on the Gap Analysis, identify opportunities to improve functionality, performance (effectiveness and or efficiency). These opportunities may include investing in technology compliant with the Departmental EA; business


process reengineering; and or collaborating with another project. Discuss whether E-Gov initiatives can be leveraged. Describe how the project could deliver services more efficiently in a web-based environment.

b. Investment Manager Recommendations

Justify if the existing system should continue in operation as is, be enhanced, or terminated. If the system is to be enhanced or terminated, summarize the actions to be taken this fiscal year.

OPDIV ITIRB Recommendation:

HHS ITIRB Decision:


Appendix J: IT Investment Rating and Ranking Criteria- HHS Priority and Quality Scoring

J.1 Purpose

To define the criteria that will be used to rate and rank the IT investments being presented to the IT Investment Review Board (ITIRB) and other executive management committees.

J.2 Background

Ongoing Departmental IT portfolio management activities require development of rating and ranking criteria to compare IT investments in support of investment management decisions. Competition for resources may only allow a portion of the proposed investments to be approved and funded. In addition, a structured investment decision making process is required by OMB. The proposed criteria will produce one standard view for comparing major IT investments across the Department. The ITIRB reviews and revises the rating criteria on an annual basis to ensure that they reflect value, risk, health of the project, and quality of the business case. The table below describes the source for each of the criteria. Each of these categories contains a structured set of criteria; for example, the quality category reflects the OMB business case scoring criteria.

J.3 Scoring Methodology

The Select Phase of the HHS Capital Planning and Investment Control (CPIC) process includes Prioritization and Quality reviews and scoring. Based on information provided by OPDIV and Enterprise Initiative investment managers via the HHS Portfolio Management Tool (PMT) and investment reviews conducted by HHS Critical Partners and the HHS Business Case Quality Review Team, each Major and Tactical investment will receive (1) a priority score without risk, (2) a priority score adjusted for risk, and (3) a quality score. Each of the three scores is normalized to a scale of zero to 100.

Category Source of Rating Criteria

CPIC Procedures Appendices - Appendix J Page 69 of 67

Priority

Calculates a priority score based upon a consideration of investment value and alignment with strategy. Can be determined with or without risk consideration.

Cost, schedule, and performance variances will be derived from the latest quarterly or monthly reports.

Value

Measures the value of the investment to the Department or Division. These criteria are defined further below in this

appendix.Risk

Measures high-level risks associated with the investment.

Quality

Measures the current quality of the business case.

Based on the OMB scoring criteria contained in circular A-11. Scores will reflect the most current version of the business case (Exhibit 300) for each of the major and tactical investments.

Alignment

Measures an investment’s level of alignment with strategy

Based on alignment with PMA, HHS Department Strategic objectives, OPDIV Strategic objectives, etc.


J.4 Prioritization Criteria

Decision Criteria

Scoring

Value Factors

P-1: Priority without Risk

How well does the investment contribute toward meeting HHS or Division objectives in terms of effectiveness?

The Priority Score without Risk is calculated based upon:

1. Alignment to Strategy 2. Investment Value

The Priority Score without Risk is calculated for each investment using the following formula:

Priority Score w/o Risk = (Alignment + Value) x 5.5.

P-1: Priority with Risk

How well does the investment contribute toward meeting HHS or Division objectives in terms of effectiveness?

The Priority Score with Risk is calculated based on:

1. Alignment to Strategy 2. Investment Value 3. Investment Risk

The Priority Score with Risk is calculated for each investment using the following formula:

Priority Score W/ Risk = (Alignment + Value) x 5.5 x ((9/Overall Risk Score) x 11/100))

Prioritization Components

C-1: Alignment to Strategy

(PMT Form 1.03.03)

Each investment manager rates his or her investment’s alignment to the President’s Management Agenda, HHS Strategic Goals, HHS IT Strategic Goals, and OPDIV strategic goals as:

High = initiative’s primary objective supports this strategic goal, Medium = some of the secondary objectives support this strategic goal, Low = some aspects of the project support this strategic goal, or Not Applicable.

The investment manager also is asked to provide the primary alignment at each level, but this is not used in the scoring formula.

Individual investments will be scored for each strategic goal using the following scale:

High 9Medium 6Low 3Not Applicable 0

An Alignment Score is calculated for each level (PMA, HHS, etc.) by averaging all the answers at each level. For example,

PMA Score = (Answer 1 + Answer 2 + Answer 3 + Answer 4 + Answer 5)/5

The HHS Overall Alignment Score is calculated using a weighted average formula and the following weights:

HHS Overall Alignment Score Weights:

PMA 3HHS 2HHS IT 1OPDIV 2

Overall Alignment Score = ((PMA Score x 3) + (DHHS Score x 2) + (DHHS IT Score x 1) +


Decision Criteria

Scoring

(OPDIV Score x 2))/ 8

C-2: Investment Value (PMT Form 1.03.07 with input from Forms 1.03.04 and 1.03.06)

The Investment Value score is calculated by using a mix of the qualitative and quantitative criteria listed below:

Quantitative criteria = 5 Year Return on Investment (ROI)

[ROI = Cumulative Net Discounted Benefits/Cumulative Net Discounted Costs]

Each investment will receive a score from 0 to 9 for ROI values using the following scale:

> 200% 9<200% and >150% 7<150% and >100% 5<100% and >50% 3<50% and >0% 20% or Negative 0

Qualitative criteria =

Contribution to the programmatic goals Impact of funding this investment Programmatic impact of not fully funding this investment

Responses to the programmatic goals contribution questions will be rated using the following scale and numerical values:

High 9Medium 5Low 2Not Applicable 0

Responses to the impact of funding question will be rated using the following scale and numerical values:

Increases Budget 2Base Neutral 5Reduce Base 9Not Applicable 0

Responses to the impact of NOT fully funding question will be rated using the following scale and numerical values:

Low 2Medium 5High 9Not Applicable 0

NOTE: In this question, users should be evaluating the NEGATIVE impact of not fully funding the investment.

The Overall Value Score will be calculated using an average formula and the following weights:

Quantitative Benefits (ROI Score) 1

Qualitative Benefits 2

Contribution to programmatic goals 1Impact of funding this investment 1Programmatic impact of cutting/not funding this investment

2


Decision Criteria

Scoring

For example, an investment with:

ROI Score = 3Contribution to programmatic goals = 5Impact of funding this investment = 6Programmatic impact of cutting or not funding this investment = 2

will have an overall value score of ((3x1) + ((5x1+6x1+2x2)/4)x2))/3= 3.5

C-3: Investment Risk (PMT Form 1.03.10 with input from Forms 1.03.08 and 1.03.09)

For major and tactical investments, the level of risk will be determined using the standard risk questions from OMB Circular A-11, Section 300:

OMB risk for all projects:

1. Schedule, 2. Initial costs, 3. Life-cycle costs,4. Technical obsolescence, 5. Feasibility,6. Reliability of systems,7. Dependencies and interoperability between this investment and others,8. Surety (asset protection) considerations, 9. Risk of creating a monopoly for future procurements,10. Capability of agency to manage the investment and 11. Overall risk of project failure

OMB risks for IT projects:

1. Organizational and change management, 2. Business,3. Data/info,4. Technology,5. Strategic,6. Security, 7. Privacy and 8. Project resources.

Users should answer all questions and select “No Risk,” as applicable. Risk areas that are not answered or where “No Value” is selected will be considered as “High” risk in the average risk scores calculation.

Each investment will be rated for each risk area using the following scale and numerical values:

Impact

High 3Medium 2Low 1No Value 3

Probability of Occurrence

High 3Medium 2Basic (Low) 1No Value 3

A score will be calculated for each risk question by multiplying the Impact score times the Probability of Occurrence score. The score will be calculated for each risk question according to the following matrix:


Decision Criteria

Scoring

Impact x Probability

High Probability of Occurrence 3

Medium Probability of Occurrence 2

Low Probability of Occurrence 1

High Impact 3 9 6 3Medium Impact 2 6 4 2Low Impact 1 3 2 1

For each set of risks (All Projects and IT Projects Scores), a Risk Score will be calculated using an average formula. For example,

All Projects Score = (Answer 1+ … + Answer 11)/11IT Projects Score = (Answer 1+ … + Answer 8)/8

An Overall Risk Score will be calculated using a weighted average formula:

Overall Risk Score = ((All Projects Score x 1) + (IT Projects Score x 1))/2

The resulting average scores are given a High, Medium and Low value based on the following scale:

High = Average overall risk score > 5.4Medium = Average overall risk score < 5.5 and > 2.4Low = Average overall risk score < 2.5

Investment Prioritization ComponentsInvestment Quality Components

The Quality Score is calculated based on:

1. Business Case Quality2. Critical Partner Review3. Cost Variance %4. Schedule Variance %

The Quality Score will be calculated for each investment using the following formula:

Quality Score = (((Business Quality Score/5)-1) + (Critical Partners Score) + (Cost Variance Score) + (Schedule Variance Score)) x 2.8

IQC-1: Business Case Quality Score

The HHS Business Case Quality Review Team will score each business case using the scoring criteria established by OMB Circular A-11, Section 300.

1. Supports the President's Management Agenda Items (AI) a. Expanding e-Gov b. Improved Financial Performance c. Strategic Management of Human Capital d. Budget and Performance Integration

2. Acquisition Strategy (AS) 3. Program Management (PM) 4. Enterprise Architecture (EA) 5. Alternatives Analysis (AA) 6. Risk Management (RM) 7. Performance Goals (PG) 8. Security and Privacy (SE) 9. Performance Based Management System (PB) 10. Life-Cycle Costs Formulation (LC)

A total score will be calculated for the entire business case ranging in value from 10 to 50.

Score Definition:

41-50 Strong documented business cases (BC) (including all sections as appropriate).


Decision Criteria

Scoring

31-40 Very few weak points within the BC but still needs strengthening.

21-30 Much work remains to solidify and quantify BC. BC has the opportunity to either improve or degrade very quickly.

11-20 Significant gaps in the required categories of the BC.

1-10 Inadequate in every category of the required BC.

IQC-2: Critical Partner Review

There are five Critical Partners (CP): Acquisition (A), Security & Privacy (S), Enterprise Architecture, Finance (F), and Human Resources (HR). Critical Partners in the five areas review the investments and score each investment using the following scale:

Approve 9Conditionally Approve 5Disapprove 0

The CP Score is calculated using an average formula:

CP Score = (A Score + S Score + EA Score + F Score + HR Score)/5.

IQC-3: Cost Variance (PMT Form 1.03.15)

Cost Variance % Score

The Cost Variance % Score is calculated based on the cost variance (CV) and budgeted cost of work performed (BCWP) for the total cost of the investment, i.e. DME plus Steady State.

Cost Variance % = (CV/BCWP) x 100.

A Cost Variance % Score is assigned based on the following scale:

Cost Variance % < – 5% = Green 9Cost Variance % < + 5% = Green 9Cost Variance % > or = – 5% and < -10% = Yellow 6Cost Variance % > or = + 5% and < +10% = Yellow 6Cost Variance % > or = – 10% = Red 3Cost Variance % > or = + 10% = Red 3

IQC-4: Schedule Variance (PMT Form 1.03.15)

The Schedule Variance % Score is calculated based on the schedule variance (SV) and budgeted cost of work scheduled (BCWS) for the total cost of the investment, i.e. DME plus Steady State.

Schedule Variance % = (SV/BCWS) x 100.

A Schedule Variance % Score is assigned based on the following scale:

Schedule Variance % < – 5% = Green 9 Schedule Variance % < + 5% = Green 9Schedule Variance % > or = – 5% and < -10% = Yellow 6 Schedule Variance % > or = + 5% and < +10% = Yellow 6Schedule Variance % > or = – 10% = Red 3 Schedule Variance % > or = + 10% = Red 3

NOTE: If BCWP and BCWS values are not entered (i.e. Null), then the Cost Variance % Score and Schedule Variance % Scores will be set to zero (0).



Appendix K: References

Assessing Risks and Returns: A Guide for Evaluating Federal Agencies’ IT Investment Decision-Making, U.S. General Accounting Office, Accounting and Information Management OPDIV, February 1997.

Capital Programming Guide, Office of Management and Budget, July 1997.

Circular A-11: Preparing and Submitting Budget Estimates, Office of Management and Budget, July 2004

Circular A-76: Performance of Commercial Activities, Office of Management and Budget, May 29th 2003.

Circular A-94: Discount Rates to be Used in Evaluating Time-Distributed Costs and Benefits, Office of Management and Budget, January 2002.

Circular A-127: Financial Management Systems, Office of Management and Budget, 1993.

Circular A-130: Management of Federal Information Resources, Office of Management and Budget, January 28, 2000.

Clinger-Cohen Act of 1996 (formerly the Information Technology Management Reform Act [ITMRA]).

Earned Value Management Systems (EVMS) Basic Concepts, Project Management Institute, Project Management Institute (PMI) Home Page

Evaluating Information Technology Investments, Office of Management and Budget, February 1995. http://www.whitehouse.gov/omb/index.html

Executive Guide: Leading Practices in Capital Decision-Making, U.S. General Accounting Office, Accounting and Information Management OPDIV, December 1998.

Fleming, Quentin W., Joel M. Koppelman, Earned Value, Project Management, Second Edition, Project Management Institute, Inc., 2000.

Grant, Eugene L., W. Grant Ireson, Principles of Engineering Economy, Fifth Edition, The Ronald Press Company, 1970.

Information Technology Investment Management: A Framework for Assessing and Improving Process Maturity (Exposure Draft), U.S. General Accounting Office, Accounting and Information Management OPDIV, May 2000. http://www.gao.gov/new.items/d04394g.pdf

CPIC Procedures Appendices - Appendix K Page 76 of 67

http://www.pmi.org/info/default.asp

http://www.pmi.org/info/default.asp


Smart Practices in Capital Planning, The Federal CIO Council, Capital Planning and IT Management Committee, Industry Advisory Council (IAC), October 2000.

CPIC Procedures Appendices - Appendix K Page 77 of 67

Date post:	06-May-2015
Category:	Documents
Upload:	khanyasmin
View:	653 times
Download:	1 times

DOC

Documents