The FDIC’s Information Technology Project Management Process

Office of Audits and Evaluations Report No. EVAL-14-001

July 2014

i To view the full report, go to www.fdicig.gov

Executive Summary

The FDIC’s Information Technology Project Management Process

Report No. EVAL-14-001 July 2014

Why We Did The Evaluation

IT projects involve all FDIC divisions and offices and are critical to the FDIC’s operations and successful accomplishment of the Corporation’s mission, goals, and objectives. In addition, the FDIC invests significant funding and internal resources in such projects. For example, as of December 31, 2013, the FDIC's incurred costs for projects completed or in process during 2012 and 2013 were approximately $111.7 million. Our objective was to (1) assess the extent to which the FDIC’s IT projects are meeting their cost, schedule, and requirements expectations; (2) identify factors that promote project success or prevent projects from meeting expectations; and (3) identify opportunities for strengthening the FDIC’s controls for monitoring IT projects. To address the first part of our objective, we obtained reports on IT projects completed or in-process during 2013. For the second and third parts of our objective, we selected six IT projects in process or completed during 2012 for in-depth review. In selecting the projects, we included those governed by the FDIC’s Capital Investment Review Committee (CIRC) and Chief Information Officer’s (CIO) Council and from a cross-section of FDIC divisions and offices. We also took into consideration factors such as the project management method employed, estimated cost, the contractor engaged to work on the project, and the extent to which there were any known problems or positive attributes. For each project, we reviewed project-related documentation and interviewed FDIC and contractor personnel involved with the projects, in the context of relevant industry practices and FDIC policies and procedures.

Background

The Office of Management and Budget (OMB) has reported that IT advancements have been at the center of a transformation in how the private sector operates—and have revolutionized the efficiency, convenience, and effectiveness with which the private sector serves its customers. However, according to OMB, the federal government largely has missed out on that transformation due to poor management of technology investments, with IT projects too often costing hundreds of millions of dollars more than they should, taking years longer than necessary to deploy, and delivering technologies that are obsolete by the time they are completed. Similarly, the U.S. Chief Information Officer has noted that too often, federal IT projects run over budget, fall behind schedule, or fail to deliver promised functionality. Many projects use “grand design” approaches that aim to deliver functionality every few years, rather than breaking projects into more manageable chunks and demanding new functionality every few quarters. The FDIC’s CIO plays a key role in both IT governance and IT project management at the FDIC. Specifically, the CIO is responsible for ensuring that all capital investment projects are consistent with the information technology strategies and objectives of the Corporation, including those related to architectural alignment, security, and resource optimization. The CIO also ensures that proposed systems development projects are adequately planned, estimated, resourced, and monitored throughout the development life cycle. The FDIC’s Board of Directors approves funding for capital investments, including IT projects, involving estimated costs of $3 million or more and receives updates on those projects and the performance of the portfolio as whole on a quarterly basis. The FDIC’s IT projects are governed by three entities—the

ii

Executive Summary

The FDIC’s Information Technology Project Management Process Report No. EVAL-14-001

July 2014

CIRC, the CIO Council, or the Corporate Management Council (CM)—depending on the cost and nature of the project. IT project management is considered to be the day-to-day discipline of organizing and managing resources (e.g., people and budget) so a project delivers intended requirements within defined scope, quality, time, and cost constraints. Implementing the process is a shared responsibility among DIT, the FDIC division or office sponsoring an IT project (client), and the IT contractor responsible for developing the application. The FDIC uses the Rational Unified Process® (RUP) as its system development life cycle methodology (SDLC) for managing IT projects. The RUP framework may be tailored to meet the specific needs of projects based on their risk (size, scope, and complexity). The RUP framework promotes iterative development, which is a flexible, risk-focused approach to software development divided into four phases and eleven disciplines. Although the RUP framework has always included aspects of the Agile methodology, DIT began promoting and applying the Agile methodology for FDIC IT projects in 2012. Agile software development supports the practice of shorter software delivery. Specifically, Agile calls for the delivery of software in small, short increments rather than in the typically long, sequential phases of a traditional SDLC waterfall approach. DIT management conducts milestone reviews at the completion of each RUP phase. Milestone reviews mark a point at which management and technical expectations should be resynchronized. These reviews should ensure projects have met the goal of each RUP phase and form the basis for determining whether the FDIC should move to the next phase of the IT project. If a project experiences significant challenges and is underperforming, the CIO Council may request a TechStat review. A TechStat review is an evidence-based review of an IT investment based on a model developed by OMB, with a focus on problem solving that will lead to corrective action to improve overall performance.

Evaluation Results

Most CIRC and CIO Council projects completed or in process during 2013 met planned schedules, were within 10 percent of annual budgeted expenses, and met user expectations. Still, perceptions and anecdotes persist that FDIC IT projects are sometimes too costly, experience delays, or do not deliver promised specifications. During our evaluation, the CIO Council used an annual budget process to monitor IT project costs. We concluded that the CIO Council could enhance its cost monitoring by evaluating total project costs against initial project budgets. Doing so would more readily show to what extent individual projects, and the portfolio as a whole, meet life-cycle cost expectations. The FDIC’s Project Management Office was developing metrics for tracking projects against initial project budgets at the time we were completing our fieldwork. Further, the Acting CIO indicated that he will continue to have dialogues with those having key roles in IT governance and project management regarding metrics being used to determine project success. Based on these ongoing efforts, we determined that a recommendation associated with these matters was not warranted. With respect to the six projects we selected for in-depth review, four of the six have been completed. Three of the completed projects met both schedule and cost expectations, while the other project missed the original estimated completion date by 1 year, and actual cost far exceeded the original budget. The two projects that are in process are both behind schedule and could, as a result, experience cost overruns.

iii

Executive Summary

The FDIC’s Information Technology Project Management Process Report No. EVAL-14-001

July 2014

As a result of our interviews and analysis of these projects, we identified the following aspects of the IT project management process that were key factors in project success or contributed to challenges, depending on whether and how well they were carried out:

Thoroughly planning and scoping the IT project. Ensuring developers understand the FDIC’s environment. Managing IT project collaboration and communication. Implementing an effective milestone review process. Preparing a dedicated testing team. Assigning independent risk managers.

Ensuring that these factors are emphasized and the related controls are in place and working during ongoing and future IT projects could provide greater assurance that the projects meet cost, schedule and requirements expectations.

Recommendation and Corporation Comments

The report contains one recommendation for the Acting CIO to: (1) advise client division and offices, IT project teams, DIT intersecting organizations, and appropriate governance bodies of the key factors in project success or challenges and related controls we identified in this report and (2) determine whether guidance in any of these areas needs to be strengthened. The Acting CIO provided a written response, dated June 25, 2014, to a draft of this report. In the response, the Acting CIO concurred with the report’s recommendation and described completed and planned corrective actions, which are responsive to the recommendation.

Contents Page

Background 2 IT Project Management Process, Roles, and Responsibilities 2 System Development Life Cycle Methodology 4 Evaluation Results 7

Extent to Which the FDIC’s IT Projects Are Meeting Their Cost, Schedule, and Requirement Expectations

Conclusion 8

11

Factors that Promote IT Project Success or Prevent Projects from Meeting Expectations Thoroughly Planning and Scoping the IT Project Ensuring Developers Understand the FDIC’s IT Environment Managing Collaboration and Communication Implementing an Effective Milestone Review Process Having a Well-Informed and Dedicated Testing Team Assigning an Independent Risk Manager to Projects

11121414161820

Conclusion and Recommendation 21 Corporation Comments and OIG Evaluation 22 Appendices

1. Objective, Scope, and Methodology 232. Glossary 263. Acronyms and Abbreviations 274. Summaries of IT Projects Included in the Evaluation 285. Corporation Comments 6. Summary of the Corporation’s Corrective Actions

3335

Tables 1. Key IT Governance Bodies 32. Key IT Project Management Parties 43. Summary of Sampled IT Projects 4. Overall BOI Rating Definitions 5. Summary of Projects Sampled

81024

Figures

1. RUP System Development Life Cycle Process and Phases 52. Milestone Meeting Rules of Engagement 173. Summary of FDIC Policies and Procedures and IT Governance

Documents 23

Federal Deposit Insurance Corporation3501 Fairfax Drive, Arlington, VA 22226

Office of Audits and Evaluations Office of Inspector General

DATE: July 14, 2014 MEMORANDUM TO: Martin D. Henning Acting Chief Information Officer /Signed/ FROM: Stephen M. Beard Deputy Inspector General for Audits and Evaluations SUBJECT: The FDIC’s Information Technology Project Management Process

(Report No. EVAL-14-001) This report presents the results of our evaluation of the FDIC’s information technology (IT) project management process. The report contains one recommendation intended to strengthen controls in areas we determined were key to project success or challenges. IT projects involve all FDIC divisions and offices and are critical to the FDIC’s operations and successful accomplishment of the Corporation’s mission, goals, and objectives. In addition, the FDIC invests significant funding and internal resources in such projects. For example, as of December 31, 2013, FDIC's incurred costs for projects completed or in process during 2012 and 2013 were approximately $111.7 million. Our objective was to (1) assess the extent to which the FDIC’s IT projects are meeting their cost, schedule, and requirement expectations; (2) identify factors that promote project success or prevent projects from meeting expectations; and (3) identify opportunities for strengthening the FDIC’s controls for monitoring IT projects. To address the first part of our objective, we obtained reports on IT projects completed or in-process during 2013. For the second and third parts of our objective, we selected six IT projects in process or completed during 2012, interviewed FDIC and contractor personnel involved with the projects, and reviewed project-related documentation, in the context of relevant industry practices and FDIC policies and procedures. We conducted this evaluation in accordance with the Council of the Inspectors General on Integrity and Efficiency’s Quality Standards for Inspection and Evaluation. Appendix 1 of this report includes additional details on our objective, scope, and methodology. Appendix 2 contains a glossary of key terms,1 and Appendix 3 contains a list of acronyms and abbreviations. Appendixes 4, 5, and 6 include summaries of IT projects we reviewed during our evaluation, the Corporation’s comments on a draft of this report, and a summary of the corrective actions being taken to address the report’s one recommendation, respectively.

1 Terms that are underlined when first used in this report are defined in Appendix 2, Glossary.

2

Background The Office of Management and Budget (OMB) has reported that IT advancements have been at the center of a transformation in how the private sector operates—and have revolutionized the efficiency, convenience, and effectiveness with which the private sector serves its customers. However, according to OMB, the federal government largely has missed out on that transformation due to poor management of technology investments, with IT projects too often costing hundreds of millions of dollars more than they should, taking years longer than necessary to deploy, and delivering technologies that are obsolete by the time they are completed. Similarly, the U.S. Chief Information Officer has noted that too often, federal IT projects run over budget, fall behind schedule, or fail to deliver promised functionality. Many projects use “grand design” approaches that aim to deliver functionality every few years, rather than breaking projects into more manageable chunks and demanding new functionality every few quarters.2 IT Project Management Process, Roles, and Responsibilities The FDIC’s Chief Information Officer (CIO) plays a key role in both IT governance and IT project management at the FDIC. Specifically, the CIO is responsible for ensuring that all capital investment projects are consistent with the information technology strategies and objectives of the Corporation, including those related to architectural alignment, security, and resource optimization. The CIO also ensures that proposed systems development projects are adequately planned, estimated, resourced, and monitored throughout the development life cycle. For purposes of better understanding the scope and results of our review, it is important that we distinguish between IT governance and IT project management as it relates to other entities, offices, and individuals that have functional roles in those areas. The FDIC defines IT governance as:

An integral part of enterprise governance which consists of the leadership, organizational structures, and processes that ensure that IT sustains and extends the FDIC's strategies and objectives. The overall objective of IT governance is to understand the issues and the strategic importance of IT. IT governance aims at ensuring that expectations for IT are met and IT risks are mitigated.

Table 1 below summarizes the entities that play key roles in IT governance as it relates to the approval and monitoring of the Corporation’s IT projects.

2 Effective planning and management of IT and non-IT capital investments are mandated by Congress and by OMB for most federal agencies. Although many of these laws and directives are not legally binding on the FDIC, the FDIC recognizes that they constitute best practices and should be adopted in whole or in part.

3

Table 1: Key IT Governance Bodies Governance Body Responsibilities

Board of Directorsa Approves funding requests for new and existing capital investment projects involving estimated costs of $3 million or more. Receives quarterly updates for individual projects as well as an assessment of the performance of the portfolio as a whole.

Capital Investment Review Committee (CIRC)b

Approves projects estimated to cost more than $3 million. The purpose of the Committee is to implement a systematic management review process that supports budgeting for the FDIC’s capital investments and ensures the regular monitoring and proper management of these investments, once funded.

CIO Councilc Advises the CIO on all aspects of adoption and use of IT at the FDIC. The Council prioritizes and selects IT projects for funding and reviews the progress of these projects on a monthly basis.

Corporate Management Council (CM) Governs projects that focus on improvements and enhancements to Division of Information Technology (DIT) products.

Project Initiation Review (PIR) Committee

Reviews all FDIC IT projects to ensure DIT management support and compatibility with the FDIC’s IT infrastructure, and avoid duplication or additional costs. Ensures that the appropriate budgetary resources, infrastructure standards, security standards, and enterprise architecture planning are in place to support new project initiatives.

Program Management Office (PMO) A resource center for clients, executives, project managers, and project team members engaged in the operations and oversight of IT projects. The PMO's mission is to continuously improve the practice and results of IT program and project management.

Source: FDIC DIT Web site. Notes: a The Board of Directors consists of the FDIC Chairman; FDIC Vice Chairman; FDIC Director; Comptroller of the Currency; and Director, Consumer Financial Protection Bureau. b The CIRC is chaired by the Chief Financial Officer (CFO) and CIO, and its members include the Chief Risk Officer, division directors, and the Director, Office of Complex Financial Institutions. c The CIO Council is chaired by the CIO and includes executive representatives of the FDIC’s divisions and offices.

IT project management, on the other hand, is considered to be:

The discipline of organizing and managing resources (e.g., people and budget) so a project delivers intended requirements within defined scope, quality, time, and cost constraints.

Implementing the IT project management process is a shared responsibility among DIT, the FDIC division or office sponsoring an IT project (client), and the IT contractor responsible for developing the application.3 Table 2 below identifies and describes the key parties involved in that process.

3 The FDIC awarded an Information Technology Application Services Basic Ordering Agreement in May 2013 to 11 contractors to develop IT projects and perform other IT-related services for FDIC divisions and offices.

4

Table 2: Key IT Project Management Parties Key Party Responsibilities

Client Program Manager Leads the requirements package development for the IT project and for the final approval of all process and software development documents, including the establishment and maintenance of roles and responsibilities.

Client Project Manager Leads the planning of the project, coordinates interactions with the other parties involved in the project, and keeps the project team focused on meeting the project objectives.

DIT Project Manager Serves as the Technical Monitor for the software contractor and is responsible for ensuring the contractor’s performance on the contract in developing the software. Also ensures that the contractor is coordinating with other DIT areas such as configuration management, enterprise architecture, infrastructure services, and quality assurance, which are known as intersecting organizations (IOs).

Contractor’s Project Manager Ensures the product is delivered by the contractor in compliance with the contract requirements and schedule.

Project Testing Team Responsible for the core activities of the test effort. Those activities include identifying, defining, implementing, and conducting the necessary testing processes as well as logging the outcomes of the testing and analyzing the results.

Independent Risk Manager The Division of Finance’s (DOF) Corporate Management Control Branch (CMCB) provides an Independent Risk Manager for CIRC and other major CIO Council IT projects. This individual maintains the risk list and provides risk management support to the project.

Source: OIG review of StarTeam documentation for six IT Projects and the FDIC’s RUP Web site.

System Development Life Cycle Methodology The FDIC uses the Rational Unified Process® (RUP) as its system development life cycle methodology (SDLC) for managing IT projects. The RUP framework may be tailored to meet the specific needs of projects based on their risk (size, scope, and complexity). Although the RUP framework has always included aspects of the Agile methodology, DIT began promoting and applying the Agile methodology for FDIC IT projects in 2012. The RUP framework promotes iterative development, which is a flexible, risk-focused approach to software development divided into four phases and eleven disciplines as shown in Figure 1.

Agile software development supports the practice of shorter software delivery. Specifically, Agile calls for the delivery of software in small, short increments rather than in the typically long, sequential phases of a traditional SDLC waterfall approach. Agile emphasizes early and continuous software delivery, the use of collaborative teams, and measuring progress with working software. For Agile to be practical, each feature must be fully developed, tested, styled, and accepted by the user before counting it as completed and moving on to the next feature. An important aspect of the Agile methodology is that the client users are involved in project development and that their feedback from testing is critical.

5

Figure 1: RUP System Development Life Cycle Process and Phases

Source: FDIC RUP Web site.

A more detailed discussion of each RUP phase follows. Inception Phase The primary goal of the Inception phase is to develop an understanding of the client’s requirements and the purpose of the IT project. The client, contractor, and DIT project manager scope the project and document the functional requirements that address the client’s business needs, such as the type of reporting needed, and non-functional requirements, such as the number of anticipated users, required operating speeds, and data capacity. The PIR Committee meets at the beginning and throughout the Inception phase with the project managers to review start-up of the initiatives, help establish priorities, resolve potential conflicts, and plan key activities to successfully initiate the project. During the Inception phase, the IT project team considers alternative solutions for achieving the client’s needs. Solutions may include purchasing an existing system from a vendor, customizing an existing product, or developing new software from scratch. It is critical for all aspects of the project to be explored, including recognizing interfaces with other systems, identifying key risks, determining acceptance criteria, and capturing the most important reporting requirements, among other things.

6

The scope, risks, potential solutions, costs, schedules, resources required, and acceptance criteria should be understood by the parties responsible for developing the project at the completion of the Inception phase. However, further knowledge gained in the Elaboration phase may clarify these aspects of the project and require adjustments. Elaboration Phase The primary goal of the Elaboration phase is to prove that the solution selected will successfully meet the requirements. The contractor further evaluates the project's architecture and determines the required resources. The contractor and the client consider possible applications of the software and costs associated with a project. The contractor also tests critical aspects of the software solution and builds a prototype of the software to validate that the proposed solution will support the IT project requirements at a reasonable cost and in a reasonable timeframe. At the end of the Elaboration phase, the product vision and requirements, and architecture should be stable; the key approaches to be used in test and evaluation are proven; major risk elements have been addressed and credibly resolved; iteration plans for the Construction phase are of sufficient detail and quality to allow the work to proceed; and actual resource expenditure versus planned expenditure is acceptable. The project may be aborted or considerably re-thought if it fails to reach this milestone. Construction Phase The primary goal of the Construction phase is to ensure that the IT software is useable and includes all necessary functionality. The contractor develops and completes the project. The contractor and DIT project manager should confirm that the client business units are ready to accept the new software. At this time, the contractor and client should complete the analysis, design and development, and testing of all required functionality. Transition Phase The primary goal of the Transition phase is to ensure that software is available for its users. The Transition phase can span several iterations and includes testing the product in preparation for release and making minor adjustments based on user feedback. At this point in the lifecycle, user feedback should focus mainly on fine-tuning the product, configuration, installation, and user-related issues. By the end of the Transition phase, lifecycle objectives should have been met, the contractor and DIT execute deployment plans, and the project should be ready to be closed out.

7

Project Reviews DIT management conducts milestone reviews at the completion of each RUP phase. Milestone reviews mark a point at which management and technical expectations should be resynchronized. These reviews should ensure projects have met the goal of each RUP phase and form the basis for determining whether the FDIC should move to the next phase of the IT project. If a project experiences significant challenges and is underperforming, the CIO Council may request a TechStat review. A TechStat review is an evidence-based review of an IT investment based on a model developed by the OMB, with a focus on problem solving that will lead to corrective action to improve overall performance. However, in some cases, a TechStat may reveal that the best course of action for an investment is that it temporarily be halted or even terminated. In addition, the CIRC meets quarterly and the CIO Council meets monthly to monitor their respective IT project portfolios. During these meetings, client and DIT project managers make presentations related to project status and request scheduling and budget adjustments, as needed.

Evaluation Results Most CIRC and CIO Council projects completed or in process during 2013 met planned schedules, were within 10 percent of annual budgeted expenses, and met user expectations. Still, perceptions and anecdotes persist that FDIC IT projects are sometimes too costly, experience delays, or do not deliver promised specifications. During our evaluation, the CIO Council used an annual budget process to monitor IT project costs. We concluded that the CIO Council could enhance its cost monitoring by evaluating total project costs against initial project budgets. Doing so would more readily show to what extent individual projects, and the portfolio as a whole, meet life cycle cost expectations. The PMO was developing metrics for tracking projects against initial project budgets at the time we were completing our fieldwork. Further, the Acting CIO indicated that he will continue to have dialogues with those having key roles in IT governance and project management regarding metrics being used to determine project success. Based on these ongoing efforts, we determined that a recommendation associated with these matters was not warranted. With respect to the six projects we selected for in-depth review, four of the six have been completed. Three of the completed projects met both schedule and cost expectations, while the other project missed the original estimated completion date by 1 year, and actual cost far exceeded the original budget. The two projects that are in process are both behind schedule and could, as a result, experience cost overruns. As a result of our interviews and analysis of these projects, we identified the following aspects of the IT project management process that were key factors in project success or contributed to challenges, depending on whether and how well they were carried out:

Thoroughly planning and scoping the IT project. Ensuring developers understand the FDIC’s environment. Managing IT project collaboration and communication.

8

Implementing an effective milestone review process. Preparing a dedicated testing team. Assigning independent risk managers.

Ensuring that these factors are emphasized and the related controls are in place and working during ongoing and future IT projects could provide greater assurance that the projects meet cost, schedule and requirements expectations. To that end, we are recommending that the Acting CIO (1) advise client division and offices, IT project teams, DIT intersecting organizations, and appropriate governance bodies of the key factors in project success or challenges and related controls we identified in this report and (2) determine whether guidance in any of these areas needs to be strengthened.

Extent to Which FDIC IT Projects Are Meeting Their Schedule, Cost, and Requirement Expectations Most CIRC and CIO Council projects completed or in process during 2013 met planned schedules, were within 10 percent of annual budgeted expenses, and met user expectations. Results were mixed as it relates to meeting schedule and cost expectations for the projects we selected for detailed review, while user satisfaction was consistently favorable. With respect to projects meeting cost expectations, we discuss in this section how the FDIC’s contracting and budgeting approaches influenced that metric. Meeting Schedule and Cost Expectations. Of the FDIC’s 34 projects active or completed during 2013, 27 were within 10 percent of their project milestones and 31 projects were within 10 percent of their annual 2013 budget. We selected six projects for review from FDIC’s inventory of IT projects completed or in-process as of December 31, 2012. As shown in Table 3, three of the completed projects met both schedule and cost expectations, while the other project missed the original estimated completion date by 1 year, and actual cost far exceeded the original budget. The two projects that are in process are both behind schedule and could, as a result, experience cost overruns. However, such overruns are largely mitigated because the FDIC uses firm fixed-price contracts for the Construction and Transition phases. As such, the FDIC is generally not obligated to compensate the contractor for costs above the contract ceiling price. Further, as discussed later, the FDIC had historically budgeted and measured the cost of projects on an annual basis, which made it difficult to measure the total actual cost of an ongoing project against its original estimated cost. Table 3: Summary of Sampled IT Projects

Name of Project

Within Schedule?

Within Contract Budget?

Project Status/Key Points

Advanced Legal Information System (ALIS)

No No Project Completed. ALIS was originally planned to be completed by first quarter 2012 at a cost of approximately $1.7 million. The CIO Council ordered a TechStat assessment in July 2012 as the project was significantly behind schedule and over-budget. The TechStat resulted in recommended corrective

9

Name of Project

Within Schedule?

Within Contract Budget?

Project Status/Key Points

actions for improving management of the project. ALIS was completed in August 2013 at a total cost of $4.7 million.

Assessment Information Management System (AIMS)

Yes Yes Project Completed. Changes mandated by the Dodd-Frank Wall Street Reform and Consumer Protection Act directly impacted the FDIC Assessment Program and required modifications to the AIMS system to address technology obsolescence risks and to revise the AIMS’ method for calculating assessments. A key to AIMS’ success was that there was 100 percent staff involvement from DOF’s Assessments Branch.

Claims Administration System (CAS)

Yes

Yes

Project Completed. CAS had strong senior management support. A key to the project’s success was that the project scope was identified early and remained constant throughout the project.

Examination Tools Suite-Supervisory Application Generating Exams (ETS-SAGE)

No Yes, but will likely

exceed budget.

Construction Phase in Process. During our evaluation, ETS-SAGE was progressing well and parties we interviewed reported favorably on the use of the Agile development approach. However, significant challenges developed during the second half of 2013 when the project team discovered that, although the IT solution worked in the test environment, it would not work in remote locations where it is most needed. As a result, the project has experienced several delays and will require a contract extension to be completed.

Identity Access Management System (IAMS)

Yes Yes Project Completed. IAMS experienced a number of challenges early in the development and the contractor appeared to have misunderstood the complexity of the project. The contractor’s project manager and other key team members were replaced and the project’s development significantly improved.

Proforma Modernization (PROFORMA)

No Yes Construction Phase in Process. The project has been significantly challenged with application, hardware, connectivity, and network performance issues. The FDIC’s current infrastructure does not support the IT solution developed by the contractor. The CIO Council approved a 5-month extension of the project completion date in January 2014.

Source: OIG interviews and review of IT project files.

Meeting Requirements Expectations. To gain perspective on the extent to which FDIC IT projects are meeting requirements expectations, we asked FDIC users that participated in testing of the IT projects in our sample whether they considered the IT project to be performing as expected. The perceived success of an IT project may change over time; however, the responses we received indicated consistent satisfaction with the performance of these projects. Even on completed projects that overcame significant challenges and delays, the users felt that the end product met or exceeded their expectations. In addition, to validate the extent a project has realized its objectives, business values, and outcomes, DIT’s PMO conducts a business outcome and lessons learned evaluation on completed CIO Council and CM projects.4 These post-project surveys indicated that the FDIC’s IT project management process is ultimately delivering software development products that meet

4 The objectives, business values, and outcomes are established at the outset of a project in the Business Proposal Outline or Project Proposal Outline by the client organization with the assistance of a DIT business analyst.

10

users’ expectations, even when IT project development is not completed within estimated milestones and budgets. The PMO assigns Business Outcome Index (BOI) ratings for each IT project based on Business Value Realized, Project Objective Attainment, and Quality of Delivery, and the PMO assigns an Overall Project Rating. As of December 31, 2013, based on ratings of 68 IT projects, the average BOI was 3.7 based on a 5-point scale indicating that IT projects generally exceeded their expected business value. Table 4 illustrates the overall BOI rating definitions for 2013 projects. Table 4: Overall BOI Rating Definitions

BOI Rating Initiative Realized Value Number of ProjectsGreater than 4 and/or equal to 5 High Business Value 32 Greater than 3 and/or equal to 4 Exceeded Business Value 25 Greater than 2 and/or equal to 3 Realized Value 9 Greater than 1 and/or equal to 2 Low Business Value 2 Source: FDIC DIT Web site.

Monitoring and Reporting Project Status. We observed that IT project delays and cost variances were monitored and reported timely to the FDIC’s governing IT committees, in accordance with CIRC and CIO Council guidelines. DIT provides the governing committees with a variety of IT project management reports, monthly or as needed, and posts monthly status reports on the FDIC Intranet. Reports are available on project status, including a monthly reporting of the current RUP phase of each project in relation to approved milestones. In addition, as discussed earlier, DIT’s BOI process evaluates the success of IT projects in meeting users’ expectations, and CMCB performs post-project reviews of CIRC portfolio projects. When projects included in our evaluation experienced challenges, actions were taken to identify causes and correct IT project management issues. As noted earlier, one of the projects included in our evaluation received a TechStat review because of CIO Council concerns regarding milestone and cost variances. The review identified various corrective actions to improve project management. Efforts to Enhance IT Project Management. In August 2012, the CIO Council adopted CMCB recommendations to improve the overall governance of projects and sharpen the focus on major projects. These guidelines included:

Tracking and monitoring total direct costs of projects, including planning and implementation project costs; discontinuing the use of division-level discretionary funds to cover project shortfalls; and allowing the CIO Council to request TechStat reviews at any time.

Using a line-of-sight approach to group projects that were previously broken into annual

phases or releases into a single fully-scoped project, and requiring new cost and milestone baselines at the end of each RUP phase for the complete project.

Designating certain CIO Council projects as “major projects” to receive greater oversight,

increasing reporting and briefings to the CIO Council on major projects, and requiring

11

PMO evaluation of projects with variances to determine if the project should be redesignated as a “major project.”

Establishing new reporting and metrics to the CIO Council and quarterly reports on

system performance and health. DIT began routinely providing the CIO Council with the total project cost in addition to the annual budgeted cost in February 2013. Still, during our evaluation, we determined that the CIO Council could do more to monitor total project costs against the initial project budget on a portfolio basis. Specifically, the CIO Council monitored non-CIRC projects using an annual budget process, which could limit insights into overall cost performance for projects spanning multiple years. While CIO Council members could request that DIT’s PMO prepare an analysis showing a project’s original budget to actual costs incurred-to-date, DIT did not maintain such information on an ongoing basis. More recently, in April 2014, the PMO revised the CIO Council Notification Report to include, among other things, total project budget information to provide council members with a financial assessment for the total investment and to allow visibility into future year budget requirements. Conclusion At our exit conference, the Acting CIO acknowledged that there remains some concern within the FDIC as to whether IT projects are being implemented efficiently and effectively—despite current metrics indicating otherwise—and is continuing to work towards implementing meaningful metrics for project success that are understood and agreed upon by those having key roles in executing and monitoring IT project management. Further, efforts to refine and improve the reporting of cost and other aspects of IT project status to the CIO Council and other program officials were well underway as we completed our fieldwork. Accordingly, we concluded that a recommendation was not warranted to address our findings in this area.

Factors that Promote Project Success or Prevent Projects from Meeting Expectations We identified six factors that either promoted IT project success or prevented projects from meeting expectations based on our interviews and analysis of the six projects that we selected for detailed review. These projects involved a variety of FDIC divisions and offices, DIT project managers, and IT contractors. A summary of the projects, including their purpose and a description of their respective project management processes, is provided in Appendix 4. Thoroughly Planning and Scoping the IT Project Selecting the Right Contractor and IT Solution. Adequate planning of the IT project begins with consideration of all available IT solutions during the Inception phase. FDIC RUP guidelines note that before scheduling the Inception milestone review, the project team should

12

Vision Document Defines the stakeholders’ view of the technical solution to be developed. It communicates the fundamental “what and why” for the project and provides a strategy against which all future project decisions can be validated.

have selected the solution architecture and determined it to be feasible from a business and technical perspective. Project managers for two projects in our sample stated that the success of their projects was directly attributable to an effective contracting process during which a variety of IT solutions were evaluated. The technical evaluation panel considered each proposal and the selection was based on the quality of the contractors’ understanding of the business needs and processes rather than on cost. On other projects that encountered numerous challenges, we were told that, in hindsight, the selection process of the IT solution and contractor was abbreviated or not fully executed and that the project team should have given greater consideration to an alternative solution or contractor. Contractors who had worked on multiple FDIC projects told us that the project works best when the business unit knows what it wants and considers a variety of contractor solutions. Understanding Project Scope and Complexity. Properly scoping the project and accurately communicating the project’s complexity is critical to project success. DIT Milestone Review Guidelines note that the scope of work should be defined, validated, and agreed upon by the business and technical stakeholders during the Inception phase. FDIC RUP guidance documents require a number of scoping-related documents, including the Vision document, software development plan, and risk lists that should be prepared or in-process at the Inception milestone review decision point. Project managers and other officials whom we interviewed indicated that many of the challenges they faced could have been avoided if the project had been better understood by all parties when it was being planned. For example, on one project, much of the contractor’s project team needed to be replaced because the complexity of the project was not adequately documented when it was proposed. That project involved multiple FDIC divisions and offices, and interacted with numerous FDIC systems. After the project began encountering problems, the contractor replaced its project manager and other lead staff, and progress on all aspects of the project greatly improved. Another project was originally cast as an upgrade; however, after the project met significant challenges, a project review determined that it should have been treated as a major overhaul. FDIC RUP guidance notes that for projects focused on enhancements to an existing system, the Inception phase is briefer than for full system development efforts. Accordingly, project teams should ensure that the scope and complexity are fully understood to preclude a project from being misclassified and not subject to sufficient planning. In those two projects, the contractor and DIT project managers were replaced with more experienced personnel and the projects’ progress significantly improved. FDIC officials we spoke to recommended that the client should ensure that the complexity of a given project is clearly identified during the planning and scoping of the project, prior to the contractor and DIT assigning personnel to a project. Contractors also told us that ensuring that the quality of the staff is compatible with the complexity of the project is critical. The FDIC RUP guidelines for conducting the Inception milestone review include reviewing resource availability, expertise, and engagement for preparedness in moving to the next RUP phase. Setting Realistic Milestones. FDIC officials we spoke to for one project we reviewed felt that, in hindsight, the challenges they faced resulted from the scope of the project not being

13

effectively communicated to FDIC senior management. As a result, the milestones established for the project were too aggressive. Unrealistic milestones negatively affected the project primarily in two ways. First, it put unneeded pressure on the project team to meet the deadlines and caused the team to reduce communication on the project. Second, because of the unrealistic deadlines, the contractors stated that they were not allowed time during the Inception phase of the project to gain an adequate understanding of the FDIC’s technical environment. FDIC RUP guidelines note that one of the objectives of the Inception phase is to develop initial cost and schedule estimates, to be followed by more detailed and reliable estimates in the Elaboration phase. The guidance also notes that cost and schedule estimates should be credible and justifiable. Focusing on Business Needs Rather Than IT Capabilities. Other comments related to planning centered on the importance of the FDIC client ensuring that the proposed solution meets the FDIC’s business need as opposed to the contractor changing the FDIC’s business process to meet a contractor’s proposed solution. FDIC project managers suggested that the client should lead the discussion with the contractor to ensure that the contractor fully understands the business requirements and should not change the business requirements to conform to limitations of a proposed solution. On one project that we reviewed, officials we spoke to said that, in hindsight, they wish that they had not allowed the contractor to lead the discussion to such a large extent. Those officials believed that this approach took them away from pursuing the original project concept to a more involved project that experienced many challenges. Contractors agreed that they should not be leading business requirements discussions, and that they should be responding to the client’s needs. These contractors noted that project development is facilitated when the DIT project manager has a good understanding of the client’s business unit. Contractors also said that DIT should be working to find IT solutions that fit the business rather than making the business process fit the IT solution. Contractors further explained that the process works best when the developers are able to meet with as many of the users as possible early on and when the users are heavily involved in the requirements phase. Contractors emphasized that it is critical for the developers to have a complete understanding of users’ needs before designing the IT solution. The FDIC RUP Web site includes key principles for business-driven development and notes the need to balance competing stakeholder priorities between developing an application that does exactly what the stakeholder wants, which may be costly and time intensive to develop versus leveraging a less-costly, more-timely packaged application that limits user requirements. To be in a position to balance needs, the project team must understand and prioritize business and stakeholder needs. This means capturing business processes and linking them to projects and software capabilities and involving the customer in the project to ensure the project team understands the users’ needs.

14

Ensuring Developers Understand the FDIC’s IT Environment Understanding Technical and Operational Issues. Ensuring that the development team has a complete understanding of the technical and operational environment is a key factor during the Inception phase to promote project success. FDIC RUP guidelines for the Elaboration phase include confirming that infrastructure impacts, such as bandwidth, storage, etc., have been identified and communicated. Comments from project managers we interviewed centered on the importance of the contractor’s understanding of the FDIC’s technical and operational environment, including hardware, software, and requirements analysis. On the two projects in our sample that were described as proceeding very well by officials whom we interviewed, project managers stated that the contractors’ understanding of the FDIC’s technical and operational environment greatly facilitated the project. On three other projects in our sample that experienced unanticipated challenges, FDIC project managers specifically stated that the contractors’ lack of understanding of the FDIC’s environment was a primary reason for the difficulties encountered. On those projects, contractors did not clearly understand security requirements, age of the FDIC’s laptop computers, firewall limitations, data migration technology, and bandwidth limitations. As a result, unexpected issues arose during system development testing in the Construction phase that caused milestone delays while the project team developed strategies to mitigate the issues. Developing Systems Within the FDIC’s IT Environment. Another common element of these three projects is that, in each case, the contractor developed the software outside of the FDIC’s IT environment. Significant deficiencies were identified when the system was tested in the FDIC IT environment. On each of these projects, the solutions that the contractors employed would succeed in an ideal working environment. However, because much of the FDIC’s work is conducted in remote locations, requires enhanced security, or involves other limitations, the developed software was incompatible with the FDIC’s current technology. FDIC project managers told us that, in hindsight, they would have ensured that the contract required the contractor to develop the IT project within the FDIC’s environment. Other IT projects where development was conducted within the FDIC’s IT environment did not have deficiencies to the extent to which those projects developed outside the FDIC did. The RUP Elaboration phase milestone review guidelines suggest understanding whether the development tools contemplated for the project have been used before at the FDIC. However, we did not identify any explicit FDIC guidance that IT projects should be developed within the FDIC’s IT environment. Managing Collaboration and Communication Managing communication is a critical aspect of IT project management. Obtaining buy-in and representation from all participants in the project’s development, marketing the benefits or reasons for the project, and managing the expectations of end users all are relevant to ensuring a successful project. Much of the feedback we received from both FDIC and contractor personnel regarding factors that facilitate IT project management centered on collaboration and communication among the FDIC project team, contractor, and IOs.

15

Intersecting Organizations

• Configuration Management • Corporate Management Control • Development Support and Monitoring

Section • Enterprise Architecture • Enterprise Information Management • Independent Test Section • Information Security and Privacy Staff • Infrastructure Services Branch • Peer Estimation Group • Program Management Office • Quality Assurance • Release Management

Holding Regular Team Meetings. For each of the projects in our sample, we asked project managers and testing team members what they felt had facilitated their respective IT projects, and if the project started over, what they would have done differently. One commonly stated factor was the level of communication among the project team, contractor, and IOs. Specifically, holding regular meetings between the contractor and business users throughout the project and early communication between the contractor and IOs were often cited. FDIC and contractor recommendations centered on holding regular program meetings early in the project with FDIC IOs, involving users in meetings early in the project and as much as possible, and holding regular discussions with FDIC officials and contractors involved in the project development. Officials we interviewed said that it is important for the testing team to be fully informed as to the time that will be required of them before they join the project. We were told that it greatly facilitated the project when everyone was consistently available during meetings and users made a dedicated commitment to participate in all meetings and discussions. One project manager told us that one of the things he would have done differently would be to press for a larger travel budget just to have everyone together in the same room because face-to-face meetings were always the most beneficial. A key principle discussed on the RUP Web site is collaborating across teams. This principle stresses the importance of fostering optimal project-wide communication achieved through proper team organization and effective collaborative environments. This principle involves motivating individuals on the team to perform at their best and creating self-managed, cross-functional teams (e.g., analysts, developers, testers) with the authority to decide on issues directly influencing their work. Cross-functional collaboration helps to break down the walls that often exist among analysts, developers, and testers. Each team member needs to understand and buy in to the mission and vision of the project. Involving Intersecting Organizations Early. Contractors told us that coordinating with the IOs and ensuring full understanding of the business function and relationships between the business unit and IT organization is a critical element in IT project management. Contractors said that DIT’s work to facilitate coordination with the IOs throughout the project greatly improved their ability to keep the project on schedule. In other cases, where projects experienced challenges, FDIC project managers told us that the IT contractor should have reached out to IOs earlier in the project. FDIC guidance related to each of the RUP phases discusses the importance of engaging the IOs well in advance of milestone reviews in order to understand and adhere to all IO requirements. Practicing Agile Coordination. One aspect of the RUP and Agile methodologies is for the project team to ensure that those participating in the project’s development are motivated and dedicated to the long-term IT project. Officials we interviewed told us that frequent meetings that apprised them of the project status and upcoming activities helped them to engage and understand how their involvement fit into the bigger picture. Testers provided positive

16

comments consistently when they understood the testing process and felt they were a key part of the solution. On one project that was behind schedule, the testers we interviewed did not have a good understanding of the testing process and did not seem enthusiastic about the IT project. Projects that involved heavy user participation also reported fewer challenges that negatively affected the projects’ schedule. On two of the projects included in our evaluation, we were told that divisional staff participated extensively in project testing, including writing their own test scripts, deciding the test schedule, and following up on results of user acceptance testing. The contractors on these projects specifically stated that the substantial participation of the user groups greatly facilitated the project. Conversely, FDIC project managers told us that when the project met challenges, these were often due to the lack of consistent communication between the contractors and sponsoring division personnel or DIT IOs, or both. Both contractor and FDIC personnel stated that early communication with the IOs is especially important under the Agile methodology. Collaboration and communication across teams is a key principle of the RUP and Agile methodology so when communication is lacking, the effectiveness of the development methodology is likely to be adversely affected. Implementing an Effective Milestone Review Process Ensuring Projects Meet All Milestone Review Requirements Before Moving to the Next RUP Phase. The RUP framework requires that milestone reviews be conducted prior to the IT project moving to the next project phase. Milestone reviews are of special importance because the FDIC includes a large number of IOs that need visibility into key aspects of IT projects to make sure that the concerns they represent are addressed adequately. All IOs are invited to milestone reviews that determine whether the project should be allowed to move forward to the next phase. Most of the officials we interviewed reported that they were satisfied with the overall milestone review process. However, we received a number of comments that the milestone review process could be enhanced to be more effective and facilitate the project transition from one RUP phase to the next. FDIC RUP guidelines for conducting milestone reviews at each of the RUP phases include specific discussion points pertaining to accomplishments; risks mitigated, accepted, and outstanding; preparedness to continue to the next RUP phase; deployment strategy (for the Transition phase); and concurrence. The RUP guidelines also include suggested clarifying questions that reviewers may wish to cover during milestone reviews. Figure 2 presents rules of engagement for conducting milestone reviews.

17

Figure 2: Milestone Review Meeting Rules of Engagement

Source: DIT Web site, Milestone Review Playbook.

As shown, the guidelines state that the project manager is responsible for ensuring that minutes are kept and circulated for approval following the milestone review. However, minutes were not available on any of the projects included in our sample. The only documentation DIT provided were PowerPoint presentation slides from the milestone review meetings. Therefore, we could not evaluate the depth of questions and answers discussed during any of the milestone reviews for the projects included in our evaluation. Circulation of minutes to milestone review participants helps to reinforce meeting expectations and action items due and to confirm meeting discussion points. Improving the Intersecting Organization Approval Process. Project managers told us that one of the most frustrating aspects of the milestone review is the IO approval process. Project managers begin preparing for the milestone review about 1 month prior to the formal meeting. Project managers hold meetings with the IO managers during which they discuss the RUP phase and project accomplishments and provide documentation supporting that the phase has been completed. The IOs then indicate their approval via e-mail or signature that the project may move to the next RUP phase. A few of the project managers we spoke with said that the procedure for obtaining approval from the IOs needs to be revised because it often requires repeated attempts to get the IOs to respond. FDIC RUP guidance on the Transition phase milestone review provides questions related to coordination with IOs, including which IOs need to be involved in system deployment readiness, level of IO participation, whether the project team has shared information with IOs, and whether IOs are fully informed of the system deployment and concur with moving forward. In addition, many of the project managers we spoke with felt that the IOs should be required to attend the milestone review meeting. Although the guidelines for milestone reviews indicate that all IOs are invited to the milestone review meeting, we heard from multiple project managers that often there is not a representative from all IOs at milestone review meetings. For example, the Inception phase of product development includes defining project scope and identifying technical challenges. Officials we interviewed on one of the projects in our sample stated that, in hindsight, they considered both the Inception and Elaboration milestone reviews as being rushed. They felt that the project should not have been permitted to move from the Inception to the Elaboration phase because there had not been adequate discussion and interaction by the

The agenda is provided in advance and at the meeting. Attendance will be recorded. The DIT Project Manager is solely responsible for running the meeting and tailoring the structure to meet

the specific needs of the project. The DIT Project Manager is responsible for ensuring that minutes will be kept, circulated for approval

afterward, and posted. No conditional approvals will be issued. Identification of action items during the meeting may require project plan revisions, and the project may not

be approved to move to the next RUP phase. In such cases, a second, reduced scope milestone review must be conducted.

Silence and non-participation equate to support and consent to proceed. Sufficient advance notice will be given to target attendees to permit appropriate participation. In the case of scheduling conflicts, target attendees may send alternates; alternates must agree to these rules

of engagement.

18

Agile development recognizes that testing is not a separate phase, but an integral part of software development.

contractor with the IOs regarding the FDIC’s IT environment. We did not note any guidance requiring IOs or IO representatives to attend milestone review meetings. Project managers also explained there were no questions during the milestone review about whether the solution would work in the FDIC’s technical environment. Other officials associated with this project stated that that they felt the Elaboration milestone review was rushed as well. They felt that more discussion and preparation with the IOs might have identified issues related to the FDIC’s technical environment prior to completing the Elaboration phase. Subsequently, this project encountered numerous challenges due to the contractor’s lack of an adequate understanding of the FDIC’s technical environment. It was suggested that greater IO participation in both the Inception and Elaboration milestone reviews may have altered the decision for the project to be approved for the next phase. We noted that the FDIC’s RUP guidelines recommend confirming that infrastructure impacts, such as bandwidth, storage, etc… have been identified and communicated. In addition, the Transition milestone review guidelines recommend discussion of whether the Physical Configuration Review (i.e., hardware, software components, operating system, configuration files) has been completed to verify that the quality assurance staging environment matches the target production environment. Making Milestone Reviews Meaningful and Comprehensive. We also received comments that the overall milestone review process should be more comprehensive. Officials we interviewed told us that the manner in which milestone reviews are conducted is geared more towards making sure the required paperwork is completed than actually ensuring that the goals of the RUP phase have been successfully completed. They further noted that while questions are raised at the milestone review meeting, participants do not sufficiently explore project details. Many managers use checklists to ensure all requirements are documented, but there is not enough done to evaluate the adequacy of the project as a whole. One project manager who was assigned to a project during the Construction phase explained that, in her opinion, the contractor was not following the RUP methodology as required because the Elaboration phase was not completed when Construction began. The project manager did not understand how the project made it through the Elaboration milestone review and said that, had the Elaboration phase been properly completed, many of the issues that plagued the project may not have occurred. As discussed earlier, the FDIC RUP Web site includes specific guidelines for discussing project accomplishments, risks, and preparedness for continuing to the next phase, to be used in milestone review meetings. Having a Well-Informed and Fully-Dedicated Testing Team The quality of user testing is an important factor that facilitates successful IT project management. This is especially critical when employing the RUP and Agile methodologies because testing occurs throughout the project lifecycle. Agile testing has been referred to as the headlights of the project because it shows the project manager where the project is and where it is headed. Testing provides information to the project manager from which critical development decisions are made so that the ability to test the software drives the IT development. FDIC personnel involved in the testing

19

Federal Challenges in Applying Agile A 2012 Government Accountability Office (GAO) report (GAO-12-681) noted several challenges that agencies face in implementing Agile. • Teams had difficulty collaborating closely. • Teams had difficulty transitioning to self-

directed work. • Staff had difficulty committing to more timely

and frequent input. • Agencies had trouble committing staff to Agile

development efforts. • Customers did not trust iterative solutions. • Teams had difficulties managing iterative

requirements. FDIC and contractor officials we interviewed expressed some of these same concerns.

process and project managers explained that the quality of the testing process significantly influences whether or not the project will meet milestones and users’ expectations. We also observed that the feedback we received from testing participants as to their level of satisfaction with the testing process matched the overall level of project success indicated to us by the project managers. Another key principle for business-driven development is focusing on continuous quality. This means that quality must be addressed throughout the project lifecycle through iterative testing. Project teams should test early and continuously throughout the system development effort. All project members should “own” quality and design the system and write code with testing in mind. Testing should be expanded as part of each software iteration and should include regression testing to make sure that defects are not introduced as new iterations add functionality. Selecting the Right Users to Test the Solution. Project managers told us that the quality of their user testing team contributed to the level of success on the project. It was critical for the testing team to include subject matter experts familiar with the program area for which the application was being developed and an adequate number of testing participants. One project manager attributed successful project completion primarily to the exceptional efforts of the testing participants. The project manager indicated that the testing participants wrote their own test scripts, decided the test schedule, and met daily during user acceptance testing. The developer was able to correct all deficiencies on schedule due to the excellent work of the testing team. A contractor on this project said organization of the testing teams facilitated the contractor’s work on the project. On other projects, the project managers told us that because of the dedicated work of the testing teams, errors were caught early in development and were quickly corrected. Training Team Members on the Agile Testing Process. Testing participants that received training on the Agile testing process prior to project testing generally had positive comments regarding the project. They explained that it is important for testers to receive training so that they understand that the testing and retesting process is normal during Agile development. In those instances when the testing process encountered problems, those individuals we interviewed indicated it was because team members participating in the testing process had not been trained on the Agile methodology. As defects were repeatedly encountered, the team members became frustrated with the overall testing process. Frustration with Agile testing is likely if testing participants do not understand how the system is being developed iteratively and are expecting a fully functional system when testing is initiated.

20

The majority of negative comments we received from testers occurred when they felt they had not been provided adequate instruction prior to the testing process. Comments from these testing participants included complaints that they had not been adequately prepared for the process, testing took too long, there were too many errors, the test scripts did not work, and they did not understand what happened to the defects they reported because nothing ever seemed to work as expected. These testers appeared to be frustrated with the process and complained that it detracted from their primary assignments. Because they had not been adequately prepared for the time requirement and did not understand the Agile testing process, these testing participants had a negative view of the IT project and were not enthusiastic about their involvement. We noted that FDIC’s RUP Web site includes information about computer-based instruction courses available through FDICLearn. RUP-related course titles include Breaking the Work Up: Iterative Development Overview and Software Requirements Specification Overview. Dedicating Team Members and Maintaining Continuity. Testers should also be adequately prepared for the time commitment required when assigned to project development testing. This is especially important under the Agile methodology because the software is modified based on users’ feedback. Those testing teams that were made aware of the time requirement and how their testing work affected the project development reported a positive testing experience. They indicated that while the testing process might become tedious at times, they understood the importance of their work and appreciated that they had the opportunity to be a part of the project development. Project managers for one of the projects in our sample told us that having a consistent team of testing participants would have greatly facilitated the project development. The ability of FDIC divisions or offices to have a designated testing team varies due to the division or office’s work requirements and resources. For example, the workload of Division of Resolutions and Receiverships (DRR) staff may fluctuate significantly due to the number of bank closings and related activities required of DRR staff. Where dedicating a consistent team of testing participants is not possible, ensuring that testing participants are fully briefed on the testing process and time requirements so that they understand the importance of their work and properly plan their involvement is a critical component of IT project management. Assigning an Independent Risk Manager to Projects Effectively managing and mitigating IT project risks is a key tenet of the RUP and Agile methodologies. Risks exist within each RUP phase that should be fully addressed before the IT project team begins work on the next phase. The project managers we interviewed agreed that when risks are not properly mitigated, especially before moving from the Elaboration phase to the Construction phase, problems are likely to arise that will delay the project. The FDIC RUP Web site discusses essential elements of RUP, including mitigating risks and tracking related issues. The Web site notes that it is essential to identify and attack the highest risk items early in the project. The risk list is intended to capture the perceived risk to the success of the project. Along with each risk should be a plan for mitigating that risk.

21

Assigning Independent Risk Managers to All Major Projects. Under the RUP methodology, the contractor’s project team should include a risk manager who tracks potential project risks and monitors the project’s progress in mitigating risks. In addition to the contractor’s Risk Manager, CMCB provides an Independent Risk Manager (IRM) for all CIRC projects and other major projects as needed. Four of the IT projects in our sample included an IRM. Project managers provided very positive feedback regarding the IRM’s participation on their projects. Officials we interviewed stated that an IRM facilitates IT project management by focusing on risk identification, providing a different perspective of risks based on factors outside the project and across many projects at once, and ensuring mitigation of risks in a timely manner. One contractor manager we spoke with indicated that having an IRM was pivotal to the success of the project and recommended that one should be assigned to IT projects estimated to last over 1 year. Others managers offered that the IRM’s ability to provide an independent perspective reduces the “finger pointing” among the client, DIT, and the contractor. Project managers agreed that it is important for the IRM to be included in all meetings and have a full view of the project. Some also felt that the IRM did not need to be an IT expert but more of a risk expert that knows what issues need to be considered that might not be evident to the project team. Ensuring IRM Concerns Are Addressed. Although officials we interviewed provided this positive feedback, we were advised that the concerns of the IRM were not always mitigated in a manner that fully addressed the intent of the risk identified. On one of the projects in our sample, even though the IRM noted many risks early on in the project, the IRM felt that the project managers worked to dismiss the concerns raised instead of working to address IRM concerns. As the IT project moved to the Construction phase of the project, it encountered numerous delays and challenges. During a lessons learned session at the completion of the project, IT managers concurred that had the concerns of the IRM been properly addressed, some of the challenges the project experienced may have been averted. Among the reasons given for two of the IT projects in our sample that experienced significant delays and challenges is that the RUP Elaboration phase objectives had not been fully completed before the project team began work on the Construction phase. As a result, significant risks were not identified prior to building the project and when they came to light, the project needed to be reworked. In hindsight, project managers believed that had the concerns of the IRM been given greater consideration, these risks may have been identified and perhaps mitigated earlier in the project development. Conclusion and Recommendation Our interviews and analysis led us to identify a number of key factors that can either make or break an IT project. Three of the six projects we sampled experienced significant delays, in part, because factors discussed in this report were not fully addressed. These same factors, however, were equally important to those projects that were completed on time, within budget, and consistent with requirements. Not surprisingly, these factors involved project planning,

IT Project Team Member Comments About IRMs “An IRM is very beneficial, especially when you have an IT project that crosses numerous divisions.” “IRMs have a different perspective from outside the project that I came to appreciate.”

22

understanding the IT environment, collaborating across teams, asking the right and difficult questions at key milestones, engaging subject matter experts, and addressing project risks early and head-on. Accordingly, we recommend that the Acting CIO: (1) advise client division and offices, IT project teams, DIT intersecting organizations, and appropriate governance bodies of the key factors in project success or challenges and related controls we identified in this report and (2) determine whether guidance in any of these areas needs to be strengthened. The most notable factors and issues include:

Considering all available IT solutions during the Inception phase; Documenting, assessing, and communicating the complexity of a proposed IT

solution to appropriate parties to ensure that contractor resources and milestones are commensurate with requirements;

Ensuring the development team completely understands the FDIC’s technical and

operational IT environment, and development occurs within that environment; Ensuring consistent collaboration among all those involved in the project and that

contractors communicate and coordinate with the FDIC’s IOs early and often; Facilitating IO approval for projects to move to the next RUP phase and their

participation in milestone review meetings; Ensuring milestone reviews fully explore the adequacy of the work performed and

that all risks are properly mitigated prior to RUP milestone approval, including those identified by the IRM;

Establishing dedicated IT project testing teams that are fully briefed on the testing

process and anticipated timeframes; and Ensuring there is an awareness of the Agile approach to system development and its

impact on implementing and measuring the progress and value of IT projects.

Corporation Comments and OIG Evaluation The Acting CIO provided a written response, dated June 25, 2014 to a draft of this report. The response is presented in its entirety in Appendix 5. In the response, the Acting CIO concurred with the report’s recommendation and described completed and planned corrective actions to address the recommendation. A summary of the Corporation’s corrective actions is presented in Appendix 6. The completed or planned actions are responsive to the recommendation, and the recommendation is resolved.

Appendix 1

Objective, Scope, and Methodology

23

Objective The objective of the evaluation was to (1) assess the extent to which the FDIC’s IT projects are meeting their cost, schedule, and requirements expectations; (2) identify factors that promote project success or prevent projects from meeting expectations; and (3) identify opportunities for strengthening the FDIC’s controls for monitoring IT projects. We conducted this evaluation from April 2013 through February 2014 in accordance with the Council of the Inspectors General on Integrity and Efficiency’s Quality Standards for Inspection and Evaluation. We performed our evaluation work at the FDIC’s offices in Arlington, Virginia and Dallas, Texas. Scope and Methodology To address our evaluation objectives, we first gained an understanding of the FDIC’s IT project management governance structure and processes, including internal controls for monitoring and reporting on the FDIC’s IT projects by reviewing relevant policies and procedures; interviewing DIT officials, program office officials, and members of the CIO Council. We also observed a number of CIO Council meetings to understand how the statuses of on-going FDIC IT projects are reviewed. Our evaluation objectives did not require that we evaluate whether the FDIC’s IT project management controls were properly designed or require that we gain an understanding or test information system controls. Further, our evaluation objectives did not require that we specifically test the implementation of internal controls or effectiveness of controls except to the extent we considered the effectiveness or implementation of controls in assessing factors that promote project success or prevent projects from meeting expectations. As explained below, we reviewed documentation for a sample of IT projects to understand the extent of documentation and not to specifically test compliance with policies and procedures and other controls. Figure 3: Summary of FDIC Policies and Procedures and IT Governance Documents

Source: OIG analysis of FDIC directives, DIT policy, and IT governance documents.

FDIC Directives and DIT Policy FDIC IT Governance Documents

FDIC Capital Investment Policy, dated September 23, 2011. FDIC Circular 1303.1, FDIC Enterprise Architecture Program, dated June 16, 2008 Policy 07-005 Systems Development Life Cycle (SDLC), dated June 15, 2007. Policy 09-004 Information Technology Project Management (Project Management Office), dated December 28, 2009. Policy 09-006 DIT Earned Value Management (EVM), dated May 1, 2009.

CIO Council Governance Guidelines, adopted revision on November 15, 2012. CIO Council Charter, revised and adopted on September 6, 2012. Charter of the Capital Investment Review Committee (CIRC), revised and adopted on October 2011. Charter for the FDIC Financial Analysis Committee (FAC), adopted on May 2007. Charter of the DIT Project Initiation Review Committee (PIRC), effective February 26, 2006.

Appendix 1

24

To assess the extent to which the FDIC’s IT projects are meeting their cost, schedule, and requirements expectations, we obtained CIO Council and CIRC reports on IT projects completed or in-process during 2012 and 2013. We focused on that defined period because it provided us a sufficient population from which to evaluate current IT project management practices and processes. To identify factors that contributed to a project’s success or difficulties, we took a case study approach. To that end, we judgmentally selected six projects for review from FDIC’s inventory of IT projects completed or in-process as of December 31, 2012. The results of a non-statistical sample cannot be projected to the intended population by standard statistical methods. In selecting projects, we included both CIRC and CIO Council projects and projects from a cross-section of FDIC divisions and offices. We also took into consideration factors such as whether the project management method employed the Agile methodology; estimated cost; the contractor engaged to work on the project; and the current project status, including the extent of any known problems or positive attributes. Table 5 summarizes key information about each of the projects in our sample, including our reason for selecting the project. Table 5: Summary of Projects Sampled

IT Project Name Project Status

Actual or Projected Total Cost

Division Sponsor Reason for Selection

Advanced Legal Information System (ALIS)

Construction $4.7 million Legal Division Legal Division project

TechStat performed Assessment Information Management System (AIMS)

Completed with update in process

$7.8 million DOF DOF project Strong user

acceptance testing Claims Administration System (CAS) 2.0

Construction $3.6 million DRR DRR project High Profile Issues reported in

prior versions Examination Tools Suite- Supervisory Application Generating Exams (ETS-SAGE)

Construction $35 million Division of Risk

Management Supervision

(RMS)

CIRC Portfolio RMS Project Agile methodology

used

Identity Access Management System (IAMS)

Completed $950,000 DIT Complex project Cost overruns and

milestone delays PROFORMA Construction $2.7 million DRR Bank Closing Tool

Agile methodology used

Project management issues reported

Source: OIG analysis of DIT status reports and Projects at a Glance.

For the IT projects selected, we performed the following procedures.

Reviewed RUP documentation to understand the purpose of the project and assess how it was being managed relative to an approved project proposal and the FDIC’s overall IT

Appendix 1

25

project management framework. RUP documentation reviewed included project plans, configuration architecture, measurement analysis plans, software architecture plans, iteration assessments, milestone review presentations, budget proposal outlines, risk lists, schedule status reports, quality control strategies, risk management plans, system security plans, requirements vision, and master test plans. We did not assess whether the applications were adequately designed or specifically review RUP documentation to test whether development policies, procedures, and guidance had been properly implemented.

Interviewed personnel in RMS, DRR, DOF, DIT, Division of Administration, and the

Legal Division who were responsible for IT project management, risk management, and user testing to obtain their perspectives on project management, development practices, testing practices, and user expectations. We also interviewed FDIC contractor personnel who serve as project managers and DIT personnel in the Delivery Management Branch and Program Management Office. In these interviews, we solicited individual views about the factors that contributed to the project’s success or prevented the project from meeting expectations and ideas for strengthening the FDIC’s controls for monitoring IT projects, and analyzed responses to identify common themes or outlier comments that warranted follow-up.

To help us identify opportunities for strengthening the FDIC’s controls for monitoring IT projects, in addition to discussions with FDIC personnel and FDIC contractor personnel, we reviewed industry guidance. Specifically, we reviewed the following:

Intel Information Systems Audit and Control Association (ISACA) publication,

COBIT 5: A Business Framework for the Governance and Management of Enterprise IT, dated 2012. This publication contains five basic principles for governing and managing enterprise IT.

Executive Office of the President of the United States, 25 Point Implementation Plan to Reform Federal Information Technology Management, date December 9, 2010. This plan covers the structural areas that impact the success rates of large IT programs across government.

Global Technology Audit Guide (GTAG) 12: Auditing IT Projects, dated March 2009.

This GTAG provides internal auditors with an overview of techniques for effectively engaging with project teams and project management offices (PMOs) to assess the risks related to IT projects.

Global Technology Audit Guide (GTAG) 17: Auditing IT Governance, dated July 2012.

This GTAG covers aspects of governance that should be in place to ensure IT supports the strategies and objectives of the organization, describes elements of effective governance and performance frameworks, and describes example controls that address IT governance risks.

Appendix 2

Glossary

26

Term Definition Agile

A group of software development methods based on iterative and incremental development, where requirements and solutions evolve through collaboration between self-organizing, cross-functional teams.

Basic Ordering Agreement A written instrument of understanding negotiated between the FDIC and a contractor for future delivery of as yet unspecified quantities of goods or services. A BOA becomes a binding contract when a task order is issued.

Business Outcome Index

An index developed by the FDIC PMO to summarize the current state of DIT’s ability to deliver business outcomes with CIO Council projects.

Business Proposal Outline A template tool used by the CIO Council for making informed decisions related to the selection of FDIC CIO Council IT projects on a yearly basis.

Deployment Plan A deployment plan defines the sequence of operations or steps that must be carried out to deliver changes into a target system environment.

Dodd-Frank Wall Street Reform and Consumer Protection Act (Dodd-Frank Act)

The Dodd-Frank Act (Public Law No. 111-203) enacted July 21, 2010, contains many provisions affecting the FDIC and its regulatory authorities over banks and the financial services industry. Certain of those provisions affect how the FDIC calculates assessments on insured depository institutions.

Intersecting Organizations FDIC Intersecting Organizations are DIT and other FDIC groups that projects interact with during a project's lifecycle.

Project Proposal Outline A governance document that documents high-level elements of a contemplated project in order to support decisions on funding and timing. The Project Proposal Outline provides a good basis for development of a project's vision.

Rational Unified Process® (RUP)

A comprehensive process framework that provides industry-tested practices for software and systems delivery and implementation and for effective project management. RUP® is the standard systems development methodology used by DIT for the IT projects it manages.

Risk List A document maintained by the Risk Manager of potential IT development risks to be addressed. The list includes mitigating tasks to be completed for each identified risk.

Software Development Plan

The Software Development Plan is a comprehensive, composite document that gathers all information required to manage the project. It includes a number of documents developed during the Inception phase and is maintained throughout the project.

System Development Life Cycle (SDLC)

The overall process of developing, implementing, and retiring information systems through a multistep process from initiation, analysis, design, implementation, and maintenance to disposal.

Technical Evaluation Panel

A panel of FDIC employees established to evaluate the written proposals for compliance with the solicitation’s technical requirements and the evaluation criteria established in the solicitation for formal contracting.

TechStat A TechStat is an evidence-based review of an IT investment typically requested by the governing IT committee when a project is underperforming. It is based on a model developed by OMB.

Appendix 3

Acronyms and Abbreviations

27

Acronym / Abbreviation Explanation AIMS Assessment Information Management System ALIS Advanced Legal Information System BOI Business Outcome Index CAS Claims Administration System CIO Chief Information Officer CIRC Capital Investment Review Committee CM Corporate Management Council CMCB Corporate Management Control Branch DIT Division of Information Technology DOF Division of Finance DRR Division of Resolutions and Receiverships ETS-ALERT Examination Tools Suite - Automated Loan Examination Review Tool ETS-SAGE Examination Tools Suite - Supervisory Application Generating Exams GAO Government Accountability Office IAMS Identity Access Management System IO Intersecting Organization IRM Independent Risk Manager IT Information Technology OIG Office of Inspector General OMB Office of Management and Budget PIR Project Initiation Review PMO Program Management Office PROFORMA Proforma Modernization RMS Division of Risk Management Supervision RUP Rational Unified Process® SDLC System Development Life Cycle

Appendix 4

Summaries of IT Projects Included in the Evaluation

28

Advanced Legal Information System (ALIS) ALIS will be the key system the FDIC’s Legal Division uses to manage matters and invoices from outside counsel firms and legal support services contractors. ALIS will replace the current Legal Integrated Management System by upgrading the foundational software from Corporate Legal Desktop to Passport. The contractor will externally host Passport. The CIO Council ordered a TechStat assessment in July 2012, as the ALIS project was significantly behind in milestone schedules and over budget. The TechStat reported that ALIS was originally planned to be completed by first quarter 2012 at a cost of approximately $1.7 million. As of the TechStat report date of August 2012, ALIS was scheduled to go live on February 4, 2013 at a total cost of approximately $3.4 million. The TechStat assessment concluded ALIS was unlikely to meet the schedule and budget estimates presented to the CIO Council based on past performance and the current challenges. The TechStat reported that:

ALIS was a major overhaul and not an upgrade. Inconsistent understanding of “complete” existed between the FDIC and the contractor. Insufficient project detail existed to reliably predict outcome. Regular development and configuration issues existed. Ineffective communication occurred between stakeholders, DIT, and the contractor. Unresolved contractual disputes existed.

The TechStat Assessment suggested that the Legal Division and DIT consider taking a number of corrective actions, including:

Ensuring alignment of personnel skills with current challenges;

Simplifying and streamlining coordination-related activities;

Conducting all-hands-on-deck workshops to state, clarify, and confirm requirements for DIT, Legal, and the contractor; agree on criteria for determining “complete;” identify and address remaining risks; and implement an effective communication strategy;

Developing concrete Go/No Go criteria;

Conducting a thorough Market Analysis, if called for;

Performing a Cost Benefit Analysis, if called for;

Appendix 4

29

Ensuring the charters and associated responsibilities of the existing governance bodies were being met; and

Inviting additional expertise to governance boards, including the CFO and a DOF Deputy Director.

ALIS ultimately was completed on August 13, 2013 at a total cost of $4,706,590. At the project close-out briefing on October 24, 2013, DIT and the Legal Division presented lessons learned to the CIO Council. Lessons learned included that:

Detailed analysis should have been performed to assess project characteristics beginning with project planning and continuing through the first two RUP phases.

ALIS was initially categorized as an upgrade of commercial off-the-shelf software, which

did not account for the data conversion and migration activities combined with the custom development of 27 interfaces.

The project team should have immediately and appropriately addressed concerns of the

Independent Risk Manager. The FDIC Contracting Officer should have been involved when issues such as contractor

staff turnover and delivery of quality products with contractors first arose. The Data Manager should have spearheaded data scrubbing activities earlier which may

have resulted in the timely completion of data migration and fewer reported defects once ALIS went “live.”

Assessment Information Management System (AIMS) AIMS enables the FDIC to comply with statutes mandating the FDIC to assess and invoice financial institutions for deposit insurance premiums that provide the income for the Deposit Insurance Fund. Using AIMS, FDIC manages the assessments process by performing operations throughout the year in support of the quarterly assessment cycle and other special assessment cycles. The assessments must be 100 percent accurate and delivered on time, every time. Changes mandated by the Dodd-Frank Act and FDIC regulations that became effective in 2011 directly impacted the FDIC Assessment Program and required unanticipated modifications to the AIMS system to address technology obsolescence risks and to revise the AIMS method for calculating assessments. The end results of AIMS development met the DOF user, stakeholder, and manager expectations because the stakeholders or users were involved in daily meetings during AIMS development and design and reviewed and performed test scripts. A key to AIMS’ success was that there was 100-percent staff involvement from DOF’s Assessments Branch. The staff wrote their own test

Appendix 4

30

scripts, decided the test schedule, and met daily during user acceptance testing. Defects found during testing were corrected before the release. DOF personnel handled the Testing Phase with very little support from the contractor or DIT. Another success factor was that DOF and DIT ensured that the contractor maintained the same IT project management team throughout a number of releases. The most challenging aspect of AIMS development was the contractor coordinating work with the FDIC’s IOs involved in the project. Because one of the IOs disagreed with the AIMS systems development approach (total rewrite versus enhancement), this IO refused to sign off on milestone reviews. AIMS was never provided the resources to perform a total rewrite because project performance had to meet the criteria and deadlines in the Dodd-Frank Act legislation.

Claims Administration System (CAS) The purpose of CAS is to have a flexible process that will support deposit claims datasets from both large and small financial institutions, decrease the amount of manual work done by Claims Agents and the Business Information Systems staff, and enable the FDIC to handle the closing of a financial institution of any size. DRR uses CAS to determine deposit insurance amounts and to process deposit insurance claims when a financial institution fails. CAS 2.0 is a technology upgrade to implement systems changes that will lead to improvements in the maintainability and stability of the CAS application, reduce DIT maintenance costs, and provide for a more efficient and intuitive user interface. CAS 2.0 development met milestones and expectations due to the excellent communication and collaboration between DRR, DIT, and the contractor. Other reasons for CAS development success included: DRR designating a team of users to be testers on the system throughout development; priority and support from client organization management; and an experienced DIT project manager. Challenges experienced were largely due to contractor staff turnover. Stakeholders generally reported that CAS development went well and that CAS met expectations. Examination Tools Suite-Supervisory Application Generating Exams (ETS-SAGE) The objective of the ETS IT project is to replace prior RMS examination reporting tools and transmittal forms and revitalize and simplify the examination process for both examiners and reviewers. The program introduces wireless onsite networks that will enhance security and accuracy of shared examination data in the bank. ETS was developed by examination staff for use by examination staff. ETS is expected to: (1) increase efficiency and reduce maintenance expenses by reducing technical complexity and by reducing the number of systems RMS examiners must use to perform their jobs; (2) eliminate data redundancy and duplicative data entry; (3) improve ease of access, reporting, and data accuracy by improving on and eliminating the examiner download process, enhancing the Report of Examination review process, and improving RMS’ Automated Loan Examination Review Tool’s (ALERT) import and mapping processes; (4) reduce risk to and improve the security of examination data; and (5) reduce risk from technological obsolescence.

Appendix 4

31

During our review, ETS SAGE was meeting users’ expectations. Because the project team developed ETS-SAGE after ETS-ALERT development, they had the benefit of lessons learned from the ETS-ALERT’s Inception and Elaboration phases. The FDIC changed contractors after ETS-ALERT’s Inception phase. This change challenged the replacement contractor team as there was insufficient time for the knowledge transfer between the original and replacement contractor teams. The stakeholders and managers all reported that the Agile software development methodology was well-suited for this type of project and that it was critical to the project’s success. DIT’s PMO personnel provided valuable support to the project and acceptance of the Agile methodology. During our evaluation, the SAGE and ALERT projects were combined. We were informed after our interviews on the ETS-SAGE project that the contractor was not aware of the complexity of the project and DIT had some difficulties preparing the testing environments. Also, it had been determined that although the IT solution worked when the testers were in an ideal environment, the solution would not work in remote locations where the solution is most needed. The project team discovered that the FDIC’s current technology does not support the IT solution developed by the contractor. As a result, the project has experienced several delays and will require a contract extension to be completed. Identity Access Management System (IAMS) IAMS was implemented to manage FDIC IT user identities, workflow-based access requests, and the systems access approval process. IAMS is a streamlined end-to-end integrated process that captures all steps of FDIC access control from the initial entry of a new FDIC employee or FDIC contractor to the time of their departure. This process ensures that accounts are set up with the proper levels of security for all users. All corporate applications supported by DIT must be tracked through IAMS. The IAMS application had a number of releases for each calendar year of our review. These releases addressed defects and include enhancements and corrective measures that will help improve the application’s efficiency and usability. Environment testing improved from beginning to end. IAMS development improved during the past year as compared to previous year’s progress. FDIC attributed the improved process to the contractor’s latest assigned IT Project Manager. During the early development stages of IAMS, the IT Project Managers assigned by the contractor experienced turnover which significantly impacted the progress and IAMS continuity. When the contractor assigned a highly skilled and engaged IT Project Manager, IAMS development improved significantly. As the IAMS environment is complex, DIT ensured that the contractor’s test environment matched the FDIC’s IAMS production environment. This was another significant challenge early in IAMS development, but this situation improved over time and is much better now. Although the contractor developed IAMS software outside of the FDIC environment, the contractor designed IAMS to exactly match the FDIC’s operational environment. Another challenge was the CIO Council budget process. Because the project was so complex and involved many intersecting systems and organizations, there were many unanticipated

Appendix 4

32

improvements required during the Inception and Elaboration phases. However, the budget dictated the amount of contractor time allocated to the project phase so the IAMS project kept going over budget because of the additional requirements.5 Proforma Modernization (PROFORMA) PROFORMA is a bank closing tool that brings failed bank financial statements into DRR's accounting system. It imports the general ledger from a failing financial institution, allows financial analysts to make final adjustments, and converts the information into a standardized accounting system. PROFORMA is also used to print reports and statements used to create the Inception balance entries for the receivership and initial starting balances for the assuming institution(s). PROFORMA is significantly behind schedule, as its development has significant environmental challenges to overcome. The original project transition date of October 2012 was revised to May 24, 2013 and extended to May 31, 2014. The project was built and initially tested by the developers at an off-site location. When the system was tested in the FDIC’s technical environment, significant challenges became apparent especially when operating at a failed bank’s off-site location. Technical issues involving the FDIC’s hardware, DIT’s testing capacities, and limitations of bandwidth availability at failed bank locations were not communicated to the developers prior to construction of the IT solution. As a result, additional costs have been incurred to upgrade the FDIC’s hardware and solutions are required to mitigate these challenges.

5 Although the project was over budget from a time perspective, because the project involved a firm fixed price contract, the project was within its cost budget.

Appendix 5

Corporation Comments

33

Federal Deposit Insurance Corporation

3501 Fairfax Drive, Arlington, VA 22226-3500 Chief Information Officer

DATE: June 25, 2014

TO: Stephen M. Beard Deputy Inspector General for Audits and Evaluations

FROM: Martin Henning /Signed/ Acting Chief Information Officer

SUBJECT: Management Response to the Draft Audit Report Entitled, The FDIC’s Information Technology Project Management Process (Assignment No. 2013-013)

Thank you for the opportunity to comment on the Office of Inspector General’s (OIG) draft report on FDIC’s information technology project management processes issued May 14, 2014. In its report, the OIG made one recommendation to the Acting Chief Information Officer (CIO), the CIO agrees with the recommendation, and actions to address the recommendation are planned or underway. Our specific response to the recommendation is provided below.

MANAGEMENT RESPONSE Recommendation 1 The OIG recommended that the Acting CIO:

“(1) advise client divisions and offices, IT project teams, DIT intersecting organizations, and appropriate governance bodies of the key factors in project success or challenges and related controls we identified in this report and (2) determine whether guidance in any of these areas needs to be strengthened. ”

Management Decision: Concur Corrective Action: The CIO believes that although due diligence is being exercised in advising stakeholders of the key factors and supporting guidance, both could be enhanced. The CIO agrees with the OIG that these key factors warrant a review to validate that stakeholders are properly advised and that current guidance is appropriate.

.

Appendix 5

34

Appendix 6

Summary of the Corporation’s Corrective Actions

35

This table presents corrective actions taken or planned by the Corporation in response to the recommendation in the report and the status of the recommendation as of the date of report issuance.

Rec. No.

Corrective Action: Taken or Planned

Expected Completion

Date

Monetary Benefits

Resolved:a Yes or No

Open or Closedb

1 Brief client divisions and offices, IT project teams, DIT intersecting organizations, and appropriate governance bodies regarding the key factors in project success, or challenges and related controls identified in the report and determine whether current guidance needs to be strengthened.

11/07/2014 $0

Yes Open

a Resolved – (1) Management concurs with the recommendation, and the planned, ongoing, and completed corrective action is consistent with the recommendation.

(2) Management does not concur with the recommendation, but alternative action meets the Intent of the recommendation. (3) Management agrees to the OIG monetary benefits, or a different amount, or no ($0) amount. Monetary benefits are considered resolved as long as management provides an amount.

b Recommendations will be closed when (a) Corporate Management Control notifies the OIG that corrective actions are complete or (b) in the case of recommendations that the OIG determines to be particularly significant, when the OIG confirms that corrective actions have been completed and are responsive.