Dana McLean - ocio.wa.gov

Public Consulting Group, Inc. Monthly Status Report Report Date: July 18, 2019

Washington State Department of Transportation (WSDOT) Customer Service Center (CSC) Back Office System (BOS)

2405 Evergreen Park Drive, SW | Olympia, Washington 98502 | P: (916) 565-8090 | F: (916) 637-8802

Date: July 18, 2019 Ms. Patty Rubstello, CSC BOS Sponsor Washington State Department of Transportation (WSDOT) Customer Service Center (CSC) Back Office System (BOS) Project RE: WSDOT CSC BOS Project Quality Assurance Report – as of July 15, 2019 (Contract #K1118, Engagement

#P0046834)

Dear Ms. Rubstello: PCG’s assessment of the CSC BOS Project is based on the professional experience and judgment of its consulting team. This report was prepared independent of project participants and stakeholders, other than inclusion of responses to QA findings. Thank you for the opportunity to provide QA services to this project. Please let me know if you have any questions or comments. Sincerely,

Dana McLean (Gerow)

Dana McLean

Director, Washington State Team Public Consulting Group | PCG Technology Consulting 916-709-3262 cell 2405 Evergreen Park Dr. SW Olympia, WA 98502 PublicConsultingGroup.com




Washington State Department of Transportation (WSDOT)

Customer Service Center (CSC) Back

Office Support (BOS) Project

Monthly Quality Assurance (QA)

Status Report

V1.0

Reporting Period: June 16, 2019 – July 15, 2019 Report Date: June 21, 2019 Contract #K1118 Engagement # P0046834




Document History

Version Date Brief Description of Modifications

1.0 07/18/2019 Initial submission of the Monthly QA Status Report

Document Reviewer(s)

Name Title

Jennifer Charlebois CSC BOS Project Manager

Matt Ringstad CSC BOS Project Manager

Document Receiver(s)

Name Title Acceptance Status Date

Jennifer Charlebois CSC BOS Project Manager

Matt Ringstad CSC BOS Project Manager




TABLE OF CONTENTS

1 Executive Summary ................................................................................................................................................................................ 1

2 Reference Documents ............................................................................................................................................................................ 4

3 Glossary .................................................................................................................................................................................................. 6

4 Scope....................................................................................................................................................................................................... 7

5 Approach ................................................................................................................................................................................................. 7

6 Assessment ............................................................................................................................................................................................ 7

6.1 Project Integration Management ................................................................................................................8

6.2 Project Scope Management .......................................................................................................................9

6.3 Project Schedule Management ..................................................................................................................9

6.4 Project Cost Management ....................................................................................................................... 11

6.5 Project Quality Management ................................................................................................................... 12

6.6 Project Human Resource Management .................................................................................................. 13

6.7 Project Communications Management .................................................................................................... 13

6.8 Project Risk Management........................................................................................................................ 14

6.9 Project Procurement Management .......................................................................................................... 15

6.10 Project Stakeholder Management ........................................................................................................... 15

Finding Status Code Definitions ...................................................................................................................................... 17

Finding Priority Definitions .............................................................................................................................................. 18

C.1 Risks ........................................................................................................................................................ 18

C.2 Issues....................................................................................................................................................... 19

TABLE OF TABLES

Table 1: Project Trending Status – Six Month View ...................................................................................................2

Table 4: Risk Priority Definitions .............................................................................................................................. 19

Table 5: Issue Priority Definitions ............................................................................................................................ 19




1

1 Executive Summary

Following is a summary of the monthly Quality Assurance (QA) risk assessment reporting conducted by the Public Consulting Group (PCG) QA team for the Washington State Customer Service Center (CSC) Back Office Support (BOS) Project for the period of June 16, 2019 through July 15, 2019.

Project Status/Summary during the Reporting Period:

The overall risk status of the project during this reporting period remains high. WSDOT has engaged with their vendor ETCC to move forward with the tolling of SR99. Taking this information into account, WSDOT and ETAN are in the process of reassessing the viability of the go-live date are establishing a new date of late 2019/early 2020. This would bring the project schedule back into alignment with the original intent of the WSDOT / ETAN contract, including allowing sufficient time for deliverable reviews and a cadence that allows for development to happen in synch with the project schedule. WSDOT and ETAN have defined interim milestones to ensure that ETAN can remain on track to deliver on time and within scope, such as: completion of development, test procedure and ICT close out, completion of all of the training materials, end to end testing and a dry run for UAT. PCG and WSDOT will continue to monitor items on the critical path to ensure they are progressing as planned within the new schedule; including: the completion of training materials and activities, system testing, data migration, infrastructure readiness, test execution and end to end testing.

WSDOT continues to proactively manage risks and has been diligently taking into account the quality of the system being built. WSDOT and ETAN executive team have determined downstream impacts on the legacy system vendor (i.e., ETCC) and managing communications regarding the new schedule and impacts to cost and resources. Additionally, WSDOT has created a project schedule to define the tasks and critical path with ETCC now that the project has moved forward with ETCC for SR99 tolling.

During this reporting period PCG made the following changes:

• Cost Management – PCG opened QA41 as recent changes have required WSDOT go

through the “Gated Funding” process, which includes the creation of a new Technology

Budget and the requirement to go through the gated approval process. These changes

impact the CSC project regarding ensuring the project’s funding and the ability of the

project to process payments.

The table on the next page is a 6-month view of the trend in status across each assessment area. Included with the table is a legend to help understand the colors and arrows included in the table.

HIGH




2

Table 1: Project Trending Status – Six Month View

QA Status Assessment Legend

Green - Poses the least risk to

project success and can generally be considered near-free of risks to quality, scope, cost and/or schedule.

Yellow – Poses a moderate risk to project success with some products or processes that present risk resulting in deficient quality, scope, cost or schedule.

Red – Poses significant risk to project success with respect to the project quality, scope, cost, and/or schedule and should be given immediate attention.

To easily track status changes between the current and former reporting periods, the following symbols will be added to the priority status rating.

Indicates the risk status has increased (worsened) since the last reporting period.

Indicates the status has been decreased (in severity) since the last reporting period.

Indicates the status has not changed since the last reporting period.

Assessment

Category

WSDOT CSC Trending Status

Feb 2019 Mar 2019 April 2019 May 2019 June 2019 July 2019 7/15/2019

Integration

Management

QA27, QA40

Scope

Management

QA26

Schedule

Management

QA28, QA36

Cost Management

QA25, QA41

Quality

Management

QA22

Human Resource

Management

QA29

Communications

Management

QA35

Risk Management

QA32

Procurement

Management

QA34, QA38

Stakeholder

Management

QA39




3

Project Summary by Assessment Area:

The table below includes a summary of each assessment area (i.e., category). Additional details on each of the findings that are being tracked in these areas can be found in Section 6 of this report.

Table 2: QA Status Highlights by Assessment Category

Assessment Category

Risk Highlights

July 2019

06/15/2019

Integration Management

High

PCG maintains Integration Management as high. ETAN and WSDOT continue to assess the go-live date, including bringing UAT, FAT and ICT into alignment with WSDOT's original expectations. PCG will continue to track Integration as a high risk until progress is seen in the interim project milestones.

PCG is tracking one (1) high risk (QA27) around ETAN’s testing processes and one (1) medium risk (QA40) around the readiness to implement.

Refer to Section 6.1 Project Integration Management for additional information on one (1) high risk (QA27) and one (1) medium risk (QA40) which PCG is tracking in this area.

Scope Management

Medium

PCG maintains Scope Management as medium. WSDOT has continued to work to proactively manage risks and has been diligent to consider the scope and quality of the system being built. PCG will work closely with ETAN and WSDOT during the reporting period to ensure that all required scope is achieved.

Refer to Section 6.2 Project Scope Management for additional information on one (1) medium risk (QA26) which PCG is tracking in this area.

Schedule Management

High

PCG maintains Schedule Management as high. The project is once again being re-baselined with a potential new go-live date for late 2019/early 2020. PCG will continue to monitor the schedule as it relates to items on the critical path, including interim milestones. PCG will continue to track this as a high risk until there is confidence in the schedule, including items identified on the critical path and associated go/no-go checkpoints. Refer to Section 6.3 Project Schedule Management for additional information on one (1) issue (QA36) and one (1) medium risk (QA28) which PCG is tracking in this area.

Cost Management

High

PCG maintains Cost Management as high. PCG continues to view the re-baselined go-live date and return to the contractual testing sequence as an opportunity for the project to mitigate the risk. PCG opened one (1) new high risk as recent Legislative mandates are requiring WSDOT to take part in "Gated Funding," which could impact the projects ability to maintain funding and pay their project costs.

Refer to Section 6.4 Project Cost Management for additional information on high risks (QA25 and QA41) which PCG is tracking in this area.

Quality Management

High

PCG maintains Quality Management as high. Vendor deliverables have improved in some areas, but issues remain around timely and complete work, particularly in the area of testing and training deliverables. If improvement continues this item will be considered for reduction.

Refer to Section 6.5 Project Quality Management for additional information on issue (QA22) which PCG is tracking in this area.




4

Assessment Category

Risk Highlights

July 2019

06/15/2019

Human Resource Management

High

PCG maintains Human Resource Management as high. Testing efforts continue to remain a high risk to the project. To date, WSDOT has been able to maintain their turn around times to review vendor documentation but the risk to WSDOT Human Resources remains high if deliverables and documentation comes in all at one time. Now that the project is re-baselining, there is an opportunity to return to the original intent of deliverable review timelines. PCG will continue to monitor the impact to resource utilization.

Refer to Section 6.6 Project Human Resource Management for additional information on one (1) high risk (QA29) which PCG is tracking in this area.

Communications Management

Medium

PCG maintains Communications Management as medium. Although WSDOT and ETAN have continued to improve communication channels, PCG would like to see this trend continue before decreasing the overall risk status to low. This includes communication around progress towards completion of the interim milestones.

Refer to Section 6.7 Project Communications Management for additional information on one (1) medium risk (QA35) which PCG is tracking in this area.

Risk Management

Low

PCG has continued to observe that the WSDOT Project Managers are actively

managing risks, including monthly risk management meetings and the continued

execution of the risk management plan and implementation of risk mitigations.

Refer to Section 6.8 Project Risk Management for additional information on one (1)

observation (QA32) PCG is tracking in this area.

Procurement Management

Medium

PCG maintains Procurement Management as medium. ETAN still has outstanding contract amendments that require finalization and with the newly re-baselined schedule, there is a potential for additional amendments.

During the current reporting period, the Test Manager position remained unfilled. ETAN secured a Test Manager that began on July 15, 2019. PCG will continue to monitor this transition during the upcoming reporting period (ending August 15, 2019).

Refer to Section 6.9 Project Procurement Management for additional information on one (1) medium risk (QA38) and one (1) low risk (QA34) which PCG is tracking in this area.

Stakeholder Management

Medium

PCG maintains Stakeholder Management as medium. Stakeholders will need to be

notified in a timely manner of the decision to re-baseline and modify the go-live date.

PCG will continue to monitor this risk and to assess stakeholder confidence.

Refer to Section 6.10 Project Stakeholder Management for additional information on

one (1) medium risk (QA39) and PCG is tracking this area.

2. Reference Documents

This section provides a list of documents referenced in the development of the Monthly Status Report:

• Institute of Electrical and Electronics Engineers (IEEE) Std. 730, IEEE Standard for Software

Quality Assurance Processes

• IEEE Std. 829, IEEE Standard for Software and System Test Documentation




5

• IEEE Std. 1012, IEEE Standard for System and Software Verification and Validation

• IEEE Std. 1028, IEEE Standard for Software Reviews and Audits

• IEEE Std. 1061, IEEE Standard for a Software Quality Metrics Methodology

• Project Management Institute (PMI®) Project Management Body of Knowledge (PMBOK®).

• International Organization for Standardization (ISO)/International Electrotechnical

Commission (IEC) 20000 Standards

• Information Technology Infrastructure Library (ITIL®) Standards

• Capability Maturity Model Integration (CMMI) Standards of the Carnegie Mellon University

Software Engineering Institute (SEI) for Development

• PCG’s Eclipse QA™ Methodology and Framework

• PCG’s QA Plan

• CSC BOS Project’s Project Management Plan

• CSC BOS Project’s internal quality reviews and lessons learned artifacts

• CSC BOS Project’s budget control and tracking documents and artifacts

• CSC BOS Project’s Master Project Schedule

• PCG’s CSC BOS Project Quality Assurance Plan

• CSC BOS Project’s Investment Plan

• CSC BOS Project’s Risk Register

• CSC BOS Project’s Organization Chart

• CSC BOS Projects RFP Appendix 2 – Business Requirements

• CSC BOS Project’s RFP Tolling Status Gauge

• CSC BOS Project’s WSDOT Planning Workbook

• CSC BOS Project’s DDI Aging Workbook

• CSC BOS Project Training Plan




6

2 Glossary

The table below lists the acronyms that appear within this Monthly QA Status Report.

Table 3: Acronyms and Definitions

Acronym Definition

CSC Customer Service Center

BOS Back Office System

CMMI Capability Maturity Model Integration

DD&I Design, Development and Implementation

DED Deliverable Expectation Document

FAT Factory Acceptance Testing

IEC International Electrotechnical Commission

IEEE Institute of Electrical and Electronics Engineers

ISO International Organization for Standardization

ITIL Information Technology Infrastructure Library

ITPS Information Technology Professional Services

LD Liquidated Damages

MOMS Maintenance Online Management System

M&O Maintenance and Operations

N/A Not Applicable

OCM Organizational Change Management

PCG Public Consulting Group

PM Project Manager

PMBOK® PMI’s Project Management Book of Knowledge

PMI Project Management Institute

PMO Project Management Office

PMP Project Management Plan

QA Quality Assurance

RFP Request for Proposal

SEI Software Engineering Institute

SOW Statement of Work

UAT User Acceptance Testing




7

3 Scope

The PCG QA team has provided an independent monthly risk assessment of the technical work products and the technical planning, management and control processes that directly support the successful implementation of the CSC BOS Project. As part of this assessment and project oversight activities, the PCG team continues to maintain a findings log and work collaboratively with the WSDOT project team to develop and implement recommendations for improvement that support the overall success of the project. The scope of project activities assessed includes the following areas:

• CSC BOS Project’s current status

• CSC BOS Project’s performance

• CSC BOS Project’s compliance with applicable standards and policies

• CSC BOS Project’s compliance with program requirements

• CSC BOS Project’s compliance with stakeholder expectations

4 Approach

PCG has taken the following four-step approach to completing the monthly QA risk assessment. This approach was tailored to be consistent with the current schedule and activities being conducted by the project:

1. Discovery – Review project documentation, work products, deliverables, and application code as well as interview key project team members to gain a thorough understanding of the assessment area. It is during this phase that applicable standards, best practices and lessons learned will be identified as evaluation criteria.

2. Research and Analysis – Research and analysis will be conducted for specific aspects of the products or processes being assessed in order to form an opinion of the validity of the proposed tool selection, design, and/or implementation approach. Once the initial analysis is completed the assessment results will be documented.

3. Clarification – Seek clarification from key project team members on specific aspects to ensure agreement and concurrence between the project, vendor and the PCG QA team.

4. Document – Document the results of discovery, research, analysis, and clarification in a Monthly QA Risk Assessment Report. These reports will contain key findings, observations and risk assessments. The PCG QA team will provide input regarding mitigation strategies, as well as any potential impact to the project if risks are not addressed

5 Assessment

Following is the detailed analysis and associated findings that resulted while completing the monthly QA risk assessment for the reporting period June 16, 2019 through July 15, 2019. Each section includes findings, recommendation(s) and significance that PCG has identified as critical to the CSC BOS Project’s success. A cumulative list of all QA findings is maintained in a separate QA Findings and Recommendations Log. This cumulative list includes updates made to existing, open findings and new findings identified during the current reporting period.




8

5.1 Project Integration Management

PCG continues to track Project Integration Management as high. There is a total of two (2) risks. One (1) high risk (QA27) around ETAN’s testing processes currently being used that are not in alignment with the approved Master Test Plan. ETAN and WSDOT are again reassessing the go-live date, including UAT, FAT and ICT to come into alignment with WSDOT's original expectations. While PCG is pleased that the schedule will be once again aligned, this will remain a high risk until the interim milestones are being met.

PCG maintains QA40 as Medium. During the current reporting period ETAN and WSDOT are reassessing the current schedule and go-live date, bringing the anticipated go-live to late 2019/early 2020. This would bring the schedule, including contractual review timelines, and remove work being done in parallel, taking into account ETCC's tolling of SR99. Additionally, WSDOT has defined interim milestones including: completion of development, test procedure and ICT close out, completion of all of the training materials, end to end testing, and a dry run for UAT. PCG would like to see these iterim milestones being met before reducing this risk.

Project Integration Management

# Type Description Priority Recommendation

QA27

Ris

k

ETAN’s revised hybrid testing execution does not align with the approved Project Master Test Plan.

High

PCG recommends WSDOT continue to ensure ETAN is executing the testing approach as defined. Consider options for schedule recovery and or concession(s) to testing, including impact to project timeline.

QA40

Ris

k

ETAN has been focused on meeting the go-live date rather than readiness of process, people and functionality.

Med

PCG recommends that WSDOT communicate to stakeholders, including the OCIO in their decision to change the go-live date. Include an understanding of the state’s tolerance for the degree of risk and threshold for the impact of any potential problems using the developed go-live checklists to support their decision. Monitor interim milestone progress and ensure that a contingency plan is defined for each interim milestone, including impact on the critical path.

Significance:

Project Integration Management is a critical area to overall project success. The significance to the risks opened here is around the project’s approach to execution, the impact to resources, and the ability of the project to maintain the schedule as well as the test activities currently on the critical path. Additionally, going live before fully tested may lead to problems in production including a lack of user confidence in the new system.

The fact that thoughtful consideration into a new go-live date was made provides the project additional time to focus on the integration of critical tasks and activities. Focusing on a go-live date rather than the readiness of processes, people, and functionality would have resulted in potential problems in production and a lack of user confidence in the new system.




9

5.2 Project Scope Management

PCG maintains the priority of QA26 as a medium risk around Project Scope Management. Due to compressed timeframes and project schedule, there remains a risk to the project scope. WSDOT has continued to work to proactively manage risks and has been diligent to consider the scope and quality of the system being built. The newly assessed re-baselined project schedule includes project scope per the contract, but PCG would like to see the interim milestones being met to instill confidence that ETAN is able to deliver all in-scope functionality. PCG will work closely with ETAN and WSDOT during the reporting period to ensure that all required scope is achieved.

PCG recommends WSDOT monitor the scope so that required scope and functionality is not negatively impacted by the new go-live date.

Until PCG is confident that the new go-live date and full scope can be delivered, PCG will maintain this as a medium risk.

Project Scope Management


QA26

Ris

k

Due to compressed timeframes and project schedule, a scope risk exists that pertinent scope may be deferred in order to meet the desired go-live date.

Med

PCG recommends WSDOT monitor the scope so that required scope and functionality is not negatively impacted by the compressed timeframes. Continue to monitor and update the go/no-go scorecard, including ensuring that ETAN is meeting the defined interim milestones.

Significance:

All required functionality for Back Office System may not be realized in the initial go-live phase, thereby requiring manual processes to handle the missing functionality. The manual processes may impact resources utilization.

5.3 Project Schedule Management

PCG continues to track one (1) issue (QA36) and one (1) medium risk (QA28) around Project

Schedule Management.

The project is being re-baselined once again, with a new go-live date of late 2019/early 2020.

PCG is in support of re-baselining the schedule since it appears to consider the original

contractual obligations, such as allowing enough time for WSDOT review of deliverables and

removing parallel activities being completed at the same time. PCG will continue to monitor

the schedule as it relates to items on the critical path and the new go-live date, including

interim milestones such as: completion of development, test procedure and ICT close out,

completion of all the training materials, end to end testing and a dry run for UAT. PCG will

continue to track this until there is confidence in the schedule, including items identified on

the critical path and the associated go/no-go checkpoints.

PCG recommends that WSDOT receive consistent (weekly) schedule updates and schedule

maintenance from ETAN. Ensure that the go-live date is achievable without compromising

quality. Confirm that items on the critical path can be assessed to display confidence that the




10

schedule can be achieved. PCG encourages WSDOT and ETAN to provide realistic, frequent

and up to date schedule maintenance, updates and assessments – even if it appears ETAN

is falling behind schedule.

PCG continues to track QA28 as a medium risk as there is no fully Integrated Comprehensive

Project Schedule. Without a fully comprehensive Integrated Project Schedule that considers

inputs and dependencies, WSDOT is not able to track the touchpoints between multiple

contracts and separate schedules. WSDOT continues to work on the development of a high-

level integrated project schedule. WSDOT has coordinated a comprehensive go-live

schedule that includes respective inputs from all collaborating contracts/work efforts as they

approach the to be defined go-live date.

WSDOT should continue to use alternative methods of tracking multiple project schedules.

PCG recommends capturing the gaps between the schedules and track inputs and to

continue to meet regularly and validate schedules, resources and scope. Continue to monitor

items on the critical path. Ensure that the changes within the delivery of SR99 are

represented.

PCG will continue to track QA28 as a medium risk until WSDOT has created an Integrated

Project Schedule, or at a minimum, a “go-live schedule” that the project is tracking to.

Project Schedule Management


QA28

Ris

k

There is no fully Integrated Comprehensive Project Schedule.

Med

PCG recommends WSDOT use alternative methods of tracking multiple project schedules until the master schedule is finalized. Capture the gaps between the schedules and track inputs.

Continue to meet regularly and validate schedule, resources, scope and schedule.

QA36

Issue

Items on the critical path (e.g., test execution, training) are falling behind without a current plan for recovery.

High

PCG recommends WSDOT monitor items on the critical path. PCG recommends that WSDOT receive consistent (weekly) schedule updates and schedule maintenance from ETAN. Ensure the go-live date is achievable without compromising quality. Confirm that items on the critical path can be assessed to display confidence the schedule can be achieved. PCG recommends that WSDOT and ETAN should work together if impacts to the schedule/schedule recovery are necessary. PCG recommends that WSDOT maintain control of the project schedule.

Significance:

Without a fully integrated, comprehensive schedule that considers inputs and dependencies, WSDOT is not able to track the touchpoints between multiple contracts and separate schedules (QA28).

Items that fall behind in schedule that are on the critical path indicate that WSDOT and ETAN must continue to discuss schedule recovery options, including adjusting the go-live date again (QA36).




11

5.4 Project Cost Management

PCG maintains QA25 as a high risk around Project Cost Management and opened one new

risk (QA41) during the reporting period. QA25 remains as a high risk. Re-baselining the go-

live and return to the contractual testing sequence continues to provide an opportunity to

mitigate this risk. PCG will monitor progress and reduce this risk if items remain on schedule

(including meeting the interim milestones) for the new go-live date and any potential cost

impacts are known.

PCG recommends that WSDOT continue to monitor the change in delivery of the test

procedures and phase sequencing to ensure budget is not negatively impacted and include

additional updates to these changes as required. PCG will continue to track this as a high

risk until it is clear that the project budget will not be impacted due to delays or the need for

additional resources.

PCG opened QA41 as a new high risk during the reporting period. Recent Legislative

mandates are requiring WSDOT to take part in the "Gated Funding," which could significantly

impact the projects ability to secure project funding and to pay project expenses. Newly

legislated budget provisions are intended to provide the authorizing environment with

additional financial detail. WSDOT will now be required to work with OCIO and OFM to

comply with the new budget template. WSDOT will not be authorized to spend FY20 money

until they complete an approved Gated Funding compliant budget that requires significant

work of agency, OFM and OCIO to get completed in time. Additional impacts include a 15%

hold back.

Project Cost Management


QA25

Ris

k

The revised approach to delivering Test Procedures and phase sequencing could cause delays to the project, resulting in increased cost.

High

Continue to monitor the change in delivery of the test procedures and phase sequencing to ensure budget is not negatively impacted and include additional updates to these changes as required.

QA41

Ris

k

Recent Legislative mandates are requiring WSDOT to take part in the "Gated Funding." Newly legislated budget provisions are intended to provide the authorizing environment with financial detail. WSDOT will now be required to work with OCIO and OFM to comply with the new budget template. WSDOT will not be authorized to spend FY20 money until they complete an approved Gated Funding compliant budget that requires significant work of agency, OFM and OCIO to get completed in time. Additional impacts include a 15% hold back.

NEW

High

PCG recommends that WSDOT continue to work closely with OCIO to ensure that compliance is achieved. Discuss if interim solutions can be defined or agreed upon with OCIO and with agency vendors.

Significance:




12

Due to the delays for the scheduled implementation there could be potential impacts to the

cost.

Without phase gate approval and continued project funding WSDOT will not be able to pay

project expenses, including vendor payments.

5.5 Project Quality Management

PCG is tracking one (1) issue (QA22) around Project Quality Management. PCG maintains QA22 as a high-risk issue. Vendor deliverables have improved in some areas, but issues remain around timely and complete work, particularly in the area of testing and training deliverables. If improvement continues this item will be considered for reduction. The go-live date is being modified extended past the previously discussed August 30, 2019 to late 2019/early 2020, and the critical path, along with interim milestones is being defined and adjusted. PCG will continue to monitor the quality of vendor deliverables but views this as an opportunity for WSDOT and ETAN to return to the intent of the vendor contract, including allowing enough time for ETAN development and WSDOT review of deliverables.

PCG recommends that the WSDOT Project Managers work closely with the ETAN Executive(s), PM(s), and Quality Assurance Manager to ensure that quality metrics are defined for all deliverables and closely follow the processes outlined in the Quality Management Plan. Furthermore, ensure that the agreed upon approach for deliverable acceptance (including stakeholder review and acceptance criteria) is timely and is being followed.

PCG will maintain this issue until ETAN is regularly submitting quality work, nearly free of defects, re-work and without multiple rounds of comment/WSDOT review.

Project Quality Management


QA22

Issue

Lack of adherence to quality standards and timely resolution by ETAN in deliverables development may cause schedule delays with the potential to affect the critical path.

High

PCG recommends that the WSDOT Project Managers work closely with the ETAN Executive(s), PM and Quality Assurance Manager to ensure that quality metrics are defined for all deliverables and closely follow the processes outlined in the Quality Management Plan.

Furthermore, ensure that the agreed upon approach for deliverable acceptance (including stakeholder review and acceptance criteria) is timely and being followed.

Significance:

Poor quality deliverables may cause schedule delays and impact the Project’s critical path

(QA22).




13

5.6 Project Human Resource Management

PCG is tracking one (1) high risk (QA29) around Project Human Resource Management.

PCG maintains this as a high risk due to the compressed timeframe for WSDOT resources

to review vendor documentation, which may cause over-utilization of existing resources.

WSDOT resources can become over-utilized when vendor documentation comes in all at

once. Testing efforts continue to remain a high risk to the project. To date, WSDOT has been

able to maintain their turn around times to review vendor documentation but the risk to

WSDOT Human Resources remains high if deliverables and documentation comes in all at

once. This risk may be reduced in the next reporting period if it is evident new go-live date

will allow enough time for WSDOT review of ETAN deliverables and documentation and that

the risk to resource utilization has lessened.

PCG recommends ensuring mutual collaboration and priority is understood by WSDOT and ETAN to ensure resources can fulfil their deliverable reviews in a timely manner. Additionally, PCG recommends prioritizing the information to be reviewed and identifying the functionality. Create a high-level version of the deliverable. Walk through with the vendor and WSDOT to streamline the Q&A process. Refer to detailed version of the documentation only when needed for additional clarity. PCG will continue to track this as a high risk until there is a reduced impact to resources as it relates to ETAN document reviews.

Project Human Resource Management


QA29

Ris

k

A compressed timeframe for WSDOT resources to review vendor documentation may cause over-utilization of existing resources.

High

Ensure mutual collaboration and priority is understood by WSDOT and ETAN to ensure resources are able to fulfil their deliverable reviews in a timely manner. Prioritize information to be reviewed. Identify functionality to be reviewed. Create a high-level version of the deliverable. Walk through with the vendor and WSDOT to streamline the Q&A process. Refer to detailed version of the documentation only when needed for additional clarity.

Significance:

WSDOT resources can become over-utilized when vendor documentation comes in all at once. There is a potential for the review process to be rushed and significant functionality could be missed.

5.7 Project Communications Management

PCG is tracking one (1) risk (QA35) around Project Communications Management. PCG maintains QA35 as a medium risk. This risk is around communication between ETAN and WSDOT and within ETAN and WSDOT, as communication has been observed to be happen in silos. Although WSDOT and ETAN have continued to improve communication channels, PCG would like to see this trend continue before decreasing the overall risk status to low. PCG is maintaining QA35 as a medium risk. Although it appears that many of the communication challenges continue to be worked through, including an increased executive engagement by ETAN, PCG would like to observe this trend to continue before reducing the




14

severity of the risk; particularly to ensure that it is clear what progress is being made towards the newly defined interim milestones.

PCG recommends WSDOT ensure the Communications Management Plan is being followed. Additionally, PCG maintains that the project should update the repository on a weekly, or bi-weekly basis at the latest. Continue to ensure that project team members and vendor staff take a broader view of their work and involve stakeholders with updates more proactively.

Project Communication Management


QA35

Ris

k

It has been observed that communication between ETAN and WSDOT and within ETAN and WSDOT is beginning to happen in silos.

Med

Ensure project team members and vendor staff take a broader view of their work and involve stakeholders with updates more proactively.

Significance:

Without insight into other workstreams, it is not always clear what updates have been made,

progress to date and critical updates.

5.8 Project Risk Management

To date the WSDOT Project Management team continues to proactively manage project risks (observation QA32). It has been observed that the WSDOT Project Management team is actively managing risk, including monthly risk management meetings and the continued execution of the risk management plan as defined. The Project Management Team provides regular updates regarding risks to the Project Sponsor and Steering Committee and risks and issues are being escalated appropriately and in a timely manner. PCG meets monthly with the Project Sponsor, Project Managers and the Project’s OCIO Consultant to discuss the status of the project, key risks and mitigation strategies. The WSDOT and ETAN executives meet regularly to facilitate effective and proactive decision making. PCG has reviewed the risk log and WSDOT has consistently tracked and updated the register. Risk Management remains an area that PCG will continue to review, especially as the project remains at a high risk.

Project Risk Management


QA32

Ob

se

rvation

WSDOT Project Management team is proactively managing project risks. It has been observed that WSDOT PM’s are actively managing risk, including monthly risk management meetings and the execution of the risk management plan as defined.

N/A

PCG recommends that the project continue to actively manage and monitor risk.

Significance:




15

Risk Management ensures the project can address and defer risk to the project.

5.9 Project Procurement Management

PCG is tracking one (1) medium risk (QA38) and one low risk (QA34) around Procurement Management.

PCG continues to track QA38 as a medium risk. As of the current reporting period, the Test Manager position remained to be covered by the Project Management team with the assistance of the ETAN Development team. ETAN confirmed that the Test Manager will be coming back on board to fulfil the Test Manager role, with a start date of July 15, 2019. PCG will continue to ensure that key project roles are being fulfilled before reducing the severity of this risk in the upcoming reporting periods.

PCG is maintaining QA34 as a low risk. ETAN still has outstanding contract amendments that need to be finalized. With the recent adjustment of the go-live date there is a potential need for additional contract amendments. Until it is clear to PCG that these amendments are being executed in a timely manner, this will remain a low risk to the project.

PCG recommends ensuring that ETAN and WSDOT are following the Change Management methodology as defined in the contract and Change Management Plan.

Project Procurement Management


QA34

Ris

k

ETAN change management is not making timely decisions to support needed contract amendments.

Low

Ensure that ETAN is following the Change Management methodology as defined in the contract and Change Management Plan.

QA38

Ris

k

ETAN has initiated a transition in the ETAN Project Management and Test Manager role.

Med

Ensure that ETAN provides the needed support to successfully transition key roles onto the CSC Project.

Significance:

A lack of rigor around Change Management lends to potential impacts to the project schedule as it relates to delays.

A change in project management can be disruptive to the project. There will need to be a knowledge transfer period, and the new PM will need to review and confirm the ETAN Project Schedule.

5.10 Project Stakeholder Management

PCG is tracking one (1) medium risk (QA39) around Project Stakeholder Management. PCG

is maintaining this risk as medium. Stakeholders need to be notified in a timely manner of the

decision to re-baseline and modify the go-live date. Since the decision to re-baseline

happened during the last day of the reporting period, PCG would like to assess how the

communication effort went, and if there have been any significant questions around, or




16

stakeholder concerns in regard to the newly re-baselined go-live date. PCG will continue to

monitor this risk and assess stakeholder confidence.

PCG recommends WSDOT continue to actively manage stakeholder expectations and

ensure timely communications happens to any/all change as it applies to stakeholders.

WSDOT should anticipate and address concerns; clarify and resolve issues. PCG will

continue to track this medium risk until it is clear that no impact to stakeholder expectations

or that confidence is restored in the go-live date.

Project Stakeholder Management


QA39

Ris

k

With the recent delay in the go-live date, stakeholders may develop lack of confidence in the successful, on time delivery.

Med

Continue to actively manage stakeholder expectations. Ensure timely communications happens to any and all change as it applies to stakeholders. Anticipate and address concerns; clarify and resolve issues.


17

Finding Status Code Definitions

A status identifies the current state of a risk or issue. This section provides a description of each of the status code values that may be assigned.

Table 4: Status Code Values of a Finding

Status Code

Value

Description

Open Identifies a risk or issue that has been opened during the reporting period either through

specific deliverable tasks or routine QA project monitoring activities.

Updated Identifies a risk or issue that was opened in an earlier report or deliverable and has

subsequently been updated based on new information. The “Update” column, also found in

the QA Findings and Recommendations Tracking Log, will provide the details of the update as

QA continues to monitor the risk or issue.

Closed Identifies a risk or issue that has been closed during the reporting period. The “Update”

column found in the QA Findings and Recommendations Tracking Log, will document the

reason for closure.

N/A A status of N/A is assigned to a neutral or positive observation. Observations are reported to

document a specific aspect of the project that was reviewed; however, did not result in a risk

or issue.




18

Finding Priority Definitions

A priority is assigned to all risks and issues. Once a priority is assigned to a risk or issue, it will be reassessed during each subsequent reporting period as part of the routine QA project monitoring activities and documented in the QA Findings and Recommendations Tracking Log.

C.1 Risks

A risk is “an uncertain event or condition that, if it occurs, may have a positive or

negative effect on a project’s objectives”. The PCG QA team will identify risks with

negative effects and expand the definition to include both conditions which may occur

and those which may not occur (e.g. lack of a well-defined requirements traceability

process could lead to delivery of an incomplete system, requiring costly and time-

consuming rework).

A key to risk management is understanding the potential risks to the project and ensuring

that these risks and risk mitigation strategies are communicated to key project

stakeholders on an ongoing basis. Risk analysis should begin early during project planning

by determining or identifying the factors that may affect the project. Risk can impact a

project in many ways: project quality, scope, cost, and schedule. Proper risk identification

seeks to determine how the risk may affect the project and to document the project area(s)

impacted by the identified risk.

Once risks are identified and characterized, both qualitative and quantitative factors are

examined. Analysis

considers the project

conditions to determine

the probability of the risk

being realized and the

impact to the project if it

is realized. Overall the

risk priority is determined

by multiplying the

probability rating times

the impact rating. The

PCG QA team will

determine the risk priority

using a risk-rating matrix

(see Figure 1) which

determines the priority of

each risk based on an

assessment of probability

of occurrence and the

magnitude of impact.

Figure 1: Risk Rating Matrix




19

The following table defines the risk priorities that will be used when identifying risks.

Table 2: Risk Priority Definitions

Risk Priority Definition

High

A priority of high is assigned if there is a possibility of substantial impact to

product quality, scope, cost, and/or schedule. A major disruption is likely,

and the consequences would be unacceptable. A different approach is

required. Mitigation strategies should be evaluated and acted upon

immediately.

Medium

A priority of medium is assigned if there is a possibility of moderate impact

to product quality, scope, cost, and/or schedule. Some disruption is likely,

and a different approach may be required. Mitigation strategies should be

implemented as soon as feasible.

Low

A priority of low is assigned if there is a possibility of low impact to product

quality, scope, cost, and/or schedule. Minimal disruption is possible, and

some monitoring is likely needed to ensure priority does not increase.

Mitigation strategies should be implemented as time permits.

C.2 Issues

An issue is a finding, often previously identified as a risk that documents an event that has

occurred and caused negative impact to the project. Issues, documented in the QA

Findings and Recommendations Tracking Log, identify the event, its impact to the project,

and updates towards resolution.

An issue’s priority is determined by its impact on the project. The following table defines

the issue priorities that QA uses when assigning a priority to an issue.

Table 3: Issue Priority Definitions

Issue Priority Definition

High

A priority of high is assigned if the issue presents substantial impact to

product quality, scope, cost, and/or schedule. A catastrophic disruption is

likely, and the consequences would be unacceptable. A different approach

is required. Mitigation strategies should be evaluated and acted upon

immediately.

Medium

A priority of medium is assigned if the issue presents a moderate impact to

product quality, scope, cost, and/or schedule. Some disruption is likely, and

a different approach may be required. Mitigation strategies should be

implemented as soon as feasible.

Low

A priority of low is assigned if the issue presents a slight impact to product

quality, scope, cost, and/or schedule. Minimal disruption is possible, and

some monitoring is likely needed to ensure priority does not increase.

Mitigation strategies should be implemented as time permits.




20

www.publicconsultinggroup.com

Date post:	31-Oct-2021
Category:	Documents
Upload:	others
View:	0 times
Download:	0 times