Volantic PMO Metrics Recommendations

8/12/2019 Volantic PMO Metrics Recommendations

1/22

PMO Metrics Recommendations

1 PMO Metrics Recommendations

Introduction

The purpose of this document is to recommend metrics to be used by the Project Management Office

(PMO) to measure and analyze their project and PMO success. The metrics are divided into Project and

PMO levels and each one is defined with description, source, tracking, analysis, and reporting

information.

The mission of the Project Management Office (PMO) is to provide the project management foundation

and services that establish an environment where:

Project success is the norm. Functional organizations gain benefit from their investments. PMO services are desired because of their consistent success and professionalism.

as an organization is recognized by the enterprise for their successful project delivery thatmeets or exceeds our customers expectationsfor agreed upon requirements, on schedule and to

budget.

Project manager professionalism continually grows through common approaches, shared learning,and mentoring.

The following project and department (PMO) metrics were identified to enable the PMO in achieving

these goals.

Project Level Metrics - Overview

Project level metrics are intended to measure the projects progress and performance. The PMO

performance measurements for project success are:1. All documented and agreed upon business objectives and project goals are achieved by the project2. The end product is fully delivered to and used by a satisfied customer3. The project achieves targeted schedule and costs while providing the defined capability.With these goals, the project level metrics are:

Project Schedule VarianceTracks and reports variance of forecasted completion date against thebaseline project completion date.

Project Cost VarianceTracks and reports variance of actual and forecasted project cost againstbaseline project budget.

Customer Project SatisfactionMeasures project success from a stakeholder perspective based on asatisfaction survey issued by PMO to IT and Business at the end of a project.

Milestone Schedule VarianceTracks and reports variance of actual and forecasted completiondates against baseline milestone completion dates.

Milestone Cost VarianceTracks and reports variance of forecasted and actual milestone costagainst baseline milestone budget.

Project Success RatingNumerical rating calculated at the completion of a project based on theSchedule, Cost and Customer Satisfaction.


2/22



PMO Level Metrics Overview

PMO level metrics show trends across all projects over the course of months. Tracking and analyzing

these trends allows the PMO to:1. Identify trends and adjust processes, tools or other items to continue to mature the organization.2. Demonstrate continuous improvement in the management of projects.The recommended PMO level metrics are:

Project Schedule SummaryMonthly statistics on project delivery, from the schedule perspective,showing the number of projects that completed ahead, over or on schedule.

Project Budget SummaryMonthly statistics on project delivery against budget showing the numberof projects that completed under, over or on-budget.

Resource UtilizationTracks and reports variance of resource usage against overall allocation. Customer Project Satisfaction SummaryMonthly statistics showing trends of customer responses. Project Success Rating SummaryMonthly summary of the numerical ratings from the projects that

closed during the previous month.

Some of the project and PMO metric recommendations may not be achievable at this time, but are

presented as a goal to work toward as the organization continues to mature.


3/22



Project Level Metrics

The following set of metrics is required at the individual project level to provide the necessary

information that will allow the project managers and management to drive successful delivery and

continuously improve performance.

Project Schedule Variance Metric

DescriptionThis metric Tracks and reports the variance between the baseline project completion date

and the forecasted project completion date.

SourceThis data is currently available through . The project manager

establishes a schedule in the tool based on work to be done and the approved project completion date

in the Charter or adjusted through the change management process. The schedule is baselined to

establish a line to measure progress against. Throughout the project, resource progress is tracked and

remaining effort is recalculated to identify potential change to the project end date. This forecasted

change is the project schedule variance and can represent either as positive (achieving the project goals

sooner) or negative (expanding beyond the baseline date) movement. The baseline completion date

should only be modified with an approved change request.

TrackingIdeally, forecasting and progress checks should be performed weekly utilizing actual and

estimates to complete (ETC) by resource. Delivery dates of intermediate deliverables should be

monitored for slippage to verify overall project schedule is not in jeopardy.

Recommendations: In order to more effectively use to track this, two best

practices need to be adopted:

1. Record Estimates to complete (ETC) for each activity by resource to reflect progressaccomplished and not simply hours spent.

2. Track schedules on a Resource Based rather than Task Basedmethod. Task based (aka FixedDuration) locks the finish date for tasks. Any slippage in hours is compared to that date.

Resource Based (aka Fixed Work) allows the finish date to adjust based on the ETC and

dependencies. This allows the project manager to identify schedule slippage earlier and make

adjustments.

AnalysisProject managers review the week to week changes in the overall projected completion date

to determine what actions, if any, are required to pull the project back on schedule. Slippage in any

intermediate deliverable completion dates should also be analyzed for root cause and possible trends.

Corrective actions should be documented and taken early on to reduce the risk of runaway projects.

ReportingCurrently, formal reporting is required monthly on the Project Status Report using

the following Red/Yellow/Green indicators:


4/22



Red if variance is > 10% of project duration (Baseline FinishActual Start)

Yellow if variance is > 5% but


5/22



Project Cost Variance Metric

DescriptionThis metric tracks and reports variance of actual and forecasted project cost against the

baseline project budget.

SourceThis data is only partially available through . The initial baseline

cost is established by the budget. It is placed in using a combination of:

A standard bill rate of $60/hour for internal resources (assessed annually) Contractor actual rates HW/SW costs from the original budget Consultant projected costs from their statement of work.True resource costs are stored in the Oracle Financials system and only reported at the project level, in

group totals.

Internal and Contractor costs are tracked in in the same method as thetracking done for the Project Schedule Variance metrics. Actual hours are entered by resources at the

activity level and remaining effort is forecasted. This is multiplied by the associated hourly rate to

calculate costs.

Consultant, HW/SW, and other costs are tracked by invoices or accrual process. As invoices are

received, the expenses can be entered into Primavera by the project manager or by the Resource

Analysts to ensure a holistic view of the project cost.

On a monthly basis the Oracle Financials information is extracted to true up the actual costs and re-

evaluate the forecast. Project Cost Variance is measured by comparing the cost of the effort to date and

the remaining effort (estimate at completion) against the baseline cost.The current cost tracking method has issues.

1. The accuracy of the standard bill rate fluctuates depending on the actual resources assigned toany given project. This results in project managers having to work toward budget, not hours.

2. Notification of invoices is often delayed, resulting in understated current costs. This can bemitigated by requiring project managers to accrue the costs based on either timesheets or

payment schedules rather than waiting for invoices.

3. Actual costs for internal labor from the Oracle Financials system may cut off as much as twoweeks before the month end reporting, again, understating the current spend.

4. Current policy states that a maximum 2% variance between the estimated SOW baseline projectbudget baseline and actual spend is acceptable. Utilizing existing tools it is difficult to track the

variance that accurately.

Recommendations:

1. Provide a weekly report or feed from the Oracle Financial system to the projects for comparisonand adjustment. Note: weekly feeds would not include accruals since these are only part of the

monthly Oracle processing. They may still reflect a 2 week lag for internal labor cost.

2. Implement an average hourly rate by role and use this rate for forecasting and reporting costs.


6/22



The primary role of each resource would need to be verified. This would bring the tracking

more in line with the reports from the Oracle Financials system. [Note: To maintain salary

anonymity, some roles with fewer individuals may need to be averaged with similar roles.]

TrackingThe tracking changes suggested for the Project Schedule Variance (ETC and Resource Based)

are applicable with the tracking for internal and contractor work.

Recommendation: To obtain a holistic view of the project cost variance, update the HW/SW costs,

consultant invoicing, and other expenses in Primavera in a timely manner to facilitate more accurate

reporting.

AnalysisProject managers review the week to week changes in the overall projected cost to determine

what actions, if any, need to be implemented to bring costs into alignment with the budget. In addition,

intermediate deliverable cost at completion slippage should be analyzed for root cause and trends.

Corrective actions should be documented and taken early on to reduce the risk of runaway projects.ReportingCurrently, formal reporting is currently required monthly on the Project Status

Report. Since cost slippage occurs day to day at the activity and deliverable level, project managers

should review the data and address issues every week.

For formal reporting the following Red/Yellow/Green indicators are used:

Red if projected costs are > 10% higher than current baseline budget Yellow if projected costs are > 5% but


7/22



2.Generate a cost variance report from Primavera using the fields Baseline Cost, At Completion

Expenses, and Variance Baseline Cost. This level of tracking allows Earned Value reporting in

Primavera.

Customer Project Satisfaction Metric

DescriptionSatisfaction Survey issued by PMO to IT and Business at the end of each project in order to

measure project success from stakeholder perspective.

Recommendation: Create a standardized survey issued to key stakeholder at the completion of each

project based on the following Source, Tracking and Analysis.

SourceIdentify a method to produce a Customer Satisfaction Survey using questions similar to the

ones listed below. Use a scale of Strongly Agree (5), Somewhat Agree (4), Agree (3) Somewhat Disagree

(2) and Strongly Disagree (1). The numeric values will be used for the Project Success Rating metric.

1. Business objectives The project team understood the business objectives of the project. The end product met the business objectives and goals.

2. Communications I was kept informed of the progress of the project. There was open communication with the project team. I received answers to my project related questions in a timely manner. The level of communication helped eliminate surprises on the project.

3. Management I was comfortable that the main project risks and issues were being managed. The project was managed to a logical schedule. Changes to the scope, schedule and/or budget were presented for discussion and approval.

4. Success The product was delivered when promised. The project costs were within budget. The product delivered included the needed capabilities to achieve our purpose.

5. Overall Satisfaction (Change scale to Very Satisfied, Somewhat Satisfied)


8/22



How would you rank your overall satisfaction with the delivery of this project?The PMO would issue the survey and request that responses be returned to an individual not directly

associated with the project being surveyed.TrackingSurvey results are collected once for each project. Project-to-project tracking and trending is

handled at the PMO level. See Customer Project Satisfaction Summary metric for more information.

AnalysisThe results of the survey are to be discussed with the project manager, ideally without

revealing the response of any given individual. If additional clarification is necessary, the PMO will

contact the respondents. Lessons learned that resulting from the review are documented and shared

with the rest of the PMO.

ReportingConsolidate reporting at the PMO level to show trend information and publish the

combined results. See Customer Project Satisfaction Summary metric for more information.

Sample Project Survey Results Graph:


9/22



Deliverable Schedule Variance Metric

DescriptionTracks the variance of actual and forecasted completion dates against the baseline

completion dates for project deliverables.

Recommendation: Implement project planning and scheduling around the concept of a Deliverable as

illustrated below.

SourceUtilize a Deliverable based project schedule approach in order to track schedule variance at a

more granular level as illustrated in the image below.

The project is built from the products (A, B, and C above) identified in the Work Breakdown Structure

(WBS). Each Deliverables has an associated baseline finish date, hours and cost estimates against which


10/22



actual and forecasted information is measured. A strict implementation of this method requires all

activities to be included in a Deliverable with associated predecessors. The impact of extending any

given activity would then change the planned end date of that Deliverable, causing a variance. Project

management aspects (status meetings, reporting, tracking, admin, etc.) should be included in a Project

Management Deliverable.

Deliverables (other than Project Management) are not allowed to run the length of the project. For

example, Requirements in the current schedule template has an activity entitled Requirements

Managementused to track the requirements through the remainder of the project. We recommend

that the Deliverable Requirements Document use an activity to Create the Requirements Traceability

Matrixin Requirements and then future Deliverables, such as Test Plan, would include an activity to

Update the Requirements Traceability Matrix.

With this approach Deliverables become checkpoints to measure planned vs. actual progress throughout

the project, allowing for earlier identification of off-track projects and enables stronger Earned Value

based measurements.

Primavera facilitates this level of tracking through the utilization of sub-phases for each Deliverable, as

illustrated in the image above.

TrackingCurrently, in the Primavera project schedule, the cost associated with consultants and

hardware / software are forecasted in total as part of the Planning phase in. Actual costs are then

entered into the Planning phase against the budgeted amount. Although this does not affect the overall

project tracking, it will impact the deliverable level tracking.

Recommendation: Include the projected costs for Consultant, Hardware and Software expenses as

components of the baseline costs for the phases and deliverables where they are projected to occur. As

the actual costs are incurred (either invoiced or accrued) they are tracked against the baseline.

Implementing this change along with the changes described in the Sourcing section above allows the

tracking performed for the Project Schedule Variance to automatically feed the Deliverable Schedule

Variance.

AnalysisProject managers review the week to week changes at the Deliverable level to determine

what actions, if any, need to be implemented to comply with the schedule. Keeping the Deliverable

completion dates from slipping keeps the overall project on track.

ReportingThe current Status Report Template has a Standard Deliverables/Milestones Schedule Statustable.

Recommendations:

1. Ensure that the Deliverables listed in the status table mirror those set up in the projectschedule.

2. Add a column to display the variance.


11/22



3. Use Red/Yellow/Green indicators to visibly show variance: Red if variance is > 1 week beyond baseline completion date and no plan created to correct. Yellow if variance is < 1 week beyond baseline completion date or > 1 week with corrective

plan.

Green if variance is ahead of schedule. Clear if on schedule.

4. Use the same reports that show Project Schedule Variance andEarned Value at the Deliverable level to report variances. This is accomplished by adjusting the

detail level displayed to include the subtasks.


12/22



Deliverable Cost Variance Metric

DescriptionTracks variance of actual and forecasted milestone cost against the baseline Deliverable

budget.

SourceCurrently project managers are not required to track variance at the Deliverable level.

Recommendations:

1. Implement the recommendations from the Sourcing section of the Deliverable ScheduleVariance metric described above. This will enable Schedule Variance tracking.

2. Establish the sub-phase level for Deliverables as illustrated inDeliverable Schedule Variance.

3. Provide a mechanism to report on cost variances between Oracle and at the Deliverable level (i.e. sub-phase). This would allow closer tracking of the planned

and actual true costs at the Deliverable rather than Project level. Options might include

Business Objects reporting; direct feed from Oracle to Primavera; using role-based resource

costs the closely resemble true costs; or other method.

TrackingOnce the recommendations give in the Sourcing section above are established, the tracking

performed for the Project Cost Variance will automatically feeds the Milestone Cost Variance.

AnalysisProject managers review the week to week changes in the individual Deliverable cost to

determine what actions, if any, need to be implemented to bring costs into alignment with the baseline.

If the Oracle feed is not changed it will be difficult to match the true costs at the Deliverable level.

Managing the cost of the individual Deliverables will keep the overall project budget in check.

ReportingThe current Standard Deliverables/Milestones Schedule Status table on the status report

will need to be expanded to include baseline and projected/actual cost. Indicator values can be defined

to show variance as:

Red if projected Deliverable cost is > 10% higher than current baseline budget Yellow if projected Deliverable cost is > 5% but


13/22



Green if the projected Deliverable cost is less than the baseline budget Clear if projected Deliverable cost is


14/22



Project Success Rating Metric

DescriptionNumerical rating calculated at the completion of a project based on the Schedule, Cost

and Customer Satisfaction.

Recommendation: Create a Project Success Rating metric showing the overall success of a project.

SourceThe rating is based upon a 5 point scale and is calculated at the completion of a project using

the Schedule Variance, Budget Variance, Overall Customer Satisfaction and the Customer Satisfaction

Survey results. The calculation is based on the following factors.

20% Schedule (5 Points)

+/- 15% from baseline = 1 Point

20% Budget (5 Points)

+/- 15% from baseline = 1 Point

20% Overall Customer Satisfaction (5 Points)

Average of the answers to the Overall Customer Satisfaction question on all surveysreturned

40% Customer Satisfaction (5 Points)

Numerical average of all answers on Customer Satisfaction Survey except the OverallCustomer Satisfaction on all surveys returned

Rating = 0.2*Schedule + 0.2*Budget + 0.2*Overall + 0.4*Customer Satisfaction

TrackingThe data for this calculation is collected manually from the Schedule Variance, Budget

Variance and Customer Satisfaction surveys for the project.

Recommendation: Create a Project Rating spreadsheet for the project manager to enter the raw data

from their individual projects as they close. This will eliminate the need to recollect that data for the

monthly summary.

AnalysisThe actual analysis for this rating is performed at the individual pieces. In the event that the

overall rating is low, review the analysis for the individual pieces to

ReportingThe score for each completed project is reported in the monthly PMO report issued to

management. Projects with scores of 4.5 or greater should receive positive recognition. The minimum

requirement to receive a 4.5 rating is two 5s and two 4s with one of the 5s is in the Customer

Satisfaction area.


15/22



APMO Level Metrics

Project Schedule Summary

DescriptionConsolidated project delivery summary statistics to show month-to-month trends on

project delivery.

Recommendation: Create a monthly Project Schedule Summary report that shows the number of

completed projects that finished early, late and on schedule.

SourceThe base for this information is in . At the beginning of the month,

schedule data is pulled for projects completed in the previous month. Required information includes

project identification (name, number, etc.), baseline finish and actual finish. (Note: Definition of finish

needs to be determined, i.e. implementation or closure). should be able toaccommodate multiple finish dates if there are multiple releases.

TrackingAssuming the recommended proper project level tracking and actual / ETC updates are

occurring, no extra tracking is necessary. A milestone should be included in the template and schedules that corresponds to the finish date (either project closure or

implementation). Predecessors should be utilized to adjust the finish date automatically based on

changes to the schedule.

AnalysisMany variables can impact the delivery of any given project, but in general the summary

information should trend toward on or ahead of schedule. Individual projects that are late should be

reviewed to determine root cause. Recurring reasons might indicate a systemic issue and would be

candidates for new processes or tools. If multiple projects fail during a given month, additional analysis

should be performed to see if there have been organization level changes that impact project delivery.

ReportingA report from will pull the project finish date information based

on the standard project completion milestone for those projects (or implementations) completed the

prior month. Reporting logic should pull data for a few previous months and update the previous graphs

to account for any changes.

The information can be displayed in a stacked column graph or other image showing on time, early and

late project counts by month.


16/22



Project Budget Summary Metric

DescriptionMonthly statistics for completed projects showing delivery against budget, graphically

displaying the number of projects that complete on, under or over budget.

Recommendation: Create a monthly Project Budget Summary report that shows the number of

completed projects that finished under, over and on budget.

SourceThe source of the planned budget information is from . Project

level baseline information is recorded at the beginning of the project and modified using the Change

Management process. The standard project tracking will account for the actual costs. Note: true

financial costs are managed in Oracle Financials. For metric tracking, is the

source of record.

Once a month, data is pulled for completed project for analysis and reporting.

TrackingTracking is done at the individual project level as described in the Project Budget Variance

metric. This metric summarizes the information from completed projects. The project level tracking

trends will be the leading indicators of this summary.

One erroneous practice observed at other companies is issuing a Change Request at or near the end of

the project to true up the costs. This eliminates any final variance and prevents organizational maturity.

This is sometimes done under the guise of transferring funds to or from other projects and called budget

management. This should be avoided.

AnalysisProject budgets can be impacted by many factors, including hardware/software costs and rate

increases. In general the summary information should trend toward more projects completing on or

under budget. Individual projects that go over budget should be reviewed to identify causes. Recurring

reasons are candidates for new processes or tools to correct the issue. If multiple projects fail during a

given month, additional analysis should be performed to see if there have been organization level

changes that impact project delivery.

ReportingStatistics are collected monthly and displayed using a graph like the clustered column

below, grouped by month. A twelve month rolling display will show trending.


17/22




18/22



Resource Utilization Metric

DescriptionTracks and reports variance of resource usage against allocation at the resource level.

SourceThe main source for this metric is in . Progress is currently using

s Resource Availability Report showing the sum of the hours for each

resource across all projects. It displays the accumulated number of hours spent and, for future time

periods, the forecasted hours. The web version allows immediate drill down to see the actual projects

each resource is assigned to.

Some Managers hold weekly meetings to discuss and change resource assignments. Project managers

are not always invited to attend or made aware of the outcome. Attending all resource management

meetings would also hamper the project managers productivity. Project managers are frequently asked

to level their resources (cut forecasted hours) to reduce over allocation. Neither one of these tactics are

effective.

Recommendation: Define and implement a Resource Request / Allocation process that allows project

managers to request resources by role and timeframe and enables managers to allocate resource to fill

the need. This will require an organizational level change.

The web version of has a resource module that allows resources to be

requested and allocated based on role. This module is not currently utilized but is being reviewed by

the Resource Analyst team. This module would allow project managers to request resources at different

percentages throughout the duration of the project, accounting for ramp up and release. Managers

could then allocate accordingly and gain a better picture of resource needs and contention.

Formalizing the process and making the information more visible allows the project managers to better

plan and forecast project completion. As part of the allocation, it is important that Managers designate

time for support and maintenance activities for existing applications.

TrackingTracking is currently managed in using the client-based Resource

Availability report. At the organization level, individual managers use various methods to manage their

resources.

Recommendations:

1. Move to the web-based reports which give immediate access to the project details for eachresource. This will allow project managers to see which other projects are vying for theirresources time.

2. Provide Managers with a tool that allows them more visibility to resource tracking informationand one that formalizes the request / allocation process. A project manager would then track

actual usage against the planed allocation for each resource. Any significant schedule changes

or project priority changes that impact the resource usage are required to be resubmitted,

analyzed and re-allocated.


19/22



3. Require that Remaining Work is updated for each activity by resource to reflect true effortremaining and not simply subtracting hours spent from the original amount. If this is not done

the roll up to the resource level will be inaccurate.

AnalysisThe practical test of the allocation will be seen in the web based

tool. This should be reviewed weekly to eliminate large over or under allocations. The details show the

projects that are impacted. Resource contention is reviewed first by the project managers and

escalated when unresolved.

ReportingResource allocation reporting comes from through the Resource

Availability Report, the allocation module, or other similar reports. Resource Managers are the main

recipients and adjustments are made with the project managers.


20/22



Customer Project Satisfaction Summary Metric

DescriptionCreates a monthly average of the Customer Project Satisfaction survey results by focus

area (Business Objectives, Communications, Management and Success).

Recommendation: Create a monthly Customer Project Satisfaction Summary report from the individual

projects complete the previous month.

SourceThe source for this metric is the project level customer satisfaction surveys. The results are

collected by month and an average for each category is taken. Consideration may be needed for late

arriving surveys.

TrackingTracking is performed manually. The results from the project surveys are collected and

entered into a spreadsheet for analysis and reporting.

AnalysisInitial analysis was performed at the project level to determine the cause of poor results. Atthe summary level, trends are assessed to determine if additional management involvement,

communication, processes or tools can help drive success from the customers perspective.

ReportingThe month by month results can reported in a number of method (one overall score for the

month, individual answers, etc.). A simple grouped column graph showing the average value by

category gives a good, condensed view of trending.


21/22



Project Success Rating Summary

DescriptionMonthly summary of the numerical ratings from the projects that closed for the previous

month.

Recommendation: Create a monthly Project Success Rating Summary report from the individual

projects complete the previous month.

SourceThe source of this data is average rating of all projects completed the previous month.

Assuming the project managers are populating the Project Rating spreadsheet recommended in the

Project Success Rating metric, the information should be readily available to produce an overall average.

TrackingInclude in the Project Rating spreadsheet columns to create the averaged rating by month.

AnalysisCheck the overall trend of the average ratings. Sharp changes should be reviewed for

corrective actions or best practices. Include interpretation of any changes (ex. fewer projects skewed

average toward extreme rate on single project) and trends.

ReportingThe monthly PMO report to management contains the scores for each of the projects

completed in the prior month. A trending chart can be generated showing the average PMO rating by

month.


22/22



Summary

The eleven metrics recommended above are intended to achieve the mission of the PMO as

demonstrated by the following table.

Mission Statement Supporting Metric

Application Development project success

is the norm.

Project Schedule Variance Project Cost Variance Milestone Schedule Variance Milestone Cost Variance Project Success Rating

Functional organizations gain benefit from

their investments.

Customer Project Satisfaction Resource Utilization Customer Project Satisfaction

Summary

PMO services are desired because of their

consistent success and professionalism.

Project Schedule Summary Project Budget Summary Customer Project Satisfaction

Summary

Project Success Rating Summary as an organization is recognized

by the enterprise for their successful

project delivery that meets or exceeds our

customers expectations for agreed upon

requirements, on schedule and to budget.


Summary

Project Success Rating SummaryProject manager professionalism

continually grows through common

approaches, shared learning, and

mentoring.


Summary

As the organization matures, the objectives and expectations will change. As they do, the metrics are to

be reviewed, adjusted and replaced as necessary to ensure continued success.

Date post:	03-Jun-2018
Category:	Documents
Upload:	sergio-ivan
View:	222 times
Download:	0 times

Volantic PMO Metrics Recommendations

Documents