Benefits Realization: Program Monitoring through Middle-out Balanced Scorecard

Approach

(“Managing Vision”)

Srinivasa-Desikan Raghavan,

Tata Consultancy Services

--------------------------------------------------------------

ABSTRACT

Large transformation programs, as part of any corporate initiatives, are always tied to their intended

‘benefits’ as objectives of the programs. Even the well-endowed Program Management practitioners

usually get sidetracked by focusing on the execution of the program and its constituent projects, failing to

follow the well-developed Benefits Management principles, for periodically tracking the benefits (to be)

realized.

Balanced Scorecard (BSC), as a Corporate Strategy Deployment tool, is usually top-down driven. In this

paper, the author intends to describe a ‘Middle-out’ approach to BSC that can be adapted to any program

execution from Benefits Realization view point, given the daily challenges of program monitoring and

control.

A case study, where ‘Middle-out’ approach of BSC was adopted, would be explained in this paper.

Benefits Realization: Program Monitoring through Middle-out Balanced Scorecard

Approach

Srinivasa-Desikan Raghavan,

Tata Consultancy Services, “Nyati Tiara”, Nagar Road, Yerawada, Pune - 411006

(Desikan.r@tcs.com)

1. Introduction: As is evident, when organizations sanction large transformation programs, it would

be on the basis of a logical Business Case. Other than the requirements driven by statutory

processes of regulatory or Government authorities, many of these programs have, besides, solid

commercial end-of-logic but also of other organizational dimensions like People, Processes and

Technology as ‘drivers’ for the justification of budgetary sanctions.

‘Business Benefits’ or simply ‘Benefits’ have come a long way from being buried under Business

Case documents into being a subject of its standing in the body of Project and Program

Management framework. From a program management view point, ‘Benefits’ are the starting

point for Terms of Reference when programs are sanctioned and initiated. But once the

program(s) get into execution mode, benefits realization should become the focus of every

program review that happens at the tactical and strategic layers of the organization. Sadly, this

may not the case at every large programs that organizations run, as, not only the practitioners are

battling the daily grind of program and project management at the operational level but also may

ignore the well-developed Benefits Management framework, as identification of right measures

may be a challenge and monitoring the results of a ‘closed’ program would involve more time and

effort.

The Balanced Scorecard (BSC, henceforth) has been in practice for Corporate Performance

Management and for strategy deployment purposes since early 1990s (Kaplan and Norton, 1992;

1993; 2000). It has been used both at corporate level as well as at the level of a project or a

program. From a ‘performance management’ view point, many corporations have adopted it for

organizational performance and for measuring individuals’ performance from a Human Resource

management view point. From the example of a corporate scorecard getting cascaded to

individual teams’ level, there are cases where BSC had also been used for ‘Project focused IT

Organization’ (Alleman, 2003). From the design point of view, there are many organizations that

exist specializing both in BSC tools and in training (2GC, 2009).

In this paper, the author explains, in brief, a typical Benefits Management life cycle from the

Project and Program Management literature (and also from what is being practiced at Tata

Consultancy Services (TCS) for their customer assignments); describes BSC, which can be a

‘middle-out’ approach compared to the traditional top down way of arriving at scorecards; and

suggests how BSC – Middle –out approach can be used for tracking Benefits Realization in large

programs.

2. Benefits Management framework: The Benefits Management Strategy documentation should

take a cue from the program business case and elaborate upon the following points:

Description of the Program's benefits and where in the organization they will occur

(People, Process and Technology dimensions, besides the corporate financials);

Benefit model showing interrelations and dependencies, amongst intended benefits;

Functions, roles and responsibilities of individuals in the Program and in the larger

organization, as the recipients and the owners of the intended benefits (for benefits

planning and realization);

Review and assessment process for measuring benefits realization; this would be

accomplished by appropriate ‘Toll Gate(s) Review(s)’, along the Program progress.

The following diagram depicts the phases in Benefits Management framework and their

relationship to Program Lifecycle.

Fig 1: Program and Benefits Management Phases

Straw

Model Metrics

Dovetail

Plan

Initiate – Plan – Mobilize – Execute - Close

a. Phases of Benefits Management: The Benefits Management Guideline is a collection of

activities, frameworks and templates designed to achieve the goal of realizing business values

from the programs. It achieves this via key phases, each with its own purpose and outputs.

Benefits Identification: Before we can plan or measure benefits, we must understand that

outcomes which are the targets of such investment. The Benefits Identification phase is all about

understanding and establishing the strategic intent and identifying the outcomes required

achieving this intent. A ‘Straw Model’ would be arrived at, identifying the relationships amongst, -

Program-end Business Benefits (the identified ‘reasons’ or the ‘intents’ for the Program

Business Case;

‘Enablers’ (the dimensions of People, Processes (IT and Business) and the Technology

drivers);

‘Intermediate Benefits’: this is a critical step for identifying those ‘deliverables’ or

‘outcomes’ of intermediate stages of the Program, which would be then the drivers for

final Program Benefits.

Benefits Analysis: The focus of this step is to quantify the benefits in terms of measurable units.

Metrics, which can be tagged to each identified benefits along with Straw Model, would facilitate

drawing up of relationships amongst these benefits, along the lines of BSC, viz., ‘lead’ and ‘lag’

indicators. This would also facilitate testing hypotheses of ‘cause-and-effect’ relationships along

various, identified benefits, taking the cue from the BSC metaphor.

Benefits Planning: A plan is then created, which includes additional details such as the owner,

the target to be achieved and the units with which to measure benefits, via related projects’

deliverables. It also includes the necessary arrangements and infrastructure needed to be in

place to realize the benefits.

From the view point of program management phases, getting aligned to Benefits Management

phases, it should be noted that any Benefits Realization Plan should be dovetailed to Program

Plan, so that activities related to monitoring benefits accrual could be integrated with Program

Plan and then monitored.

Benefits Realization: Realizing the benefits is then achieved by monitoring progress towards the

planned outcomes, linked to the deliverables of appropriate project or group of projects. An

example could be a new Billing System or an Inventory System, whose efficiency parameters like

Speed, Capacity or Number of Transactions can be numerically verified and be compared with

the earlier systems. Deviations from plan can be picked up early, with appropriate corrective

action being planned and taken up. Throughout this process the business case should be

updated and maintained as there will inevitably be differences between what was initially

proposed and what is attainable as the program progresses.

Benefits Reporting: Reporting of metrics related to the identified-benefits (‘actual versus

planned’) is provided to the Governance body in accordance with the plan, facilitating

accountability of performance of new systems in place, as deliverables and outcomes of the

program are getting realized.

The final review and evaluation of benefits should ideally be completed after the business

transformation has fully completed and changes are embedded into the subsequent operations,

as ‘business as usual’. This is can take several months after a system ‘go-live’.

The following section explains the BSC from a middle-out approach to designing score cards.

3. BSC design – the ‘Middle out’

There are cases in literature (Alleman, 2003), where BSC was used as a pure Project

Management element, complementing the traditional project management and control

mechanism. But the design of BSC was attempted from a top down approach. Goold et al.

(1994), describe three types of ‘Parenting Styles’, viz. strategic planning, strategic control and

financial control, for the roles and responsibilities between corporate and organisational units.

These types of styles also influence the role the corporate would adopt in the design and usage

of BSC across corporate and business units (De Waal, 2007). Usually for large programs, we can

adopt a method that has parallel to ‘strategic control’ style, wherein the corporate (the Program

Vision, in this case) would influence the design of scorecard, but it would be the Business

Benefits Plan that would influence the usage of it.

When a program gets sanctioned, the program sponsor, steering committee, business

stakeholders and the program and project managers go through the Benefits Management steps

(Identify, Analyse, Plan) to develop a Straw Model, Metrics and their BSC relationships and thus,

a first cut BSC for the program, much akin to a Corporate BSC is arrived at.

In this approach, the program core team would work out multiple iterations, to arrive at individual

scorecards across the individual projects (re-using many prevailing project measures) and

connecting them to the Program BSC, to arrive at a consensus that is aligned with the proposed

Benefits metrics. One would be able to retain many measures that can be used at individual

project teams’ level, while choosing the ones that would get ‘aggregated’ at program level

Benefits scorecard. Thus, program Benefits Realization expectations (metrics) were typically

‘cascaded’ downwards as BSC measures (from Program Benefits BSC to Projects Teams’

Delivery – Benefits BSC), while re-using typical project delivery measures for ‘aggregating’

upwards.

3.1. Characteristics of BSC design – the ‘Middle out’

The design process is typically recursive at each time when a new project team is added to the

program portfolio and / or when the Program Benefits or Business case is changed due to

unexpected events; and that the participating project team contributes to the BSC Benefits BSC

design more, by way of carrying forward their set of existing measures. Thus we would prefer to

call the design approach the ‘Middle-out’, compared to top down mode of designing scorecards.

The following steps would describe the process of this design approach:

1. Start-up / or from a previous steady state phase: Existing islands of projects in the Program

portfolio, (with Identified & Analysed Benefits, independent Service Level Agreements (SLA), Key

Performance Indicators (KPI) and measures) focus on their operational efficiency, project

management and control, besides monitoring for Risk management and Benefits Management.

2. Coalescence phase: When a new program is sanctioned, driven by the program vision and goals,

coalescence comes into play. The steps in this phase are -

1. Select pilot projects that have similar and comparable SLAs and KPIs and Delivery

Benefits metrics.

2. Derive ‘tactical themes’ as opposed to Corporate Strategic Themes. (The example is –

Project Delivery Benefits as opposed to Program Vision BSC).

3. Develop Strategy Map and the Benefits Straw Map from the new Delivery Benefits goals

and the identified program benefits to derive the new set of Benefits measures.

4. Assign targets with tolerance ranges (Green / Amber / Red) for the finalized measures

that would drive the projects’ and program goals to fruition.

5. Analyse new risk/impediments profiles and mitigation plans.

6. Derive the new governance model, Benefits Straw Model and Metrics; get approval for

the same.

3. Communication Phase: Publish the Program performance and Benefits Scorecard to

stakeholders and draw up communication and change management plans. (Town hall meetings,

Training, Kiosks for Demonstrations, etc. as required).

4. Implementation Phase: Go Live and monitor. (Closure / start steady state phase)

5. Iterate from ‘Coalescence’, when new projects join.

The following figure (Fig. 2) depicts the design elements.

Fig 2: Strategy Maps for BSC Design Middle-Out

We can compare the traditional Balanced Scorecard approach (the first generation) with the

middle-out approach in the following way (Table 1).

BSC Top Down BSC Middle-Out

Starts at Organizational top;

Corporate Vision driven SLAs / KPI

Focus on Customer – Vendor

Relationship, Portfolio / Program

Objectives; Benefits driven SLAs

Long Term planned (3-5 years) Short Term focused (1-2 years)

Start from Financial goals

(Perspective) and derive other

Perspectives. Identify Strategic

Initiatives (as relevant).

Start from Customer Expectations on

Portfolio & program Benefits and

distribute Delivery metrics and SLAs

across relevant BSC Perspectives

Usually top-down approach to BSC ‘Middle-out’ design’; iterative process of

top-down (from Portfolio Benefits and

design SLAs) and bottom-up, where the

quantum of contribution is more from

Projects’ level (Delivery Benefits and

operational parameters for arriving at

measures and targets).

Strategy Maps are enablers for

BSC design; they validate the

Strategic Themes.

Strategy Maps and Benefits Straw maps

drive the design.

Changes to Dash board measures

are generally minimal at Corporate

level BSC.

Flexible to changes to measures or their

targets both at Projects’ level and at

‘Internal Processes’ Perspective of

individual Scorecards.

Table 1: Comparison of BSC Design approaches

Thus, the Program Benefits Scorecard can evolve from the relationship to individual projects’

level BSCs easily. Also, the scorecard structure (parent – children scorecards) can be extended

to more projects / changed Business Cases, at different ‘coalescence’ phases, as the maturity of

Benefits Management in the program organization grows.

3.2. Performance Index

For the purpose of monitoring performance, as well for the purpose of rewards recognition, the

individual measures can be given ‘weights’ (though, during the time of piloting, the weights can be

set to a value of 1) and their performance deviation can be measured at regular intervals. The

individual measures’ performance values can then aggregated for specific BSC (4) perspectives,

as well as at individual scorecard levels. Thus we would have various ‘weighted performance of

measures’, which are called Performance Indices (PI) on the scorecards. This idea can help the

program in a significant way, by comparing PIs across various perspectives, across scorecards

as well as across individual teams.

Thus Benefits Realization can be monitored through a set of ‘lead’ and ‘lag’ measures across

various projects that make up the program, while PIs can also help in identifying the cause-and-

effect’ hypotheses.

4. Recommendation for BSC Implementation

The launch preparation phase would typically last about 4 weeks, when internal marketing

campaign should be conducted. The project teams, project and program manager(s), and the

program core team would then freeze the scorecard elements (that include the benefits’

measures, negotiated targets and their target deviation zones) and address the program

roadblocks and issues. The key elements for scripting a success story of Program Benefits BSC

implementation are the town hall meetings with individual teams and stakeholders, content-rich

collaterals for internal marketing purposes and self-running demonstration kits from the program

portal for the user groups.

In the final mode of governance, we should superpose the new Benefits BSC based program

review, while retaining the existing delivery performance review mechanisms at the Program /

Project level, wherever required. This would help the program to track important program specific

benefits’ measures, while facilitating the need-driven data drill down at individual projects’ level.

Also more importantly, the adoption of BSC dashboard would facilitate better corporate

communication on business, program and project specific audiences, while parallel tracking the

Benefits metrics for their performance.

5. Some closing thoughts

Our focus in this paper has been that BSC dashboard will be the link in corporate communication

amongst business, IT-program and IT-project teams that have both delivery based and benefits

based BSC metrics, given the importance of tracking the program goals through their benefits.

Given below (Fig. 3) is an example of Program BSC (for the sake of confidentiality, we have

masked the actual numbers; data points from April-09 onwards were re-constructed for display).

Fig 3: BSC Program scorecard

***

Sr # Performance Measure Unit KPI Target Frequency Apr '09

Tre

nd May

'09 Tre

nd

Jun '09

Tre

nd

Finance

1 TCO Savings (Direct / Indirect) $ N Half yearly

2 Sprint Burnup rate $ Y Monthly

Customer 0.80 0.80 0.80

3 Customer Satisfaction Index (Overall) % Y 90% Half yearly 84% 84% 84%

4 CSI - Most important parameters rated low % Y 10% Half yearly 50% 11% 11%

5CSI - Most important Service & Business Goals

parameters rated high% Y 80% Half yearly 86% 86% 86%

6 Customer Appreciations # N Monthly 8 9 6

7 Customer Complaints # N Monthly 0 0 3

8 Quality of Service (from annual survey) # N Yearly

Process & Delivery 0.54 0.54 0.66

9 Post Delivery Defects # Y 5 Monthly 2 4 4

10 Sprint review Meeting attendance % Y 100% Monthly 100 100 90

11 Monthly Governance % Y 100% Monthly 33% 67% 60%

12 Velocity (per Sprint) (Total) # Y 15 Monthly 14 14 14

13 Velocity (per Sprint) (per Scrum team) # Y 3 Monthly 3 3 3

14 Burn down deviation (per Sprint) % Y 5% Monthly 8.0% 8.0% 9.0%

15 SLA compliance to response time (Resources SLA) % Y 95% Monthly 99.7% 99.6% 99.4%

16 SLA compliance to resolution time (Resources SLA) % Y 95% Monthly 97.0% 96.6% 96.7%

Learning, People & Competency 0.40 0.90 0.90

17 Compliance to minimum competency level % Y 100% Monthly 80% 80% 80%

18 Unplanned Attrition in critical phases # Y 0 Monthly 1 0 0

19 Upload activity of assets into KM system # N Monthly 0 0 0

20 Reference activity of assets in KM system # N Monthly 0 0 0

Portfolio Performance Index 0.58 0.58 0.72

0

References

2GC (2009), “Performance Management & 3rd

Generation Balanced Scorecard”, 2GC Active

Management, <www.2gc.co.uk>, (Accessed 02/01/2010).

Advanced Development Methods, (2003), “Scrum Methodology: Incremental, Iterative Software

Development from Agile processes”, <www.controlchaos.com> (Accessed 02/01/2010).

Alleman, G.B. (2003), “Using Balanced Scorecard to Build a Project Focused IT Organization”, in

Proceedings of Balanced Scorecard Conference, San Francisco.

De Waal, A. (2007), “Strategic Performance Management: A managerial and behavioural

approach”, Palgrave Macmillan, New York.

Goold, M., Campbell, A. and Alexander, M., (1994), “Corporate Level Strategy: Creating value in

the multibusiness company”, Wiley, New York.

Kaplan, R.S. and Norton, D.P. (1992), “The balanced scorecard: measures that drive

performance”, Harvard Business Review, January-February 1992, pp. 71-80.

Kaplan, R.S. and Norton, D.P. (1993), “Putting the Balanced Scorecard to Work”, Harvard

Business Review, September – October, 2-16.

Kaplan, R.S. and Norton, D.P. (2000), “The Strategy-Focused Organization: How balanced

scorecard companies thrive in the new business environment”, Harvard Business School Press,

Boston, Massachusetts.

Sliger, M. (2007), “A Project Manager’s Survival Guide to Going Agile”, Rally Software

Development Corp. < www.rallydev.com > (Accessed 02/01/2010).

================================end of text==============================