JPO Training Programme

October-November 2011

“Results Based Management: Theory

and Application”

Marielza Oliveira

Operations Support Group/Executive Office

Organizations get what they focus on.

If an organization wants to get high-level results (national impact), then this is what

it must focus on.

Key Message 1

MEANING & PURPOSE OF RBM

“A broad management strategy aimed at achieving improved performance and

demonstrable results ” RBM helps us to answer 4 related questions:

1. Why does the organization exist, and is it serving its

purpose?

2. Whose needs is the organization intended to serve, and is

it serving those needs effectively?

3. What is the organization supposed to deliver, and is it

delivering this effectively?

4. What are the resources available, and is the organization

using them efficiently?

10

Holistic RBM

Str

ate

gic

Pla

nnin

g 1. Identifying clear and measurable objectives

2. Selecting indicators

3. Setting explicit targets

Perf

orm

ance

Measure

ment

4. Developing performance monitoring systems

5. Reviewing, analyzing and reporting actual results vis-à-vis targets

Results-B

ased (

Perf

orm

ance)

Managem

ent

6. Using evaluation findings to generate lessons and increase the understanding of strengths, weaknesses and comparative advantages

7. Using performance information for internal management accountability, learning, resource allocation decisions including human resources management, and reporting to stakeholders and partners 7-9

THE RBM LIFE-CYCLE APPROACH

10

WHAT IS THE “RESULT” FOR WHICH GOVERNMENT, UNDP AND OTHER PARTNERS ARE WORKING?

“Real and meaningful improvement in people’s

lives and the opportunities open to them.”

1. Not just for a few persons, but for the majority (or for a significant part of the population)

2. Change and improvements must be sustainable – national ownership and capacity

13

Evolution of RBM in UNDP • Decision 98/1 - adopted guiding principles for sharpening the

programmatic focus of UNDP, and requested that the Administrator operationalize them, including a mechanism for implementation, impact measurement and evaluation.

• Decision 98/23 - directed UNDP to develop a multi-year funding framework integrating programme objectives, resources and outcomes within the corporate priorities and focus.

– Consistent with the recommendation of the SG to the UNGA that MYFFs be used throughout the United Nations system as pledging mechanisms under which donors could link their financial contributions to results, programme performance and aid effectiveness.

– First UNDP MYFF presented to the EB in 1999 initiated the transformation of UNDP into a results-based organization

• Business Plan 2000-2003 + MYFF 2000-2003

• MYFF 2004-2007

– 5 Strategic Goals, with 30 Service Lines

• 4 focus areas

– Poverty and MDG (12 outcomes)

– Democratic Governance (9 outcomes)

– Crisis Prevention and Recovery (9 outcomes)

– Environment and Energy (4 outcomes)

• 5 cross-cutting issues

– Capacity development

– Gender equality

– South-South cooperation

– National ownership

– Aid effectiveness

• Coordination – 8 outputs: improving UNDS

effectiveness, efficiency and coherence; strengthening ownership of UNRC system; improving UNRC knowledge management and capacities; resource mobilization and reporting; strengthening partnerships.

• Management – 12 outputs: accountability; staff

security; financial and audit management; programming arrangements; talent and career management; gender parity; RBM; knowledge management

• South-South cooperation – 3 outputs: mainstreaming SSC;

enhancing contributions to development; increasing effectiveness

Strategic Plan 2008-2011 (then 2013)

How efficient is UNDP as an

organization? (How well to we manage

inputs: HR, finances, ICT, premises,

etc.?)

How well does UNDP contribute to

development results?

Human Development

IRF – Operational Efficiency

IRF – Operational Effectiveness IRF – UN Development Coordination

DRF – evaluative evidence

Available Resources

Results-based management at UNDP (before MTR)

• Fuel, food and financial/economic crises since 2008 = tighter aid budgets, focus on foreign policy objectives

• New events – MDG Summit

– International agreements in Cancun and Nagoya

– Rio + 20

– Least Developed Countries Conference

– Beijing + 15

• Need for all development organizations to more clearly articulate “Value for Money” – donors are “ranking” partners to make funding decisions

• Rise of MICs and emerging donors

Changes in Global Scenario

Mid-Term Review • The most extensive analysis UNDP has ever done of

its performance, over a period of time

– Results-Oriented Annual Reports (ROAR) narratives submitted by the various UNDP programming units at global, regional and country levels

– Statistical analysis of outcomes and expenditures at country level

– Findings from decentralized evaluations

– Case studies, assessments (ex: 30 MDG assessments)

– Partner surveys (UNDP survey, Multilateral Organization Performance Analysis Network, etc)

– Trends detected in new Country Programme Document submissions

Poverty and MDGs

Democratic Governance

Crisis Prevention and Recovery

Environment and Sust. Development

Substantive findings

Evaluation recommendations • Programmatic “how”

– Leverage downstream results for transformational change so that they inform policy, learning, and possible scale-up or replication

– Need to better identify and codify successful experiences

– Work more across practices and across regions instead of in silos

– Mainstream national capacity development throughout programming

– Improve knowledge management & South-South Cooperation

• Operational “how” – Streamline internal processes to reduce burden and enhance

efficiencies and effectiveness at country level

• Faster procurement processes

• Improved talent management

2008-2010 Country expenditures by Focus Area

Focus area 2008 2009 2010

Poverty & MDGs $ 1,199 $ 1,117 $ 1,205

Democratic Governance $ 1,414 $ 1,445 $ 1,133

CPR $ 612 $ 576 $ 1,158

Environment & Energy $ 373 $ 413 $ 406

Unlinked $ 158 $ 188 $ 324

Total $ 3,757 $ 3,752 $ 4,328

* In USD millions

2010 country expenditures by source of funds (percentages by focus area in each source of funds)

Focus area Regular

Resources

Other Donor

Resources

Local

Resources

Poverty & MDGs $ 158

(33.1%)

$ 738 (24.5%)

$ 335 (41.5%)

Democratic

Governance $ 154

(32.3%)

$ 713 (23.7%)

$ 281 (35.5%)

CPR $ 75

(15.9%)

$ 932 (31.0%)

$ 36 (4.8%)

Environment &

Energy $ 29

(6.2%)

$ 268 (8.9%)

$ 100 (12.0%)

Unlinked $ 60

(12.5%)

$ 356 (11.8%)

$ 47 (6.3%)

Total $ 478

(100.0%)

$ 3,007 (100.0%)

$ 801 (100.0%)

* In USD millions

2010: Outcomes representing 50% expenditures by Typology

All

countries

SDS = 4 LDC = 7

+ other

LIC = 5 L-MIC =

4

U-MIC =

3

NCC = 2

+ other

Pov + MDG Pov +

MDG

Pov +

MDG

Pov +

MDG

Ineq +

MDG

Ineq +

MDG

Ineq +

MDG

Private

sector

AIDS

integrate

P/M/E

MDG

Private

sector

Public

services

Public

services

“Other”

AIDS funds Elections Local dev AIDS

funds

Conflict

recover

Mainst EE Mainst

EE

Elections Conflict

gov CD

Public

services

Public

services

Mainst EE

Public

services

NHRI CD Mainst

EE

Conflict gov

CD

“Other”

Mainst EE Conflict

recover

Mainst

EE

Red = Poverty & MDGs / Blue – Democratic Governance

Black = CPR / Green = Environment & Energy

• Our stakeholders want to know

– That UNDP is sharpening its strategic direction, relevance in different settings, and focus on transformational development results

– What is UNDP’s specific contribution as reflected in a more robust results chain and accompanying monitoring and reporting system

– What/how we’re doing: predictability, transparency and clarity of reporting so everyone can understand – both what is succeeding and what is not succeeding, and where work is in progress

• Upcoming conversations with the EB will be “existential” for UNDP! What/how we “do now” greatly affects our positioning.

EB Dynamics and Expectations

• Address recurring evaluation findings:

– Leverage downstream results for transformational change: inform policy, scale-up, replicate, learn – and tell the story

– Manage and make visible our knowledge, identify and codify successful

experiences

– Find the substantive space that brings practices together (e.g., poverty/environment), don’t work in silos – and tell that story

– Mainstream national capacity development throughout programming – and tell that story

– Highlight South-South cooperation – and tell that story

– Streamline processes to reduce burden and enhance value for money

• Improve programme management, starting with more serious use of measurement (2800 outcome indicators, only 10% measured) and learning (including from evaluation); invest in high quality CPDs that are meaningful for and underpin the entire programme cycle

• Avoid fragmentation and increase alignment of projects in support of outcomes

• Improve monitoring , document it to support stronger and more meaningful evaluations; make better use of evaluation findings; invest in improving the quality of decentralized evaluations and management response; and learn about what works, what doesn’t work, and how that can improve our effectiveness as a development partner – and tell that story

Our commitments to the Executive Board

Current UNDP Results Map

RCA Key Results (Outputs)

RCA Key Results (Outputs)

RCA Key Results (Outputs)

RCA Key Results (Outputs)

Strategic Plan Development Results Framework Institutional Results Framework

Integrated Work Plan (IWP) Unit Level Results (Outputs)

Programme Resources

Management Resources

Corporate Level

Global, Regional and Country Level

Unit (HQ/CO) Level

Staff Level

Evaluation

Audit

Output

Development Results

UNDAF/CPD/CPAP Outcomes

Output Output

BSC Results (Outputs)

Strategic Plan 2008-2011, extended to 2013 (post MTR)

• 4 focus areas

– Poverty and MDG (7 outcomes)

– Democratic Governance (9 outcomes)

– Crisis Prevention and Recovery (6 outcomes)

– Environment and Energy (3 outcomes)

• Development Effectiveness – 5 outputs: quality of programming,

knowledge management, capacity development, gender equality, SSC

• Coordination – 3 outputs: UNRC system

management, ownership and coordination

• Management – 7 outputs: effective leadership,

accountability, HR, partnerships, input management, security, oversight & internal controls

Results-based management at UNDP (post MTR)

Level 1 – How efficient is UNDP as

an organization? (How well to we

manage inputs: HR, finances, ICT,

premises, etc.?)

Level 3 – How well does UNDP

contribute to development

outcomes?

Level 2 – How effective are UNDP

operations? (in CD, KM, SSC,

partnership building, resource

mobilization, etc)

Human Development

IRF – Operational Efficiency

DRF - Development Effectiveness IRF – Operational Effectiveness

IRF – UN Development Coordination

DRF – evaluative evidence

Available Resources

Transformational results

Value for money, SROI

Key Message 2

RBM is about much more than outcomes/ outputs/ indicators being pasted into a

matrix...

It’s about having a clear articulation of what you want to change and how to make it

happen.

Problem Analysis

GATHERING INFORMATION About development problems from existing sources,

Incl. national treaty reports and observations and

recommendations from treaty bodies

ANALYSIS Of root causes,

capacity issues &

their linkages

ASSESSMENT Shortlist major development problems

for deeper analysis

Negative outcomes, manifestations of

problems, unfulfilled rights

Root causes Beliefs, attitudes, culture, traditions Natural resources, natural disasters

Political and economic systems, ideologies, conflict

Underlying causes and capacity issues

Policies, laws, budgets

Systems for service delivery

Behaviors and practices, low household incomes

Immediate causes

Affecting individuals and households

Causality analysis: “why?”

38-42

Problem Tree

Malnutrition &

death

Inadequate dietary

intake Disease

Insufficient

food security

Inadequate Maternal

& Child Care services

Insufficient health services

& unhealthy environment

Political, Ideological,

Economic structures

Resource Control

+

Organizational structures

Root

causes

Underlying

causes

Immediate

causes

Manifestations

Formulate problem in a neutral manner

• “Minorities and marginalized groups do not have the right to vote.” Versus

• “Minorities and other marginalized groups do not participate in elections.” Or

• “Low levels of participation by minorities in elections.”

33-35

• A theory of change is a strategy for change.

• It explains all the major things that need be in place in order for development change to occur.

• It is not about what one Agency must do, but what all partners and non-partners must do to make real change happen .

• It is not about projects, but all the different types of interventions necessary for change to happen.

THEORY OF CHANGE

44-48

WHAT SOLUTION?

If a problem is caused

by conditions three

If a problem is caused

by conditions three

All three conditions

must be addressed

WHAT SOLUTION?

MAKING THE STRATEGIC CHOICE The Value, Support, Capacity Model

VALUE SUPPORT CAPACITY & COMPARATIVE

ADVANTAGE

Solving this problem would

bring significant value to the

country/region.

1. It is a global, regional,

national priority

2. It supports the

country/region in

achieving an MDG or

other major development

priority

3. There is regional or

national ownership of

the issues.

Would we have support to

work towards solving this

problem:

1. It is in line with our

mandate (Executive

Board or Senior

Management Support)

2. We can count on

government and others

to partner with us.

3. We can count on the

support of those with

decision-making power

and resources.

Would we have the capacity

and comparative advantage

to work on the problem:

1. We have or can put in

place capacity quickly to

address the problem.

2. We have the skills and

resources to provide

support more effectively

or efficiently than others

3. We have unique

resources and/or

attributes (e.g., neutrality,

legitimacy, reputation,

convening role)

37

• Programme design / planning - serves as a tool to develop programme strategy and to explain programme concepts and approach.

• Programme implementation – provides the elements for monitoring and performance improvement.

• Programme evaluation and reporting – presents program information and progress for accountability and communications.

THEORY OF CHANGE

Strategic Priority Setting: “Inclusion”

UNDP

comparativ

e

advantage

Major

national

challenge

Alignment of

key actors to

support action

1. “Just do it”/strategic priority

2. Potential high priority, if

consensus can be built

3. Potential high priority, if others

cannot meet demand and internal

capacity development is feasible

4. Lower priority: does not meet

major national challenge

Value

Support

Capacity

37

Who is involved in defining the solution?

ENGAGING STAKEHOLDERS IN

PLANNING FOR RESULTS

Who should be involved in planning for and implementing change?

Always ask: “whose voice is normally not

heard on this issue?”

25-29

STAKEHOLDER ANALYSIS – STEP 1 Stakeholders (examples) Interest in Activity Nature of Interest

(+ or -??)

Office of the Prime Minister Greater citizen participation +

Universities Political culture and civic behaviour +

Main political parties Free and fair elections

Opportunities for greater influence?

+

+/-

Church umbrella organization Ethics in politics, fairness +

NGO group(s) Fairness. Greater influence +

Private sector organization(s) Opportunities for influence, fairness +/-

Minority group(s) representatives Opportunities to participate +

Youth umbrella organization Opportunities to participate, fairness +

Electoral administrative body Maintain neutrality/independence +

International observer group Fairness +

Citizens Organisations Rights of citizens, fairness +

Women’s organization Rights of women, fairness +

Informal political leaders/”dons” Threats to their power - 25-29

STAKEHOLDER ANALYSIS – STEP 2

Stakeholders (examples) Importance

(Scale of 1 to 5, 5 =

highest)

Influence

(Scale of 1 to 5, 5 =

highest)

Office of the Prime Minister 5 5

Universities 3 2

Main political parties 5 4

Church umbrella organization 3 2

NGO group(s) (e.g. a watchdog NGO) 4 3

Private sector organization(s) 3 4

Minority group(s) representatives 5 1

Youth umbrella organization 5 1

Electoral administrative body 4 3

International observer group 1 3

Citizens Organization’s 5 2

Women’s organization 5 2

Informal political leaders/”dons” 2 4 25-29

STAKEHOLDER ANALYSIS – STEP 3 (Importance and Influence Matrix)

Group 1: High

Importance/Low Influence

Stakeholders.

Group 2: High

Importance/High Influence

Stakeholders.

Group 3: Low

Importance/Low Influence

Stakeholders.

Group 4: Low Importance/High

Influence Stakeholders.

Importance

Influence 25-29

Develop a strategy for each relevant group

Case Study:

Women’s Political Participation

The Ministry of Social Affairs in Erbamia has written to UNDP asking for support on the issue of women’s political participation. Women are

generally poorly represented in all levels of the political system and are also under-represented in elections and other forms of engagement in

decision-making. In the letter, the Ministry points to the problem of inadequate support for women from civil society organizations. The letter

further specifies that the Ministry would like to move forward in developing a $500,000 project that would help train and equip Civil Society

Organizations that support women political activists. The project would also support the opening of a women’s political resource centre in the

National Institute of Public Administration.

How should UNDP respond? What should be done? Who should be involved in defining solutions?

Draft a letter of response to the Ministry.

• What are RISKS?

• What are ASSUMPTIONS?

What could affect program performance?

Definitions

“Risk corresponds to a future event, beyond our control, that may impact the achievement of UNDP's objectives. ” Risks may, however, also present new opportunities.

Assumptions – things that need to be in place for the programme or project to be successful, which are outside of our control, but which we believe have a moderate to high probability of occurring. Assumptions are positive – we assume they will happen. Risks are negative – we think they might happen and negatively affect our efforts.

Risks can be assessed based on impact and probability.

Since future impacts can be both positive and negative, risks can take the form of either threats that would prevent UNDP from achieving its objectives, or opportunities that would enhance the organization’s ability to achieve its objectives.

49-51

Score Rating Likelihood of occurring

5 Expected >80%

4 Highly likely 60% - 80%

3 Likely 40% - 60%

2 Not likely 20% - 40%

1 Slight < 20%

Probability of risk occurring

Score

Rating

5 Critical

4 Very high

3 High

2 Moderate

1 Low

Risk impact

Impact on development

results?

Impact on programme and

operations?

Impact on safety and security?

Impact on Reputation?

Financial impact?

Time for Recovery?

What would be the Organizational and Operational Scope?

Risk impact

Critical impact on outcomes in several focus areas

Inability to perform mission critical programmes and operations for more than six months

Loss of life (Staff, partners, general population)

Reports in key international media for more than one week

Senior management

level

Potentially irrecoverable impact

Organization-wide; inability to continue normal business operations across the organization

Significant impact on outcomes in several focus areas.

Significant disruptions in programmes and operations for one month or longer

Loss of life due to accidents/ non-hostile activities

Comments in international media/forums

Bureau Director level

Recoverable in the long term (i.e. 24-36 months)

Significant ongoing interruptions to operations in 2 or more offices or locations

Significant impact on outcomes in one focus area

Significant disruptions in programmes and operations for less than one month

Injury to UN staff, partners or general population

Several external comments within a country

Manager level 3

(i.e. normally above

USD 100.000)

Recoverable in the medium term (i.e., 12-24 months)

Significant impact within 1 or more departments, offices or locations

Moderate impact on outcomes in one focus area

Disruptions in programmes and operations for less than one week

Loss of infrastructure, equipment and other assets

Isolate comments within a country

Manager level 2

(i.e. normally below USD

100.000)

Recoverable in short term, (i.e., within 12 months)

Moderate impact within 1 or more departments, offices or locations

Low impact on outcomes in one focus area

Moderate disruptions to programmes and operations

Damage to infrastructure, equipment and other assets

Isolate comments within a region of a country

Manager level 1

(i.e. normally below

USD 30.000)

Temporary

Limited impact within 1 or more departments, offices or locations

Likelihood

Imp

act

1 2 3 4 5

1

2

3

4

5

Decision to accept risk by RC/RR

Decision to accept risk by RC/RR in

consultation with ASG

Decision to accept risk by USG

Unacceptable risk

Decision to accept risk by ASG

Decision making and accountability

Guidance for decision-making:

Do the expected gains from an activity justify the associated risk exposure?

Wow, what a marathon! Thanks for your attention today.

What are your questions?