OECD 2016 Performance and Results Survey

Lisa von Trapp

12th Annual Meeting of the OECD SBO Performance and Results Network OECD Headquarters, Paris

24-25 November 2016

OECD Performance Budgeting Survey 2016

• Fourth OECD PB Survey

• Coverage: 65 questions on

– Performance budgeting

– Spending review (second iteration)

– Evaluation

• Respondents mainly from CBAs

• Response rate: 32 countries

• Some data cleaning is ongoing

Part I – Performance Budgeting

Key Observations

Performance budgeting (PB) 1. Performance frameworks are the norm

in OECD 2. Most countries set performance targets

but number and coverage varies 3. Institutions across government report

being more engaged in performance budgeting

4. Accountability and transparency: key priorities and benefits

5. Management responses to poor performance still more likely than budgetary consequences

6. Mixed signals on whether on whether countries believe performance budgeting has improved the quality of public finances

Performance Budgeting Performance frameworks are the norm in OECD

Source: 2016 OECD Performance Budgeting Survey (preliminary results, subject to revision)

17 6 2

3 4

Yes (25)

No (9)

compulsory for line ministries & agencies compulsory for line ministries only

Optional

No standard framework, Line ministries have

their own

None

Performance Budgeting Most countries set performance targets but number and coverage varies

Coverage and responsibility for setting targets

38%

27%

35%

All programmes

Most programmes

Only priorityprogrammes

0 10 20 30

Internal Audit

Other

Supreme Audit

Legislature

Chief Executive

CBA

Agencies

Line Ministries

Number of responding countries

2016

2011

2007

Performance Budgeting

Institutions across government report being more engaged in performance budgeting

Source: 2016 OECD Performance Budgeting Survey, Q20 preliminary results, subject to revision)

Which institutions play important roles in the performance budgeting process?

0

10

20

30

40

50

60

70

80

90

2016

2011

Performance Budgeting Slight increase in use of performance information in budget negotiations but…

Source: 2016 OECD Performance Budgeting Survey (preliminary results, subject to revision)

How often do the CBA and line ministries use the following kinds of performance information in their budget negotiations?

1 2 3 4 5

Independent performance information

Spending Review

Statistical information

Line Ministries/Agencies performance evaluations

Line Ministries/ Agencies operational data and performancereports

Line Ministries'/Agencies' financial data

2016

2011

NEVER ALWAYS

Performance Budgeting …accountability and transparency seen as the the key priorities and benefits

Ranking of the importance of factors behind introducing PB and effectiveness of PB system in advancing them

Index

Setting service delivery targets

Legal compliance

Coordinating government-wide strategic objectives

Facilitating evaluations/value-for-money studies

Allocation and prioritisation

Parliamentary budget scrutiny

Culture of performance

Transparency

Accountability

Effectiveness

Rationale

Low High

Source: 2016 OECD Performance Budgeting Survey, Q22-23 (preliminary results, subject to revision) Note: Index calculated as high=5, medium-high=4, medium=3, low-medium=2, low=1, N/A=0. Depicted as sum of all OECD member countries.

Performance Budgeting Management responses to poor performance still more likely than

budgetary consequences

NEVER ALWAYS

If performance targets are not met, how likely is it that the following consequences are triggered?

Source: 2016 OECD Performance Budgeting Survey, Q37 (preliminary results, subject to revision)

1 2 3 4 5

Pay cut for headProgramme eliminated

Programme transferred to othersBudget decreasesBudget increases

Budget freezesNegative consequences for leadersMore training for programme staff

More staff assigned to…New leadership brought in

Spending review or evaluationMore intense monitoring in future

No consequencesPoor performance made public

2007

2011

2016

Performance Budgeting Mixed signals on whether on whether countries believe performance

budgeting has improved the quality of public finances

Has PB contributed to improved quality of public finances in your country? Number of countries

Source: 2016 OECD Performance Budgeting Survey, Q37 (preliminary results, subject to revision)

5

1

10

13

5

-5

-2

-7

-2

-6

Strongly agree

Somewhat agree

Somewhat disagree

Strongly disagree

No benefit

Non-quantifiable benefits

Quantifiable benefits

Performance Budgeting Certain sectors stand out where performance information is used more

frequently and with higher impact on budgetary decisions

Top 4

Environment

Social affairs

Health

Education

Bottom 4

Internal operations

Trust in government

Inclusiveness

Gender equality

Source: 2016 OECD Performance Budgeting Survey, Q37 (preliminary results, subject to revision)

Performance Budgeting Greatest challenges to implementation

1. Lack of performance culture

2. Lack of resources

3. Lack of capacity/training

4. Lack of accurate/timely data

5. Lack of info on efficiency

Part II – Spending Review

Key Observations

Spending review (SR)

1. More countries than ever conduct spending reviews

2. The CBA tends to lead spending review exercises

3. Spending review outcomes are not always clear

Spending Reviews More countries than ever conduct spending reviews

Source: 2016 OECD Performance Budgeting Survey, Q56 (preliminary results, subject to revision)

0

2

4

6

8

10

12

2000 2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011 2012 2013 2014 2015 2016

No

. of

spe

nd

ing

revi

ews

acro

ss O

ECD

re

spo

nd

en

ts e

ach

ye

ar

Comprehensive (20-100% of total government spending)

Broad (5-20% of total government spending)

Narrow (0-5% of total government spending)

Spending Reviews

The CBA tends to lead spending review exercises

Three governance models for spending review Percentage of spending review functions performed by ‘centre of government’ institutions (CBA or President/PM’s office)

Source: 2016 OECD Performance Budgeting Survey, Q54 (preliminary results, subject to revision) Note: Japan uses a committee-based review model.

0%

20%

40%

60%

80%

100%

PL DK NL DE SE US SL IT CH LX FI MX BE FR CAN GR UK PT IR NZ

Ce

ntr

alis

atio

n in

dex

Pres/PM CBA

Pres./PM centric (3)

CBA centric (10)

Line ministry centric (6)

Spending Reviews Spending review outcomes are not always clear

Source: 2016 OECD Performance Budgeting Survey, Q57 (preliminary results, subject to revision)

47%

33%

20%

All met

Some met

None met

Cannot be assessed

27%

33%

27%

13%

Were the objectives of the spending review met?

Fiscal objectives

Output and outcome objectives

Spending Reviews Major challenges around availability and quality of PI for spending reviews

Source: 2016 OECD Performance Budgeting Survey , Q58 ( preliminary results, subject to revision)

0 10 20 30 40 50 60

Lack of ICT

Gaming

Lack of political support (legislative)

Lack of capacity (e.g. available staff)

Lack of framework

Lack of support (senior civil service)

Lack of capability (e.g. technical expertise in terms of analysis or…

Lack of political support (executive)

Lack of attention on implementation/execution

Lack of time (e.g. short time frame for implementation)

Poor quality of performance information/data

Lack of performance information/data

What have been the greatest challenges to ensuring spending reviews are successful?

Part III – Evaluation

Key Observations

Evaluation and Audit

1. Evaluation is more

decentralised than PB and SR

2. Evaluations continue to have a limited budgetary impact

Evaluation Evaluation is more decentralised than PB and spending review

Source: 2016 OECD Performance Budgeting Survey, Q42 (preliminary results, subject to revision)

0%

5%

10%

15%

20%

25%

30%

35%

40%

LineMinistries /

Agencies

CBA SupremeAudit

Institution

Otherexternal

body

Governmentevaluationservice(s)

Legislature

Which organisations are responsible for conducting evaluations?

Evaluation Evaluation is more decentralised than PB and spending review

Many organisations within OECD member countries have expanded their roles in evaluation since 2011 Percentage of organisations

Source: 2016 OECD Performance Budgeting Survey, Q42 (preliminary results, subject to revision)

0

20

40

60

80

100Chief Executive

CBA

Line Ministries

AgenciesLegislature

Supreme Audit

Internal Audit

2016 2011

Evaluation Evaluations continue to have a limited budgetary impact

How impactful is information generated through the following types of ex post evaluation on budget decisions?

Source: 2016 OECD Performance Budgeting Survey, Q30 (preliminary results, subject to revision)

0

2

4

6

8

10

12

14

16

Always Usually Occasionally Rarely Never

Nu

mb

er o

f O

ECD

res

po

nd

ents

Used in CBA-Line Ministry negotiations

Used in Line Ministry-Agency negotiation

Evaluation Evaluations continue to have a limited budgetary impact

Countries indicated the following challenges:

– Evaluations have considerable delays between policy

implementation and evaluation publication, limiting their relevance to policy and budget makers.

– The evaluation process can often create added bureaucracy and drain on existing financial and human resources. The benefits may not, at times, justify the effort, weakening the legitimacy of evaluation as an information gathering tool.

– Meagre results in evaluation are partly an indication of evaluation as a technically difficult exercise, but it are also symptomatic of poorly aligned incentives.

For more information

OECD Senior Budget Officials Network on Performance and Results please visit:

www.oecd.org/gov/budgeting/seniorbudgetofficialsnetworkonperformanceandresults

Thank you!