1

Where are we at in the Bank’s Public Management & Employment work?

Nick Manning

Jurgen Blum

May 3nd, 2012

Overview

• The numbers suggest quantity more than quality

• But we have some rebuttals• But, between us, we have some other

problems• Where to from here?

3

The numbers suggest quantity more than quality

The Bank seems to have been quieter about its PM&E work recentlybut the number and lending volumes for this work show no consistent decrease

The IEG reported in 2008 that CS reforms perform below average (based on improvements in CPIA ratings)

CPIA 13 - Quality

of budget a

nd financial m

anagement

CPIA 15 - Quality

of public

administra

tion

CPIA 14 - Efficie

ncy of r

evneue m

obilization

CPIA 16 - Corru

ption, transp

arency

and accountabilit

y0

10

20

30

40

50

60

70

80

Any relevant lendingno relevant lending

Percent of IDA and IBRD countries with CPIA improvements

The IEG report 2008 that CS reforms perform below average (based on improvements in CPIA ratings)

7

But we have some rebuttals

Defense A – it’s no different to most other PSM reforms

-15.00%

-10.00%

-5.00%

0.00%

5.00%

10.00%

15.00%

20.00%

-7.00%*

4.00%

-9.00%*

15.40%**

2.00% -1.00%

Tax projects are the only projects that consistently perform above average (compared to non-PSM projects) at statistically significant levels (5 percent confidence level)

Defense B – even if we control for other factors

Tax is the only contents characteristic that remains a statistically significant predictor of project performance (IEG outcome ratings), when controlling for other contextual and management variables

Administra

tive and ci

vil se

rvice

reform

Decentra

lization

Public exp

enditure

, financia

l management a

nd procu

rement

Tax polic

y and administ

ration

Other acc

ountability/a

nti-corru

ption

Other public

secto

r gove

rnance-4

-202468

1012

-0.963 0.883-1.99

9.74**

0.42 1.34

Percentage point change in project success rates associated with a 10 percent increase in PS thematic share

Defense C – we’re working in particularly tough environments

Compared to non-PSM projects, (upstream) PSM projects are particularly targeted to :

(i) the Africa (AFR, 30.5 percent) and Latin America and the Caribbean (LAC, 30.5 percent) region (see Table 12) and

(ii) towards countries with significantly lower GDP per capita growth (1.6 percent for PSM versus 3.4 percent for non-PSM projects);

(iii) with significantly higher aid flows than non-PSM projects (11.4 versus 6.3 percent); (iv) with a slightly higher degree of political and civil liberties (Freedom House ratings of political and civil liberties are about 0.25 to 0.3 points lower for countries with PSM projects, on a 7-point scale, see Table 10).

(iv) By contrast, there is no conclusive evidence of a “needs-based” upstream PSM project targeting towards countries with lower initial administrative capability.

11

But, between us, we have some other problems

Shhh…. But we’re not sure what we are trying to fix

We know what formal changes

we can make

We know the sorts of results that we would

like to see

We know the sorts of results that we would

like to see

We are uncertain

about what needs to

happen in between!

There is no shortage of propositions about the behaviors that we are looking for

OECD has a set of propositions around these behaviors/upstream performance competencies:1. Workforce planning and management2. Core values3. Staff performance and capacity4. Cooperation between levels of government

The HRM AGI work also had propositions:5. Attract and retain required staff6. Depoliticized, meritocratic HRM practices7. Performance-focusing HRM practices8. Fiscally sustainable wage bill9. Ethical behavior10. Effective working relationships with other cadres

CPIA also has some very detailed propositions

1. Management of operations within the core administrationa. HRM practices within the core administration are structured around education background and merit criteria

clearly and publicly defined. [Q15a1]b. There are clear mechanisms within the public administration to ensure accountability of staff and avoid

conflict of interest and abuse of power. [15a2, Q16c1, Q16c2]c. There is an effective arbitration system for violations of the HRM regime and its decisions are made publicly

available in a timely manner. [Q15a3, Q16b1]d. The employment regime for the core administration is effective in attracting, retaining and motivating

competent staff. [Q15a4, Q15a5, Q15a6]

2. Quality management in policy and regulatory managementa. The policy process is transparent and ensures credibility in social and sectoral policy pronouncements.

[Q15b1]b. There is quality and credibility in the actions of sector regulators. [Q15b2, Q15b4]c. Sector regulators are held accountable for their decisions or lack of. [Q15b5]d. Decisions and regulations are made public available in a timely manner. [Q16b1]

3. Coordination of the public sector HRM regime outside the core administrationa. In the public sector outside of the core administration, HRM practices are structured around merit criteria that

are clearly and publicly defined and the employment regime is effective in attracting, retaining and motivating competent staff. [Q15c1]

b. There are clear mechanisms within the public administration to ensure accountability of staff and avoid conflict of interest and abuse of power. [Q15a2, Q16c1, Q16c2]

c. The aggregate public sector wage bill is not high by comparison with other similar countries. [Q15c2]

What do they all have in common?

We don’t know that they are correct! (although they are not unreasonable!)

16

Where to from here?

Research directions (1)finding out more about what tends to work in general through new metrics

Maybe replicate PEFA – balancing use with research

1. A rapid expansion since 2005:135 countries, 325 assessments in March 2012 2005 2006 2007 2008 2009 2010

05

101520253035404550

8 11 1523

12233

59

5

10

13

13

17

1918

14

7

Led by EC Led by Other Led by WB

2. Growing Nr. of repeat assessments

135 first 52 second 8 third generation

Note: PEFA assessments are shown by lead agency. The year refers to the main mission when most of the interviews for the PEFA assessment were conducted. The number of assessments reported includes both "finalized" and "draft assessments". While every effort was made to take stock of PEFA Assessments comprehensively, some assessments may have been omitted in the dataset.

Research directions (2)finding out more about what tends to work in general through Impact Evaluation

1. IEs provide great value-added for evidence-based reform design, by rigorously identifying the effect of an intervention

2. The PSM IE research agenda is lagging compared to other sectors. 3. There is significant opportunity for expanding the IE research agenda on

Public Sector Governance – and the Bank should play a leading role in this.

4. This is true despite a typically small “n” in PSM reform. • By contrast to health or education reforms, that can be applied

selectively to schools or health centers to construct a counterfactual, PSM reforms often focus on central agencies, such that n is small or a randomized roll-out difficult.

• However, recent research demonstrates that IE methods can successfully be applied to learn about the effects of upstream reform elements – e.g. tax inspector incentives or the impact of performance-related pay schemes.

Move towards more standardized clusters of project indicators to encourage peer learning

ISPMS

Research directions (3)finding out more about what tends to work in general by learning from projects

19

NOT: a straight-jacket! (not standardizing indicators)

BUT: A way of better peer-learning from projects throughstandardized “menus” of indicators, e.g. organized by:

1. What is the specific reform area? (pay reform, regrading, agency creation etc.)

2. Where in the results-chain does the indicator measure?

20

1. Focus on the functional problem, rather than the solution

2. Engage stakeholders in identifying functional problems and binding constraints

3. Use political economy analysis prospectively

4. Use available evidence and accepted theory on whether a reform will fix the problem

Operational directions (1) finding out what is likely to work here through more structured diagnostic work

21

Benefit from the results-based-lending instrument (PforR) for PSM reform

The Sierra Leone Experience:

• A different, problem-focused engagement process

• Results dialogue as a vehicle for encouraging MoF and line-ministries to engage in joint problem-solving

• Balancing quantity and quality in indicator design is not easy

• Planning to build in Impact Evaluation design

Operational directions (2)finding out what is more likely to work here through more flexible instruments

In sum• The numbers suggest quantity more than quality• But we have some rebuttals• But, between us, we have some other problems• Where to from here?

o Let’s find out more about what tends to work in general – new metrics, impact evaluation, standardized project indicator buckets

o Let’s focus also on what is likely to work here – better diagnostics and more flexible instruments