Use of PEFA assessments

ICGFM Conference

Miami – May 21, 2009

PEFA Secretariat

2

Harmonize information needs• Common information pool

Inform PFM reform formulation and priorities

Compare to and learn from peersMonitor results of reform

efforts

What can countries use the PEFA Framework for?

3

Harmonize information needs Inform PFM reform formulation

and prioritiesCompare to and learn from peersMonitor results of reform

efforts

What can countries use the PEFA Framework for?

4

PEFA reports for reform formulation (1)

PEFA report is one – out of several - inputs– Identification of main strengths and weaknesses – and

potential impact on budgetary outcomes – Other factors: political economy, institutional, culture,

constitution/legal, resources, capacity at entry Ownership means government decisions on priorities

– Government to consider all factors in deciding priorities– Reform dialogue with donors to allow ample space

Do not use Indicator scores simplistically – A low score is not sufficient justification for high priority reform – Other factors: relative importance of subject,

complexity/timeframe for improvement, interdependence with other elements, weakest links

5

Implement PFM reforms

Recommend PFM reform measures

Identify main PFM

weaknesses

High level performance

overview

Coverage of PFM Performance Report in the PFM Reform Cycle

Investigate underlying

causes

Formulate PFM reform

program

Identify main PFM

weaknesses

Recommend PFM reform measures

PFM-PR

6

PEFA reports for reform formulation (2)

Complementary analysis to PEFA required– Detailed analysis of underlying causes needed for

formulation of detailed action plan– Limit such analysis to priority areas – Drill-down tools – some exist, others under development

PEFA is preparing guidance on use of PEFA reports in reform formulation – Stock-taking of conceptual approaches used– Country case studies illustrating good practice

7

Purposes of Standard Diagnostic Tools

PER CFAA CPAR Fiscal ROSC

TA report

FRA PFM-PR

Review of public expenditure policies & budgetary outcomes

X

Review of political incentives

X X

High level overview of PFM performance

X

Identification of PFM strengths/weaknesses

X X X X X X

In-depth analysis of capacity factors

X X X X X

Recommendations for reform

X X X X X

Assess risk to public funds

X X

Track progress over time * * * X * if performance indicators are incorporated

8

Findings of Norad-managed self-assessment presented to OECD-DAC in December 2007

The assessment showed low scores for seven areas of PFM system performance

Ministry of Finance reaction:– Weaknesses in procurement practices and follow-up to

external audit findings need to be addressed.– Three areas of low scoring not considered priority at

present (Multi-year program/sector budgeting, Limited extent of internal audit, no consolidated overview of risks from autonomous agencies and public corporations)

– Two indicators scored low but are municipal responsibilities; central government will not get involved.

Country case - Norway

9

Harmonize information needs Inform PFM reform formulation,

prioritiesCompare to and learn from peersMonitor results of reform

efforts

What can countries use the PEFA Framework for?

10

PEFA reports for country comparison (1) PEFA Framework developed for in-country use

– ‘Summary assessment’ provides nuanced overview of strengths and weaknesses as basis for reform prioritization

– No method given for arriving at one measure for ‘overall performance level’

Wide interest in country comparison using aggregation– Researchers - learning on determining factors – Donors - aid allocations – Governments - peer learning

Aggregation requires three decisions– Conversion from ordinal to numerical scale– Weighting of indicators (generally and by country)– Weighting of countries (for country cluster analysis)

11

PEFA reports for country comparison (2)

No scientifically correct or superior basis for deciding conversion and weights exists

– Each user takes those decisions on individual opinion– PEFA program will not endorse any particular method

Preferred method of country comparison:– A nuanced comparison of two assessment reports – Consider country context, ensure comparison of like with like

In case aggregation is desired:– Be transparent on aggregation methods used– Discuss reasons for choice– Sensitivity analysis to illustrate impact on findings

12

Harmonize information needs Inform PFM reform formulation

and prioritiesCompare to and learn from peersMonitor results of reform

efforts

What can countries use the PEFA Framework for?

Repeat Assessments

As at March 2009, thirteen ‘real’ repeat assessment undertaken

Expected to become common from 2008/2009 onward i.e. three years after the first series of baseline assessments

What we want to determine?

Specific changes in system performance – What has changed?– How much?

Comparison of Indicator Scores

Indicator scores will provide a crude overview of changes over time, but …– Dimensions may change differently– Performance may not always change enough

to change the score (use of arrow)

So more detailed explanation is required

Other Possible Reasons for Change in Scores

Changes in definitions Improved availability of or access to

information Different information sampling &

aggregation Scoring methodology mistakes in

previous assessment

Reporting on Progress Made

Explain all factors that impact a change in rating indicator-by-indicator

Identify the performance change Ensure that any reader can track the

change from the previous assessment – what performance change led to the change in a rating

Reporting on Progress MadeIndicator Score

2005Score 2007

Performance change Other factors

PI-1 C B Performance appears improved based on

2005: last deviations 6%, 11%, 18%

2007: 5%, 11%, 6%

Not clear if all external project funds were excluded from data for 2005 assessment but probably insignificant issue.

PI-4 (i) A C Performance change is uncertain, despite reported arrears increase from 1% in 2005 to 6% in 2007.

2005 assessment used data on pending payment orders only, not overdue invoices.

Organizational Requirements Performance tracking to be clearly

reflected in the TOR Use of same assessment team desirable

but rarely possible The lead agency of the previous

assessment, if different, should assist with access to previous assessors’ notes

… and be part of the reference group for the repeat exercise

20

PEFA incorporated in PFM reform monitoring system– baseline 2005, repeat assessment 2007, planned follow-up

2010 Important performance improvements 2005-2007 in budget

execution– revenue administration, cash management, internal controls

Improvements resulted from:– Reforms already well under way in 2005 (e.g. IFMIS/SISTAFE

and revenue administration)– Small managerial/admin changes (including quick-wins

identified on the basis of 2005 assessment)– New reform initiatives in pay-roll control (identified from the

2005 assessment as an important neglected area of reform)

Country case - Mozambique

Thank you for your attention