Measuring performance: What?
• The objectives of PFM-(cf. module 1.1):• The 3 specific objectives
Aggregate fiscal discipline Allocation of resources in conformity with policy objectives Operational efficiency
• PFM “traditional” values: Regularity, timely, fair procedures
• The role of Parliament in democratic society, accountability, transparency
Measuring PFM system performance: tools?
4
PER CFAA CPAR ROSC EC Audits FRA OECD/DAC NPA PEFA PMFReview of public expenditure policies and budgetary outcomes X
Review of political incentives X XHigh Level overview of PFM performance X XComprehensive review of PFM functions XIdentification of PFM strengths and weaknesses
X X X X X X X X
Compliance Testing X X XIn-depth analysis of capacity factors X X X X
Recommendations for reform X X X X XAssess risk to public funds X XTrack Progress over time X X
Measuring PFM performance: PEFA
5
PEFA Strengthened approach
1.Encouraging a country-led reform program defined within a Government Reform Strategy and Action Plan
2.Ensuring a harmonised and donor aligned program of support for the country reform program
3.Adopting a common assessment framework applied to a common pool of PFM information for measuring and monitoring PFM reform progress overtime
PEFA Initiative launched
Paris Declara-tion
Mainly IMF interest in
PFM systems
More donors
PEFA Framework
1983
Interest e.g. WB, regional
banks
1998 2001 March 2005
June 2005
•Strengthened approach developed by 8 donors and shared by many others
•Need for result orientation in development of PFM systems and harmonization of PFM analytical work
•Many donors show interest in PFM because of increased preference for budget support as aid modality
Measuring PFM performance: PEFA
• Focus on high level performance indicators applied to central government level
• Widely applicable: Relevant to countries at all levels of development
• Assessment must be evidence based• Capable of calibration to capture progress over
time
7
The PEFA Design Principles
• Comprehensive: cover all aspects of the PFM cycle• Use data that can be collected cost effectively• Define the achievement based on the 3 objectives• Incorporate a measure of the impact of Donor
practices on PFM performance
8
The PEFA Design Principles
PEFA
• 6 critical areas of a PFM system are distinguished
• On which performance is measured with a standard set of
• 28 government performance indicators
• 3 donor indicators, reflecting donor practices influencing the government’s PFM systems
• Most indicators are composed of various dimensions: in total 72 dimensions
• Calibration and Scoring
• Specific calibration of scores using a four point ordinal scale (A, B, C and D)
• Intermediate scores (B+, C+, D+) for multi-dimensional indicators, where dimensions score differently
• Arrow ▲ can indicate an improvement not reflected in change of indicator score
• Two scoring methods:
• Method M1 ‘weakest link among dimensions’
• Method M2 ‘average of dimensions’
PEFA Scorings
Budget formulation and preparation
13
Indicators PI-11:Orderliness and Participation in the annual Budget Process
Scores the existence and adherence to a fixed budget calendar, the utilisation of clear and comprehensive budget circulars whose preparation involves political inputs
Budget formulation and preparation
14
Indicators PI-12: Multi-year perspective in fiscal planning, expenditure and budgeting
Scores multi-year forecasts, links to the annual budgets; existence of fully costed sector strategies, the frequency and scope of debt sustainability analysis carried out and the linkages between budgets and forward expenditure estimates
Predictability and Control
15
Indicators PI-16: Predictability in the availability of funds for commitment of expenditures: scores the extent to which cash flows are forecast and monitored, ceilings for expenditure commitment are reliably communicated to spending ministries and the frequency and adjustments to budget allocations
Indicators PI-17: Recording and Management of Cash Balances: scores the regularity, comprehensiveness and timeliness of reporting on domestic and foreign debt, and on cash balances; and the controls instituted for contracting loans and guarantees
Predictability and Control
16
Indicators PI-18: Effectiveness of Payroll controls: scores the degree of integration between the personnel database and the payroll, the timeliness of updates to the personnel records and payroll, the effectiveness of internal controls to make changes to personnel and payroll records and the utilization of payroll audits to guard against ghosts and double dippers
Indicators PI-19: Competition, Value for Money and Controls in Procurement: scores the degree of utilization of the open tender method, the justification of the use of less competitive methods and the existence and operation of a procurement complaints mechanism
Predictability and Control
17
Indicators PI-20: Effectiveness of Internal Controls for non-salary Expenditures: scores the effectiveness of a commitment control mechanism and the comprehensiveness, understanding of other expenditure control procedures and the degree of compliance to the expenditure rules
Indicators PI-21: Effectiveness of Internal Audit: scores the coverage of internal audit and the degree to which system audits are carried out; adherence to a fixed schedule of reporting and the extend to follow up and management response to findings.
Accounting, Recording and Reporting
Indicators PI-22: Timeliness and Regularity of Account Reconciliation: scores the regularity and timeliness of accounts reconciliation and the clearance of suspense accounts and advances
Indicators PI-24: Quality and timeliness of in-year budget reports: scores the scope and consistency of reporting format with the budget of the in-year budget reports; the regularity and timeliness of the issuance of in-year budget reports and on the accuracy of the data
Indicators PI-25: Quality and timeliness of Annual financial statements: scores the completeness of financial information vis-à-vis revenue, expenditure, financial assets and liabilities; the timeliness of the submission of final accounts and the accounting standards adopted
External Audit and Legislative Oversight
19
Indicators PI-26: Scope, Nature and Follow-up of External Audit : scores the audit coverage, audit standards and range of audits performed; the timeliness of audit submittals to the legislature and the extent of audit follow up
Indicators PI-27: Legislative Scrutiny of the Annual Budget Law: scores the scope of legislative scrutiny beyond details of revenue and expenditure; the establishment and respect of well defined procedures; the adequacy of the time allotted for the legislature to review the budget and the adherence to strict rules on the extent and nature of amendments to the budget
Indicators PI-28: Legislative Scrutiny of External Audit Reports: scores the timeliness of examination of audit reports, the extent of hearings on key audit findings and the the issuance of legislative follow up actions and implementation by the executive
The PEFA Iceberg – how comprehensive?
20
PEFA Assessment
Fixed asset register
Supply chain management
Financial administrative network
Capacity
ProcurementPolitical context
Market
Quality of expenditure management
Engaged civil society
The weakest link?
21
the strength of a foundation is determined by it weakest footing.
The performance of a PFM system is determined by the strength of its weakest PFM activity.
As a general rule PFM scores, like the strengths of foundation footings, cannot be averaged
Movements in scorings may vary only slowly over time
The weakest link (M1 method) will contributes further to slow changes in indicator scorings
Significant improvements may not be reflected in a score change
‘Cherry picking’ of performance indicators without substantial improvements
PEFA in practice
22
Budget Support and PEFA
23PFM Progress over time
Measurements in progress over time using PEFA
Requirements for demonstrated progress to facilitate GBS disbursements
Agreed reviewable milestones