Bottom-Up Administrative Reform: Designing Indicators for a Local Governance Scorecard in Nigeria Africa Region Working Paper Series No. 68 June 2004 Abstract

This paper provides a historical overview of the role and framework of governance assessment in World Bank operations, as well as an informative, process oriented and technical description of the design of such a framework in the Nigerian public administration reform environment. The paper outlines the process and rationale behind the development, modification and refinement, by the World Bank and other international agencies, of governance performance indicator categories, from the broadly defined “first generation” indicators to the more precise, meaningful and effective “second generation” ones. Through the Nigerian case study, the paper also illustrates the integration of a governance assessment scorecard into a community driven development operation, and the analytical and participatory process through which the scorecard was developed. Emerging findings explicit in this paper suggest that governance indicators, both qualitative and quantitative, ought to be simple, measurable, directly related to specific areas of governance, based on accessible data, universal to and acceptable by most government authorities in the particular country in which the framework is applied.

The Africa Region Working Paper Series expedites dissemination of applied research and policy studies with potential for improving economic performance and social conditions in Sub-Saharan Africa. The Series publishes papers at preliminary stages to stimulate timely discussion within the Region and among client countries, donors, and the policy research community. The editorial board for the Series consists of representatives from professional families appointed by the Region’s Sector Directors. For additional information, please contact Paula White, managing editor of the series, (81131), Email: pwhite2@worldbank.org or visit the Web site: http://www.worldbank.org/afr/wps/index.htm. The findings, interpretations, and conclusions expressed in this paper are entirely those of the author(s), they do not necessarily represent the views of the World Bank Group, its Executive Directors, or the countries they represent and should not be attributed to them.

Pub

lic D

iscl

osur

e A

utho

rized

Pub

lic D

iscl

osur

e A

utho

rized

Pub

lic D

iscl

osur

e A

utho

rized

Pub

lic D

iscl

osur

e A

utho

rized

Pub

lic D

iscl

osur

e A

utho

rized

Pub

lic D

iscl

osur

e A

utho

rized

Pub

lic D

iscl

osur

e A

utho

rized

Pub

lic D

iscl

osur

e A

utho

rized

mailto:pwhite2@worldbank.orghttp://www.worldbank.org/afr/wps/index.htmAdministrator31271

Africa Region Working Paper Series # 68

Bottom-Up Administrative Reform:

Designing Indicators for a Local Governance Scorecard in Nigeria

Talib Esmail

Nick Manning Jana Orac

Galia Schechter

June 2004

Authors’Affiliation and Sponsorship

Talib Esmail, World Bank. Email address: tesmail@worldbank.org Nick Manning, World Bank. Email address: nmanning@worldbank.org Jana Orac, Consultant Email address: janaorac@yahoo.com Galia Schechter, Consultant Email address: gschechter@worldbank.org

The authors would like to thank the Trust Fund for Environmentally and Socially Sustainable Development (TFESSD) - formerly known as the Norwegian Trust Fund (NTF) - for their support during the early stages of developing and testing the governance

indicators. During the course of developing the various versions of the scorecard, helpful comments were received from Gary Reid, James Anderson, Michael Stevens, Elizabeth Moriss-Hughes, Peter Papka and Andrew David. We are grateful to Yoshiko Masuyama for formatting this paper and to Dirk Prevoo for agreeing to finance the printing costs from the supervision budget of the LEEMP.

mailto:tesmail@worldbank.orgmailto:nmanning@worldbank.orgmailto:janaorac@yahoo.commailto:gschechter@worldbank.org

Abbreviations and Acronyms CDD Community Driven development CY Calendar Year DFID Department for International Development DAC Development Assistance Committee ESW Economic and Sector Work FY Fiscal Year IBRD International Bank for Reconstruction and Development IDA International Development Association IMF International Monetary Fund LEEMP Local Empowerment and Environmental Management Project LGA Local government authority NGO Non-governmental organization NPRS National Poverty Reduction Strategy OECD Organization for Economic Cooperation and Development PAC Public Affairs Center SIGMA Support for Improved Governance and Management SPAI Stability Pact Anti-Corruption Initiative TTL Task team leader UN United Nations

Table of Contents

PURPOSE OF THIS PAPER .......................................................................................... 1

INTRODUCTION............................................................................................................. 2

PART I: HISTORICAL EVOLUTION OF GOVERNANCE INDICATORS........... 2 The emergence of a “first generation” of governance indicators .................................. 2 The emergence of a “second generation” of governance indicators.............................. 4 Combining second generation indicators with scorecard methodology ........................ 6

PART II: THE NIGERIAN GOVERNANCE SCORECARD ..................................... 7

Objectives and potential benefits .................................................................................... 7 The scorecard development process ............................................................................... 9 The first version of LGA scorecard for Nigeria............................................................ 10 The indicators in the first version ................................................................................. 10 Constraints influencing the simplification of the scorecard ........................................ 12 The simplified version of the scorecard ready for application ..................................... 13 The assessment process................................................................................................. 15 Potential risks of implementing the scorecard .............................................................. 17 Conclusion and Next Steps ........................................................................................... 19 References..................................................................................................................... 36

Purpose of this Paper

A local governance scorecard was designed during the preparation of a community driven development (CDD) project in Nigeria (Local Empowerment and Environmental Management Project – LEEMP). The scorecard was designed as a practical tool for operations. It is not an instrument for empirical study, in which significant time and resources can be spent on statistical sampling and research.

The purpose of this paper is to lay out the logic and thought processes leading up to the design of key governance-related indicators (both quantitative and qualitative) that will be used by LEEMP to further its intention of administrative reform of local governments. As the application of the scorecard in Nigeria takes place, this working paper will be updated with further analysis and lessons learned regarding the indicators and methodology. The authors hope that this paper will be of practical relevance to Task Team Leaders (TTLs) of the growing number of CDD operations in which local government administrative reform may not be the central focus, but rather an important objective through which to contribute to sustainable community development and improved service delivery. This paper begins with a historical overview of governance assessment in World Bank operations. It continues with a description of the evolution of governance indicators from the rather broadly defined “first generation” indicator categories to the more precise and practical “second generation” indicators. We believe that this background provides useful information which TTLs may find helpful when explaining a similar methodology to their clients. The second section of this paper presents a case study of the evolution of the local governance scorecard in the LEEMP. It describes the initial (or some might say utopian) set of indicators that were developed and the rationale for their selection. Following field testing of the initial set of indicators, the paper describes how a much more simplified version of the scorecard had to be developed as the implementation constraints became apparent. The section ends with a brief conclusion and a description of the next steps in the application of this scorecard.

1

Jana OracDo we still want to say this, given the low likelihood it will actually happen plus the change in project staffing? Maybe we can soften it to offer updates to the website note, if it still up, or updates available via project documents/task manager. Or be totally realistic and cut all reference to updates.

Introduction

In contrast to many countries in Sub-Saharan Africa, Nigeria has considerable wealth by virtue of its large petroleum reserves. It is officially a “blend” country; eligible for both IBRD loans and IDA credits. The stark reality, however, is that the IDA budget envelop available to Nigeria (US$200 million for FY03) is trivial compared to the overall public expenditure budget (about US$10 billion for CY03). Obviously, the most significant impact on reducing poverty will have to come from increased efficiency and effectiveness of public expenditures and not from IDA resources. The LEEMP will provide about US$70 million in IDA resources to nine States in Nigeria. Undoubtedly, the credit amount is not sufficient to benefit all local governments and communities in all the participating States. Therefore, the LEEMP has decided to establish an incentive system that uses a local governance scorecard based on second generation indicators to identify local governments whose levels of competence and commitment qualify them to participate in LEEMP. The aim of this approach is to provide incentives for local governments to provide social and economic services to their constituencies while encouraging communities to demand such services from their local governments. In targeting its resources to better performing local government authorities (LGAs), IDA thus hopes to encourage government to emulate similar standards when allocating public expenditure budgets.

Part I: Historical Evolution of Governance Indicators

The emergence of a “first generation” of governance indicators

The World Bank’s attention to the role of governance in the success and sustainability of development efforts emerged in the early 1980’s. At that time, it became increasingly apparent that the effectiveness of both adjustment and investment operations among Bank borrowers was heavily impeded by governance factors related to poor management. In 1989, a Bank publication entitled Sub-Saharan Africa: From Crisis to Sustainable Growth: A Long Term Perspective Study (1) drew attention to the crisis of governance in Africa and reflected the growing interest in issues of governance in general and, in particular, as they relate to economic development. A more comprehensive analysis of the role of governance in development was presented in the early 1990’s with the issuing of a milestone World Bank publication entitled Governance and Development (2). The paper further

2

expanded the debate about the relationship between development success and the quality of government action by proposing a framework of performance categories for governance assessment. Among the numerous categories proposed, four primary areas of governance were identified as consistent with the Bank’s mandate to promote sustainable economic and social development: (3) • Public sector management – establishing a direct correlation between

prospects of development and the capacity of the public sector to manage the economy and deliver public services;

• Accountability – involving the transparent and consistent correspondence between public policy, successful implementation, efficient allocation and use of public resources;

• The legal framework for development - creating an enabling legal infrastructure that is conducive for growth oriented economic actions and efficient use of resources; and

• Information and transparency - enhancing open access to information and credibility of public sector management while reducing uncertainty among all stakeholders regarding risks associated with development endeavors.

The decade of the 1990’s was characterized by a great deal of effort by international agencies, commercial firms and NGOs to clearly define characteristics, incentives, and obstacles to and overall conditions of good governance. Such efforts played a key role in the development and expansion of the “first generation” of indicators that pointed to broad weaknesses and problematic areas of government and its operating environment. For example, in cases of national government assessment, evaluators could apply a “first generation” indicator such as “corruption” to examine government accountability, and would consequently attribute an aggregate score to overall government performance in that area. Such indicators were instrumental in defining some of the broad areas of governance initially proposed, and in enabling the drawing of comparative analysis of the quality of governance across nations. They were also helpful in the analysis of political risks associated with government performance faced by foreign investors. Additional types of “first generation” indicators continued to emerge as research and analysis were conducted to identify the relationship between decentralization and quality of governance. In 1998, a World Bank working paper entitled Applying a Simple Measure of Good Governance to the Debate on Fiscal Decentralization (4) further proposed an elaborate governance assessment framework. The framework proposed a measurement of quality of governance based on the impact of its exercise of power on the quality of life enjoyed by its citizens. While somewhat similar to the one cited in Governance and Development, the new framework elaborated upon the broad categories of performance and added several more narrowly defined indicators. The four observable governance dimensions proposed by this expanded framework included the following: • Citizen’s voice - ensure political transparency and voice for all citizens; • Government orientation - provide efficient and effective public services; • Social development - promote health and well being of its citizens; and

3

• Economic management - create a favorable climate for stable economic growth.

Other quantitative analyses have linked governance to a variety of development outcomes using subjective and broad indicators of corruption or the rule of law (e.g. Ades and DiTella 1996; Chong and Calderon 1999; Hall and Jones 1999; Johnson, Kaufmann, and Zoido-Lobaton 1998; Kaufmann, Mehrez, and Schmukler 1999; Knack and Keefer 1995; Mauro 1995; Rodrik 1997; and Tanzi and Davoodi 1997; Wei 1997). (5)

The emergence of a “second generation” of governance indicators

Such an evolving range of “first generation” of governance indicators drew attention to the crucial role of governance in development. However, these indicators were quickly realized to be of limited value in identifying high-payoff reforms and in building ownership for reform by developing country governments. Indeed, in March 2000, a joint UN/OECD/World Bank/IMF forum reached a consensus that “first generation” indicators and frameworks of analysis had a meaningful but limited value, and that “these first generation governance indicators have helped to draw attention to the right issues, but the explosion in their numbers has not been accompanied by improved insight into practical ways forward for reforming governments.” (6) More specifically, a paper (7) prepared for the 2000 Forum explicitly argued that the existing indicators indeed point to problems in governance, and yet a “6.2” aggregate score in transparency, or a “5.34” aggregate score in corruption are unhelpful in identifying operational solutions and thus leading to performance improvement and capacity building. “For example, a low score on a rule of law index implicate multiple policy and institutional culprits. It is undeniably a problem but it does not naturally suggest what the solution might be or even who should implement it. (8)” The most critical problems associated with “first generation” indicators were identified to be the following:

• They often do not provide precise and objective measurement of governance

related challenges and performance trends; • They do not point to particular institution or institutional arrangements as the

cause of governance challenges; • They do not isolate particular causes to broad problems; and • Information provided is insufficient to facilitate the identification of

appropriate operational solutions and performance improvement processes.

Thus, with the help of the Department for International Development, United Kingdom (DFID), the World Bank started work in the Fall of 2000 on the development of institutionally specific and transparently generated governance and corruption indicators. This paper summarizes some modest, but significant progress within this project that seeks to lower the temperature on a key question in governance: ‘how can it be measured in a way that promotes constructive change?’

4

The work has been undertaken in collaboration with the Development Assistance Committee (DAC) of the OECD. The DAC has been working to establish indicators that measure movement towards the meeting of major UN conference goals since the early 1990s. In "Shaping the 21st Century: The Contribution to Development Cooperation" (OECD 1996), the DAC confirmed the goals and set out the areas of democratic governance that it saw as essential for achieving these goals. These areas are good governance (including public sector management, rule of law, corruption and military expenditure); human rights; and democratization and participatory development. In 1996, the DAC ad hoc working group reported that it had failed to forge a consensus around some core indicators for these issues. Demand for measurements of progress increased significantly, particularly from the development agencies, and a DAC proposal to a joint OECD/UN/World Bank meeting on Agreed Indicators of Development Progress in February 1998 re-launched the process. However, at the Joint UN/OECD/World Bank/IMF International Development Forum hosted by OECD/DAC in March 2000, there was again a consensus that there was not yet a meaningful set of indicators of participatory democracy and good governance that was politically acceptable to governments. Nevertheless, there was strong sentiment that disciplined work on identifying meaningful and acceptable set of “second generation” indicators should continue, as it was recognized that the alternative was the increasing use (at least outside DAC) of indicators that were non-transparent and/or unacceptable to governments. To qualify as pragmatic, transparent, particular, and result oriented, second generation indicators had to comply with certain established standards. Among the standards proposed by Knack, Kugler and Manning (2001) are whether: • the indicator implicates particular institutional arrangements; • it describes characteristics of good governmental processes; or • the indicator lends itself to valid and reliable measurement.

The development of “second generation” indicators intensified as part of the World Bank’s contribution to the good governance pillar of the Balkans Stability Pact Anti-corruption Initiative (SPAI). As part of the Pact, South East Europe became a pilot region for testing the feasibility of gathering sufficient data on more specific, replicable indicators to construct governance “scorecards.” Bank economic and sector work on the Balkans countries and other sources were comprehensively reviewed to gather information to construct the indicators. In June 2001, a joint World Bank/OCED-SIGMA mission to the region sought data directly from governments. This work provided some indication that governmental transparency with regard to data collection and dissemination can itself be a useful indicator measure of accountability. Although data were obtained only for the Balkans countries that are participating in the SPAI, initial evidence from this data collection and analysis effort suggests that accountability tends to be associated with pro-poor outcomes. However, the governance scorecard approach for monitoring governance via peer review mechanisms has not been implemented by the SPAI, due to a combination of dissatisfaction among participants with the proposed indicators, the scorecard peer monitoring approach, or with the SPAI itself.

5

As types and examples of “second generation “ indicators began to emerge, these have been integrated into several recent World Bank operations. For instance, two public sector reform loans in Albania and Macedonia have used “second generation” governance indicators in monitoring the progress of public administration reforms. The Government of Albania has adopted a comprehensive policy reform program to strengthen Albania’s weak institutional and governance capacity. This policy reform program was supported by a $45 million Structural Adjustment Credit approved by the World Bank in June of 1999, followed by an ongoing $9 million Public Administration Reform . The objective of the project is to provide required resources for technical assistance, training, goods and incremental operating costs that are needed to implement the Government’s Institutional and Public Administration Reform agenda effectively. The project will include a number of performance indicators, some of which are intended to focus the Government’s attention on the longer term objectives of its reform effort, and others are intended to capture the more immediate and concrete progress that can reasonably be expected to be achieved during project implementation. Similarly, a $15 million Public Sector Management Adjustment Credit project in Macedonia includes provisions for monitoring progress using indicators such as the public sector wage bill, vertical compression of civil service salaries. Another example of the integration of “second generation” governance indicators is found in Cambodia’s National Poverty Reduction Strategy (NPRS) for 2003-2005, which provides for monitoring progress via objectively-measured indicators, including some in the area of “institutional strengthening and good governance.” Some of the progress indicators include increases in civil service salaries, reductions in the number of civil servants, in the size of the armed forces, and in defense expenditures. Because of the emphasis on promoting gender equity in the NPRS, the progress indicators in the area of governance also include increasing the number of female prosecutors, and increasing the share of parliamentary seats held by women.

Combining second generation indicators with scorecard methodology

The concept of the scorecard was pioneered by the Public Affairs Center (PAC) in Bangalore, India in 1993, and has since been broadened in scope and applied both nationally and internationally. In fact, it has emerged in recent years as a highly effective empirical means of measuring the performance of service providers, relying on public awareness and participation to generate collective action and bottom up pressure against poor service delivery and rent seeking. The scorecard methodology is usually not intended to replace a more comprehensive, government specific diagnostic, but rather is utilized as a tool for comparative and systematic assessment and change catalysis of governments based on participation, awareness and collaboration. (9) The use of the scorecard as a monitoring mechanism by the SPAI represents an innovative application of second generation governance indicators through a simple, pragmatic and participation-based tool for government assessment. The scorecard methodology has been used successfully to provide policy makers and

6

Jana OracHave we clearly described what exactly this concept is? If not, I suggest a very succinct line, maybe inserted right into this sentence. Something along the lines of: “ The scorecard methodology – an independent assessment that draws upon feedback from various stakeholders and is periodically updated and publicized – is usually not …..”

their constituents with an opportunity to independently evaluate performance associated with the provision of basic public services. The marrying of the scorecard methodology with second generation indicators has, therefore, been a natural evolution of the efforts to apply a mechanism for both assessment and reform of governments. Specifically, two main commonalities exist between the scorecard approach and the evolutionary thought process leading to the second generation of governance indicators: • Civic engagement and participation: One of the common links between the

scorecard and the emergence of second generation indicators is their mutual focus on civic participation. Not only do emerging second generation indicators and elements traditionally used in scorecards focus to a great extent on government’s responsiveness and accountability to its citizens, but also, and even more so, the methodologies involved in identifying and applying the appropriate indicators, and the expected process of consequential behavior change within government relies extensively on citizen participation and engagement.

• Incentive based performance improvement: While many changes occurred

throughout the years in the definition of good governance and its corresponding indicators, it has been consistently agreed that governance is a continuum, requiring incentives and constant tending in order to improve. The role of citizens, therefore, is critical in demanding and influencing good governance through awareness, education, mobilization and participation. The scorecard methodology too, has been designed on the premise that public expectations motivate governments while placing certain constrains on its actions. The scorecard, therefore, encourages citizens, particularly the poor, to express their preferences and to hold public officials accountable for translating their preferences into results. The utilization of scorecard methodology with objective and transparent second generation indicators is, therefore, expected to provide civil society and stakeholder communities with a mechanism for influencing governance and the policy making process.

Part II: The Nigerian Governance Scorecard

Objectives and potential benefits

There are a total of 36 states and 774 LGAs in Nigeria – an average of 21 LGAs per state. Following the enactment of the Constitution of the Federal Republic of Nigeria (1999), democratic elections were held for the first time at all three tiers of government. All local governments have been established on the “presidential model” – the chair of the LGA is directly elected by eligible voters in the local government area, and governs with the assistance of commissioners who are appointed to head local government departments. The local government council is the legislative arm of the LGA. Members of the council are elected from single member wards (i.e. districts). The term of both the chair and council of the LGA is currently three years. LGAs constitute the weakest tier of government in the federal system. However, there is considerable variance in performance amongst LGAs.

7

Although administrative reform is not its principal focus, LEEMP recognizes the tremendous impact that all levels of the public administration have on community development and service delivery. Since local governments are closest to the citizenry and receive significant public funding, improved performance at this level has particular potential to contribute to sustainable community development. As part of its design, LEEMP strives to change the paradigm that currently dominates community-local government relations. The credit seeks to empower communities to view local governments as entities that should serve and be accountable to the electorate, and to demand that local governments act accordingly. Likewise, local governments that wish to participate in LEEMP must demonstrate an orientation toward public service and a willingness to work in partnership with communities. Through the assessment and selection process the scorecard will serve as a tool to distinguish governments that are heavily corrupt and/or dysfunctional from those where administrative systems governing policy making, budgeting and basic service provision might be insufficient but are essentially present. The local government scorecard has been specially developed to advance these objectives. The scorecard will be used to identify local governments for participation in LEEMP. As a community-driven development project, LEEMP will channel funds directly to selected communities within those LGAs (only) that pass the scorecard assessment, without going through LGA accounts. LEEMP funds would not go through LGA accounts because they lack financial and accounting systems, as well as staff capacity. Bank Financial Management Specialists were skeptical as to whether any LGAs would meet the necessary fiduciary requirements (at least initially) to enable funds to flow through their accounts. However, even without funds flowing through their accounts, local governments will benefit from the prestige of participation, as well as having the opportunity to build stronger relationships with communities and further enhance their own administrative capacities. Selected communities will be able, with external facilitation and in coordination with their LGAs, to identify, manage and implement their own development priorities within a pre-assigned budget envelope. The scorecard may also serve some secondary objectives. These include:

• Gathering information that can be used to target LEEMP’s local government

capacity building efforts; • Encouraging modest behavioral changes by mandating specific steps (related

to scorecard criteria) that local governments must undertake, once accepted, in order to remain active participants;

• Potentially, encouraging behavioral changes by local governments that wish to improve their performance in order to be accepted to LEEMP in future; and

• Informing decisions about the type and level of LEEMP-specific responsibilities that a local government should be assigned.

The scorecard may bring more general benefits:

8

Increase the voice of rural communities by explicitly considering their feedback in LEEMP’s selection of local governments; Educate governments and citizens in participating states about how local governments should behave; and Communicate to local government politicians and officials that they are under real scrutiny, and that their performance can have real consequences. The extent to which these secondary objectives and general benefits are met will become evident only after the scorecard is launched.

The scorecard development process

The scorecard’s development included interviews of communities, LGAs, and non-governmental organizations in several states (August 2000), analysis and review by World Bank staff, consultative workshops with stakeholders from six states and field visits (November 2001), and field testing (May 2002).

• Initial interviews: In an effort to understand what were the key constraints to

local governments fulfilling their constitutionally mandated role, a short questionnaire was developed covering a variety of topics related to planning, budgeting, service delivery, reporting and governance. The purpose of the questionnaire was to guide Bank staff (who were not necessarily public sector management specialists) in consistently asking the “right” questions when in the field. As part of a Bank preparation mission, staff carried out semi-structured interviews with communities, NGOs, local government chairmen, local government councilors, state level civil servants and others stakeholders. Whenever possible, responses were triangulated in order to verify their authenticity.

• Analysis and review by World Bank staff: Interview results were analyzed

over a period of three months with the aid of a public sector management specialist. Knowledge gaps were partially filled through semi-structured interviews with other members of the Country Team from a variety of sectors. In some cases, on-going economic and sector work (ESW) on Nigeria was used to gain further insight into issues such as the state governance environment. The outcome of this stage was an embryonic scorecard based on five primary categories of performance indicators (described in full in section below and also in Annex 1).

• Stakeholders consultation workshops: In October of 2001, a series of

workshops took place inviting community, NGO, LGA and state representatives of six states (one from each geo-political zone in Nigeria). The primary aim of these workshops was to test the reaction to the indicators in the embryonic scorecard and, if possible, to reduce the number of indicators and obtain greater specificity in measuring performance. Considerable time was also spent on discussing how the scorecard might be implemented in order to prevent collusion amongst those being assessed in an effort to subvert the objectivity of the assessment. With hindsight, the stakeholder workshops were not particularly cost-effective in achieving their

9

objectives. However, they did serve to convince federal government of the overall acceptability of the scorecard approach by states and local governments.

• Field application and testing: In May of 2002 extensive field testing of the

proposed indicators and the assessment methodology took place in several of the LEEMP participating states in Nigeria. The primary objective of the testing was to determine which indicators would be most effective given the constraints. It was also designed to refine the methodology and adjust it as necessary to reflect local conditions. The outcome of this process was a significant reduction in performance categories and indicators with a much leaner and simpler scorecard.

The first version of LGA scorecard for Nigeria

The initial set of indicators was largely based on a combination of subjective (perception based) criteria from an analysis of field interviews with multiple stakeholders, as well as objective (empirical) criteria based on tested second generation indicators and international good practice concerning administrative reform. We consider it useful to present these indicators here to demonstrate their breadth of coverage and the rationale for choosing them. While subsequent iterations of the scorecard drastically reduced the number of indicators, the long list presented here and also in Annex 1 may assist TTLs in making informed choices as to which are more applicable to their particular context.

The indicators in the first version The set of indicators initially proposed covered the following broad categories: a. State governance environment for the LGA; b. Budget formulation, execution and reporting; c. Participation and planning; d. Project implementation capacity; and e. LGA personnel capacity.

The following are examples of indicators prescribed under each category, and the overall rationale for their application: (A complete table of indicators and their rationale and performance classification categories is available in Annex 1). State Governance environment for the LGA: Suggested indicators for this category included the following: a. Effective legal framework; b. Agreement on inter-governmental revenue transfers (state to LGA); c. Autonomy; and d. Relationship with state ministry of local government affairs or equivalent. The overall rationale for these indicators was based on the assumption that sustainable capacity and effective performance of local governments are directly

10

related to the consistency, transparency, extent of collaboration, and support between local and state governments. More specifically, in order to operate effectively and systematically, local governments must have a supportive and effective legislative environment, a reliable and consistent system of resource transfer, and a decentralized and autonomous operational environment based on which LGAs can effectively respond to local interest. Budget formulation, execution and reporting: Suggested indicators for this category included the following: e. Formal and comprehensive budget; f. Effective project prioritization; g. Effective revenue collection of both taxes and users fees; h. Appropriate flexibility in budget allocations; i. Effective accounting and audit; and j. Sound financial management practices.

The overall rationale for these indicators was based on the premise that formal and comprehensive LGA budget planning and administration, ingrained in a realistic and effective revenue collection and expenditure monitoring system, ought to provide the mechanism for identifying clear and realistic priorities, and matching them with available resources. Additionally, by maintaining some flexibility in the budgeting system and applying effective auditing practices, LGA staff should be able to respond to changing budgetary constraints while ensuring effective and transparent use of funds.

Participation and planning: Suggested indicators for this category included the following:

k. Consultation, responsiveness and redress vis-à-vis communities; and l. Effective LGA Council

The overall rationale for these indicators was based on the premise that consultation with communities and extensive communication and collaboration with community members enables LGAs to serve their constituencies in an accountable, responsive and transparent manner, thus truly reflecting community interests and priorities. Such participation ought to be consistent and systematic with credible oversight by the elected LGA Council.

Project Implementation capacity: Suggested indicators for this category included the following:

m. Community participation in project implementation; n. Involvement of community based organizations and non-governmental

organizations; o. Transparent and open procurement; p. Active monitoring and evaluation of projects; and q. Hiring of community members. The overall rationale for these indicators was based on the premise that community participation and involvement in project design and implementation

11

of LGA financed investments may foster a sense of ownership and commitment among community members toward development efforts, will enable community members to compete fairly and openly for procurement contracts, enable the LGA to benefit from knowledge and experience of civil society organizations, and thus promote overall quality and sustainability of development efforts and community empowerment. Personnel and administration: Suggested indicators for this category included the following:

r. Appropriate staffing levels; and s. Administrative structures and processes are well regulated and based on

sound principles.

The overall rationale for these indicators was based on the premise that a LGA should be a responsible and efficient employer ensuring appropriate levels of employment within the context of realistic needs, overall priorities, and budget constraints. Additionally, transparent and reliable government functioning are critical factors in overall quality and consistency of performance.

Constraints influencing the simplification of the scorecard

During the consultation workshops and field testing using the above set of indicators, a number of constraints were highlighted which necessitated a substantial revision and simplification of indicators for the scorecard.

Local governments are embedded in state and federal contexts (political, legal and administrative) that heavily influence their operations, including the extent to which even a willing local government can improve its operations in certain areas. Some notable examples of intra-governmental influence include the fact that local governments rely for most of their funding on transfers from the federal account whose timing or (more usually) amount can vary significantly from what was expected; that salaries of senior LGA officials are set at the federal leveli; and the nature of relationships between the state and LGAs can often be characterized as unbalanced, arbitrary or based on party political considerations. This meant that indicators related to the State governance environment for LGAs were not acceptable despite the view of many public sector specialists that the extent of contextual constraints on local government performance were some of the most important elements used to distinguish between bad and well performing governmentsii. Politicization of local governments: After years of military dictatorship and severe politicization, Nigeria’s administration -- including local governments -- has an entrenched culture of rent-seeking and self-interest, with little incentive to focus on serving the public. As a result, local governments are generally

i In early 2001, the federal government approved a many fold increase in LGA councilors’ salaries, with the result that salary expenditures largely exhaust LGA budgets. ii Such constraints placed by higher authorities often encourage mechanisms of transparency, auditing and accountable reporting that are significant contributors to good governance - Getting a grip on governance using “second generation” indicators/ (World Bank: Knack, Kugler, Manning, Dec. 2000)

12

distrusted and held in low esteem by the public, due to past experience with service failures and mismanagement. Data availability and reliability: In general, in Nigeria reliable data and information on government operations (at any level) are difficult to obtain. This restricted the ability to use reports and publications for establishing objective and quantitative indicators. Complexity of indicators: It was evident that the framework of categories and indicators were often not meaningful and understandable for non public sector management specialists in government who would be required to oversee implementation of the scorecard. Difficulty in identifying human resources with appropriate skills to carry out the assessment: One of the most important challenges requiring indicator simplification was the insufficient number of national consultants/enumerators with the right skills to understand the scorecard and to carry-out the assessment.

The simplified version of the scorecard ready for application

The initial set of indicators have undergone several revisions. It was apparent during field testing in May 2002 that, due to the constraints mentioned above, effective implementation of the assessment would only be feasible if there was a substantial simplification in the range of indicators. Therefore the following areas of LGA performance were identified as most appropriate for inclusion in the scorecard:

t. Financial management and budgeting practices; u. Outreach to communities; and v. Effective implementation of projects that benefit communities.

Detailed questions were evolved under each of the categories listed above. The key considerations were that, collectively, the questions needed to be consistent with the methodology for building second generation indicators, namely: (i) the indicator must implicate particular institutional arrangements; (ii) it must describe characteristics of good governmental processes; and (iii) it must lend itself to valid and reliable measurement. In order to form an accurate picture of each LGA, information will need to be gathered from multiple sources. Taken together, these would form important checks and balances on the information from any one source. The main sources of information are: w. Official data budget: A set of indicators will be calculated from formal

budgets and actual outturns. Figures will be taken from the approved budget estimate (published by the LGA and obtained from the state Ministry of Local Government or equivalent) for the most recent year available.

• Interviews with communities: In each LGA, three communities will be

interviewed regarding the LGA’s public service orientation, as well as

13

evidence of effective project implementation. The interviewees will be recognized leaders: the traditional ruler; an executive member of the Community Development Association or Town Union; the leader of a women’s organization; and a representative of an ethnic minority or vulnerable group such as the physically handicapped.

• Interviews with senior career staff of LGAs: In each LGA, the director of

administration, treasurer, and internal auditor will be interviewed regarding compliance with important financial and administrative procedures, and quality of LGA operations.

• Interviews with state officials, and state records on LGA reporting:

Relevant senior state officials will be interviewed at the outset regarding LGA performance, including reporting compliance. The extent to which an LGA submits key reports in a timely manner will also be examined using records of the Ministry of Local Government’s Inspectorate and Monitoring unit or equivalent.

• State audit reports: The State Auditor General will be asked to classify

each LGA as acceptable, marginal, or unacceptable, based upon the most recent completed annual audit.

Assigning scores based on responses Scores would be assigned to sets of questions that focus on the same issue. After all the field work is completed and reports are compiled for each LGA, each individual respondent’s responses to each distinct set of questions will be assigned a score of 0, 1, or 2.

• 0 indicates a negative situation • 1 indicates a marginal/indifferent situation • 2 indicates a positive or promising situation

The overall score of each LGA would be based upon a simple formula comprising of: (a) an average score from all three communities to yield a Responsiveness to Communities (RC) Score; (b) an average score for LGAs based on respondent responses will yield an overall Administrative Operation (AO) score; (c) financial and budget indicators will be rank-ordered prior to assigning a point score and the average score on all indicators will yield a Financial & Budgeting (FB) score; and (d) Information on Reporting compliance (R) will be scored on the same scale as interview responses.

14

Jana OracI can’t remember why it was necessary to state this, but in any case, no harm. We’ll have to remind the international consultants to develop questions for state officials, and think about whether their responses should be a separate element of the formula, or be integrated into each existing element as the state response. Or considered separately from the formula. Any thoughts?

The overall scoring formula is: LGA score = (2RC + 1 AO)/3 + FBiii + R

LGA feedback is weighted less heavily than community feedback in order to account for cases where senior LGA staff do not give frank responses, and to emphasize the importance of community satisfaction. Placement of LGAs in performance categories Local governments will be placed in one of three categories of performance based on the following rationale: • Acceptable / most promising: Reasonable goodwill and cooperation between

LGA and communities. LGA complies reasonably well with the most important state standards for reporting and procurement; Evidence that LGA has participated in or facilitated at least some beneficial projects in communities; Minimal or low levels of mismanagement, fraud, or theft of public funds; Communities in accepted LGAs from this category will be considered for active participation in LEEMP (i.e. qualified communities can access LEEMP financing); Staff of all LGAs in this category will be eligible to attend LEEMP-sponsored training.

• Marginal: Communities are indifferent to the LGA. Few projects in

evidence, or projects are not fully acceptable to communities; Fraud, theft or mismanagement of public funds are not uncommon; No communities will be eligible for LEEMP financing; However, staff of these LGAs may be eligible to attend LEEMP-sponsored training.

• Unacceptable under any circumstances: Antagonistic relationship with

fundamental absence of goodwill and cooperation between LGA and communities, and serious dysfunction within the LGA itself; Lack of meaningful cooperation between LGA and state, due to LGA failures; Credible allegations of large-scale, outright fraud, theft or severe mismanagement of public funds; LGA attempted to interfere with objectivity of the scorecard assessment; Staff of these LGAs are ineligible to participate in LEEMP-sponsored training, since the LGA’s extreme dysfunction indicates that it is not receptive to such efforts.

The assessment process

Apart from simplifying the scorecard, it was also agreed with the federal implementing agency for LEEMP that it would contract a Nigerian consulting firm (or NGO), as well as international consultants (preferably one to three individuals) to handle implementation of the assessments in the nine states. The Nigerian consulting firm will be responsible for recruiting qualified, capable interviewers to carry out the assessments; cooperating closely with international consultants; ensuring quality and objectivity of fieldwork; and preparing detailed

iii The Financial & Budgeting component may include scores for audit performance. Alternatively, audit issues may become a separate component in the formula, in order to stress the importance of audit compliance.

15

assessment reports on each LGA as well as a draft ranking/classification of LGAs based on assessment findings. As a quality control measure, the final payment will be subject to a satisfactory independent check on the completion and quality of the assessments. The international consultants will be responsible for reviewing the draft design of the entire scorecard process as well as the technical instrument (i.e. scorecard questionnaire) and making any refinements necessary to finalize and implement the scorecard; designing and delivering training sessions for field interviewers; and overall quality control of implementation of the technical instrument. The assessment process is, therefore, designed to be conducted by a trained team of qualified individuals and to be based on qualitative and quantitative indicators, administrative and survey based information collected from community and LGA members, as well as official records. The scorecard will be used first for a baseline evaluation of all rural governments in the participating states, and for subsequent follow up evaluations. A follow up evaluation will be done mid-way through the credit’s five year term, and at the end, if a follow up credit is approved. Local governments that do not qualify for active participation in LEEMP in the course of the baseline evaluation will still be able to benefit from project funded technical assistance and practical training seminars.

The assessment team ought to be multi-sectoral, objective and well trained. A Nigerian consulting firm will be responsible for recruiting qualified and capable interviewers to carry out the assessment process while ensuring quality and objectivity of the entire assessment process, and preparing detailed reports in each LGA as well as draft ranking/classification of LGAs based on the assessment findings. The methodology through which data will be collected will include a combination of administrative and survey based research.

The administrative approach would be applied to public indicators related to such elements as financial management and reporting standards. The survey approach on the other hand, would be primarily applied toward more private indicators such as those associated with participation and planning. The collection of administrative data regarding each LGA will take place prior to and during the assessment process, and particular information to be collected is to be related primarily to budget analysis and audit and reporting practices. Scoring is to be based for the most part on a simple and easily quantifiable scale representing a continuum of bad to good performance. Survey data was designed with the following key criteria for the development of specific questions: (i) a clear rationale exists for inclusion of each variable in the questionnaire; (ii) the questions that will be asked in order to measure that variable ought to be readily understood by survey respondents, and their responses ought to capture the variable intended to be measured; and (iii) the variable should exhibit a reasonable amount of variation across LGAs. Although possibly quantifiable, survey questionnaires were designed as a qualitative source of data for assessment purposes.

16

Separate from issues of the design and application of the scorecard as a technical tool -- and thus largely outside the scope of this paper -- there remain questions regarding how the results of this technical exercise will be managed by each participating state and the federal government. Such issues are particularly important since participating states are effectively borrowing LEEMP funds from the federal government. As such, they have an important voice in setting an objective and appropriate process for managing the technical results of the scorecard exercise. It is foreseen that officials responsible for LEEMP at the federal and state levels will jointly devise sample options for this process, as well as specifying steps that the weakest categories of LGAs will be required to take in order to improve their performance. Each state will then chose the most appropriate option for its own process. All states will be required to publish the details of the process they selected, as well as the findings of the assessment.

Potential risks of implementing the scorecard

The challenges and risks in applying such an instrument in the Nigerian environment are great. Interventions such as the scorecard must proceed with care, and be ready to reorient rapidly in response to actual developments. The scorecard should be used cautiously, with the intent to do good but also a commitment to do no harm. The following is a list of the most critical risks and how the LEEMP has sought to mitigate some of these:

Biased responses from LGAs. Since selection of specific LGAs for project participation depends on their assessment using the scorecard, there is a risk that members of LGA will be reluctant to reveal true information about any negative performance. Mitigation means: (i) Information based on interviews with LGA chairman will not be directly factored in the overall assessment; (ii) Meetings with senior civil servants will be held in private. Additionally, senior civil servants are usually appointed by the state, therefore there is a great likelihood that such individuals will speak their mind freely; and (iii) For the purposes of scoring LGAs, the weight given to information collected from LGAs is less than that collected from communities. Biased responses from communities. Communities may also be inclined to provide unrealistic and favorable information about the performance of their LGAs in order to increase the likelihood of their participation in the project. Mitigation means: (i) There is no immediate connection between the communities that are interviewed and the ones that are eventually selected from within the jurisdiction of selected LGAs. Once LGAs are approved for participation, the selection of communities will depend on several additional criteria, including poverty level, ability to organize themselves, etc.; (ii) Multiple members of the community, representing various interest groups will be interviewed; and (iii) From field experience thus far it appears that communities are not hesitant to speak out truthfully about their LGAs.

17

Reliability of budget reports. As budgeting is not a robust practice among LGAs in Nigeria, there is some risk that the budget information collected will be composed of pro-forma numbers that are not very meaningful. Mitigation means: (i) Audits will be assessed to cross check budget information; (ii) information will be collected on reporting compliance; and (iii) LGAs will be assessed in comparison to others in their states that follow the same rules of analysis and reporting. Unreliable information. There is a risk that any one of the sources of information and actual items of information collected will be unreliable. For example, it is known that while in some states in Nigeria the audit reports are up-to date, it is also clear that in some states they are not. Mitigation means: Information is collected and is cross-checked through multiple sources. Thus even if some sources or types of information turn out to be unreliable in a given state, the remaining information should be sufficient to draw conclusions. Quality of assessment process. There is a chance that the quality of the assessment process will be poor and/or captured by various interest groups. Mitigation means: (i) The assessment process is designed to be conducted by trained consultants rather than by government employees; (ii) final payment to consultants will only be made after a random check of the results of an LGA in each state by international consultants from a different firm than recruited the enumerators; (iii) consultants, both mangers and field interviewers, will be trained and guided regarding recruiting standards and work in the field; and (iv) selected consultants are expected to comply with certain ethnic and linguistic standards that will facilitate their work in diverse environments. Non-cooperative local governments. Some LGAs may refuse to participate in the assessment process. Mitigation means: No LGA is forced to participate. Each can simply withdraw eligibility for the project. Negative distortion of LGA performance: There is a risk that the types of questions asked could indirectly encourage biased mismanagement (negative distortion) in future performance of LGAs depending on the perception of LGAs’ assessment process and timing, and given the availability of “shortcuts” or quick unsustainable “fixes” that could be assumed just in time for the next assessment. Mitigation means: Each question’s potential to result in negative distortions was considered, and as much as possible, questions were designed so that the behavioral responses they might plausibly inspire in LGAs would be positive or, at worst, neutral. Additionally, budget analysis will be conducted based on assessment of formal documents rather than interview questions that can give misleading ideas to LGAs regarding quick, unsustainable corrections.

18

Conclusion and Next Steps

Administrative reform is a complex and long-term process. Even when significant political commitment and resources are brought to bear, it does not come easily and is easy to subvert. There are few easy answers or simple instructions. Many general principles have been identified, but these require intensive work and perseverance in order to be transferred into actions and results on the ground. The LEEMP Local Governance Scorecard will be applied immediately after project effectiveness in the autumn of 2003. This is strategically timed following the general elections in Nigeria during April 2003 so as to avoid any disturbances, misinterpretation or misuse of the scorecard for short-term political purposes. Results of the first round of LGA assessment will be analyzed and provided as an update to this working paper by the Autumn of 2004. Consistent with the LEEMP’s LGA selection procedures and built-in incentive structure, the scorecard will be reapplied prior to the project’s mid-term review.

19

Jana OracClearly, this time frame has been superceded. Can we get an update from the new task manager?

Jana OracIn keeping with earlier comments – I’d vote for not making any unrealistic promises re updates, and certainly this date is untenable.

Annex: 1 -1st Draft of Scorecard: Full List of Indicators, their Rationale and Performance Classification

A. State governance environment for the LGA

Rationale Weak Neutral/average Positive

1

Effective legal framework Sustainable capacity within LGAs requires an effective legislative framework, consistent with the constitution.

Legal arrangements that determine the political control of LGAs, the administrative responsibilities (particularly including hiring and firing of employees) and the intergovernmental fiscal relations are either absent, contested or ignored in practice.

Necessary legal arrangements are present but sometimes flouted.

Necessary legal arrangements are present and generally followed in practice.

2 Agreement on inter-governmental revenue transfers (State and LGA; also relevant for federal government)

Compliance with explicit transfer requirements improves predictability of funding to LGAs, enabling them to tackle medium- to long-term objectives and activities, rather than focusing only on short-term/small-scale needs or emergency measures.

State and LGA largely fail to transfer revenues (e.g. percentage of state planning fund, income taxes collected by LGA, ) to other level of government, as required by law. Timing of transfers is highly unpredictable, and amount often differs from what was promised; LGA ability to plan and undertake activities and contracts severely restricted. Revenue predictability is in the bottom quintile of all LGAs in that state.

Some revenue transfers are carried out according to regulations (timing, amount), while others are not (or not consistently). LGA able to undertake reasonable level of activities, but prevented from doing some larger-scale projects principally due to such uncertainties. Revenue predictability is neither in the top nor bottom quintiles of all LGAs in that state.

Requirements for revenue transfers (timing, amount) generally respected in practice. Budget transfer process does not pose major problems for implementing activities and contracts. Revenue predictability is in the top quintile of all LGAs in that state.

3 Autonomy Effective decentralizationrequires that sub-national governments have the autonomy (disciplined by accountability) to respond to local interests.

The combined effect of the legal arrangements, the arrangements for intergovernmental transfers, and the nature of political control is to produce a LGA that has little or no genuine autonomy from the state government .

LGA exercises reasonable amount of autonomy, although problems persist in some areas.

LGA enjoys full autonomy envisioned by the constitution and federal and state laws.

20

A. State governance environment for the LGA

Rationale Weak Neutral/average Positive

4 Relationship with state Ministry of Local Government Affairs or equivalent (State and LGA)

Cooperation between State and LGA levels facilitates effective decentralization and service delivery.

LGA has little or no contact with state government. State excludes or provides minimal support to LGAs that are of different political affiliation or lack personal connections. LGA disregards state guidelines and requirements. Persistent lack of communication between states and LGAs results in significant overlapping activities, or gaps in coverage, in areas where the two levels share responsibility.

Some channels for LGA-state communication and cooperation exist, and there are regular attempts to use them. However, weaknesses in these mechanisms commonly result in overlap/gaps in activities. Political affiliation remains a significant influence on standard administrative processes and relationships.

Regular and effective communication between LGA and state. State and LGAs inform each other of actions, participate in joint planning when needed and communicate to resolve gaps or overlaps in cases where they share responsibility for a particular service or area. Both state and LGA fulfill their responsibilities toward each other (e.g. in budget submission process). LGA has voice in state policy and specific LGA-related processes.

B. Budget formulation, execution and reporting

Rationale Weak Neutral Positive

5 Formal and comprehensive budget

A formal and comprehensive LGA budget (including all revenues/expenditures and entities) provides the mechanism for matching priorities with available resources.

No formal budget other than an account of the transfers from state or federal government.

Formal budget but covering only a small proportion of LGA activity.

There is a LGA budget that encompasses great majority (80-90%) of the LGA activities, with relatively few extra budgetary funds and with donor funds generally included in the budget.

6 Effective projectprioritization

Given budget constraints, projects should target clear priorities. Project budgeting should address both capital and recurrent costs, to ensure completion and operation – not abandonment – of projects.

Selection of projects and allocation of budgets done with a short-term (annual) time horizon, and does not adequately provide for completion and operation of projects in future years.

Quality of project selection and funding allocation varies. In some cases, medium- and long-term implications are considered and provided for in the budget.

Project selection/funding decisions take into account operation and maintenance costs, not just investment costs. Selection of projects is not unduly influenced by political considerations or personal ties.

21

B. Budget formulation,

execution and reporting Rationale Weak Neutral Positive

7 Effective revenuecollection of both taxes and user fees

Local revenue collection, to the full extent permitted by law, provides budgetary resources and encourages local accountability and participation.

LGA fails to exercise revenue-raising powers assigned to it by law, or is unable to exercise such powers effectively. Estimated collection rates on indicative taxes/fees are in bottom quintile of all LGAs in that state.

LGA exercises some but not all revenue-raising powers. Collection rates (on indicative taxes/fees) are neither in the top nor bottom quintile, and could be significantly improved through better management.

LGA consistently exercises revenue-raising powers assigned to it. Collection rates are in the top quintile, and are reasonable for the context.

8

Appropriate flexibility in budget allocations

LGA staffs should be able to respond to changing circumstances when these are well-justified, while otherwise maintaining confidence that budget allocations will generally be respected.

Excessive flexibility or rigidity in allocating funds, with management problem as a result. Audit reports show that the deviation from appropriations is in the top quintile for all eligible LGAs in that state.

Some instances in which excessive rigidity or flexibility lead to administrative burdens, negative impacts on activities, planning, etc. Audit reports show that the deviation from appropriations is neither in the top nor bottom quintile.

Changes in budget allocations are possible with good cause. Clear procedures regulate such changes, which are subject to a reasonable but not excessively burdensome level of oversight Audit reports show that deviation from budget appropriations is in bottom quintile.

9 Effective accounting and auditing (no. 1)

Audit and accounts are essential links in the resource management chain that promotes effective use of funds and good services.

Generally poor quality of audits. Full accounts (including financial statements) are not submitted at all, or submitted too late, thereby hampering effective audit. Reliability of account information is low. Regulations that specify audit frequency and projects/accounts to be audited are not respected, and may also be inadequate. No private independent audit available. The delay in circulation of audited financials statements is in the top quintile for all LGAs in that state.

Generally reasonable quality of regulations, audits and reports, given capacity constraints, but could be improved. Enforcement mechanisms need to be strengthened to promote compliance. Private independent audit is available in certain circumstances, but not used often enough (for cost or other reasons). Accounts are often submitted incomplete or late; some information is unreliable. Audit able to proceed in many but not all instances. Delay in circulation of audited financial statements is neither in the top nor bottom quintile.

Regulations meet generally-accepted standards for audit coverage. Audits and reports are thorough and rigorous. Private auditors can be used as necessary. Full, accurate accounts are submitted in a timely manner. All (or almost all) projects and accounts subjected to audits as required by regulations. The delay in circulation of audited financials statements is in the bottom quintile for all LGAs in that state.

22

B. Budget formulation, execution and reporting

Rationale Weak Neutral Positive

10 Effective accounting and audit (no. 2)

Transparency andresponsiveness: Audit recommendations rarely or never implemented. No sanctions in place or applied for failing to implement recommendations. Audit findings available only to limited audience. Reports not published, and not available to communities in an accessible/comprehensible format.

Some recommendations are implemented. Enforcement could be reasonably be strengthened without imposing undue burdens. Audit findings are available in principle, but obtaining them in practice requires effort. Presentation is not in a format useful to communities.

Audit recommendations are discussed and implemented. Sanctions applied if fail to implement recommendations without cause. Audit reports published and widely circulated. Reports available to LGA, state, communities and public in a reasonably comprehensible format.

11 Sound financialmanagement practices

Financial management, including monitoring and control, is an essential link in the resource management chain. It should include: (i) quality and timeliness of reporting to local government Council; and (ii) timely reporting from the LGA to State Bureau of Local Government Affairs and State Assemblies.

LGA is lax in meeting its financial obligations (for reasons unrelated to intergovernmental transfers) and in monitoring and controlling its financial commitments. Significant arrears on required payments. Required frequency of financial reporting is too low, and excessive delays in submitting reports hamper the timely identification and resolution of problems.

LGA often meets financial obligations but also fails to do so with unacceptable frequency. Arrears are a fairly frequent occurrence. Some monitoring and controlling of financial commitments occurs on an ad hoc basis, but is not widely practiced. Problems with financial reporting are commonly reported.

LGA meets financial obligations in a timely manner, and monitors and controls financial commitments effectively to prevent overruns. Arrears on required payments are rare. Financial reporting carried out annually and in a timely manner.

23

C. Participation & planning Rationale Weak Neutral Positive 12 Consultation,

responsiveness and redress vis-à-vis communities

Consultation enables LGAs to serve community interests, promotes accountability and effectiveness of resource use, including increasing viability of specific projects by tailoring them to explicit community priorities. Clear, accessible channels for feedback and redress increase the likelihood that community needs will be addressed and promote accountability to constituents.

Consultation non-existent or rare. Narrow consultation, not representative of broader community interests and stakeholders. Segments of community (e.g. women, ethnic minorities, disadvantaged groups) excluded. Community’s stated needs and preferences not considered in project planning. Community reports lack of contact. Communities neither involved in nor adequately informed of funding allocation decisions, including cutbacks and other measures to resolve budget shortfalls. Avenues for contesting decisions, filing complaints or seeking redress are unclear or non-existent. Communities rely significantly on personal connections to make their case to LGAs, and believe formal channels to be largely ineffective.

LGA makes good faith attempts at consultation though these may not be as systematic, institutionalized or as far-reaching as would be ideal. LGA solicits some community participation in some but not all areas of budgeting. Level and/or extent of participation should be increased. Some avenues for redress are in place and have some credibility with communities. Formal channels may not be fully defined or functional in practice, but informal methods mitigate this shortcoming to some degree.

Regular, open consultation with a representative sample of stakeholders, including official leaders, community-based organizations, NGOs and citizens. Measures taken to include women, minorities and disadvantaged populations. Community needs and preferences are an important factor in selection and design of projects. Community satisfied with level of involvement and transparency. LGA requests and considers community input in budget decisions. LGA informs communities of shortfalls and their causes, and proposed means of mitigating them. Clear avenues exist for contestability, grievance and redress related to LGA decisions and activities. Communities are aware of and use channels for communicating with LGA, report receiving responses, and believe that their feedback is given serious consideration.

24

C. Participation & planning Rationale Weak Neutral Positive 13

Effective LGA Council The elected council is the primary vehicle for communicating community interests and concerns

LGA council plays no role in planning, or is seen to be comprehensively captured by other interests or otherwise unrepresentative

LGA council plays some role in planning, but is captured by other interests on some issues.

LGA council plays an effective role in community consultation, planning and budgeting.

D. Project Implementation capacity

Rationale Poor Neutral Positive

14 Community participationin project implementation

Participation fosters a sense of ownership, thereby promoting quality and long-term viability of projects.

LGA manages project implementation without involving communities. Significant number of projects abandoned or under-utilized because they do not respond to community needs or are not welcoming of community members. LGA does not solicit or negotiate community contributions and buy-in even for those small- and medium-scale projects that would benefit from community labor or funds. Community communications to LGA regarding project implementation go unanswered, or receive inadequate responses.

Community involvement is variable. Some communities are favored by the LGA. Some LGA entities place importance on promoting participation while others do not. Community contributions are negotiated in some cases, but could reasonably be agreed in additional instances as well. Community commitment and ownership vary accordingly. Some communications are answered. Community may need to resort to informal channels (e.g. personal contacts) or repeatedly request information.

Community members involved in all phases of LGA-sponsored project, including monitoring and evaluation, operation and maintenance. Community members utilize and are satisfied with completed projects. When feasible (i.e. small and medium-scale projects) Community contributes matching funds and labor to projects, as agreed with LGA. Community feels ownership of project and commitment to its sustainability. LGA has a functioning system that responds to community requests, proposals or complaints in a timely and appropriate manner.

25

15 Involvement of

community-based organizations and non-governmental organizations

Direct involvement of civil society organizations brings in expertise and promotes government responsiveness.

CBOs and NGOs are present in the community, but LGA does not involve them, or utilize them effectively in relevant projects.

Some involvement of CBOs and NGOs, and other civil society actors. Opportunities for greater involvement exist.

CBOs and NGOS effectively involved in relevant projects. LGA actively seeks partnerships.

16 Transparent and openprocurement

Competitive and open procurement processes increases the likelihood of quality and affordability in contracting, and accountable use of funds.

Procurement procedures are unclear or non-existent. Contracts awarded based upon personal or political considerations rather than merit.

Procurement procedures exist but fall short of what could reasonable be done in the context. Procurement outcomes are often acceptable but favoritism is often suspected.

Procurement processes conducted according to explicit and publicized criteria, rather than based upon personal or political favoritism. Multiple evaluators. Bidders notified of outcomes, which are also available to public. Processes in place to handle complaints.

17 Active monitoring and evaluation of projects

Careful oversight reduces the scope for abuse and facilitates project quality.

Inadequate requirements and regulations. Regulations not respected in practice. Records not kept or not reliable.

Mixed record on project monitoring and evaluation. In some cases (e.g. certain types of projects or LGA entities, or certain staff) it is adequate, but there is significant discretion and variation in implementation.

For each project, there are clear arrangements and responsibility assignments for monitoring and evaluation of activities and expenditures. Monitoring and evaluation are carried out to an acceptable standard.

18 Hiring of communitymembers

Employment brings direct economic benefits to the community, promotes ownership and sustainability.

Community members are not considered for jobs on projects in their community.

Some cases of community members being hired.

Community members receive fair consideration and are hired for jobs on projects in their community.

26

E. Personnel and

administration Rationale Poor Neutral Positive

19 Appropriate staffing levels Government should be a responsible employer, ensuring that employment costs are restrained to an affordable level without unduly harming performance.

Incidence of severe over- or under-staffing results in budget problems (e.g. crowding out of operations and maintenance), rampant inefficiency or inability to deliver services. Wage bill as a proportion of total LGA expenditure is in the top quintile of all LGAs in that state.

Pockets of over- or under-staffing exist or appear from time to time. Such situations are remedied slowly or not at all. Wage bill as a proportion of total expenditure is neither in the top nor bottom quintiles of all LGAs in that state.

Staffing numbers are appropriate for organization’s mission. Wage bill as a proportion of total LGA expenditure is in the top quintile of all LGAs in that state.

27

Annex: 2: Revised scorecard: Summary of principal indicators and near-final interview questions

Original rationale Basis for inclusion Questions in near-final technical instrument A. Financial

management; Budget formulation, execution and reporting

1. 2.

Effective accounting and response to audits. Sound financial management practices

Audit and accounts are essential links in the resource management chain that promotes effective use of funds and good services. Financial management, including monitoring and control, is an essential link in the resource management chain. It should include (1) quality and timeliness of reporting to LG council and (2) timely reporting from LGA to state.

In practice, state audits are outside of LGA control and vary in timeliness and quality. However, where robust audits are available, they – along with other information such as reporting and insider feedback -- can provide insights into LGA financial practices: e.g. compliance with regulations, generalquality of LGA accounts, willingness to act on findings, instances of egregious financial misconduct.

• If so, how often does this happen – often, sometimes, never?

Auditor general • Based on the most recent completed audit, would you say this LGA’s

compliance with financial regulations is: acceptable, marginal, or unacceptable.

• When the audit points out problems, does this LGA make good faith attempts to rectify those problems? Always, usually, sometimes, rarely, never.

Director, State LG Inspection unit • How well does the LGA follow financial regulations, and keep its accounts?

Acceptable, marginal, unacceptable Director of Administration, Treasurer • In this LGA, have you ever been asked to co-sign a check that you didn’t

believe was legitimate? Yes, No.

• In such cases, what happens if you point out the problem? Is it rectified, or are you overruled?

• Did this situation – being pressured to issue or co-sign illegitimate checks – ever arise in the other LGAs you served in?

• Have you ever had occasion to point out to the chair that the LGA’s proposed

actions would violate state or other regulations? • If so, how often does this happen – often, sometimes, never? • In such cases, are your remarks generally accepted or not? Was the problem

rectified, or were you overruled? • In general, how well do staff of this LGA comply with requirements to return

unused money from advances, or submit receipts for expenditures from such advances, by the appropriate deadline? That is, do you have problems with people not returning money by the deadline, or not submitting receipts after they have collected an advance? (serious, moderate, or minimal problem)

28

Original rationale Basis for inclusion Questions in near-final technical instrument • What about the chairman and councilors? (serious, moderate, or minimal

problem) Internal Auditor • In general, to what extent does the LGA try to comply with key [the most

important and reasonable] financial regulations – almost always, usually, never.

• Have you ever had occasion to point out to the chair that the LGA’s proposed actions would violate state or other regulations?

• If so, how often does this happen – often, sometimes, never? • In such cases, are your remarks generally accepted or not? Was the problem

rectified, or were you overruled? Director, State LG Inspection unit • For each of the two key reports [insert specific title]:

o Based on your records, did the LGA submit the most recent such report on time? Yes, No.

o If no to above: was the delay: acceptable, marginal, unacceptable – or: minor, moderate, severe

Director of Administration, Treasurer • What is the date [month] of the most recent [REPORT NAME] that you

submitted to the state? 3. Transparent and open

procurement

Competitive and open procurement processes increase the likelihood of quality and affordability in contracting, and accountable use of funds.

In practice, state procurement rules may have problematic aspects (e.g. outdated thresholds, delays in approvals from higher levels) but best efforts at transparent, competitive procurement remain an important indicator of LGA’s financial practices and accountability.

Director, LG Inspection unit • How well does the LGA respect state procurement regulations, including

requirements to seek state approval? Acceptable, marginal, unacceptable Director of Administration • In the specific area of procurement, that is the tendering of contracts, would

you say that the LGA complies with procurement regulations or not? This includes both thresholds/authority as well as other processes, such as the involvement of various committees in the selection process. Complies: Always, usually, sometimes, rarely, never.

• How many tenders over 500,000 Naira has the LGA issued in the past year (or, if in office less than a year, in the time since you arrive here)?

• Could you show us copies of the announcements for these tenders, and tell us how/where you publicized them?

• Did the LGA obtain state approval for all of these tenders? 4. Formal and A formal and LGAs are required to Director of Administration, Treasurer

29

Original rationale Basis for inclusion Questions in near-final technical instrument comprehensive budget [Sound and robust budget process]

comprehensive LGA budget (including all revenues/expenditures and entities) provides the mechanism for matching priorities with available resources.

have a formal budget, and do produce this document, though not always in a timely manner. However, the approach to budgeting and the process followed are not necessarily robust – e.g. unrealistic projections made for prestige reasons; under-funding of fundamental priorities -- service-delivery needs -- in favor of overhead costs, or excess employment. The budget process is within the LGA’s control, and there are benefits to encouraging a more considered and timely process. LGA borrowing is an opaque area. To the extent it does occur, the scorecard probes compliance with state regulations as well as restraint and soundness of borrowing, as with other expenditure decisions – e.g. to cover a shortfall in federal transfers in order to pay wages vs. financing non-essential

• When [month only] was the budget for this year approved by the LGA council?

• If the budget estimates have not yet been approved, what is the r