Post on 03-Dec-2014
description
transcript
Anti-corruption and Result-Based Management (RBM)UNDP
2012
In plain language…
RBM helps us to connect what we do to what we want to achieve
RBM also tells us how we’ll know if we’ve achieved it
What is results based management?
A management strategy that aims at ensuring:• that activities achieve desired results
(Performance monitoring is a critical element)• How well results are being achieved• What measures are needed to improve the process
Results
Goal
Impact
Outcome
Outputs
Activity
Indicator
RBM Terminology (UNDG Approved)
Changes in a state or condition which derive from a cause-and-effect relationship. There are three types of such changes (intended or unintended, positive and/or negative) which can be set in motion by a development intervention – its output, outcome and impact.
The higher-order objective to which a development intervention is intended to contribute
Positive and negative long-term effects on identifiable population groups produced by a development intervention, directly or indirectly, intended or unintended. These effects can be economic, socio-cultural, institutional, environmental, technological or of other types
The intended or achieved short-term and medium-term effects of an intervention’s outputs, usually requiring the collective effort of partners. Outcomes represent changes in development conditions which occur between the completion of outputs and the achievement of impact.
The products and services which result from the completion of activities within a development intervention
Actions taken or work performed through which inputs, such as funds, technical assistance and other types of resources are mobilised to produce specific outputs
A tool to measure evidence of progress towards a result or that a result has been achieved.
Advantages of RBM
• Improved focus on results instead of activities
• Improved transparency
• Improved accountability
• Improved measurement of programme achievements (performance rather than utilization)
• Enhanced strategic focus
Challenges
• Difficult to apply causal logic
• Difficult to learn
• Difficult to integrate
• Difficult to revise (... or reluctance to revise? )
• Difficult to measure
• Difficult to ‘attribute’ (at outcome level, the UN is partly accountable but not wholly
responsible)Go to typology
Linking analysis with results
Impact: (e.g., corruption reduced)
Outcome: (e.g., improvements in the performance of government institutions as well as monitoring my citizens)
Output: (e.g., tangible changes in any elements in the capacity of government institutions and population (information, attitudes/motivation, risks, resources)
Linking Analysis to results:
Translating the problem tree into results
Causality Analysis Results chain
(Problem tree) (UNDAF matrix)
Immediate causes Impact (long-term)
Underlying causes Outcomes (medium-term)
Root causes Outputs (short-term)
1. To effectively analyze problems and their causes, using evidence and lessons including key institutions/capacity gaps
4. To establish performance monitoring and evaluation systems (including risk management)
5. To capture and monitor actual results vis-à-vis targets
6. To clearly identify issues/bottle necks, generating lessons regarding what works/ what doesn’t work
2. To develop right indicators that measure capacity changes and performance improvements
3. To determine baseline and setting targets
Linking analysis with results: Why results frameworks are relevant for anti-corruption in UNDAF?
UNDAFs?
efficiency
effectiveness
Impacts
Outcomes
Outputs
Activities
Inputs
RESULTS
INFLU
ENCE
Performance indicators
Pla
nn
ing
Mo
nito
ring
Linking Analysis with Results: The Results Chain
Result Chain – Quick Recap
Inputs
Activities
Outputs
Outcomes
Impact
Indicator
Resources available - money, staff, facilities, equipment, technical expertise
What is done with the inputs - holding seminars, producing manuals
Services or products produced as a result of activities – 100 staff trained
s Changes, effects due to the activities and outputs – higher skill levels
Long-term results – quality of government services, less corruption
Measure progress towards a result over time that indicates positive, negative, no change with respect to progress towards a stated target
Corruption and Anti-Corruption Indicators
What is an indicator?
A tool to measure evidence of progress towards a result or that a result has been achieved.
Types of Indicators
Quantitative statistical measures:
• Number of• Frequency of• % of• Ratio of• Variance with
Qualitative judgments or perceptions:
• Alignment with• Presence of• Quality of• Extent of• Level of
Considerations
• Indicators do not exist in a vacuum – they should always be related to results
• Need a balance of quantitative and qualitative• Some results are more suitable for indicators than
others• It takes time to get indicators right
A Key Challenge:
Is it possible to meaningfully measure corruption, which has no universal definition, but has different forms, typologies, manifestations, determinants, causes and symptoms?
“What gets measured, gets managed.” - Peter Drucker. So, the UN should demonstrate results, results and results.
The mushrooming industry of indicators: Trying to measure perception, impact,
existing gaps, integrity, enabling environment, etc.
80 90888682 84 96 98 00 02 0492 94 06
International Country Risk Guide
Corruption Perception Index
GovernanceMatters
7876
CPIA (WB)
1974
Freedom in the World
Afrobarometer
Bertelsmann Transformation
Index
Bribe Payers Index
BEEPS
CIRIHuman Rights
Database
Commitment to Development
East Asia Barometer
GAPS in Workers’ Rights
Gender Empowerment
Measure
Eurobarometer
Global Accountability Report
Global Competitiveness
Index
Global Integrity Index
Index of Economic Freedom
Journalists killed
Media Sustainability
Index
Opacity Index
Open Budget Index
Polity
Press Freedom
World Governance Assessment
Are there enough tools and methodologies?
YES
Are these existing corruption and anti-corruption indicators helpful?
It depends -
Recall considerations
Understanding the tools and indicators
Global Indices:
CPI; WB Governance Indicators, Freedom House Indicators
Regional indices: Latin Barometer; Afrobarometer
Country level indices
Understanding corruption and anti-corruption tools and indicators (contd.)
Corruption Transparency/Accountability/Integrity
Diagnostic Assessments
Institutions
Processes
Sectors
Local level
Compliance monitoring
Perception
Experience/ victimisationPublic
opinion
Experts
Public sector
General population / vulnerable groups
Public sector
Private sector
Mapped by Transparency International
“Not everything that counts
can be counted, and
not everything that can be
counted counts.”
– Albert Einstein
Challenge is to find the right indicators!
Use and misuse of indicators for RBM
1. No indicator is perfect and thus standard; all have advantages and disadvantages. However, global perception or cross-country indicators are not useful for tracking progress at the country level.
2. In terms of quality and usefulness of data, it is advisable that the country-based and nationally-owned corruption measurement/assessment indicators be used.
3. Similarly, experience-based indicators (victimization surveys) are advisable over the perception indices. However, victimization surveys data are expensive, too.
1. The perception data (e.g., score cards) at the country level has been often used as a proxy.
2. If the country level data/indicator are not available, regional surveys such as Afrobarometer or Latin Barometer can also be useful.
3. The selection of indicator mainly depends on anti-corruption outcomes and outputs, availability of data and the cost associated with gathering data.
DOs and DON’Ts of capturing adequate or satisfactory results
Bad Good
No supporting indicators, programme logic or data
Increase evaluability of anti-corruption projects, prospective evaluations
Weak, activity-oriented indicators Focus on a few good results oriented (outcome and output) indicators, and disaggregate data
Major cause-effect (attribution) leaps from output to impact level
Stop reliance on macro-level global indices (CPI) and develop country level indicators
No evaluation/logical framework to capture results
Mixed methods. Pragmatic approach to statistical design (best fit) and strong qualitative support
Worst: Claiming impact when there is no result chain or framework
Adapt impact evaluation methods to characteristics of anti-corruption projects
Performance Indicator Selection Criteria
• Validity - Does it measure the result?
• Reliability - Is it a consistent measure over time and, if supplied externally, will it continue to be available?
• Sensitivity - When a change occurs will it be sensitive to those changes?
• Simplicity - Will it be easy to collect and analyze the information?
• Utility - Will the information be useful for decision-making and learning?
• Affordable – Do we have the resources to collect the information?
Illustrative example of the results chain of an Anti-corruption initiative (adapted from a country example)
Stakeholder dialogue regarding corruption issues, review of current audit mechanism and gaps, assessment and development of the capacities of supervisory body, budgetary and funding support; etc.
(money, staff, facilities, equipment, technical expertise)
Government supervisory functions and audit processes are established
Anti-corruption institutions/ supervisory bodies are enforcing the law
Media has channels to publish corruption related information
Information regarding incidences of corruption is accessible to the public
Public government officials, citizens, etc. have reduced their engagement in corruption practices
Reduced corruption and increased public trust towards
public institutions
Citizens have mechanism to report on corruption incidences
Government agencies are exchanging information about corruption incidences
Corruption data management system is put in place
Knowledge, Innovation and Capacity Group, BDP, UNDP (April, 2013)
Inputs
Activities
Outputs
Outcomes
Impact
UNDAF’s Results Matrix
• Currently many UNDAFs restrict the aspect of anti-corruption to the objective of governance / human rights / democracy. Anti-Corruotion is hardly an output in case of MDGs, environment, conflict prevention, disaster reduction, etc.
• Although corruption is identified in some of the UNDAFs documents, the RBM is very weak.
Anti-corruption in UNDAF’s
Anti-corruption in UNDAF’s
Option 1 A: UNDAF RESULT MATRIC WITH OUTCOME LEVEL ONLY (Mongolia)
Strategic priority - Governance and Human rights
Outcomes Indicators, Baseline, TargetOutcome 1: Representation, accountability and transparency of governing institutions strengthened
Output 1.1. Enabling policy environment created for effective decentralization and increased functional capacity of local governments to deliver service.
Capacity of local governments to deliver services
Output 1.2.
Increased capacity to implement the UN Convention Against Corruption
Compliance with UNCAC provisions on corruption prevention
Output 1.3. Increased civil society participation in key national processes and strengthened state-citizen engagement for accountable and responsive governance
Feedback mechanism of state-CSOs-state Social dialogue between government, workers and employers
Bad or good?
What is the underlying vision behind the reform/policy? What are the objectives pursued? Which results do we expect to see?
What is the logic underpinning the reform/policy design? (from inputs to outputs, outcomes and impact)
What are the preconditions for success? Which behavioural assumptions are being made? How will political, cultural or economic factors affect potential for change?
Key questions you need to consider when formulating the results matrix
Results Matrix
Reduction in corruption
Corruption cases are
investigated to required standard
Substantiated cases are sent
to court and prosecuted
Other law enforcement
inst. cooperate.
Witnesses are willing to come forward
Sufficient resources
and mandate
Donor demands
for outputs
Legislative
obstacles
Result Chain with Risks and Assumption
Group Work 2 (15 minutes)
1. Put together a coherent, logically sequenced result chain to introduce a Code of Conduct (CoC) in ministries (yellow pieces)
2. Identify where indicators fit at the right level of results in the chain.
Inputs Activities Outputs Outcome Impacts
Output indicators
Outcome indicators
Impact indicators
Key messages
• Corruption and anti-corruption measurements and assessments provide useful information for country analysis, which is an integral part of UNDAF process.
• Corruption and anti-corruption measurements and assessments also strengthen results by providing indicators to measure, monitor, and report on change.
• However, it is also important to make sure that the appropriate indicators are used to measure progress and results.