Date post: | 30-Dec-2015 |
Category: |
Documents |
Upload: | leila-quinn |
View: | 17 times |
Download: | 2 times |
1
Development Effectiveness Development Effectiveness at the World Bankat the World Bank
ODI 04/00
““...we will not measure our performance by dollars lent ...we will not measure our performance by dollars lent or projects approved, but by our development impact -- or projects approved, but by our development impact -- results on the ground. I cannot overstate the results on the ground. I cannot overstate the importance of this change.” importance of this change.”
James D. WolfensohnJames D. WolfensohnAnnual Meeting SpeechAnnual Meeting Speech
October 1996October 1996
2
The Development Environment The Development Environment Has EvolvedHas Evolved
Ascent of GlobalCapital Market
Greater Pluralism
End of Ideological Divide
Advent of Civil Society
Before Now
Money Bank
Planned Development
Conditionality
Resource Transfer
Full Service Bank
ParticipatoryDevelopment
Knowledge Management
and Partnerships
DevelopmentEffectiveness
3
The Very Concept of Development The Very Concept of Development Has ChangedHas Changed
FiftiesFifties SixtiesSixties SeventiesSeventies EightiesEighties NinetiesNineties
ReconstructionReconstruction GrowthGrowth Basic NeedsBasic Needs AdjustmentAdjustment Capacity BuildingCapacity Building
Technical AssistanceTechnical Assistance
ProjectsProjects
Sector InvestmentSector Investment
Adjustment LoansAdjustment Loans
Country Assistance Country Assistance StrategyStrategy
EngineeringEngineering
FinanceFinance
PlanningPlanning
NeoclassicalNeoclassical
Multi-disciplinaryMulti-disciplinary
DevelopmentDecades
Major Objective
Main Instruments
Dominant Discipline
Physical
NaturalHuman
Social
4
World Bank Performance Criteria Reflect World Bank Performance Criteria Reflect the New Development Agenda:the New Development Agenda:
andand
Governance: social development, economic management, public sector reform, etc.
Poverty/genderpopulation,
health,nutrition
Education
Human economy
Environment
Rural development
Infrastructure
Physical economy
Private sectordevelopment
Finance
Market economy
Energy/miningSocialprotection
Socially SustainableSustainableEnvironmentally Growth
5
The Meaning of Development The Meaning of Development Effectiveness Has Changed TooEffectiveness Has Changed Too
Development Effectiveness connotes a Development Effectiveness connotes a demonstrable and efficient contribution to demonstrable and efficient contribution to economically sound, socially responsible economically sound, socially responsible and environmentally sustainable growthand environmentally sustainable growth
6
OED methods are best practiceOED methods are best practice
According to two major reviews of the MDB’s According to two major reviews of the MDB’s evaluation methods and practices:evaluation methods and practices:
Bank implementation of evaluation criteria are Bank implementation of evaluation criteria are closest to the agreed criteria of all MDBsclosest to the agreed criteria of all MDBs
(according to Wyss report (according to Wyss report Harmonization of Evaluation Criteria Harmonization of Evaluation Criteria Among the MDBsAmong the MDBs, 1999), 1999)
The Bank is a leader in completion reporting and The Bank is a leader in completion reporting and performance review among all MDBsperformance review among all MDBs
(according to Eriksson report (according to Eriksson report Comparison of MDB Completion Comparison of MDB Completion Reporting and Performance ReviewReporting and Performance Review , 1996), 1996)
7
How is Development Effectiveness How is Development Effectiveness Assessed? Assessed?
At the project level, OED assesses five main criteria of development effectiveness:
Results
1.Outcome2. Sustainability
3. Institutional development impact
Inputs
4. Bank performance5. Borrower performance
Country conditions, exogenous factors
8
The outcome assessment The outcome assessment
““DOING THE RIGHT THINGS”DOING THE RIGHT THINGS”Relevance: Are objectives responsive to country needs andBank priorities?
““DOING THINGS RIGHT”DOING THINGS RIGHT”
Efficacy: Are we achieving objectives shared by the Bank and its partners?
Efficiency: Are we using Bank and partner resources economically?
Definition:the extent to which a project has achieved or is expected to achieve its major relevant objectives efficiently; rate of return threshold of 10% used when possible
6-point scale:•highly satisfactory•satisfactory•marginally satisfactory•marginally unsatisfactory•unsatisfactory•highly unsatisfactory
9
The sustainability assessment The sustainability assessment
Evaluation of 9 criteria on the risks to project net benefits over time:
technical environmental financial government ownership economic stakeholder ownership social conditions institutional support resilience to exogenous factors
Definition:The resilience to risk of net benefits flows over the long term
3-point scale:•likely•uncertain•unlikely
10
The institutional development The institutional development impact assessment impact assessment
Evaluation of 2 basic criteria:
capacity development of an organization
improving institutional arrangements (i.e., “rules of the game” such as the regulatory framework)
Definition:the improvement in a country’s ability to make effective use of its human, financial, and natural resources
3-point scale:•substantial•modest•negligible
11
What is the unit of account?What is the unit of account?
Investment projectsInvestment projects Sector investment programsSector investment programs Adjustment loansAdjustment loans Non-lending servicesNon-lending services Country assistance strategies (CAS)Country assistance strategies (CAS) Sector strategiesSector strategies
12
SupervisionReports
Mid-termReviews
Imple.Completion
Reports
PortfolioImprovement
ProgramReviews
PerformanceAudit Reports
and ImpactEvaluations
Regions/OPR
Regions/OPR
Regions/OPR
QAG OED
ARDE
RegionsAFREAPSASECAMNALAC
NetworksHDDESSDFPSI
PREM
Country AssistanceStrategy and Compacts
Sector Assistance Strategiesand Thematic Progress Reports
Lending ServicesNonlending
Services Lending ServicesNonlending
Services
ActivityCompletionReports onNonlending
Services
SectorImpactEval.
Networks/OED
AidCoordination
andCofinancing
Reviews
Regions/QAG/OED
ResearchActivity
CompletionReports
Trust FundsCompletion
Memos
Regions/OED
DEC/Networks
Regions/NetworksOED
13
Evaluation in The World BankEvaluation in The World Bank
OED is independent: it reports to the Board of OED is independent: it reports to the Board of Executive DirectorsExecutive Directors
5,000 projects have been evaluated since OED’s 5,000 projects have been evaluated since OED’s creation in 1970 creation in 1970 (644 operations, worth $52 billion in real terms, (644 operations, worth $52 billion in real terms, in FY98 -FY99 alone)in FY98 -FY99 alone)
The function spans a variety of sectors The function spans a variety of sectors (agriculture, energy, finance, transport, (agriculture, energy, finance, transport, water supply, urban development)water supply, urban development)
It evaluates projects, programs, processes, and policiesIt evaluates projects, programs, processes, and policies
14
Project Performance Project Performance Measurement SystemMeasurement System
• consistentconsistent• objectiveobjective
• continuouscontinuous• professionalprofessional
• credibilecredibile• coherentcoherent
(but need further (but need further harmonization)harmonization)
Quality at entryQuality at entry
Quality at exitQuality at exit
Monitoring duringMonitoring duringimplementationimplementation
OED
QAG QAG
15
Real Time Evaluation Means Real Time Evaluation Means Quality AssuranceQuality Assurance
Two Complementary ComponentsTwo Complementary Components Portfolio Improvement ProgramPortfolio Improvement Program Quality Assessment ProgramQuality Assessment Program
– Quality at EntryQuality at Entry– SupervisionSupervision– Economic and Sector WorkEconomic and Sector Work
16
It also Means Quality ManagementIt also Means Quality Management
Target managerial attention on “high risk” Target managerial attention on “high risk” clustersclusters
Regular monitoring and feedback to staff and Regular monitoring and feedback to staff and managersmanagers
Partnering to disseminate lessons of experiencePartnering to disseminate lessons of experience Linking evaluation to staff/managerial Linking evaluation to staff/managerial
evaluations and strategy (a long term goal)evaluations and strategy (a long term goal)
17
Where are we?Where are we?Where are we?Where are we?
Quality at exitQuality at exit have been improving over the 90s have been improving over the 90s Data on the active portfolio promises future improvementData on the active portfolio promises future improvement However, borrower implementation performance stagnantHowever, borrower implementation performance stagnant
1994 1995 1996 1997 1998-99*
Exit fiscal year
50
60
70
80
90
Per
cent
sat
isfa
ctor
y
OED - Outcome performance (exiting)
1997 1998 1999
Active fiscal year
50
60
70
80
90
Per
cent
non
-ris
ky
QAG - Non-risky portfolio (active)
18
(% satisfactory)(% satisfactory)
Exit fiscal yearExit fiscal year
RegionRegion 1990-93 1994-97 1998-99*1990-93 1994-97 1998-99*
AFRAFR 54 54 55 55 60 60 (+)(+)
EAPEAP 80 80 84 84 81 81 (-)(-)
ECAECA 73 73 72 72 83 83 (+)(+)
LACLAC 64 64 78 78 82 82 (+)(+)
MNAMNA 70 70 66 66 70 70 (+)(+)
SASSAS 73 73 67 67 68 68
BankwideBankwide 66 66 68 68 72 72 (+)(+)
Project Outcome by RegionProject Outcome by RegionProject Outcome by RegionProject Outcome by Region
19
Sustainability has been Sustainability has been improving...improving...Sustainability has been Sustainability has been improving...improving...
Sustainability Assessments
0
10
20
30
40
50
60
% Likely % Uncertain % Unlikely
Exit FY94-97Exit FY98-99*
47 49
3337
2014
20
… … as well as institutional as well as institutional development impact.development impact.… … as well as institutional as well as institutional development impact.development impact.
Institutional Development Impact Assessments
0
10
20
30
40
50
60
% Substantial % Modest % Negligible
Exit FY94-97Exit FY98-99*
31
3946 48
24
13
21
Regional ComparisonsRegional Comparisons
Exit FY90–93 Exit FY94–97 Exit FY98–99*
* Results are preliminary for FY99 exits (43 percent coverage).
Sustainability (percent likely)
0 10 20 30 40 50 60 70 80 90
AFR
EAP
ECA
LCR
MNA
SAR
ID Impact (percent substantial)
0 10 20 30 40 50 60 70 80 90
AFR
EAP
ECA
LCR
MNA
SAR
22
(% satisfactory)(% satisfactory)
Exit fiscal yearExit fiscal year
NetworkNetwork 1990-93 1994-97 1998-99*1990-93 1994-97 1998-99*
ESSDESSD 59 59 68 68 63 63 (-) (-)
FPSIFPSI 68 68 65 65 72 72 (+) (+)
HDNHDN 78 78 72 72 75 75 (+) (+)
PREMPREM 66 66 73 73 78 78 (+) (+)
BankwideBankwide 66 66 68 68 72 72 (+) (+)
Project Outcome by NetworkProject Outcome by NetworkProject Outcome by NetworkProject Outcome by Network
23
Network ComparisonsNetwork Comparisons
* Results are preliminary for FY99 exits (43 percent coverage).
Exit FY90–93 Exit FY94–97 Exit FY98–99*
Sustainability (percent likely)
0 10 20 30 40 50 60 70 80 90
ESSD
FPSI
HDN
PREM
ID Impact (percent substantial)
ESSD
FPSI
HDN
PREM
0 10 20 30 40 50 60 70 80 90