Date post: | 27-Mar-2015 |
Category: |
Documents |
Upload: | dylan-tucker |
View: | 215 times |
Download: | 0 times |
ECA: New Frontiers in Achieving Results* Plenary April 1, 2008, Improving the Lives of People in ECA
Vinod Thomas, Director-General Independent Evaluation Group
*This presentation is based on IEG’s recent evaluation findings. Ann Flanagan and Jiro Tominaga contributed to its preparation.
Progress and shortfalls in world development
► Per capita economic growth averaged 2% word-wide, 4% in developing countries and 6% in ECA (2000-06)
► Income levels diverged across countries, but converged across populations (due to rising income in populous China and India)
► Social indicators – infant mortality, school enrollment – tended to converge worldwide since 1990
► The proportion of the poor declined since 1990, but the absolute number of poor is still around 1 billion
► Environmental degradation continued to rise everywhere -- especially pollution, land degradation, deforestation
► Progress in corruption has been slow in most countries, while stronger in ECA (1999-2006)
Growth, distribution and poverty
-20
-15
-10
-5
0
5
10
15
20
Mol
dova
Camer
oon
Seneg
al
Burkin
a Fa
so
China
-Urb
an
Alban
iaPer
u
Sri La
nka
Ukrain
e
Armen
ia
Niger
ia
Lith
uani
a
Jord
an
Roman
ia
China
-Rur
al
Domin
ican
Repub
lic
Turke
y
Brazil
Hondu
ras
Bolivi
a
Zambia
Urugu
ay
Pakist
an
Geo
rgia
Mad
agas
car
Ch
an
ge
in P
ov
ert
y
Change in poverty due to growth in household income Change in poverty due to change in distribution
Attribution of results is not easy
InternationalFinancial
Institutions
LocalPolicymake
rs
Policies CountryOutcome
s
KnowledgePolitical Economy
Development Effectiveness
Private CapitalFlows
I. Connecting operations to results
ECA Projects & AAA•Moldova: AIDS Control•Turkey: Earthquake•Romania: Rural Educ•Bulgaria: Rev. Admin•Kyrgyz: Social Prot •Ukraine: People’s Voice Etc.
EquityEfficiency
Sustainability
Quality of People’s Lives
Project-Program
Public-Private
Country-Global
Knowledge-Lending
→ → →2008 → → → → →2015→ → → → → 2030 → → →Time
1. WB projects outcomes improving
0% 20% 40% 60% 80% 100%
AFR
EAP
ECA
LCR
MNA
SAR
Percent Satisfactory (Weighted by Commitment)
FY1998-2007
FY1987-1997
Source: Staff calculations, IEG Database
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
1975 1980 1985 1990 1995 2000 2005Exit FY
BANK ECA
Percent Satisfactory (Weighted by Net Commitment)
…. but with program disconnects
Summary Result Matrix
SatisfactoryCAE
Unsatisfactory
CAE
SatisfactoryPortfolio
58.8% 29.4%29.4%
UnsatisfactoryPortfolio
5.9% 5.9%
Summary Result Matrix
Satisfactory CAE
Unsatisfactory
CAE
Satisfactory Portfolio
55.8% 35.1%35.1%
Unsatisfactory Portfolio
2.6% 6.5%
World Bank ECA
Source: Staff calculations based on IEG project ratings (IEG Database) and IEG CAE ratings (1985-2006)
2. Private sector can complement outcomes
Development outcome andIFC’s profitability (ECA)
Additionality and project development results (ECA)
Note: ECA region by number of projects based on 178 projects from 1996-2007 XPSRs.
Need public-private coordination for better results
IEG reports document crucial gaps in WB-IFC collaboration, but there are examples of linkages:
► Leasing: IFC leasing projects and technical assistance contributed to the Bank’s rural agriculture leasing projects in Azerbaijan, Kazakhstan, Ukraine
► BTC pipeline: In Azerbaijan the Bank focused on transparency and fiscal management while IFC concentrated on mobilizing consortium, social and environmental sustainability and SME linkages.
► Sub-national finance: The Bank maintained relationships with regional governments. IFC added value with its ability to finance without a sovereign guarantee.
3. Knowledge, lending and transition
► AAA rated as moderately effective or better in influencing government decisions by 2/3 of the MIC report client respondents: e.g., Ukraine’s WTO accession process
► Integration with country strategies: knowledge services provided entry for relationship with the government on public sector reform
► Knowledge-led strategies: QAG ratings of AAA programs in 8 countries with high budgets for knowledge lower than those with lower shares of AAA • Difficult to use AAA on its own as a strategic instrument• Some countries to agree on cost-sharing for AAA based on multi-year
program – Kazakhstan JERP
► IEG’s forthcoming AAA evaluation: 12 cases including Romania and Serbia. • Role in informing Bank strategy? Link to lending? When AAA can be
decoupled from lending? Adequacy of results framework and follow up in AAA?
Source: IEG, Development Results in Middle-Income Countries; and Evaluation of World Bank Support for Public Sector Reform
Lending & policy: PSR lending, CPIA 99-06
► ECA has the highest rate of CPIA governance score improvements for countries getting PSR lending
► The rate of improvement for non-borrowers is almost as high
► External factors e.g. EU accession may be in play
PSR Loan
Yes No
AFR 70% (30) 47% (15)
EAP 70% (10) 56% (9)
ECA 90% (20) 86% (7)
LCR 75% (20) 25% (8)
MNA 57% (7) 0% (2)
SAR 50% (6) 0% (1)
Total 73% (93) 48% (42)
Source: WB CPIA Scores and IEG staff calculationsNote: Total number of countries in parenthesis
4. Regional/global and country programs► With the growing importance of trans-border issues,
regional programs (such as building swifter trade routes Southeast Europe) and global programs (such as GEF) have vast potential
► At $3.5 billion in 2006, regional programs account for less than 1 percent of the Bank’s total project and partnership financing.
► Effectiveness hampered by lack of connection between country programs and regional/global initiatives.
► M&E especially weak in regional and global programs
Source: IEG, Annual Report on Operations Evaluation 2004 and 2005.
II Role of M&E in the results agenda
Projects with higher outcome ratings have better M&E ratings, FY2001-2007
1.671.78
2.00
2.62
3.00
1
2
3
4
Unsatisfactory ModeratelyUnsatisfactory
ModeratelySatisfactory
Satisfactory HighlySatisfactory
IEG Outcome Rating
Av
era
ge
IEG
M&
E R
ati
ng
Source: IEG Database
Results chain and program effectiveness
A well articulated results chain would (ARDE 2006):
► Results-based CASs, if done well, puts the focus on how projects add up to country impact
► Emphasize realism in setting objectives – a problem in 50% of Country Assistance Strategies (FY2001-FY2005) evaluated by IEG
► Track key synergies across sectors – for example, infrastructure for education results or environment for health results
► Focus on capacity building – after all, country M&E determine the application of a results chain in countries
ECA’s experience in M&E
► Performance-based budgeting of various types being followed in Czech Republic, Kazakhstan, Russia, Slovenia and Turkey
► Especially EU accession countries are following OECD standards; some are becoming donors -- Czech Republic, Latvia, Poland and the Russian Federation. Greater accountability requires M&E capacity
► Projects: The South East European trade and transport facilitation project introduced electronic forms showing duties due. Turkey brought extra-budgetary funds (that had undermined fiscal discipline) into the budget and parliamentary scrutiny
Impact evaluation can strengthen M&E
► IE can promote or question programs – a crucial role of knowledge• Found the T&V extension system used in Kenyan agriculture to have no
impact on agricultural production
► It tracks linkages across sectors – including unintended and indirect• Showed the linkage between a nutrition program in Bangladesh and
secondary schooling for girls education
► It de-politicizes decision-making – contributes to sustainability• Evidence-based case for the Mexican Opportunidades program
made it difficult for politicians to discontinue support for after change of government
► Initiatives at the Bank: Development Impact Evaluation Initiative; IFC Advisory Services Impact Evaluation Initiative; Spanish World Bank Trust Fund for Impact Evaluation; African Impact Evaluation Initiative; IEG Impact Evaluation Program
► Challenges ahead: be demand-driven; enter programs ex ante; be theory based; focus IE on the most relevant issues
ECA lags in impact evaluation
Source: IEG staff calculations (draft), DIME Database and Poverty Impact Evaluations DatabaseNote: The Poverty Impact Evaluations Database contains evaluations conducted by both Bank and Non-Bank staff.
LCR has the most number of completed and ongoing evaluations
23 29 21
92
024
5318
31
383
30
20
40
60
80
100
120
140
AFR EAP ECA LCR MNA SAR
Nu
mb
er o
f ev
alu
atio
ns
Completed before 2006 Completed in 2006 or later Ongoing
Four cutting edge challenges in ECA► Project-program: Especially where Bank lending
share is small, there is a premium on exploiting linkages across projects and sectors for the greatest country impact
► Public-private: It is vital to develop public-private partnerships and to make the most of public and private linkages
► Knowledge-lending: The knowledge-lending axis is exploited differently across countries
► Country-global: The country-based approach must be reformed to integrate tough issues of climate change, governance, migration…
Thank You