Improving Institutional Capability and Financial Viability to Sustain Transport - An Evaluation of World Bank Group Support since 2002
Presentation at
Civil Society Policy Forum
April 17, 2013
Midori Makino
Independent Evaluation Group
Objective and Definition of Sustained Transport
►Objective of the evaluation is to assess the effectiveness of the World Bank Group (WB, IFC, and MIGA) support to countries to sustain transport
►Transport is sustained when key outcome indicators either achieved or almost achieved by the end of the project were sustained or improved beyond project closure
►Examples of Indicators include accessibility, travel time and costs (intercity highways, rural roads), congestion (urban and air transport, ports), rail traffic (railways) etc.
2
Evaluation Questions
1. To what extent have World Bank Group strategies and operations aimed to ensure sustained transport?
2. To what extent have countries implemented; (i) policies and regulatory framework, (ii) institutional framework, (iii) financial arrangements, and (iv) sector management and capacity building activities supported by the World Bank Group to sustain transport?
3. How effective have these measures been?
4. What factors have determined success?
3
Evaluation Framework
Physical infrastructure Transport Sustaining Measures
Sustained Transport
Poverty Alleviation
Economic Growth
Outputs Outcomes Development
Impact
Economy of the Country, Demographic Changes, Natural Disasters, and Other External Factors
Intermediate Outcomes
Financial Viability Institutional Capability
Country Priorities and Political Economy Factors
World Bank, IFC, and MIGA Support
Inputs
4
Evaluation Methodology/Building Blocks
1. Review of World Bank Group sector and country strategies
2. Portfolio review of World Bank Group operations approved from FY 2002-11 (573 lending , over 1,000 non-lending)
3. Nine field- and eleven desk-based country focused project assessments covering 68 projects
4. IEG’s 30 project performance assessment reports carried out during the past five years
5
Key Evaluation Findings
6
Sustained Transport of World Bank Projects Varies by Country Income and Sub-sector
52%
78% 67%
57% 46%
33% 33%
0%
57%
0%
20%
40%
60%
80%
100%
Low and Lower middle Income (n=54; c=15)
Upper middle
and High Income (n=14; c=5)
Intercity highways
(n=36; c=14)
Ports (n=7; c=7)
Urban Transport
(n=13; c=11)
Aviation (n=6; c=5)
Rural Roads (n=15; c=13)
Railways (n=15; c=11)
All Projects (n=68; c=20)
% of Projects with Sustainable Transport Outcomes, Based on Field and Desk Based Assessment of Project Results
(n=number of projects, c=number of countries)
7
Projects that Planned and Implemented Measures were more likely to achieve Sustained Transport Outcomes
8
0%
25%
50%
75%
100%
Sector policy and regulatory framework
Institutional framework
Sector Management and Capacity building
activities
Financial arrangements
% of World Bank Transport Projects that achieved Sustained Transport Outcomes by Implementation Status of Measures
Planned and implemented measure Planned but not implemented measure
0
10
20
30
40
50
60
70
80
90
100
Sector policy and regulatory framework
Institutional framework
Sector management and capacity
building
Financial arrangements
% of World Bank Transport Projects that Included Measures to Sustain Transport by Fiscal Year of Approval
FY02-06 FY07-11
Sustained Transport is emphasized in Sector and Country Strategies, but it is not well reflected in Projects
9
The Share of Projects with Sustained Transport in Objectives and Projects that Identified Risks have declined
1
0
0%
5%
10%
15%
20%
25%
30%
35%
40%
45%
Percentage of Projects with Sustainability as part of their Objectives
Percentage of Projects that identified maintenance risks at appraisal
% of World Bank Transport Projects that included Sustained Transport in their Objectives and that identified maintenance risks
at appraisal by Fiscal Year of Approval
FY02-06 FY07-11
Determinants for Success
1
1
► Transport is more likely to be sustained when such emphasis is reflected in project preparation and design:
Sustained transport in objectives
Measures to sustain transport in components
Identification of maintenance shortfall risks
Determinant 1: Focus in Project Design
1
2
Determinant 2: Accountability and Incentive Structure
►Transport is more likely to be sustained when a clear accountability framework and incentive structures for operations and maintenance are in place, in cases such as: Contractual arrangements and user-pay principles,
often featured in public-private partnerships such as operations supported by IFC
Output- and performance based road contracts
Microenterprise/community participatory models in rural roads
1
3
Determinant 3: Policy Level Interventions
►Transport is more likely to be sustained with broad based policy level interventions supporting investments operations:
Development policy operations, public expenditure
reviews, and other analytic and advisory activities and policy dialogue complementing specific investment loans
Multi-modal planning and coordination at the country’s sector level, especially in planning railways
Road network models under budget constraints in highways
1
4
Determinant 4: Client Ownership
►Transport is more likely to be sustained with continued WBG support to ensure strong Government ownership: Continuous and sequential engagement taking into
account sector capacity
Attention to political economy by identification of stakeholder concerns, communication, and consultation
Mainstreaming of project activities in government systems
1
5
Other Evaluation Findings
►World Bank, IFC, and MIGA approaches to sustaining transport are different
►World Bank, IFC, and MIGA play complementary roles but there are missed opportunities for synergies
►Transport is more likely to be sustained when managed by the transport sector
►There is no systematic assessment of sustained transport outcomes beyond project closure
1
6
Recommendations for
World Bank Group Management
1
7
1: Increase Focus in Project Preparation and Design
► Increase the focus on sustained transport in projects with sector reform objectives and components
►Systematically carry out ex-ante risk analysis and mitigation for operations and maintenance
► Link investment operations with sector-wide reforms through complementary activities
►Transport components managed by other sectors should be integrated into transport sector plans and strategies
1
8
2: Improve Long-Run Financial Viability
►Engage policy makers for maintenance funding at high level
►Critically assess the viability of investments especially for railways
►Support to urban transport should include financial analysis of overall system and pay attention to affordability concerns.
1
9
3: Strengthen Institutional Capability
►Mainstream proven accountability and incentive frameworks
►Encourage continuous and sequential engagement and support government-led reforms
►Support governments to put in place a process for systematically monitoring and evaluating sustained transport outcomes
2
0
Thank You for Your Attention
2
1
The complete evaluation is available on IEG’s website:
http://ieg.worldbankgroup.org/publications