+ All Categories
Home > Documents > African Development Bank - PCR EVALUATION …Ethiopia’s potential to achieve high economic growth...

African Development Bank - PCR EVALUATION …Ethiopia’s potential to achieve high economic growth...

Date post: 23-Mar-2020
Category:
Upload: others
View: 2 times
Download: 0 times
Share this document with a friend
25
1. BASIC INFORMATION a. Basic project data Project title: Electricity Transmission System Improvement Project code: P-ET- FA0-008 Instrument number(s): loan 2100150023451 and grant 2100155019269 Project type: Investment Sector: Energy Country: Ethiopia Environmental categorization (1-3) : 1 Processing Milestones Key Events Disbursement and Closing date Date approved: 06/12/2010 Cancelled amount: UA 3,593,441.16 Original disbursement deadline: 31/12/2015 Date signed: 20/12/2010 Supplementary financing: n/a Original closing date: 31/12//2015 Date of entry into force : 24/10/2011 Restructuring: n/a Revised disbursement deadline: 30/4/201 Date effective for 1st disbursement: 07/12/2011 Extensions (specify dates): 1/01/2016- 30/4/2017 Revised closing date: 30/4/2017 Date of actual 1st : 07/12/2011 b. Financing sources Financing source/ instrument (MUA) Approved amount (MUA) : Disbursed amount (MUA) : Percentage disbursed (%): Loan: UA 93,750,000.00 90,156,558.84 96.17 Grant: UA 58,000,000.00 57,704,391.93 99.49 Government: UA 23,000,000.06 23,000,000.00 100 Other (ex. Co- financiers): TOTAL : 174,750,000.06 170,860,951.37 97.7 Co-financiers and other external partners: n/a Execution and implementation agencies: Ethiopian Electric Power (EEP) c. Responsible Bank staf Position At approval At completion Regional Director N.Matondo-Fundani Gabriel NEGATU Sector Director H. Cheikhrouhou H. Baldeh BATCHI Sector Manager E.B. Nzabanita Richard Humphery NDWIGA Task Manager E.B. Nzabanita Girma Mekuria Alternate Task Manager Solomon Abebe PCR Team Leader Mohamed Alin (consultant) PCR Team Members Amare Hadgu (Consultant) d. Report data PCR Date : December 2017 PCR Mission Date: From: 20 November 2017 To: 17 December 2017 PCR-EN Date: December 2017 PCR EVALUATION NOTE FOR PUBLIC SECTOR OPERATIONS
Transcript
Page 1: African Development Bank - PCR EVALUATION …Ethiopia’s potential to achieve high economic growth through increased electrification provides a good rationale for the Bank’s support

1. BASIC INFORMATION a. Basic project data Project title: Electricity Transmission System Improvement Project code: P-ET-FA0-008

Instrument number(s): loan 2100150023451 and grant 2100155019269

Project type: Investment Sector: Energy Country: Ethiopia Environmental categorization (1-3) : 1

Processing Milestones Key Events Disbursement and Closingdate

Date approved: 06/12/2010

Cancelled amount: UA 3,593,441.16 Original disbursement deadline: 31/12/2015

Date signed: 20/12/2010

Supplementary financing: n/a Original closing date: 31/12//2015

Date of entry into force : 24/10/2011

Restructuring: n/a Revised disbursement deadline: 30/4/201

Date effective for 1st disbursement: 07/12/2011

Extensions (specify dates): 1/01/2016-30/4/2017

Revised closing date: 30/4/2017

Date of actual 1st : 07/12/2011b. Financing sources

Financing source/instrument (MUA)

Approved amount(MUA) :

Disbursed amount(MUA) :

Percentage disbursed(%):

Loan: UA 93,750,000.00 90,156,558.84 96.17 Grant: UA 58,000,000.00 57,704,391.93 99.49 Government: UA 23,000,000.06 23,000,000.00 100 Other (ex. Co-financiers): TOTAL : 174,750,000.06 170,860,951.37 97.7 Co-financiers and other external partners: n/a

Execution and implementation agencies: Ethiopian Electric Power (EEP) c. Responsible Bank staf

Position At approval At completion Regional Director N.Matondo-Fundani Gabriel NEGATU Sector Director H. Cheikhrouhou H. Baldeh BATCHI Sector Manager E.B. Nzabanita Richard Humphery NDWIGA Task Manager E.B. Nzabanita Girma Mekuria Alternate Task Manager Solomon Abebe PCR Team Leader Mohamed Alin (consultant) PCR Team Members Amare Hadgu (Consultant)d. Report data PCR Date : December 2017 PCR Mission Date: From: 20 November 2017 To: 17 December 2017 PCR-EN Date: December 2017

PCR EVALUATION NOTE FOR PUBLIC SECTOR OPERATIONS

Page 2: African Development Bank - PCR EVALUATION …Ethiopia’s potential to achieve high economic growth through increased electrification provides a good rationale for the Bank’s support

Evaluator/consultant : Ananda Covindassamy

Peer Reviewer/Task Manager:

2. PROJECT DESCRIPTIONSummary from Appraisal Report including addendum/corrigendum or loan agreement, and taking into account any modification that occurred during the implementation phase.

The purpose of the project was to improve the provision of power supply in terms of quantity and quality through the enhancement of transmission capacity, reduction of system losses and provision of alternative electricity paths.

a. Rationale and expected impacts:Provide a brief and precise description on the project/programme rationale (concerns/questions raised), expected impacts and the intended beneficiaries (directly or indirectly impacted by the project/programme). Highlight any change that occurred during the execution phase.

Rationale:The intervention of the Bank has been requested by the Government of Ethiopia (GOE) to support the country’s efforts to sustain double digit GDP growth projected at 11% per annum for the next decade.Ethiopia’s potential to achieve high economic growth through increased electrification provides a good rationale for the Bank’s support to this project. The GoE attaches high priority to the provision of adequate cost-effective and reliable electricity supply and rapidly scaling up electricity coverage and access to all its citizens. The broad GoE’s goal under GTP are to support a robust economic growth that enables the achievement of the MDGs, and address the basic energy needs of the poor in order to improve their quality of life.

The project expected impacts are: improved availability of reliable and sustainable electricity for economic and social development, improved quality of life, and improved regional integration through reinforcing interconnections with neighboring countries.

b. Objectives/Expected Outcomes:Provide a clear and concise description of the project objectives, expected outcomes, and intended beneficiaries. In so doing, highlight anyrevision/amendment.

Objectives:To enhance socio economic development of Ethiopia by increasing availability and reliability of electricity. To improve the provision of power supply in terms of quantity and quality through the enhancement of transmission capacity, reduction of system losses and provision of alternative electricity paths.

Outcome:Increased availability of reliable and affordable electricity to rural, urban, industrial and regional consumers. from 2.1 million in 2010, to 3.6 million in 2015;Sustained real GDP growth rate in Ethiopia at a minimum of 11% over the medium term. Increased power trade to over. 2100GWh/p/a.

Page 3: African Development Bank - PCR EVALUATION …Ethiopia’s potential to achieve high economic growth through increased electrification provides a good rationale for the Bank’s support

Women’s burden reduced.

c. Outputs and intended beneficiaries:Provide a clear and concise description the expected outputs and intended beneficiaries. In so doing, highlight any revision/amendment.

Expected Outputs:Four 230 kV transmission lines constructed. Eight new Substations constructed. Seven existing substations extended. Addis Ababa Master Plan delivered. Capacity to transfer 300 MW to eastern part of the country, 320 MW to the southern and south western part of the country and 240 MW from the northern part of the country to the national grid All project affected persons compensated and/or relocated.

Outputs were not changed during project implementation and were delivered.

Intended Beneficiaries:Industrial, commercial, agricultural and domestic consumers nationwide. EEPCo.

d. Principal activities/Components:Provide a clear and concise description of the principal activities/components. In so doing, highlight any revision/amendment.

Transmission lines: construction of, on a turnkey basis, the following four 230 kV transmission lines. (i)140 km Metu - Gambela line (ii) 352 km Koka-Hurso transmission line (iii) 141km AlamataMuehuoni-Mekele line (iv) 315 km Wolkite -Hossana–Alaba, GilgelGibe I–Jimma-Agaro-Bedele transmission line

Construction/rehabilitation of sub-stations: This component comprises extension of seven substations at Metu, Bedele, Koka, Alamata, Mekele, Welkite and Gilgel Gibe and Construction of eight new substations at Gambela, Hurso, Awash, Muehuoni, Alaba, Hosahina, Jimma and Agaro. Three 400 kV substations will be reinforced around Addis Ababa at Sululta, Sabatta II and Legetafo and one 400 kV substation at Debre Markos located in the north west of Addis Ababa.

Consultant/supervision: This component includes procurement of the consultancy services for supervision of the construction contracts and for the preparation of distribution master plan for Addis Ababa and Audit service.

Compensation and resettlement: The Project has been categorized in ADB environmental and social category I. This component consists of implementation of mitigation measures and compensation of people affected by the project.

3. PROJECT PERFORMANCE ASSESSMENT

RELEVANCE

a. Relevance of the project development objective: Evaluation of the relevance ex-ante and ex-post (including during the implementation phase). The relevance of the project objective (during the evaluation ex-ante and the post-evaluation) in terms of alignment with country’s development priorities and strategies, the beneficiary needs (including any changes that may have occurred during the implementation), applicable Bank sector strategies, the Bankcountry/ regional strategy, and general strategic priorities of the Bank. This criterion equally assesses the extent to which the project’s development objective was clearly stated and focused on outcomes and the realism of the intended outcomes in the project setting.

Based on the fact that the project development objectives are too broad and only remotely related to

Page 4: African Development Bank - PCR EVALUATION …Ethiopia’s potential to achieve high economic growth through increased electrification provides a good rationale for the Bank’s support

the project, as they are all dependent on numerous other investment and decisions, the relevance of the DO in relation to the project is therefore rated Satisfactory. The PCR gives a rating of Highly Satisfactory considering the DO in general as contributors to the national poverty reduction strategy rather than in relation to the project.

The high level project objective is to enhance socio economic development of Ethiopia by increasing availability and reliability of electricity. Detailed objectives were to improve the provision of power supply in terms of quantity and quality through the enhancement of transmission capacity, reduction ofsystem losses and provision of alternative electricity paths.

The project development objectives are aligned with the Government of Ethiopia (GOE) to support the country’s efforts to sustain double digit GDP growth projected at 11% per annum for the next decade.The GoE attaches high priority to the provision of adequate cost-effective and reliable electricity supply and rapidly scaling up electricity coverage and access to all its citizens. The linkage between the brod national objectives of GoE and the project ia nevertheless rather indirect.

b. Relevance of project design (from approval to completion):The evaluator should provide an assessment of the relevance of the project design regardless of the one provided in the PCR. The evaluator will also comment on the PCR conclusion for this section, and will provide an evaluation of the relevance of the project design. The latter assesses the soundness and the timing of eventual adjustments, or technical solutions to ensure the achievement of the intended results (outcomes and outputs), the adequacy of the risk assessment, environmental and social protection measures, as well as the implementation arrangements. For Programme Based Operations (PBO), an assessment will be made on the relevance of the prior actions, the policy dialogue and the extent to which the operation could have been more pro-poor in its design.

The project design was sound and remained appropriate through-out the implementation to ensure the achievement of intended results (outcomes and outputs). The Bank's experience from implementing previous projects in the energy sector served as the basis for this project design. The risk assessment reflected Bank experience and evaluated fairly the main threats to the project. The relevance of project design is therefore rated Satisfactory, sane as the PCN.

The relevance of project design, cannot be fully assessed, as the documentation does not indicate how the project components tally with the optimized sector development plan: it is only indicated that the selected project components were part of the development plan of EEPCO: it is stated in the Appraisal document that “The implementation of the proposed project is therefore critical in ensuring the evacuation of power from generation sources to load centers and also facilitates the export of energy to Sudan and Djibouti and is thus a crucial part of the transmission expansion plan.”, which is plausible but would be more convincing if there was a reference to an optimized long term transmission development plan prepared by reputable parties to demonstrate that the priority was to build the selected lines and sub-stations as top priorities..

The project was developed through participation of key stakeholders whose involvement continued during implementation. Also, the project activities were fully integrated in the operations of the executing agency (EA). These design features contributed to the successful implementation of the project activities, Awareness campaigns and participatory assessments such as discussions with local leaders, public village meetings and interviews with focus groups were held.

EFFECTIVENESS

Page 5: African Development Bank - PCR EVALUATION …Ethiopia’s potential to achieve high economic growth through increased electrification provides a good rationale for the Bank’s support

c. Effectiveness in delivering outputs :Evaluation of the extent to which the project achieved its stated results (obtained from the logical framework) based on the last Implementation Progress and Results Report (IPR) and by considering accurate reporting of direct or indirect evidence on intended and unanticipated outputs. In the absence of sufficient data (as direct evidence), indirect evidence (such as project outcomes and other pertinent processes/elements of the causal chain) should be used particularly in the evaluation of the extent to which the project is expected to achieve its stated results/ objectives. The absence of sufficient data to assess the effectiveness should be indicated (and clearly detailed in the PCR quality evaluation section). The PCR score should equally be indicated in this section.

The project delivered most of the intended outputs, but some of them were not fully delivered at PCR time. Hence, the rating is Satisfactory. The PCR gives a rating Highly Satisfactory, as it does not take into account the problem that some outputs have not been fully delivered. . The project outputs and achievements are the followingFour 230 kV transmission lines constructed. DeliveredEight new substations constructed. Not fully deliveredSeven existing substations extended. Partly deliveredAddis Ababa Master Plan delivered. Fully deliveredCapacity to transfer 300 MW to eastern part of the country. Fully delivered170 MW to southern and south western. Fully delivered240 MW to Nnorthern part of country. Fully delivered15 MW to South west part of the country (Metu- Gambela) from grid. Fully deliveredAll project affected persons compensated and/or relocated. Fully delivered

d. Effectiveness in delivering outcomes:Evaluation of the extent to which the project achieved its intended set of outcomes (including for Program Based Operations (PBOs) where complementary measures are necessary for their implementation, namely public awareness, policy dialogue and institutional arrangements for instance). The evaluator should make an assessment based on the results of the last project Implementation Progress and Results (IPR). The evaluator shall indicate the degree to which project outcomes (intended and unanticipated) as well as reasons for any eventual gap were discussed in the PCR.

The effectiveness in achieving outcomes is rated Satisfactory because although it can be argued the project contributed to delivering the outcomes (some of which were not achieved), the linkage cannot be documented. The PCR rating is also Satisfactory.

The project outcomes were: “an increased availability of reliable and affordable electricity to rural, urban, industrial and regional consumers” but only three of the outcome indicators for 2015 listed in the Logframe of the Appraisal Report relate to outcomes. The PCR used a different set of performanceindicators, adding GDP growth, power trade, job creation, women employment and deleting power losses, although no indication concerning the justification of the change is given. The present assessment is based on the PAD indicators

Increase in power supply lines: achievedIncrease in number of customer connections from 2.1 million in 2010, to 3.6 million in 2015: not achievedIncreased access to electricity from 41% in 2010 to 75 % in 2015: not achievedReduction of power losses to less than 12%: not achieved.

These indicators except the power supply lines cannot be directly linked to the project and depend upon numerous other factors. They may even have been achieved without the project.

Page 6: African Development Bank - PCR EVALUATION …Ethiopia’s potential to achieve high economic growth through increased electrification provides a good rationale for the Bank’s support

e. Project development outcome: The ratings derived for outcomes and output are combined to assess the progress the project has made towards realizing its development objectives, based on the rating methodology recommended in the Staff Guidance Note on project completion reporting and rating (see IPR Guidance Note for further instruction on development objective rating).

Overall, the rating of the project development outcomes based on outcomes and outputs is rated Satisfactoryas most of the outputs were delivered and there was progress toward achieving, but only partly, the outcomes. The PCR rating is the same.

The achievement of project outputs and outcomes is measured both through the physical implementation of the project, and the impact in terms of outcomes. The project fully achieved its output targets with the construction of all transmission lines, substations and ancillary installation; the study included in the project was also completed. The outcome reporting in the PCR firstly, differ from the indicators in the appraisal report, with no reason given for the change, secondly, they relate toglobal sectoral or macroeconomic indicators, such as sustainable new customers of EEPCO; increase electricity access rate; GDP growth, etc. These indicators of outcome cannot be linked clearly and directly to the project but there was progress toward delivering on these outcomes. It is possible the project contributed to their achievement, but it can be argued that observed changes are not linked to the project. It is likely the project contributed to the observed outcomes, although its contribution cannot be measured. In addition, the reporting of the first four outcomes/outputs (the completion of the lines) may have been misinterpreted, as for each line, the reported outcome is the full theoretical capacity of the line, although it is unlikely they were all utilized 100% (except lot 4).

f. Beneficiaries:Using evidence, the evaluator should provide an assessment of the relevance of the total number of beneficiaries by categories and disaggregated by sex.

The direct beneficiaries of the project cannot be identified. Indirectly, all consumers may benefit from improvement in the transmission system, or only consumers from some regions: the documentation does not permit to segregate consumers and to assert with certainty that consumers and what consumers benefitted from the project. In the absence of an optimized transmission plan, the beneficiaries cannot be identified, except for the number of females employed in project sub-stations.

The main beneficiaries mentioned in the PCR are the clients of the power Utility Ethiopian Electric Utility (EEU); e.g. household, industries, commercial, Agriculture and domestic consumers. The documentation fails to identify if, and how many consumers may have benefitted from the project, and what categories and gender. The increase in the number of consumers depends on numerous other investments in distribution and generation and cannot be attributed to the project alone, particularly considering that there is no indication the infrastructure was selected as part of the sector optimized development plan. The project is likely to have contributed to the changes measures at national level, but for an unknown amount.

However, the proportion of professional staff at sub-stations that are female which is a category which can be clearly linked to the project, have not been fully achieved.

g. Unanticipated additional outcomes (positive or negative, not taken into consideration in the project logical framework) :

Page 7: African Development Bank - PCR EVALUATION …Ethiopia’s potential to achieve high economic growth through increased electrification provides a good rationale for the Bank’s support

This includes gender, climate change, as well as social and socio-economic- related issues. Provide an assessment of the extent to which intended or unanticipated additional and important outcomes have been taken into consideration by the PCR. The assessment should alsolook at the manner the PCR accounted for these outcomes.

The PCR lists the following additional outcomes:“Additional outcomes: Although client tracking survey was not conducted to assess the number of indirect jobs created by the project, the PCR mission observed that the indirect employment opportunities were created in the following categories: factory workers, farm/plantation workers and house help.

The project had a significant positive impact on the environment, as a result of utilization of clean and relatively cheap renewable Hydroelectric and other renewables, which displaced the use of imported oilused for running diesel generators and decreases utilization of traditional households cooking firewood (biomass). The industrial activity in the capital Addis Abba and other regions have increased and created new opportunities for employment and growth, thus improving the income and quality of life inthe Project area.

On the negative side there were seven individuals accidentally electrocuted during the Transmission lines construction period.”All additional outcomes listed are speculative and cannot be evaluated or traced back to the project with reasonable certainty, as it is not possible to assess what would have happened without the project, therefore, should not be considered as Additional Outcomes. The evaluator considers there were no Additional outcomes.

EFFICIENCY

h. Timeliness:The timeliness of project implementation is based on a comparison between the planned and actual period of implementation from the date of effectiveness for first disbursement. For Programme Based Operations (PBOs), the timely release of the tranche(s) are assessed through this same criterion.

Timeliness of the project was Unsatisfactory due to a 16 month delay for a total initial implementation period of 60 months. PCR rating is the same.

The implementation period of the Electricity Transmission Systems Improvement project exceeded theplanned implementation schedule by 16 months. The Project encountered a number of challenges which affected timely implementation of its activities.

The project encountered procurement and implementation delays in the execution of contracts, in particular long project execution, expiry of contractual completion time with the contractors, and finalizing the assessment and decision on claims for extension of time and related costs which were put forward by the contractors. Delay in timely assessment and settlement of compensation/resettlement claims of PAPs has also been a source of the project execution delay.

Lengthy procurement for the civil works subcontractors by the turnkey contractors was another cause for the delays in implementation and a temporary local currency cash flow shortage from the contractors and client counterpart funds contributed to the delays. These delays were also mainly the

Page 8: African Development Bank - PCR EVALUATION …Ethiopia’s potential to achieve high economic growth through increased electrification provides a good rationale for the Bank’s support

results of late mobilization of sub-contractors and a lack of efficiency in planning, coordination and site supervision activities by the main contractors.

i. Resource use efficiency:Provide and assessment of physical implementation (based on outputs delivered) against resources used (based on cumulative commitments) at completion for all contributors to the project (the Bank, Government, and others). This criterion would normally not apply to PBOs, as there is often no direct link between the outputs and the amount of contribution (in which case the rater would indicate N/A).

Overall resources were used efficiently, with significant savings achieved,although some uncertaint remains as some physical works are not completed. The resource use efficiency is therefore rated Satisfactory. The PCR rating was also Stisfactory.

All planned project outputs were delivered with the exception of some civil works in the sub-stations under lot-2 contract. There was a total project savings of UA 7.47 million both from the loan and the Grant. The client requested the utilization of the savings from the Loan to purchase power transformers and battery chargers for use of the project to which the bank gave its no objection. At time of the PCR mission, there was still UA 3.9 undisbursed funds that were to be cancelled. The physical implementation is in line with the resources used.

j. Cost-benefit analysis:Provide an assessment of the timeliness of the development outputs, and the extent to which costs of the costs have been effective and havebeen provided in the most efficient manner. The PCR rating should be discussed. The evaluator should verify whether the benefits of the project (achieved or expected) exceed its actual costs. To achieve this, evidences will mainly be based on a comparison between Economic Rates of Return (ERR) calculated at appraisal, the mid-term review and completion. When commenting PCR ratings, the degree of utilization of valid sources for evidence justifying the rating assigned should be taken into consideration. The evaluator should ensure of the validity of assumptions and that the same model was used for the calculation of others ERRs. For PBOs for which this calculation model does not apply, an assessment could be done with regards to the contribution of policy reforms to economic growth. In the absence of sufficient evidence, an appropriate rating should be assigned.

There is no indication the infrastructure included in the project are part of an optimized development plan, nor a transparent cost benefit analysis. The Cost Benefit analysis is therefore rated Unsatisfactory. The Satisfactory rating given in the PCR is not warranted or justified.

At Appraisal the rate of return of the project was included in the appraisal report. There was no indication how the economic analysis was conducted nor on the basis of the evaluated rate of return. Considering the complexity of calculation of economic benefits of transmission lines, details should have been provided to give comfort that the methodology used was appropriate. The fair evaluation of the benefits at appraisal and at PR stage could not be carried out, nor the validity of the assumptions retained. This weakness would have been identified by Bank management at appraisal stage, particularly considering that the Appraisal report does not give details on the content and validity of the sector long term development plan, nor the origin of the project, except that it was a priority project for EEPCO.

At completion, the PCR mission was unable to find the same model used to calculate the ERR and FIRR at appraisal to quantitatively reassess the validity of cost benefit analysis conducted at appraisal, highlighting the concern with the methodology and quality of the economic and financial evaluations. The Bank’s PCR guidelines recommendation that the same model at appraisal be used for the PCR could not be complied with. For the evaluation of the PCR, no documentation was made available concerning the project economic analysis.

Page 9: African Development Bank - PCR EVALUATION …Ethiopia’s potential to achieve high economic growth through increased electrification provides a good rationale for the Bank’s support

k. Implementation progress:The assessment of the Implementation Progress (IP) on the PCR is derived from the updated IPR and takes into account the all applicable IP criteria assessed under the three categories : i) Compliance with covenants (project covenants, environmental and social safeguards and audit compliance), ii) project systems and procedures (procurement, financial management and monitoring and evaluation), and iii) project execution and financing (disbursement, budget commitments, counterpart funding and co-financing).

Project implementation is Satisfactory on account of fulfillment of most commitment, compliance with procedures and financial execution, allowing for results. The PCR rating is the same.

Project covenants (Project covenants, environmental and social safeguards and audit compliance): All covenants were fulfilled. The Project also complied with auditing requirements and submitted annual audit reports to the Bank during its operation.Project systems and procedures (procurement, financial management and M&E): This PCR mission considered the M&E systems was one of the challenging areas faced by EEP and the Executing Agency during the project implementation (i) lack of systematic collection of data necessary to evaluate the Project’s log frame (ii) Adequate data not collected for some of the major project benefit, beneficiaries and impact indicators as per the specific requirement of the Project log frame. Project execution (disbursement, budget commitments, counterpart funding and co-financing): Sufficient financial resources for the Project came from the Bank and the government of Ethiopia (EEP). However, few challenges occurred in the local currency funding (counter-parts funds, cashflow shortage) from the client and contractors was not provided on a timely basis during the implementation. This has caused delays in the construction works of contractors.

SUSTAINABILITY

l. Financial sustainability:Provide an assessment of the extent to which funding mechanisms and modalities (eg. Tariffs, user fees, maintenance fees, budgetary allocations, other stakeholder contributions, aid flows, etc.) have been put in place to ensure the continued flow of benefits after completion, with particular emphasis on financial sustainability. For PBOs, the assessment should focus on financial sustainability of reforms, as well as the Bank’s policy dialogue to promote financial sustainability of the reforms.

The financial sustainability of the project depends upon the financial performance of EEP. Under the present situation, the financial sustainability og EEP is weak, hence the financial sustainability of the project is rates Unsatisfactory. The PCR rating is Satisfactory, although the financial sustainability of EEP is not established.

The PCR mission assessed the financial sustainability of EEP to the extent of continued flow of benefits after project completion. Their conclusion was that EEP is still not viable financially without Government support. This situation puts in jeopardy the capacity of EEP to maintain adequately the assets built under the project in the medium to long term.

The corporation cannot be financially sustainable anytime soon. The government of Ethiopia realizing the impact of the supply of electricity on the national economy, poverty alleviation in rural areas and access to reliable and affordable electricity, influence adjustments of the tariff price that the

Page 10: African Development Bank - PCR EVALUATION …Ethiopia’s potential to achieve high economic growth through increased electrification provides a good rationale for the Bank’s support

corporation can charge to its clients. Also, the country-wide rural electrification coverage policy puts ahuge burden on EEP operating expenses and capital investment. The initial selling price of electricity was placed at an average equivalent rate to US cents 0.06/kWh. However, due to the devaluation of thecurrency the prevailing rate went down to approximately USD 0.03/kWh. This rate is too low for EEP to cover its operating expenses and improve its long term sustainability.

Although this situation may not be specific to Ethiopia and EEP, the financial sustainability of the project cannot be considered satisfactory, contrary to the conclusion of the PCR.

m. Institutional sustainability and strengthening of capacities:Provide an assessment of the extent to which the project has contributed to the strengthening of institutional capacities – including for instance through the use of country systems – that will continue to facilitate the continued flow of benefits associated with the project. An appreciation should be made with regards to whether or not improved governance practices or improved skills, procedures, incentives, structures, or institutional mechanisms came into effect as a result of the operation. For PBOs, this should include an assessment on the contributions made to building the capacity to lead and manage the policy reform process; the extent to which the political economy of decision making was conducive to reform; the Government’s commitment to reform; and how the design reinforced national ownership.

Considering that on the job training was significant throughout the project and EEP’s technical staff isexperienced, the institutional sustainability of the project is Satisfactory. PCR rating was also Satisfactory.

The Executing Agency for the project was former Ethiopian Electric Power Corporation (EEPCo) and successor EEP which has successfully implemented similar projects with various development institutions. EEP is well staffed and equipped to preserve the investment and sustain asset value of the project. The project contributed to strengthening institutional capacity within EEP including social and environmental safeguard management. They benefited by working with experienced consultants, and most importantly exposure to challenges. Furthermore, the Bank trained the PIU staff on procurement and project management. The existing institutional set-up and staff capabilities would be sufficient to ensure the continued proper maintenance of the completed Electricity Transmission Systems Improvement project, if adequate financial resources were available.

n. Ownership and sustainability of partnerships:Provide an assessment of whether the project has effectively involved relevant stakeholders, promoted a sense of ownership amongst the beneficiaries (both men and women) and put in place effective partnerships with relevant stakeholders (eg. local authorities, civil society organizations, private sector, donors) as required for the continued maintenance of the project outputs. For PBOs, the assessment should measure the extent to which the Government’s capacity to conduct consultations during policy dialogue and the extent to which the Bank supported the Government in deepening the consultation processes.

Despite the observation that dialogue between the Bank and the Government and EEP concerning the utility’s financial performance (and the project long term sustainability) seem not to have produced practical results, ownership of the project by EEP and the Government, and the consultative process with stakeholders and communities is Satisfactory, as rated in the PCR.

The assessment of the PCR team was that: ”All stakeholders were involved to varying degree in the implementation of the project. The project was also developed through an extensive participation, and consultations were performed during project preparation and conceptualization mainly through

Page 11: African Development Bank - PCR EVALUATION …Ethiopia’s potential to achieve high economic growth through increased electrification provides a good rationale for the Bank’s support

Environmental and Social Impact Assessment (ESIA) and Resettlement Action Plan (RAP). Local participation was quite visible within the works supervision.

The project offered an opportunity for training staff and interns from universities and technical colleges transfer of expertise in electricity transmission construction & installation technologies to localpersonnel working with the contractors and EEP’s project staff.” This assessment is correct, although consultation was mainly in the socio-environmental area rather than technical. However, as the origin of the project is directly and entirely with EEP who selected the lines and sub-stations to be built, one can safely assume that extensive consultation occurred between the Bank team and EEP.

o. Environmental and social sustainability:Provide an assessment of the objectivity of the PCR rating on the project’s implementation of environmental and social mitigation/enhancement measures with regard to the Environmental and Social Management Plan (ESMP), the capacity of country institutions and systems, as well as the availability of funding to ensure the environmental and social sustainability of the operation. This criterion would normally only apply to Environmental Category I and II projects.

Special attention was put on minimizing the environmental impact of the project and handling carefully compensation of PAP. The Environmental and social sustainability is correctly rated as Satisfactory in the PCR.

The Ethiopia Electricity Transmission System Improvement Project has been classified as an Environmental Category-1, in accordance with the Bank’s Environment and Social Assessment Procedures (ESAP) and the Involuntary Resettlement Policy. This category is based on the voltage level of the lines, which is 230 kV exceeding the Bank’s threshold of 110 kV and spanning on aggregate over 900 km, and has potential impact of displacing large population in various settlements.

The key environmental issues associated with the project included adverse impact on large predatory birds which are mostly affected by high voltage transmission lines. The Bank team has made public consultations with the affected community households and elders. All participants have commonly uttered their positive attitudes and good impressions on the project. Along the transmission line route, there were no environmentally protected areas to be affected. The number of Project Affected Persons (PAP) increased during project implementation, due to the delay in project implementation which interfered with the crop season. All PAPs were compensated adequately.

Approved Environmental & Social Management Plan (ESMP) for all lots incorporated in all contract agreement documents and implementation was monitored by the supervision consultant and representatives. Compensation & RAP activities had been implemented as per the approved plan.

4. PERFORMANCE OF STAKEHOLDERS

a. Bank performance:(Preparation/approval, ensure of Quality at Entry (QAE) : quality of the supervision, completion) : Provide observations on the objectivity of the PCR ratings and feedback provided by the Borrower, and if necessary, re-assess the Bank’s performance throughout theproject cycle (design, implementation, completion) by focusing on evidence from the PCR in relation to 7 criteria defined in the PCR

Page 12: African Development Bank - PCR EVALUATION …Ethiopia’s potential to achieve high economic growth through increased electrification provides a good rationale for the Bank’s support

Guidance Note.

Balancing the weak project preparation with the satisfactory supervision, the overall assessment of the Bank performance is Satisfactory, as rated in the PCR.

The frequency of the Bank supervision of the project activities was adequate. However, according to the appraisal report a Mid-Term Review of the project was to be held not later than 18 months after the loan approval. The Mid-Term Review was to inform any adjustments to the project design to ensure that project objectives are achieved.

At the same time, the Supervision consultants was required to prepare and submit to EEP and to the Bank final commissioning reports at the completion of their assignment. The PCR mission correctly observed that, there was no Mid-term review carried out by the Bank nor was there a supervision consultant report (due to termination of the consultancy services contract before projects completion period); that would have served as an input in the preparation of the Bank’s PCR.

The PCR did not assess the quality of project preparation by the Bank team. The Project Appraisal Document presented two weaknesses: (1) it failed to indicate the origin of the project and detail preparatory work which would have led to the selection of the project components; it merely indicates that the project scope was decided by EEPCO as part of their investment plan; and (2) the Appraisal report did not detail the approach and methodology adopted for the preparation of the economic and financial analyses; this, combined with the inability of the PCR team to obtain the models underpinning the stated economic and financial returns constitute a major weakness in Bank project preparation work.

b. Borrower performance: Provide observations on the objectivity of the PCR ratings, and if necessary, re-assess the Borrower’s performance throughout the projectcycle (design, implementation, completion) by focusing on evidence from the PCR in relation to questions defined in the PCR Guidance Note.

The PCR correctly rated the Borrower’s performance as satisfactory.

Government: The Government performance was satisfactory. However, there was a delay in the civil works under the Lot-2 sub-station contract which was caused mainly by the re-scoping of the civil works at substation, that was transferred from the contractor to EEP. The escalation of the compensation to the farmers was another factor (as the sites were cash crop areas). This demanded an injection of extra local currency to complete the remaining civil works that was resolved in the process with the consultation of the Bank.

Project Implementing Agency: Notwithstanding, the delays caused by less than fully satisfactory performance of some contractors and consultant (engineer), the Executing Agency ensured good quality of the project physical outputs and implemented most of the activities planned.. However, there was a lack of awareness and capacity for the inclusion of gender mainstreaming in project design and implementation, as well as shortage of trained staff in the area of environment and social safe guards.

Page 13: African Development Bank - PCR EVALUATION …Ethiopia’s potential to achieve high economic growth through increased electrification provides a good rationale for the Bank’s support

EEP staff have extensive experience in the implementation of donor projects including the Bank Group.

c. Performance of other stakeholders: Provide observations on the objectivity of the PCR ratings, and if necessary, re-assess the other shareholders’ performance throughout theproject cycle (design, implementation, completion) by focusing on evidence from the PCR in relation to relevant questions specific to each stakeholder (co-financiers, NGO, contractors and service providers).

Considering that the project was completed 16 months behind schedule, largely due to weaknesses in the performance of some contractors, the performance of other stakeholders is considered Unsatisfactory. The Satisfactory rating granted in the PCR does not take sufficiently into account the impact of delay and poor performance of some of the contractors.

Significant delays by some of the contractors were experienced in the implementation of some components of the project, the contractor of lot-3 (ABB), completed the rehabilitation and expansion ofexisting sub-stations in Addis Area on time and transmission lines contractors Lot-1 (KEC Int.), Lot-2 (EnergoInvest) and Lot3 (Kalpataru) completed within granted extension of time by the client . On the other hand long delays were experienced in the sub-station construction handled by CYMI (Lot-1) & ALSTOM (Lot-2) and transmission line Lot-4 (SAE-Gammon). The Government of Ethiopia/EEP werenot satisfied with the performance of these contractors. The performance of the Consultant was also less than fully satisfactory.

5. SUMMARY OF OVERALL PROJECT PERFORMANCE

a. Overall assessment: Provide a summary of the project/programme’s overall performance based on the PCR 4 key components (Relevance, Effectiveness, Efficiency and Sustainability). Any difference with the PCR and the reasons that have resulted in them should be mentioned. For caseswith insufficient evidence (from the PCR and other documents) available, the evaluator should assign a partly satisfactory rating (to berevised) until a post project performance evaluation (e.g. PPER, PER or PRA) is complete.

Due to the uncertainty concerning the justification of the selected infrastructure, the weak sustainability and implementation delay, but on the other hand, nearly full completion below budget, the project overall rating is Satisfactory. The project performance concerning its relevance is rated Highly Satisfactory in the PCR, but does not sufficiently take into account the implementation delay, weak economic and financial evaluation, and poor sustainability.

In terms of Effectiveness, the project is rated Satisfactory, as per the PCR.Concerning Efficiency, the rating of the project is Unsatisfactory, because of a less than satisfactory performance in Timeliness (16 months delay). But the economic justification is estimated Unsatisfactory, whereas the PCR finds it Satisfactory. The conclusion is that in terms of Efficiency, the project rating should be slightly below the PCR rated level, as Unsatisfactory.The project sustainability is a weak point, because of the poor financial performance of the utility and the lack of clear prospects for improvement. The assessment of the PCR of a sustainability above Satisfactory is therefore to be challenges. Sustainability in rated Unsatisfactory

Overall, due to the weaknesses in quality at entry, delay in implementation and uncertain sustainability, despite a good control of resources and satisfactory completion of the project, the overall

Page 14: African Development Bank - PCR EVALUATION …Ethiopia’s potential to achieve high economic growth through increased electrification provides a good rationale for the Bank’s support

project rating is Unsatisfactory, whereas the PCR’s overall rating is above Satisfactory.

b. Design, implementation and utilization of the M&E (appreciation of the evaluator):Provide an assessment of planned and actual cost of the design, implementation and utilization of the M&E system. Design : To which extent the project M&E system was explicit, adequate and realistic to generate and analyse relevant data ; Implementation : To which extent relevant data was collected – Elements of M&E implementation and effectiveness in the PCR ; Utilization : degree of utilization of data generated for decision-making and resource allocation – elements of M&E utilization in the PCR.

The M&E system is flagged as a weak aspect of the project. However, as the project audits found the project financial management satisfactory, the rating of the M&E system is Satisfactory.

6. EVALUATION OF KEY LESSONS LEARNED AND RECOMMENDATIONS

a. Lessons learned: Provide a brief description of any agreement/disagreement with all or part of the lessons learned from the PCR after analysis of the project performance with regards to each of the key components of the evaluation (Relevance, Effectiveness, Efficiency, and Sustainability). List the PCR main new and/or reformulated pertinent (and generic) lessons learned for each of these components here. It is recommended that no more than five lessons learned are discussed. Key questions and targeted audience must also be specified for eachlesson learned.

The Lessons Learned mentioned in the PCR are:

1. Design changes introduced during implementation resulted in additional work in some of the lots (i.e Lot-1 transmission line contract). These additional works attracted the contractors claims for time extension and time related costs. It is recommended that in the future, EEP/Bank subjects all projects design and tender documents to technical audit internally and by Independent consultants before procurement of works for any deficiencies in the design and tender documentation preparation.

2. The PIU need to take early actions to evaluate contractor’s preparedness and understanding of work volume and have debriefing session on the work schedule.

3. Full integration of project activities in the operations of the executing agency and through involvement of the senior management of the executing agency in the design and implementation of project activities are essential for the timely and successful completion of project activities.

4. Proper baseline studies should be undertaken before, during, and after the completion of a project to enable proper documentation of a project’s impact on the environment

5. Strengthen the link of implementing agencies with the Federal Integrated Infrastructure Development Coordinating Agency (FIIDCA) to reduce the right of way (ROW) issue

6. Close follow-up and periodical review of the performance of project consultant (manager) is required to take timely corrective action as per the provision of the consultancy contract agreement.

Page 15: African Development Bank - PCR EVALUATION …Ethiopia’s potential to achieve high economic growth through increased electrification provides a good rationale for the Bank’s support

The six lessons learned listed in the Lessons Learned of the PCR are sound and can contribute to improve the quality of future projects. A lesson concerning Sustainability and the need to require that the utility is, or will become shortly financially sustainable should be added.

b. Recommendations: Provide a brief description of any agreement/ disagreement with all or part of the recommendations from the PCR. List the PCR main new and/or reformulated recommendations (requiring more actions by the Borrower and/or the Bank) here.

1. The future rehabilitation work should be done after intensive risk assessment and stakeholder’s analysis study to avoid complex problems during design and construction periods.

2. AfDB’s future interventions should ensure that borrowers disaggregate bank project

information and data collection from their other businesses, this will enable the PCR evaluators to assess the actual performance of the project against results based Log-frame at appraisal.

3. The EA should plan periodical capacity building program for the operation & maintenance personnel and Bank’s assistance is also vital to implement the designed institutional capacity building plan.

The recommendations of the PCR are well taken and adequately cover the technical aspects. However, the recommendations are expected to cover sustainability. In this respect, the PCR is mute. Recommendations should be provided concerning how to deal with financially unsustainable utilities and to protect Bank financed assets. Linkage of investment operations with policy based lending operations, creating Special Purpose Vehicles with ring-fenced revenues for Bank financed assets should be suggested.

7. COMMENTS ON PCR QUALITY AND TIMELINESSThe overall PCR rating is based on all or part of the criteria presented in the annexe and other: The quality of the PCR is rated as highly satisfactory (4), satisfactory (3), unsatisfactory (2), and highly unsatisfactory (1). The timeliness of the PCR is rated as on time (4) or late (1). The participation of the Borrower, co-financier, and the bank’s external office(s) are rated as follows: Very Good (4), Good (3), Fair (2), Poor (1).

The quality of the PCR is rated Satisfactory.

The quality of the PCR and its timing are adequate (dated November 2017 with a closing date of April 2017). The positive aspects are the coverage of the technical and implementation aspects of the project.The weaker points are the handling of Development objectives/Outcomes/Outputs, the change in the metric for evaluating outputs/outcome, and the cursory review of the economic and financial analysis ofthe project.

8. SUMMARY OF THE EVALUATIONThis is a summary of both the PCR and IDEV ratings with justification for deviations/comments. Appropriate section of the PCR Evaluation should be indicated in the last column in order to avoid detailed comments. The evaluator must provide a reasonable explanation for each criterion the PCR rating is not validated by IDEV. Consequently, the overall rating of the project could be “equally satisfactory”.

Criteria PCR PCREN Reason for disagreement/ Comments

RELEVANCE 4 3 Some details in project justification are

Page 16: African Development Bank - PCR EVALUATION …Ethiopia’s potential to achieve high economic growth through increased electrification provides a good rationale for the Bank’s support

missing making it difficult to be fully convinced of the relevance of all project components

Relevance of project development objective 4 3 DO are very broad and make it impossible to evaluate the contribution of the project

Relevance of project design 3 3

EFFECTIVENESS 3 3

Development objective (DO) 3 3

EFFICIENCY 2.75 2

Timeliness 2 2 Delay was kept within reasonable limits considering the size and nature of the project

Resource use efficiency 3 3

Cost-benefit analysis 3 2 There was no indication concerning the methodology and identification of benefits

Implementation progress (IP) 3 3

SUSTAINABILITY 3.25 2 Poor financial performance casts a doubt on project overall sustainability in the medium term.

Financial sustainability 3 2 The utility is not financially sustainable, andprospects for improvement are unclear

Institutional sustainability and strengthening of capacities

3 3

Environmental and social sustainability 4 3 Despite a good overall socio environmental performance, there were some problems with the compensation of PAPs

OVERALL PROJECT COMPLETION RATING

3.2 2 Sustainability and quality at entry remain concerns and implementation delay was substantial.

Bank performance: 3 2 Quality at entry was less than satisfactory due to lack of explanation concerning the origin of the project and its economic/financial justification. On the other hand, performance during implementation was satisfactory

Borrower performance: 3 3

Performance of other shareholders: 3 2 The performance of some contractors and Consultants was less than Satisfactory

Overall PCR quality: 3

Page 17: African Development Bank - PCR EVALUATION …Ethiopia’s potential to achieve high economic growth through increased electrification provides a good rationale for the Bank’s support

9. PRIORITY FOR FUTURE EVALUATIVE WORK: PROJECT FOR PERFORMANCE EVALUTION REPORT, IMPACT EVALUTION, COUNTRY/SECTOR REVIEWS OR THEMATIC EVALUATION STUDIES:

- Project is part of a series and suitable for cluster evaluation

- Project is a success story

- High priority for impact evaluation

- Performance evaluation is required to sector/country review

- High priority for thematic or special evaluation studies (Country)

- PPER is required because of incomplete validation rating

Major areas of focus for future evaluation work:

a) Performance evaluation is required for sector/ country review

b) Cluster evaluation (institutional support)

c) Sector evaluation (budgetary support or public finance management reforms)

Follow up action by IDEV: Identify same cluster or sector operations; organize appropriate work or consultation mission to facilitate a), b) and/or c).

Division Manager clearance Director signing off

Data source for validation: Task Manager/ Responsible bank staff interviewed/contacted (in person, by telephone or

email) Documents/ Database reports

Attachment:

PCR evaluation note validation sheet of performance ratings

List of references

Page 18: African Development Bank - PCR EVALUATION …Ethiopia’s potential to achieve high economic growth through increased electrification provides a good rationale for the Bank’s support

Appendice 1

PROJECT COMPLETION REPORT EVALUATION NOTE Validation of PCR performance ratings

PCR rating scale:

Score Description4 Very Good – Fully achieved with no shortcomings3 Good – Mostly achieved despite a few shortcomings2 Fair – Partially achieved. Shortcomings and achievements are roughly balanced1 Poor – very limited achievement with extensive shortcomings

UTS Unable to score/rateNA Non Applicable

Criteria Sub-criteriaPCRworkscore

IDEVreview

Reasons for deviation/comments

RELEVANCE Relevance of the projectdevelopment objective (DO) during implementation

4 3 Difficult to assess without details on project origin and demonstration the project is part of the least cosetransmission development plan.

Relevance of project design (from approval tocompletion)

3 3

OVERALL RELEVANCE SCORE 3.5 3 Some details in project preparation are missing making it difficult to be fully convinced of the relevance of all project components

EFFECTIVENESS* Effectiveness in delivering outcomes. Provide the detail of each outcome, and its actual versus expected rate of achievement. Fill in PCR rating for each and discuss /justify Outcome1 Increase in power supply lines: achieved

4 Outcome achieved

Outcome2 Increase in number of customer connections from 2.1 million in 2010, to 3.6 million in 2015

2 Outcome not achieves

Outcome3 Increased access to electricity from 41% in 2010 to 75 % in 2015

2 Outcome not achieved

Outcome4 Reduction of power losses to less than12%

2 Outcome not achieved

Effectiveness in delivering output. Provide the detail of each output, and its actual versus expected rate of achievement. Fill in PCR rating for each and discuss /justify

Page 19: African Development Bank - PCR EVALUATION …Ethiopia’s potential to achieve high economic growth through increased electrification provides a good rationale for the Bank’s support

Criteria Sub-criteriaPCRworkscore

IDEVreview

Reasons for deviation/comments

Output1 Four 230 kV transmission lines constructed

4

Output2 Eight new substations constructed

3 Partly delivered

Output3 Seven existing substations extended

2 Partly delivered

Output4 Addis Ababa Master Plan delivered

4

Output5 Capacity to transfer 300 MW to eastern part of the country

4

Output6 170 MW to southern and south western

4

Output7 240 MW to Nnorthern part of country

4

Output8 15 MW to South west part of the country (Metu- Gambela) from grid

4

Output9 All project affected persons compensated and/or relocated

4

Development objective (DO)

Development objective rating

3 3

Beneficiaries

Beneficiary1 All consumers

n/a Direct beneficiaries cannot be identifies, nor the amount of benefits. Asserting that all Ethiopian consumers benefit is an unsubstantiated assumption.

Beneficiary2 Women employed in substations

2 Number below target

Unanticipated outcomes (positive or negative not considered in the project logical framework) and their level of impact on the project (high, moderate, low)Institutional development

3

Gender 2 Number of women employed below target

Environment & climate change

4 3 Despite a good overall socio environmental performance, there were some problems with the compensation of PAPs

Poverty reduction 3

Private sector development

n/a

Page 20: African Development Bank - PCR EVALUATION …Ethiopia’s potential to achieve high economic growth through increased electrification provides a good rationale for the Bank’s support

Criteria Sub-criteriaPCRworkscore

IDEVreview

Reasons for deviation/comments

Regional integration 4

Other (specify)

EFFECTIVENESS OVERALL SCORE 3 3

EFFICIENCY Timeliness (based on theinitial closing date)

2 2 Delay was kept within reasonable limits considering the size and nature of the project

Resource used efficiency

3 3

Cost-benefit analysis 3 2 There was no indication concerning the methodologyand identification of benefits

Implementation progress (from the IPR)

3 3

Other (specify)

OVERALL EFFICIENCY SCORE 2.75 2 Quality of cost benefit analysis and financial analysis low and delat in implementation

SUSTAINABILITYFinancial sustainability

3 2 The utility is not financially sustainable, and prospects for improvement are unclear

Institutional sustainability and strengthening of capacities

3 3

Ownership and sustainability of partnerships

3

Environmental and social sustainability

4 3 Despite a good overall socio environmental performance, there were some problems with the compensation of PAPs

*The rating of the effectiveness component is obtained from the development objective (DO) rating in the latest IPR of the project (see Guidance Note on the IPR). The ratings for outputs and outcomes are determined based on the project’s progress towards realizing its targets, and the overall development objective of the project (DO) is obtained by combining the ratings obtained for outputs and outcomes following the method defined in the IPR Guidance Note. The following method is applied: Highly satisfactory (4), Satisfactory (3), Unsatisfactory (2) and Highly unsatisfactory (1).

Criteria Sub-criteriaPCRWorkscore

IDEVreview

Reasons for deviation/comments

BANK PERFORMANCE

Proactive identification and resolution of problems at differentstage of the project cycle

3 2 Initial problems in design and implementation delay justify Unsatisfactory rating

Use of previous lessons learned from previous operations during design and implementation

2 Weakness in quality at entry

Promotion of stakeholder participation to strengthen ownership

3

Enforcement of safeguard and fiduciary requirements

4

Page 21: African Development Bank - PCR EVALUATION …Ethiopia’s potential to achieve high economic growth through increased electrification provides a good rationale for the Bank’s support

Design and implementation of Monitoring & Evaluation system

3

Quality of Bank supervision (mixof skills in supervisory teams, etc)

3

Timeliness of responses to requests

3

OVERALL BANK PERFORMANCE SCORE 3 2 Quality at entry was less than satisfactory due to lack of explanation concerning the origin of the project and its economic/financial justification. On the other hand, performance during implementation was satisfactory

BORROWER PERFORMANCE

Quality of preparation and implementation

3

Compliance with covenants, agreements and safeguards

3

Provision of timely counterpart funding

3

Responsiveness to supervision recommendations

3

Measures taken to establish basis for project sustainability

2

Timeliness of preparing requests 3

OVERALL BORROWER PERFORMANCE SCORE 3 3

PERFORMANCE OF OTHER STAKEHOLDERS

Timeliness of disbursements by co-financiers

3

Functioning of collaborative agreements

3

Quality of policy dialogue with co-financiers (for PBOs only)

3

Quality of work by service providers

2

Responsiveness to client demands 3

OVERALL PERFORMANCE OF OTHER STAKEHOLDERS

3 2 The perform ace of some contractors and Consultants was less than Satisfactory

The overall rating is given: Very Good, Good, Fair and Poor.

(i) Very Good (HS) : 4(ii) Good (H) : 3(iii) Fair (US) : 2(iv) Poor (HUS): 1

DESIGN, IMPLEMENTAION AND UTILIZATION OF MONITIRING ANDEVALUATION (M&E)

Criteria Sub-criteriaIDEVScore

Comments

M&E DESIGN M&E system is in place, clear, appropriate and realistic

3 Physical M&E was satisfactory, but M&E for monitoring achievement of DO, Outcomes was weak

Page 22: African Development Bank - PCR EVALUATION …Ethiopia’s potential to achieve high economic growth through increased electrification provides a good rationale for the Bank’s support

Criteria Sub-criteriaIDEVScore

Comments

Monitoring indicators and monitoring plan were duly approved

3

Existence of disaggregated gender indicator

2 Only for contractors

Baseline data were available or collected during the design

3

Other, specify

OVERALL M&E DESIGN SCORE 3M&E IMPLEMENTA-TION

The M&E function is adequately equipped and staffed

3

OVERALL M&E IMPLEMENTATION SCORE 3M&E UTILIZATION

The borrower used the tracking information for decision

3

OVERALL M&E UTILIZATION SCORE 3OVERALL M&E PERFORMANCE SCORE

3

PCR QUALITY EVALUATION

CriteriaPCR-EVN

(1-4)Comments

Page 23: African Development Bank - PCR EVALUATION …Ethiopia’s potential to achieve high economic growth through increased electrification provides a good rationale for the Bank’s support

QUALITY OF PCR

1. Extent of quality and completeness of the PCR evidence and analysis to substantiate the ratings of the various sections

3

2. Extent of objectivity of PCR assessment score 3

3. Extent of internal consistency of PCR assessment ratings; inaccuracies; inconsistencies; (in various sections; between text and ratings; consistency of overall rating with individual component ratings)

3

4. Extent of identification and assessment of key factors (internal and exogenous) and unintended effects (positive or negative) affecting design and implementation

3

5. Adequacy of treatment of safeguards, fiduciary issues, and alignment and harmonization

3

6. Extent of soundness of data generating and analysis process (including rates of returns) in support of PCR assessment

2

7. Overall adequacy of the accessible evidence (fromPCR including annexure and other data provided)

3

8. Extent to which lessons learned (and recommendations) are clear and based on the PCR assessment (evidence & analysis)

3

9. Extent of overall clarity and completeness of the PCR

3

Other (specify)

PCR QUALITY SCORE 3

PCR compliance with guidelines (PCR/OM ; IDEV)

1. PCR Timeliness (On time = 4; Late= 1) 3

2. Extent of participation of borrower, Co-financiers & field offices in PCR preparation

3

3. Other aspect(s) (specify)

PCR COMPLIANCE SCORE 3

*** rated as Very Good (4), or Good (3), or Fair (2), or Poor (1)

Page 24: African Development Bank - PCR EVALUATION …Ethiopia’s potential to achieve high economic growth through increased electrification provides a good rationale for the Bank’s support

References

Electricity Transmission System Improvement Project Project Quarterly Status Report October to December 2012

ELECTRICITY TRANSMISSION SYSTEM IMPROVEMENT PROJECT (ETSIP) PROJECT COMPLETION REPORT December /2017

BTOR Supervision mission May 2014

BTOR Supervision mission April 2015

BTOR SREP mission May 2017

BTOR Supervision mission April 2016

Environmental and Social Supervision Mission_Audits

Ethiopia - ETSIP - 2100150023451 - 2100155019269 - Audit Report 2016 - Memo FM

Ethiopia - ETSIP - 2100150023451 -2100155019269 - Audit Report 2017 - Memo SNFI2

Ethiopia Electricity Transmission System Improvement draft final-rev04 PR comment

Ethiopia Electricity Transmission System Improvement draft final-rev05 PR comments

Ethiopia_-_AR_Electricity_Transmission_System_Improvement_Project__19_11_2010

ETSI Project _Grant Management Letter_ Audit 2013-14

ETSI Project_Loan_Audit Report 2013-14

ETSI Project_Loan_Compliance Report_2013-14 Audit

ETSI Project_Loan_Management Letter_2013-14 Audit

ETSI_ORPF.2 Memo_On the audit report of 2013-2014 Fiscal Year

ETSIP QPR March 2016

ETSIP-Monthly Project Report JUNE 2013- 2013 REV 00 (AK)Rev PPC

PCN Ethiopia_PR10764

Quarterly (2nd quarter of 2015) report of Electric transmission System Improvement project

Page 25: African Development Bank - PCR EVALUATION …Ethiopia’s potential to achieve high economic growth through increased electrification provides a good rationale for the Bank’s support

RDGE_ETHIOPIA_ETSIP_BPPS_EN

RDGE_PCR ETHIOPIA_ETSIP_BPPS_EN

Supervision ratings of ETSIP from SAP


Recommended