Operation Performance Evaluation Review

Road Sector Reform 1 and 2

Russia(A technical cooperation operation)

January 2007

ab0cdEvaluation Department (EvD)

RUSSIA ROAD SECTOR REFORM 1 AND 2 (TC) OPERATION PERFORMANCE EVALUATION REVIEW (OPER)

PREFACE

The subject of this OPER is the technical cooperation (TC) operation: Russian Federation – Road Sector Reform 1 and 2 in the committed amounts of EUR TC1 (47,895), TC2 (38,901), TC3 (877,060), TC6 (129,900), TC7 (120,000), and TC8 (198,200) funded respectively through the TC1 (Swedish government), TC2 (Danish government), TC3, TC6, TC7 and TC8 (EU-EBRD Investment Preparation Facility). The Operation Leaders (OLs) of this TC were successively TC1 (Zhanar Rymzhanova), TC2- TC8 (Elena Gordeeva) of the Transport team. They carried out their work with the support of consulting firms. The Operation team and other relevant Bank staff commented on an early draft. The Basic Data Sheet on page iii of this report is complementary to this OPER and designed to be read together. The evaluation was carried out by Nicolas Mathieu, Senior Economist of the Evaluation Department, assisted by Mr. James Dent, Consultant. Information on the operation was obtained from relevant teams and departments of the Bank and its files as well as from external sector and industry sources. Fieldwork was carried out in November 2007. Post-evaluation selection and process Selection of an operation for post-evaluation by EvD uses the following criteria: relevance to the Bank's likely future operations; lessons-learned potential; size of the Bank's investment commitment/exposure; balance among countries of operation; balance among sectors and types of operations; relative priority of investment operation OPERs within EvD's overall work programme priorities and resources. The Bank's post-evaluation process is described in Chapter 8 of the Operations Manual. The responsible Operation Leader first writes a TC SCR/SPR. The SCR/SPR report serves a self-evaluation function and establishes the basic facts and lessons from the operation's implementation outcome and future prospects. EvD’s independent evaluation follows, using the SCR/SPR as one of several inputs.

RUSSIA ROAD SECTOR REFORM 1 AND 2 (TC) TECHNICAL COOPERATION PROGRAMME

TABLE OF CONTENTS

Page PREFACE ABBREVIATIONS AND DEFINED TERMS ii BASIC DATA SHEET iii 1. INTRODUCTION 1 2. PROJECT RATIONALE 2 2.1 Road management and safety 2 2.2 Road finance 3 3. PROJECT OBJECTIVES 3 3.1 Road management and safety 3 3.2 Road financing 4 4. ACHIEVEMENT OF OBJECTIVES 4 4.1 Road management and safety 4 4.2 Road financing 5 5. OVERALL ASSESSMENT 6 6. TRANSITION IMPACT AND RISK 8 7. ADDITIONALITY 11 8 BANK PERFORMANCE 11 9. KEY ISSUES AND LESSONS LEARNED 11 9.1 TC project design and implementation 12 9.2 Sector policy dialogue 12 LIST OF APPENDICES

Appendix 1 Operations Performance Ratings Appendix 2 Transition Impact Analysis

RUSSIA ROAD SECTOR REFORM 1 AND 2 (TC) TECHNICAL COOPERATION PROGRAMME

ii

ABBREVIATIONS

EvD Evaluation Department FGup Federal Road Management Agencies IFI International Financial Institution OPER Operation Performance Evaluation Review PCR Project Completion Report PPP Public-private partnership Rosavtodor Russian State Road Administration SMEs Small and medium-sized enterprises TC Technical cooperation TOR Terms of Reference US$ United States dollar XMR Expanded Monitoring Report

DEFINED TERMS

the Bank European Bank for Reconstruction and Development the Client State Road Administration of the Ministry of Transport of the

Russian Federation, or Rosavtodor the OPER Team Staff of the Project Evaluation Department and the

independent sector consultant who jointly carried out the post-evaluation.

the Operation Team The staff in the Banking Department and other respective

departments within the Bank responsible for the Operation appraisal, negotiation and monitoring, including the PCR.

the Project The overall objectives of the TC components evaluated were

to assist the State Road Administration in developing institutional and policy reforms in the road sector.

RUSSIA ROAD SECTOR REFORM 1 AND 2 (TC) OPERATION PERFORMANCE REVIEW

BASIC DATA SHEET

Operation Codes TC1 25561

TC2 25562 TC3 25563 TC6 30099 TC7 33592 TC8 26963

Location: Russia Operation: Various aspects of road sector reform Sector: Transport Type: Technical cooperation Facilitators: Sweden, Denmark, EU-EBRD Investment Preparation Facility, Canada Bank Unit: Transport A. Funding TC TC commitment Commitment number. Commitment title Total disbursed

(EUR)1

TC1 SWEF-2002-01-02 Assistance with best practice recommendations in the area of road sector financing

47,895

TC2 DEN-2002-01-01 Scoping of an assistance programme in the area of road safety

38,901

TC3 EIPF01-2002-11-02 Russian Road Sector Reform 1 – quality management

877,060

TC6 EIPF01-2004-12-07 Russia Road Sector Reform 2 – road sector financing in Russia – long-term strategy

129,900

TC7 EIPF01-2004-12-09 Russia Roads 2: federal road management restructuring plan

95,896

TC8 EIPF01-2004-12-08 Russian Roads 2: preparation of a PPP strategy for roads

198,200

1 Amounts disbursed up to 6 March 2007. Data taken from Data Warehouse/Cognos.

iii

RUSSIA ROAD SECTOR REFORM 1 AND 2 (TC) OPERATION PERFORMANCE REVIEW

B. Procurement Mode Sources by country Consultant services TC1 Direct selection Sweden TC2 Direct selection Denmark TC3 Evaluation of proposals Sweden TC6 Selection from shortlist Russia TC7 Selection from shortlist United Kingdom TC8 Selection from shortlist United Kingdom C. Visits Type of visit No. of visits Person-days EvD/OPER 1 8

iv

OPERATION PERFORMANCE EVALUATION REVIEW RUSSIA ROAD SECTOR REFORM 1 AND 2 (TC)

1. INTRODUCTION Since 2000, the Bank has assisted the Russian State Road Administration, the “Federal Road Agency” or “Rosavtodor”, in carrying forward reforms in the road sector through six technical cooperation projects (TCs).1 The first three were associated with a loan of US$ 229 million to Russia, approved in July 2002 for road construction in St Petersburg and Russia’s Far East. The other three were under a second loan approved in June 2003 amounting to US$ 290 million for additional road construction and further assistance to reforms in road financing and management. Two other TCs (TC4 and TC5) handled the supervision of the road construction. This Operation Performance Evaluation Review (OPER) concentrates on the six TCs aimed at policy reforms and institution building.2 They are listed below in chronological order: Under the Russia Road Sector Reform 1 project:

1. Assistance with best practice recommendations in the area of road sector financing; started in January 2002 and completed in December of the same year (TC1).

2. Scoping of an assistance programme in the area of road safety; also started in January 2002 and completed in May of the same year (TC2).

3. Quality management; contract signed in November 2002 and completed in December 2005 (TC3).

Under the Russia Road Sector Reform 2 project:

1. Road sector financing in Russia – long-term strategy; contracted in December 2004 and completed October 2005 (TC6).

2. Federal road management restructuring plan; contracted in December 2004 and completed in December 2005 (TC7).

3. Preparation of a PPP strategy for roads; also contracted in December 2004 and completed in December 2005 (TC8).

Among the six components of the technical cooperation, three of them cover safety, quality management and organisation restructuring (TC2, 3 and 7). The three others relate to road maintenance and construction financing (TC1, 6 and 8). They are grouped accordingly in the evaluation.

1 The Federal Road Agency (Rosavtodor) was created in accordance with Decree No. 314 of the Russian Federation President dated 9 March 2004 “On the System and Structure of the Federal Executive Bodies”. 2 Two TC components, labelled TC4 and TC5 for reference, provided in 2003 assistance to the supervision of the construction of road sections under the projects. They deal with the technical and engineering aspects of road construction. They are not included in this evaluation as they are associated with the construction performance of the roads.

Page 2 of 13 OPER: Russia Road Sector Reform 1 and 2 (TC)

2. PROJECT RATIONALE 2.1 Road management and safety Road safety (TC2). The federal road network in Russia has a length of approximately 800,000 km. Since the break-up of the Soviet Union, traffic levels have grown substantially, as has the number and rate of road traffic accidents. Rosavtodor was concerned about this trend in accidents and approached the EBRD for assistance in addressing the problem. In 2002 Rosavtodor wished to engage a consultant to assist it in reviewing the existing situation and any ongoing initiatives in this area, and to recommend an outline road safety plan that the state administration could implement. Road management (TC3). Rosavtodor is responsible for administration and management of the federal road network. The network is widely dispersed and poses particular technical problems because of harsh climatic conditions and difficulty with communications and logistics. As part of the general drive to move more towards a market economy, Rosavtodor has been striving to improve the effectiveness and efficiency of its operations. This includes re-evaluating many of its procedures and standards. Rosavtodor approached the Bank for assistance in 2002 to improve performance monitoring, management systems and quality management. Regional Agency Network (TC7). The federal road management agencies (known as “FGups”) were established at regional level in the 1980s and have been responsible for managing the bulk of the federal road network.3 The rest of the network, mainly in more remote areas, is managed by territorial road administrations and, in some cases, private companies. The FGups are state-owned enterprises reporting to Rosavtodor and operating under contract to Rosavtodor’s regional directorates. Originally their main activity was road construction. However, with the collapse of the Soviet Union, the volume of new construction was reduced significantly, leaving most of the FGups with excess capacity and outdated equipment. In addition, many of them are in charge of loss-making asphalt factories and social housing for the workers.4 Rosavtodor was committed to improving the performance of the FGups. Since 2003 it has collected and reviewed comprehensive data on their financial performance. This has led to the introduction of financial standards and, in some cases, replacement of the head of the FGup. Rosavtodor also invested RUR 100 billion (US$ 3 billion) in new equipment and transferred around 70 per cent of the social housing to municipalities. This involved transferring apartments that were in good condition and was facilitated by allocations from

3 There are 376 FGups. 4 Restructuring of the asphalt factories is a complex issue. Under federal law there is a standard which requires that asphalt be delivered from within 60 km of the site where it is to be used. In cities such as Moscow, asphalt can be bought on the open market, in principle, allowing asphalt factories to be closed, merged or sold. However, this is not possible in more remote areas. Investment in special vehicles allowing asphalt to be delivered further out in the field was foreseen to solve the problem. In addition, road management contracts awarded to the FGups were tendered on a yearly basis. These were input-based and contract prices were determined according to centrally established rates. The rates were low and this, combined with low road density in Russia, means there was little competition for most of the contracts. This, in turn, affected the efficiency and quality of road management/maintenance.

Page 3 of 13 OPER: Russia Road Sector Reform 1 and 2 (TC)

the Federal budget. The remainder that was already in poor condition was increasingly difficult to transfer, as allocations from the Federal budget for refurbishment were being reduced. Rosavtodor requested assistance from the Bank at the end of 2003 to complete the restructuring of the FGups. 2.2 Road financing

At the end of the 1990s Russia had reached a critical point in its approach to financing road construction and maintenance. Wider reforms in the Russian tax system had resulted in the previous system of funding being replaced with largely inadequate, direct budget subsidies.5 Rosavtodor had a vested interest in having an adequate, reliable and sustainable source of financing for its road network management activities. Accordingly, Rosavtodor approached the Bank in 2002 for the assistance which led to TC1. TC6 was essentially an extension of the work begun under TC1. Rosavtodor was actively participating in the continuing debate with regard to road user taxation and wished to further develop a comprehensive strategy of road financing reform for the longer term.6 At the end of 2003, Rosavtodor requested that the Bank provide assistance to finalise recommendations in these areas. With the rapid growth in motorisation in Russia in the early 2000s, the demand for improvements to the federal road network had increased, and was exerting significant pressure on the state budget resources. The Ministry of Transport and Rosavtodor consequently envisaged new financing arrangements in the form of public-private partnerships (PPPs) through open-tendered concession contracts. At the end of 2004, while two interim regulations on the tolling of federal roads had been enacted in 1999, federal legislation enabling concession contracts to be awarded was not yet in place. A general concession framework law was, however, being developed under the leadership of the Ministry of Economy. Rosavtodor asked the Bank to help assess the potential of PPP projects in the road sector under current and/or planned legislation and, based on this assessment, to design a PPP strategy and identify a pilot PPP project that the Bank could finance. This was the focus of TC8. 3. PROJECT OBJECTIVES 3.1 Road management and safety In TC2 on road safety, the consultant had to produce an outline of the road traffic safety action plan consistent with initiatives being undertaken nationally and based on

5 Prior to the reforms the road sector was financed by extra-budgetary funds; there was one federal road fund and more than 89 regional funds. These road funds were mainly financed by way of a 2.5 per cent tax on the turnover of enterprises (referred to as the “road user tax”), a fuel and lubricants sales tax, a vehicle sales tax and an annual tax on vehicles (“vehicle owners tax”). The turnover tax and fuel and lubricants tax yielded revenues for both the federal and regional road funds. The regional road funds, furthermore, received transfers from the federal road fund in the form of subsidies, which made up in approximate terms 30 per cent of the total income of the regional road funds. 6 Rosavtodor has been actively involved in the development of the new approach to taxation of road users and has a vested interest in having an adequate, reliable and sustainable source of financing for its road network management activities.

Page 4 of 13 OPER: Russia Road Sector Reform 1 and 2 (TC)

international best practice. The objectives of TC3 were to monitor the performance of road network management 7 and introduce quality management systems. These objectives should have been achieved in two phases: (1) being the design of the systems and (2) being the trials. Phase 1 included proposals for (i) an upgrade of existing systems in monitoring performance of road network management in key management areas to be agreed with Rosavtodor; (ii) improvements in road network information and data collection; and (iii) an appropriate system for quality management for priority areas. Phase 2 was to test new performance monitoring and quality management systems. For TC7 the consultant had to prepare a report on a comprehensive restructuring plan for the federal road management agencies (FGups). The plan was to have included key decisions to be taken by Rosavtodor, the Ministry of Transport, other government ministries and the Duma, and include a detailed implementation timetable. The consultant had to (i) review the existing organisational structure of the FGups, their resources, equipment and facilities; (ii) assess the current status and scheduling of road works, and identify reasons for any backlogs; (iii) review the efficiency of current contracting for road works; (iv) assess the operational and financial performance of the FGups;8 and finally (v) prepare a restructuring plan with a timetable. 3.2 Road financing The main deliverables from the assignment of TC1 included a review of the road network needs, the current system of road financing, expenditures and allocation mechanisms, the international best practice and recommendations for future road financing mechanisms. In TC6 the purpose of the consultancy was to move the road financing reform agenda forward in the longer term. The consultant was to investigate proposals for the introduction of taxes on heavy vehicles and provide recommendations for the long-term development of taxes. He had to analyse current systems used in other countries to fund road sectors, and provide recommendations on the introduction of similar financing systems in Russia. He would assess the impact of recent changes in the funding of regional and federal roads and make recommendations for further reforms in the long-term. In TC8 the objectives were to assist Rosavtodor with: (i) an assessment of the feasibility of developing PPP projects in the road sector under current and/or planned legislation and identifying any weaknesses in the legislative framework which need to be addressed; and (ii) taking account of the legal review, the development of a PPP strategy for the road sector, identifying a pilot PPP project suitable for financing by the Bank and providing a timetable for developing further PPP projects. 4. ACHIEVEMENT OF OBJECTIVES 4.1 Road management and safety The consultant selected for the road safety contract, TC2, started his assignment upon contract signature in January 2002 and completed it in due time four months later. He successfully covered all the areas identified in the assignment, established a good working relationship with the client, Rosavtodor, and his final report was very much appreciated.

7 Road network management covers operations, maintenance, renewal and development. 8 Operational performance included maintaining network quality, ensuring road safety, rapid response in emergency situations, project management and quality control of road works.

Page 5 of 13 OPER: Russia Road Sector Reform 1 and 2 (TC)

Rosavtodor adopted in 2003 an outline road safety action plan that reflected the recommendations of the report. New road safety standards have been implemented across the network of federal roads – though in some cases not to their full extent due to financial constraints. For TC3 on road network management the final consultant report for Phase 1 was submitted in December 2003 after several revisions. In spite of these revisions, the analysis and recommendations provided did not adequately cover all areas outlined in the TOR and Phase 1 could not be completed. A new version of the report, following further comments, was re-submitted by the consultant to the client and to the Bank on 29 March 2004. However, a reorganisation of Rosavtodor took place at the same time, which again delayed the finalisation of Phase 1 until the end of 2004. The consultant and the client then initiated a range of meetings in December 2004 in order to proceed to Phase 2 and finish the task. Finally, in March 2005, a “no objection” to proceed to Phase 2 was issued, which led to a successful completion of the full report in December 2005. The contract for TC7 on the restructuring of the FGups was executed within one year (as per contract implementation schedule) after signature in December 2004. The client expressed satisfaction on both content and quality of the report. 4.2 Road financing The task for TC1 was completed 11 months after signature in December 2002. The consultant met all objectives. His strategy and implementation plan reviewed the current situation of the road sector financing for regional and federal roads and suggested possible steps to be adopted for their improvement. Under TC6 the selection of consultants was put on hold in March 2004 when Rosavtodor began to be reorganised. The contract between the client and the consultant was finally signed in December 2004. As per contract conditions, the work started in April 2005 and was finished in October 2005 to the satisfaction of the client. The TOR for TC8 were developed in close cooperation with the client. The client’s input during the design stage increased the relevance of the TC work and facilitated its implementation. The task was completed in December 2005, within one year of the contract signing and in accordance with the implementation schedule, receiving approval of the client. Regarding the private sector involvement in road construction financing, TC6 and TC8 provided several building blocks to facilitate the establishment of PPPs: a commentary on the legal framework, the elaboration of a sound project pipeline, and the necessary pre-feasibility studies for a pilot project. The legal analyses focused on the major uncertainties that could make concession contracts more difficult to design, negotiate and implement. The legal recommendations were of a high quality and were carefully reviewed by Rosavtodor, together with other advice from legal firms, IFIs and government agencies which had been sought by the Russian government independently. The findings from the legal group financed by the TC were in general similar to the opinions expressed later by other prominent law firms.

Page 6 of 13 OPER: Russia Road Sector Reform 1 and 2 (TC) Table 1 indicates that all TCs components except one have been fully disbursed: Table 1: Level of disbursements of the TC components TC1 to TC8 (as of 6 December 2006)

Commitment number

Commitment name Commit-ment stage

Commit-ment (€)

Disburse-ment (€)

Road management and safety

TC2

DEN-2002-01-01 Scoping of an assistance programme in the area of road safety

Closed 38,901 38,901

TC3

EIPF01-2002-11-02

Russia Road Sector Reform 1 – quality management

Closed 877,060 877,060

TC7

EIPF01-2004-12-09

Russia Roads 2: Federal road management restructuring plan

Disbursing 120,000 95,896

Road financing

TC1

SWEF-2002-01-02 Assistance with best practice recommendations in the area of road sector financing

Closed 47,895 47,895

TC6

EIPF01-2004-12-07

Russia Road Sector Reform 2: road sector financing in Russia – long-term strategy

Closed 129,900 129,900

TC8

EIPF01-2004-12-08

Russian Roads 2: preparation of a PPP strategy for roads

Closed 198,200 198,200

Total 1,411,956 1,387,852

The TC project achievements of both safety and road financing components are rated successful. 5. OVERALL ASSESSMENT All six road management and safety and road financing components were relevant: they addressed the client needs for institutional strengthening at the time they were expressed by the client. The components were also effectively delivered, and most of them delivered in a reasonable time period. The components which responded to implementation problems (TC3 and TC7) were subject to corrective actions by the Bank, the client and the consultants, to yield final products which were well appreciated by the client. The reorganisation of Rosavtodor in 2004 created a major challenge for the continuity of all TCs since the new management brought into Rosavtodor was not initially fully aware of the purpose and the advancement of the TC work. In spite of the difficulty the TC tasks were carried to completion. Regarding the transition impact of the TC programme, the TC2 component generated skills transfer and new business practices on road management and safety at the corporate level. The same applies to TC3 for monitoring systems and quality control. The impact initially

Page 7 of 13 OPER: Russia Road Sector Reform 1 and 2 (TC)

extended to other public administrative units and contractors that were reached through the implementation of the reforms. In addition, some impact could be observed directly at the sector level from the road financing TC components TC1, 6 and 8 in the areas of frameworks for markets, especially road budget finance and user taxation, concession laws, public sector contracting and PPP processes. The recentralisation of decision making in the public sector has in recent years led to a slowdown in the implementation of the sector reforms, especially the decentralised components. Much of the debate about road budget finance and user taxation is continuing as serious deficiencies remain in comparison with international good practice. The PPP response to road financing, however, is being further developed in Russia. While the TC8 outputs continue to be relevant, the government has taken for the time being a different approach to the legal framework and decided on a pilot project different from the project supported by the TC. 5.1 Rating compared with PCR The ratings of the OPER confirm the ratings of the PCRs for the early components of the TC programme, but they tend to differ from the PCRs on the more recent components, mostly due to the assessment of transition impact. Although transition impact is generally fine at ‘corporate level’, it is not so strong at sector level regarding the level of advancement of reforms of regional agencies (TC7), road budget finance and the quality of the PPP framework (TC6 and TC8).9 The OPER confirms that the Bank handling of the TC was good, since Bank staff adjusted their lending and TC approaches to changing conditions in the road sector in Russia. This factor, together with a good performance in achieving the TC objectives, contributes to the overall success of the TC programme. In spite of some less positive influence of the marginal to satisfactory rating of transition impact for TC6, 7, and 8, the overall performance of the TC programme is rated as successful10.

9 On TC8, the Banking team’s position is that the transition impact should be ‘good’ and overall rating ‘successful’. As a result of the work undertaken on this assignment, the legal recommendations were taken by the government for consideration in the preparation of the draft Concession Law. Furthermore, this work allowed the Bank to foster its dialogue with the government (respective key ministries). The Concession Law was effected in 2005, which is well beyond the TC objectives. Whilst the team accepts to some extent that there are still a number of deficiencies in the approved Concession Law, it has also been confirmed by western lawyers that the approval of the Concession Law, even in its present format, is adequate to allow the implementation of PPP projects in Russia and, from that point of view, it is an important step forward. Any deficiencies in the law are expected to be rectified as part of the working process. In addition, M4 Don road section was selected by Rosavtodor for initial technical due diligence as a potential pilot project. This road remains on the government’s list of potential PPPs, albeit not the first pilot project currently tested in Russia. It should be noted, however, that the objective of the assignment was to transfer know-how on the evaluation of the project’s suitability for PPP financing; it was not a pre-requisite of the TC assignment that the evaluated project should be the first PPP project to be implemented in Russia. 10 The overall rating is calculated as the weighted sum of the ratings of each component, the weights being the amounts disbursed for each one of them. In that case, the successful TC3 on road quality management is carrying the heaviest weight. If the relevance of each component towards institution building, a qualitative appreciation, would have been used instead, giving a higher priority to road financing issues and especially the PPP component, then the weight of the TCs related to road budget finance and PPP would have been higher and the overall rating less favourable (partly successful only). In all cases, it is acknowledged that the

Page 8 of 13 OPER: Russia Road Sector Reform 1 and 2 (TC) Table 2: Summary rating table for TC performance

PED Evaluation Team PCR (or progress report) Rating 1/

Achievement of objectives

Transition impact 3/

Bank handling 2/

Overall rating 1/

Road management and safety

TC 2 Successful Fully achieved Good Good Successful TC 3 Successful Fully achieved Good Good Successful TC 7 Successful Fully achieved Marginal to

satisfactory Good Partly

successful Road financing TC 1 Successful Fully achieved Good Good Successful

TC 6 Successful Fully achieved

Marginal tosatisfactory

Good Partly successful

TC 8 Successful Fully achieved

Marginal to satisfactory

Good Partly successful

1/ Rating is highly successful, successful, partly successful, unsuccessful; PCR ratings available for TC1 and TC2, PR ratings for TC3. 2/ Rating is excellent, good, satisfactory, marginal, unsatisfactory, highly unsatisfactory 3/ Rating is excellent, good, satisfactory, marginal, unsatisfactory, negative 6. TRANSITION IMPACT AND RISK 6.1 Impact at corporate level. The impact at the corporate level depends mainly on the TC2, TC3 and TC7 components, which generated skills transfer and new business practices within Rosavtodor. The transfer of know-how and new management practices in the area of road safety occurred through a sequence of reports, seminars and discussions with the consultant. It successfully contributed to further work being carried out by the client. The same applies to monitoring systems and quality control for road maintenance and construction. Awareness on new organisational requirements and the ways to proceed were also well understood by Rosavtodor and other departments of the Ministry of Transport. Some demonstration effects from the above-mentioned TC components also occurred within the corporation. The findings of the studies were diffused across the main departments of Rosavtodor, beyond those which were initially concerned with the technical aspects of the TC.11 The awareness of more effective approaches to management and controls were used as reference for Rosavtodor to pursue its administrative reform. The transition impact at corporate level is rated as good.

lower transition impact at sector level for some TCs is to a large extent due to policy and business environment factors beyond the reach of Bank staff and TC programme recipients. 11 This applies especially to the road safety, the management systems and quality control studies, and the pre-feasibility study for one of the PPP pilot projects (M4).

Page 9 of 13 OPER: Russia Road Sector Reform 1 and 2 (TC)

6.2 Impact at the sector level The impact observed at the company level could potentially spread beyond the boundaries of the federal road administration, given its strong presence in the country. However, for both road management and financing, the transition impact of the TCs substantially deviated from what was originally foreseen in the consultant reports mostly as a result of a changing business policy climate. This appears in the areas of skills transfer, framework for markets, private ownership and new business standards. 6.2.1 Skills transfer. The transfer of skills carried out by the management and safety TC components (TC2, TC3 and TC7), at least in the early years of the TC programme, went beyond the boundaries of the federal road administration, Rosavtodor, to reach the other public administrative units and contractors that were involved in the implementation of the road sector reform. Even when the application of road safety standards was delayed,12 or when the reorganisation of the FGups took a different direction than was originally expected,13 skills transfer and new business practices had to be secured to carry the reforms forward. It appears, however, that the spread of new skills has slowed down in recent years with the return of more centralised management practices in the public sector. 6.2.2 Frameworks for markets. The road financing TC components (TC1, TC6 and TC8) generated some impact at sector level in the area of frameworks for markets, especially road budget finance, concession laws, contracts and PPP processes. On budget finance, the recommendations in TC1 made it clear that, given the federal budget reform goals at that time, a new approach to road financing was needed in order to contend with growing maintenance and construction needs. The underlining principles were that users should pay for road maintenance and the private sector would contribute to the long-term financing of road construction. In the early 2000s, the Russian government had decided to abolish the Federal Road Fund and to remove the direct allocation of enterprise turnover and other profit tax proceeds to the financing of road maintenance and construction. The consultant’s report recommended the establishment of a second generation road fund in which the road users, especially the heavy goods vehicle users, would be more involved in road financing. The application of the user financing principle was also present in TC6 and TC8. Several government reforms went in the direction recommended: the taxation of car owners became more substantial and effective and it was recommended that the taxation of trucks should be made proportional to their carrying capacity.14 However, the second generation

12 Delays in the nationwide application of road standards occurred when additional federal funds were not immediately available to cover the equipment or construction costs. 13 The FGups were restructured, but not to the very end of what the report recommended. Instead of being directly privatised, the assets of the FGups were transferred to a public property agency. The assets remain “frozen” as publicly owned for several years until a new decision is made to dispose of them. Meanwhile, a large proportion remains under the management responsibility of the federal road administration, Rosavtodor, which needs a critical mass of physical road maintenance capabilities to fulfil its mandate nationwide. 14 On 24 July 2002 the State Duma approved the federal act No. 100 – FZ, which resulted in the following changes: (i) an increase of 45 per cent of the federal excise tax on fuels and lubricants, with 60 per cent of this tax to be transferred to the federal budget; (ii) conversion of the “vehicle owner’s tax” to a “transport tax” together with substantial increases in the minimum rates. The regional road funds were to be financed

Page 10 of 13 OPER: Russia Road Sector Reform 1 and 2 (TC)

fund did not materialise.15 The proceeds from the user taxes were redirected to the federal budget and Rosavtodor only received unrelated central budget appropriations to finance road maintenance. While these appropriations have recently increased (although they are still significantly lower than pre-reform levels), a financing gap remains between current resources and needs for maintenance and rehabilitation. Presently, the principle that users should pay for road services came back partially with the PPP approach in TC8. Several key legal recommendations contained in TC8 were not taken on board by the Russian authorities when the PPP law was passed in 2005. This has left major uncertainties in the minds of foreign investors. In addition, the M4 road selected by the Bank and Rosavtodor as a suitable pilot project and for which pre-feasibility studies were included in TC8 will no longer be the pilot. While the M4 remains on the government’s list for potential PPPs, the St Petersburg Western High-speed Diameter project is now being developed as the first project and may now be in need of technical support. 6.2.3 Private ownership. The impact of the reforms of the FGups in TC7 took a different track than what was implied by the recommendations of the consultants. The FGups have not taken the direction of privatisation. Their assets have been frozen in a government-owned asset management company for at least five years.16 6.2.4 New business standards. The new standards to implement the recommendations of the same TC2, TC3 and TC7 respectively on road safety, the road network monitoring systems and quality controls and new decentralised management practices were initially applied nationwide. The move towards decentralisation has slowed considerably in recent years due to the policy changes at the highest levels of government. 6.3 Risk attached to remaining potential transition impact The reforms supported by the TCs implied further decentralisation of decision making and contracting responsibilities in the public administration of Russia. In both management and finance terms, because the growing reluctance at high levels within the Russian government for support of more decentralised decision making, the risk attached to the remaining transition impact of the TCs is presently quite large. At the moment, the interest shown by the private sector in bidding for St Petersburg Western High-speed Diameter appears to be high and the completion of this first PPP project, if successful, could have a large demonstration effect. It could serve as a basis for defining any gaps in existing concession legislation and overall project preparation as may be applicable. Nevertheless, the extent to which the provisions of new PPP contracts for road construction will really attract the private sector into a durable collaboration remains largely unknown. Taking into account the positive initiatives in this area and the current risk attached to them, the overall

by (i) the “transport tax”; (ii) revenues from the excise tax on fuel and lubricants; (iii) revenues from the enterprise profits tax; (iv) proceeds from land rent and charges; and (v) appropriations from the federal budget. 15 The federal road fund ceased to exist in 2001. The regional funds were abolished in 2003. The tax revenues previously earmarked to them in the federal law of 2002 were then reassigned to the federal government. Since then all the proceeds to finance the road sector originated from the federal government. 16 A large proportion of the frozen assets remains under the management responsibility of the federal road administration, Rosavtodor, which needs a critical mass of physical road maintenance capabilities to fulfil its mandate nationwide.

Page 11 of 13 OPER: Russia Road Sector Reform 1 and 2 (TC)

transition impact at sector level at this stage is rated as satisfactory. 7. ADDITIONALITY The Bank properly exerted its relative advantage in collecting donor funds to strengthen private sector oriented initiatives in the road sector in Russia. In terms of project design and functioning the contribution of the Bank was unique among the numerous advisory activities that were offered to the Russian government in that they focused on private sector involvement through a wide range of diverse applications: organisation, management, financing and allocation of resources. Additionality is rated as verified in full. 8. BANK PERFORMANCE

Since the beginning, the Bank involved the client heavily in the TC process to ensure

ownership that would in turn facilitate the acceptance and the implementation of the proposed reforms. In TC2 the client took an active part in the drafting of the TOR and in the consultant’s selection, and worked closely with him during the assignment. The Bank facilitated the good relationship with the consultant and the client. As a result, the client strongly supported the conclusions of the report on road safety. The initial client’s commitment in TC3 on road management was good as well. The assignment was put on hold in 2004 due to government reorganisation and the appointment of new counterparts at Rosavtodor who in turn required time to become familiar with the Bank’s loans and ongoing TC components, and hence the TC was only completed in late 2005. Nevertheless, the Bank actively contributed to the successful outcome of the advisory task in maintaining a continuous contact with both parties throughout this period.

The TC7 component dealt with a delicate subject by putting emphasis on the reorganisation of the regional agencies (FGups). The Bank handled it with tact and in due time. The client was satisfied with the TC process. During the assignments of TC1, TC6 and TC8 on road financing, both client and consultants received from the Bank the support required for a successful TC project implementation. The consultant’s contacts with the Bank and the client were effective. The consultant adapted well to changes in the objectives of his assignment, especially in TC8 where the outcome of the development PPP reforms was not in the client’s direct control and difficult to predict. Overall the Bank performance is rated as good. 9. KEY ISSUES AND LESSONS LEARNED Several positive outcomes of this TC programme bring lessons for stating the conditions that lead to successful TC project preparation and implementation of the sector reforms. There are also important lessons on the preconditions for policy dialogue when many stakeholders are involved in the reform process. The following lessons have been applied to a large extent by the Bank teams that carried out the TC programme. They are mainly addressed to new teams which may start supporting sector reforms in other countries, especially early transition economies.

Page 12 of 13 OPER: Russia Road Sector Reform 1 and 2 (TC) 9.1. TC project design and implementation 9.1.1 Client’s ownership of the programme. In this TC programme, the Bank fully acted upon demands of the client and thoroughly discussed the subjects of the studies before getting started. The diversity and the complementarity of the studies provided a sector reform approach that was relevant to the client. As a result, the client maintained a strong and continuous interest in the programme throughout the five years of provision of advisory services. Lesson: Fully involve the client in the formulation of the TC programme. When the TC aims at major institution building, the selection of the topics of the studies that will lead to institutional changes should be thoroughly discussed and agreed upon with the client ahead of any design of the various components. This is to ensure its full collaboration to the successful implementation of the TC assignment all along the path of the reform. 9.1.2 Client’s participation in project design. Another positive aspect of the TC was the client’s active involvement in the implementation of the TC. This was due to his thorough acquaintance of the TC before implementation started, which in turn can be explained by his direct involvement in the design of the TC components. Lesson: Directly engage the client in the preparation of each TC component. When the scope of the TC covers sector-wide reforms that require a complete knowledge of the issues facing the sector, the client should directly contribute in the preparation of the terms of reference and his views should be taken into account to make sure that all the key dimensions of the sector reform are included. This greatly facilitates ownership at implementation and creates a solid ground for transition impact to occur. 9.1.3 Dealing with staff changes among project counterparts. Some TCs took longer than expected because of government changes. While these changes were outside the control of the EBRD and Rosavtodor, EBRD staff had to spend even more time and energy to keep the TC rolling. Lesson: Make provision for risk of discontinuity in client relations. When a TC programme runs over several years and involves major institution building, there is some probability of staff turnover among TC project counterparts, including high-level government officials. When this occurs, EBRD staff must make extra efforts to keep the TC programme on track. It is essential that EBRD management frees the necessary resources for this to happen. 9.2 Sector policy dialogue 9.2.1 What may happen when the client receives legal advice from various sources. For the preparation of the Concession Law, the government requested separately the advice of many stakeholders, including several international law firms, one of them financed by the TC programme (TC8). In the end, the government produced a text on its own that did not include a number of the key recommendations expressed by the international legal community. Had these recommendations been accepted, the legal framework might have been more attractive to private investors. The extent of recommendations acceptance by governments is unpredictable and depends largely on the government’s willingness which, depending on a client’s profile, is sometimes well beyond the EBRD’s or any third party’s

Page 13 of 13 OPER: Russia Road Sector Reform 1 and 2 (TC)

control or influence. Lesson: Coordinate legal advice with other law firms. When providing legal advice for the preparation of a framework law on private sector participation to public investments, to the extent possible, the EBRD should be aware of other legal advice initiatives in the same area and try to coordinate with them in order to help preserve a minimum of unity in the scope and content of the recommendations that the government will receive. 9.2.2 The case of complex reforms requiring the consensus of several government units. The Bank succeeded in articulating its position on the draft Concession Law and communicating it to Rosavtodor. There were many high-level meetings, correspondence exchanges between the Bank’s senior management and the Russian Ministry of Economic Development, in particular during the stage of Concession Law drafting. However, the acceptance of recommendations by the government sometimes depends on its profile and willingness to accept third party views. The road administration had some difficulty in further carrying the message and influencing other major units in the government that were also involved in the preparation of the law, especially the Ministries of Finance and Economic Development (TC8). Lesson: Raise the Bank’s advocacy at the highest levels to promote sector reforms. When a government prepares legislation that may affect a number of sectors and involves stakeholders in the high decision making spheres of various administrations, the Bank, while working with one main interlocutor, should liaise with as many of the other stakeholder groups as possible in order to extend its advocacy and increase its impact. This may require several interventions in the field of senior Bank management. 9.2.3 How to optimise Bank interventions when a reform is ongoing. Occasionally, by the time consultants are mobilised, the reform has moved on and the government has already taken initiatives in the very same areas where advice is to be given (TC6 and TC8). Often, the decisions taken by the government are difficult to reverse. They might not have been taken, however, if the TC advice had been available earlier. Lesson: Provide timely advice throughout the reform process. When the government is in the middle of a sector reform and asks for TC support, the Bank should be prompt in mobilising resources and finding consultants that can step in and provide real time advice. 9.2.4 The TC programme is completed, but the sector reform is not. The new Concession Law is now enacted, and a substitute PPP pilot project is on the way. Rosavtodor may soon seek new advice on how to further proceed with the concession agreement, the finalisation of the tender process, the formulation of the awarded contract and the first steps of contract implementation. Lesson: Keep close contact with the former client after the completion of a TC programme. When the Bank has made a major technical assistance contribution to a sector reform and the corresponding consultancy contracts are completed, the Bank staff should keep in close contact with the former client to be able to respond to any new demand for advice that may further strengthen the sector reforms.

APPENDIX 1

OPERATION PERFORMANCE RATINGS RUSSIA ROAD SECTOR REFORM 1 AND 2 (TC)

Performance Indicator

Rating OVERALL TRANSITION IMPACT: Skills transfer and new business practices on road management and safety initially extended to other public administrative units and contractors that were reached through the implementation of the reforms. In addition, some impact could be observed directly at the sector level from the road financing TC components in the area of budget finance, concession laws, contracts and public-private partnership processes. The recentralisation of decision making in the public sector in recent years has led to a slowdown in the implementation of the sector reforms, especially the decentralised components. (Excellent, Good, Satisfactory, Marginal, Unsatisfactory, Negative)

SATISFACTORY

ENVIRONMENTAL PERFORMANCE OF THE PROJECT AND SPONSOR: (Ratings: Excellent, Good, Satisfactory, Marginal, Unsatisfactory, Highly Unsatisfactory)

N/A

EXTENT OF ENVIRONMENTAL CHANGE: (Ratings: Outstanding, Substantial, Some, None/Negative)

N/A

ADDITIONALITY: The Bank properly exerted its relative advantage in collecting donor funds to strengthen private-sector-oriented initiatives in the road sector in Russia. In terms of project design and functioning the contribution of the Bank was unique among the numerous advisory activities that were offered to the Russian government in that they focused on private sector involvement through a wide range of diverse applications: organisation, management, financing, and allocation of resources. (Ratings: Verified in all respects, Largely verified, Verified only in part, Not verified)

VERIFIED IN ALL RESPECTS

PROJECT FINANCIAL PERFORMANCE: (Ratings: Excellent, Good, Satisfactory, Marginal, Unsatisfactory, Highly Unsatisfactory)

N/A

COMPANY FINANCIAL PERFORMANCE: (Ratings: Excellent, Good, Satisfactory, Marginal, Unsatisfactory, Highly Unsatisfactory)

N/A

FULFILMENT OF PROJECT OBJECTIVES: All six road management and safety and road financing components were fully completed. The reorganisation of Rosavtodor in 2004 created a major challenge for the continuity of the TC since the new management who was brought into Rosavtodor was not initially fully aware of the purpose and the advancement of the TC work. In spite of the difficulty the TC tasks were carried to completion. (Ratings: Excellent, Good, Satisfactory, Marginal, Unsatisfactory, Highly Unsatisfactory)

GOOD

BANK HANDLING: The Bank handling of the TCs led to on-time deliveries for most of the component, and tangible results for all of them, to the satisfaction the client. (Ratings: Excellent, Good, Satisfactory, Marginal, Unsatisfactory, Highly Unsatisfactory)

GOOD BANK’S INVESTMENT PERFORMANCE: (Ratings: Excellent, Good, Satisfactory, Marginal, Unsatisfactory, Highly Unsatisfactory)

N/A

OVERALL PERFORMANCE: (Ratings: Highly Successful, Successful, Partly Successful, Unsuccessful)

SUCCESSFUL

APPENDIX 2

TRANSITION INDICATORS FOR EX-ANTE AND EX-POST APPLICATION RUSSIA ROAD SECTOR REFORM 1 AND 2 (TC)

TI

checklist categories

STEPS OF RATING TRANSITION IMPACT EX-POST

Short-term verified impact

Longer- term transition

impact potential

Risk to potential

TI

STEP I: CHANGE BY THE PROJECT AT CORPORATE LEVEL

Rating1

Rating2

Rating3

3 Private ownership N/A N/A N/A

5

Skill transfers. The transfer of know-how and new management practices in the area of road safety, monitoring systems and quality control for road maintenance and construction, new administrative organisation occurred through a sequence of reports, seminars and discussions with the consultant.

Good Good Low

6 Demonstration effects. To some extent, the findings of the studies were diffused across the main departments of Rosavtodor, beyond those which were initially concerned with the technical aspects of the TC. The awareness of more effective approaches to management and controls were used as reference for Rosavtodor to pursue its administrative reform.

Satisfactory Satisfactory Low

7

New standards for business conduct. The establishment and installation of new management systems contributed to the further modernisation of management practices in Rosavtodor.

Good Good Low

STEP II: TRANSITION IMPACT AT THE LEVEL OF INDUSTRY AND THE ECONOMY AS A WHOLE

Rating Rating Rating

1 Competition N/A N/A N/A

2 Market expansion N/A N/A Medium 3 Private ownership. The transfer of assets of the regional

agencies (FGups) to the private sector is very slow. These assets have been temporarily “frozen” in a newly created public sector entity.

Marginal Satisfactory High

4

Frameworks for markets. Current practices of financing road maintenance and construction are presently remotely linked to the principle that users should pay for road services. But there is the possibility of some improvement in the future through PPP projects. In addition, several key recommendations from the legal advice on PPPs were not taken on board by the Russian authorities when the PPP law was passed in 2005.

Marginal Satisfactory Medium

5 Skills transfers. The skills developed within Rosavtodor in the areas of road management and financing are still being diffused across the road sector, in the regions or other public administrations. These are occurring at a slower rate since other external factors intervened to change the course of the reforms.

Satisfactory Satisfactory Medium

6 Demonstration effects N/A N/A N/A

7 New standards for business conduct (see 5 above) Satisfactory Satisfactory Medium

SUMMARY OF VERIFIED, POTENTIAL AND RISK RATINGS Satisfactory Satisfactory Medium

OVERALL TRANSITION IMPACT RATING:4 Satisfactory

1 This range is: Excellent/Good/Satisfactory/Marginal/Unsatisfactory/Negative. 2 This range is: Excellent/Good/Satisfactory/Marginal/Unsatisfactory/Negative. 3 This range is: Low/Medium/High/Excessive. 4 This range is: Excellent/Good/Satisfactory/Marginal/Unsatisfactory/Negative.