Evaluation of the Tacis Interstate Programme in...

Evaluation of theTacis InterstateProgramme inEnvironment

Final report

30 September, 2000

This report is prepared by the Netherlands Economic Institute (NEI BV, Rotterdam), in co-operation with MOB (Nijmegen), and MilieuKontakt Oost-Europa (Amsterdam) for the EvaluationUnit of the Joint Service for external relations of the European Commission (SCR).

The authors retain sole responsibility for the report and the views expressed are their own. Assuch the contents of the report do not necessarily reflect the views of the EuropeanCommission. The European Commission does not guarantee the accuracy of the data includedin this report, nor does it accept any responsibility for any consequence of their use.

TIPE evaluation

TIPEfinalreport (af5524fr2.doc)

1

Table of contents

Table of contents 1

List of abbreviations 5

Executive summary 7

1 Introduction 151.1 Background 151.2 Structure of the report 151.3 Evaluation objectives and key questions 151.4 Approach and other methodological aspects 171.5 Acknowledgements 19

2 Programme description and analysis 212.1 Introduction and the problems addressed by the TIPE 21

2.1.1 Introduction 212.1.2 The environmental problems in the Soviet legacy 21

2.2 The objectives of the TIPE 222.2.1 Description 222.2.2 Analysis and findings 24

2.3 Programme intervention logic 252.3.1 Description 262.3.2 Analysis and findings 27

2.4 Programme content 272.4.1 Description 282.4.2 Analysis and findings 28

2.5 Programme organisation and management 302.5.1 Description 302.5.2 Analysis and findings 31

2.6 Conclusions 32

3 Analysis and findings at project level 353.1 The “Aral Sea” Programme 35

3.1.1 Introduction 353.1.2 The Tacis project 363.1.3 Conclusions 363.1.4 Recommendations 37

3.2 The “Black Sea” Programme 383.2.1 Introduction 383.2.2 The Tacis support 393.2.3 Conclusions and lessons learned 403.2.4 Recommendations 41

3.3 The “Caspian Sea” programme 423.3.1 Introduction 423.3.2 The Tacis support 423.3.3 Conclusions 433.3.4 Recommendations 44

3.4 The “Development of Common Environmental Policies”programme 443.4.1 Introduction 443.4.2 The Tacis support 453.4.3 Conclusions 463.4.4 Recommendations 48

TIPE evaluation

TIPEfinalreport (af5524fr3.doc) 2

3.5 The programme for “Raising Public EnvironmentalAwareness” 483.5.1 Introduction 483.5.2 The Tacis project 493.5.3 Conclusions 503.5.4 Recommendations 51

3.6 The programme for “Establishing Regional EnvironmentCentres” 523.6.1 Introduction 523.6.2 The Tacis support 533.6.3 Conclusions 543.6.4 Recommendations 55

4 Analysis and findings at programme level: Part I: issuesof content 574.1 Some figures 574.2 The political economy of environment 574.3 Validity of the programme intervention logic 58

4.3.1 Think global, plan interstate, act local? 584.3.2 The SWOT of the TIPE 59

4.4 Programme relevance issues 604.4.1 The relevance of different types of projects 60

4.5 Programme effectiveness 614.6 Programme efficiency 614.7 Programme sustainability and impact 62

Impact evaluation and feedback 634.8 Overall assessment of programme value added 64

5 Analysis and findings at programme level Part II: issuesof management 655.1 Project cycle management issues 65

6 Conclusions and Lessons learned 676.1 General conclusions 676.2 Supporting conclusions at programme level 68

7 Recommendations for future programming 697.1 The rational for continuation of TIPE 697.2 Mobilise (member state) support for environment 697.3 Considerations for Tacis programming (2000-2006) 697.4 Continuation of ongoing actions 707.5 Improving relevance in programming 707.6 Recommendations for future programme management 71

Annex A Guideline to the methodology used 75A.1 The content of annex A 75A.2 Consultation with the Client 75A.3 Key questions, and specific issues for the evaluation 76A.4 Information sources, barriers, limitation to the scope of

the study 77A.4.1 Information sources 77A.4.2 Barriers 78A.4.3 Limitations 78

A.5 Commissioning of sub-studies, and ToRs 78A.6 Impact measurement and indicators 79

A.6.1 The model 79A.6.2 The term impact, and impact classification 79A.6.3 Impact dimensions, and indicators 80

A.7 Evaluation criteria and performance indicators 80A.8 Project logical frameworks and evaluation matrices 81

Wider Objective 83Immediate objective 83Intervention logic, REAP continued 90

TIPE evaluation


Intervention logic, NRECS 91

Annex B Experts involved in the evaluation and outputs 93

Annex C People met and interview notes 95

Annex D Documentation and literature 101

Annex E Terms of Reference 105

Reports available on request only

Interim report, 28 November 1999Project synthesis reports, 25 February 2000

TIPE evaluation


TIPE evaluation


List of abbreviations

AP Inter-state Action Plan/ProgrammeASBP Aral Sea Basin Programme (GEF and Tacis sponsored project)BSEP Black Sea Environment ProgrammeCSEP Caspian Sea Environment ProgrammeDCEP Common Environmental Policies, a project from AP (1995)Commission Commission of the European UnionEC European CommissionESF Environmental Support FacilityEU European UnionGEF Global Environment FacilityINOGATE A pipeline interstate projectIP Indicative ProgrammeJEP Joint Environment ProgrammeNEAP National Environment Action PlanNewrec The phase of the REC project, which aimed at establishing an international

organisation with one central headquarters with small national offices in variousNIS

NIS Newly Independent States on the territory of the former Soviet UnionNREC National regional environment centresREAP Raising Environmental Awareness of the Public & Media Development, (project)REC Regional Environment CentreRSP Regional Seas ProgrammeTEAP Other acronym for the REAPTIP Tacis Inter-state ProgrammeTIPE Tacis Inter-state Programme in Environment, this is a component of the TIPTOR Terms of ReferenceUNEP United Nations Environment ProgrammeUNDP United Nations Development ProgrammeTRACECA Tacis transport corridor interstate programmeWARMAP Water Resource Management Programme, a number projects, incl. the Aral SeaWARMIS Water Resource Management Information System (quantity management data

base)WUFMAS Water Use and Farm Management Survey (Aral sea basin)WEAP Widening the Environment Action Plans (project)

TIPE evaluation


TIPE evaluation


Executive summary

1 Introduction

1.1 This report sets out the main findings, analysis and recommendations of the team1, whichundertook the evaluation of the Tacis Interstate Programme in Environment (TIPE). The evaluationcovers both content, and management aspects of the programme.

1.2 The Terms of Reference was prepared by the evaluation unit of SCR, and focussed thestudy on relevance, effectiveness, efficiency, and impact and sustainability of the policies andstrategies adopted by the Commission in the framework of the Tacis Interstate programme actionsfor environment. On the basis of an ex-post analysis of key projects, the evaluation exercise hasgenerated recommendations for future programming and has come up with suggestions for possibleimprovements in the TIPE programme cycle. This executive summary highlights, and presents themain results of the evaluation in four sections: programme description, findings, overall conclusions,and recommendations.

2 Programme description

2.1 The Tacis Interstate Programme in Environment (TIPE) is a component of the TacisInterstate Programme (TIP), and the regular TIP programming, budgeting and implementationprocedures apply. The content of the TIP has changed over time. Currently, it comprises three maincomponents: interstate environment, interstate networks (INOGATE, TRACECA), and third pillarissues (Justice and Home affairs).

2.2 The main orientations/themes, objectives, budgets, and project content of the TIP aredetermined on an annual basis, and are presented in the annual TIP Strategy paper and TIP ActionPlan (AP). The general objective and identity of the Interstate Programme is formulated by theCommission as follows:

“The Inter-State programme aims to assist the New Independent States (NIS) and Mongolia toidentify and pursue solutions to problems which are of an inter-state nature. As such, it operateson a basis analogous to the European Union's concept of subsidiarity since it only includesactions, which are best undertaken on a multi-country, rather than on a national, level. Projectsincluded in the Inter-State programme are designed to complement and reinforce thoseundertaken in the Tacis national programmes, Cross Border Co-operation and Small ProjectsProgrammes.”

2.3 Over the period 1992-1997, the Commission has allocated 160 Million Euro to the TIP, andapproximately one third of these resources for environment. The breakdown was as follows:environment 52,5 MEURO; networks 106 MEURO; third pillar issues 3,5 MEURO.

2.4 International initiatives in the field of environment have provided the main policy orientationfor the TIPE. The “Environment for Europe process”, and International donor initiatives on regionalseas (Aral sea, Black sea and Caspian sea); as well as other immediate relief programmes(Chernobyl) have determined the TIPE programming strategy and have shaped the project portfolioin the period 1992-1999.

2.5 In the last two years of the 1990’s the TIPE programming focus has changed somewhataway from the regional seas programme towards, the Environment for Europe process and

1 The evaluation team comprised staff members from the Netherlands Economic Institute , MOB -

Mobilisation for Environment, and MilieuKontakt Oost-Europa.

TIPE evaluation


especially towards investment promotion as emphasised in the latest Environmental Ministersconference in Dobris. The tabulation below shows this.

Table S.1 TIPE - Themes, objectives and resource allocation

Themes & Objectives Funding 92-97 Funding 98-99

The environment for Europe process• Public awareness raising 8.4 Million Euro 3.5 Million Euro• Investment promotion 3.7 Million Euro 8.0 Million Euro• Policy integration of environment 6.1 Million Euro 0.0 Million Euro

International regional seas conventions• Seas programmes 26.8 Million Euro 0.0 Million Euro

Other immediate relief programmes• Chernobyl relief 7.5 Million Euro 2.0 Million Euro• Trans-boundary rivers 0.0 Million Euro 4.0 Million Euro• Biodiversity 0.0 Million Euro 1.5 Million Euro

Total funding 52.5 MEURO 19.0 MEUROTotal all Tacis programmes 3000.0 MEURO 950.0 MEUROTIPE funding in percentage of Tacis 1.7% 2.0%

Source: chapter 2, Table 2.1

2.6 With the TIPE the Commission has followed a strategy of “fewer projects for more effect”. Alimited number of projects, have received funding for longer periods of time. Some parameters ofthe eight key projects in the portfolio resulting from the 1992-1997 Action Plans are listed below inTable S.2.

2.7 In the period 1992-1997, the Commission has allocated around 2.5 percent of its total Tacisprogramme funding for environment projects. The TIPE has been the main Tacis channel with some70% of the funds. Other channels include the Tacis National programmes (15%), the Cross-BorderProgramme (10%), and various small project facilities (5%).

3 Main findings

3.1 The evaluation team analysed the performance of six long running projects, which togethermake up some 80% of the TIPE project portfolio over the budget period 1992-1997. To assessproject performance, the SCR standard evaluation criteria were used, including relevance,effectiveness, efficiency, impact and sustainability. The data collection comprised literature reviews,file research, stakeholder interviews, and a review of expert opinions/sub-studies, especially,commissioned for this evaluation from NIS academics, NIS civil society, and representatives fromthe seven countries visited during the evaluation (Russian Federation, Ukraine, Moldova, Georgia,Azerbaijan, Uzbekistan, Kazakhstan)2.

2 Details on methodology are presented in Annex A of the main report

TIPE evaluation


Table S2 – TIPE portfolio and evaluation findings at project level

Project portfolio 1992-1997Total TIPfunding in1992-97

portfolio inMEURO

Share in1992-97

portfolio inpercentage

Relevance

Effectiveness

Efficiency

Sustainability

Impact

Aral sea basin environment programme3 10.75 20.5 % M M L LBlack sea environment programme 9.50 18.1 % H M L M/LCaspian sea programme 6.50 12.4 % H L M M/LRaising Environmental Awareness – REAP 5.50 10.5 % H M H MNew Regional environment centres – NREC 2.40 4.6 % M L L LEnvironmental action planning – REF/CEP 6.15 11.7 % H M M MInvestment promotion – WEAP 3.70 7.7 %Chernobyl affected areas/HACCP food 7.50 14.3 %Other 0.50 0.2 %

No performance ratingmade in the evaluation

Total 52.5 100.0 % H M/LSource: Table 4.1Notes: H= High/good performance found against the evaluation criterion; M= Medium/acceptable performancefound; L=Low/poor performance found.

3.2 Project specific evaluation findings can be characterised as follows:♦ The Aral Sea will die without a change in policy. A clear, commonly pursued conservation

objective, and a joint action plan are lacking in the international commitment and project aid. Inthis setting international aid efforts including, the Tacis Aral sea programme support can becharacterised as misguided. (see chapter 3.1).

♦ The Black Sea programme has lost its initial momentum. The Black sea ecosystem and itsmultipurpose functionality (tourism, fisheries) can be saved from further degradation, but thisrequires much higher priority than at present, for environmental management (e.g. wastemanagement) at national levels. Funding problems, and co-ordination problems slow down theimplementation of the Black Sea Environment Action Plan. Moreover, the necessary link ofBSEP aid actions with the national environment action plans is virtually non existent. (seechapter 3.2);

♦ The ecological threats to the Caspian Sea from oil exploration and transport are enormous andgrowing. The Tacis support aims at contributing to the preparation and adoption of a CaspianSea Environmental Action Plan. This is an essential first step in a complex road to safeguardingthe ecosystem from the booming oil business; (chapter 3.3);

♦ Plenty of visible activities, and a wide civil society focus, is effectively the sub-title of the REAPevaluation report. The project has introduced new approaches and original working methods,which have been made available to local civil society organisations for “Raising EnvironmentalAwareness of the Public in the NIS (see chapter 3.5)”;

♦ How a means has become an end in, and of itself, is effectively the sub-title of the evaluationreport on three years Tacis support for the establishment of New Regional Environment Centresin various NIS countries. The NRECs were perceived as independent, non-partisan,international organisations promoting information exchanges between stakeholders onenvironmental issues. In Moldova a national REC was set up fairly quickly. In the othercountries (Russian Federation, Ukraine, Georgia) it took three long years before the NRECswere founded (see chapter 3.6);

♦ The Common Environmental Policies project provided long term resident EU expert support toMinistries of Environment in the NIS for National Environment Action Planning (NEAPs) relatedactivities. This project has made “a positive, be it small, contribution to NIS capacity building” inthe field of environmental management (chapter 3.4).

3.3 The performance analysis of the project portfolio, and the analysis of the TIPEprogramming cycle, support the following findings at general programme level.

3 Details on this categorisation of projects are provided chapter 2, table 2.1

TIPE evaluation


3.4 Programme relevance♦ Environment in the NIS has been a low priority for the Commission and for its NIS partner

states in the Tacis programme. Total funding for environment has been no more than 3% ofTacis resources at the maximum. In the Tacis national programmes environment was notconsidered a programming area before 1997. Thus the interstate programme has been themain channel for TA in this field.

♦ The Commission Services, and especially DGXI, have over the years not developed acommunication or orientation paper for EU environment policy towards the NIS. As a result theTIPE content is only to a limited extent influenced by EU environmental policies. The opposite isalso true, the experiences of the TIPE have contributed very little to shaping Commission policy,and strategies on environmental issues in the NIS as reflected in the PCA agreements.

♦ With the TIPE, the Commission has addressed environmental problems which are highlyrelevant, and are priorities from a global environmental heritage perspective (e.g. regionalseas). These projects, and the emergency relief programme for Chernobyl affected areas, canbe considered as truly of inter-state character. They are congruent with the identity of the TIP inthe sense that that the problems require solutions based on the principles of pooling resources,common approaches and technologies of the EU, other donors and NIS partners.

♦ The Commission has also used the TIPE to provide funding for projects which are less in linewith its inter-state objective (REAP, CEPs, WEAP/JEP) These projects have been given a“inter-state” coating, based on ascribed common problems and assumed efficiency gains in NISwide implementation. This inter-state rationale proved to be weak in the evaluation. Theevaluation team, however, recognises that the Commission had no options. Without access tothe Tacis national programmes, an inter-state disguise was the only way to provide Tacisprogramme funding for the projects originating from the Conferences of EU and NISenvironmental ministers in Dobris, Sophia, and Arhus.

3.5 Programme effectiveness, efficiency, impact and sustainability♦ High relevance does not imply success, the performance of the programme on key evaluation

criteria has been substandard. Assessed on a “high- medium-low” scale, the efficiency,effectiveness, sustainability & impact of programme outcomes was found by the evaluationteam to range between low and medium.

♦ This evaluation has not succeeded in clearly demonstrating the presence or absence ofprogramme impact. This is not surprising, of course, because most Tacis funded projects areelements of bigger projects, which aim to gradually bring environmental problems under control,and in due course put a stop to the environmental degradation process that is a heritage ofSoviet times in the NIS.

♦ As an alternative to ex-post impact measurement, the potential for impact has been assessedThis has been done on the basis of sustainability indicators for the different stages of the policycycle supported by the TIPE projects. The outcome for the six interstate projects assessed bythe evaluation team is presented in Table S.3.

TIPE evaluation


Table S.3 Potential for impact, as measured by progress achieved in different stages of thepolicy cycle

Stages in the policy cycle Project perormances

Stage 1: problem identification. This covers a range of activitiesincluding risk analysis, information supply and public awareness raising

Tackled well in allcases

Stage 2: policy formulation. This covers setting targets, mobilisation ofsupport at (Inter-)state level, the formulation and endorsements of ActionPlans and AP funding commitments

Tackled well in allcases

Stage 3: Implementation capacity development. This includes theestablishment of (Inter-)state organisations, supporting legislation andthe creation of organisations with a mandate for AP componentimplementation

Properly covered in50% of the cases

Stage 4: Implementation, and progress monitoring. This covers concrete“abatement” investments, enforcement of legislation and publicawareness raising aimed at changing the behaviour changes in targetgroups

Properly covered in50% of the cases

Stage 5: Impact and policy feedback. This comprises activities such as:measuring changes in the environmental status of the identifiedproblems, analysis of the results, and policy adjustment

Too early to measure

3.6 The table illustrates that the Tacis projects are mainly contributing to the preparation stagesfor environmental action. Few are designed to have an immediate impact on the problemsidentified. As such the direct impact is low. The projects do have a potential for impact in the longerrun, provided that the actions needed in stage three and stage four of the policy life cycle arefunded in due course. This will, however, require domestic budget and investment funds as well asTA and investment support from international donors. In this context it is important to note that itmay take decades to install an effective environmental management system for the regional seas.Likewise the initiatives originating from the Environment for Europe process deal with issues thatmay require decades of attention and co-operation before they are fully built into the operation ofsocieties in the NIS.

3.7 Programme cycle management issues♦ The Commission has neither a formal, nor an informal, interstate counterpart for the

programming of the TIPE resources. The Tacis national co-ordinators are consulted in theannual Interstate Action Plan preparation process but they have only a little influence on thefinal project selection, and project preparation.

♦ Government agencies for environment, including the Ministries of environment have neither aformal, nor a practical role in the project management cycle for the environment component inthe TIP. One Minister interviewed for this evaluation was quoted in saying: “No one knows whatthese programmes are all about”.

♦ The programme logic of the TIPE is generally sound, but some of the assumptions were foundto be invalid and this reduces programme performance. Firstly, the subsidiarity principlepresumes the existence of interstate level institutions as NIS partners for TIPE projects; thesedo not exist in the field of environment. Secondly the assumed additionality or synergy betweeninterstate and national programme projects does not materialise in environment. Thirdly, donorco-ordination does work at the policy formulation stage but is ineffective at projectimplementation level; and lastly, EU technology/know-how-based TA is not sufficientlysupplemented by access to investment resources and absorption capacity development in theNIS to have impact.

TIPE evaluation


4 Overall conclusions

4.1 For the public sector in the NIS environment is at present an issue of low priority. NIS wide,the environmental authorities have lost funding, staff and policy influence. Public opinion polls, andresearch papers (Zabelin, 1999), indicate that ten years after the fall of the Berlin wall,environmental considerations in the NIS play a smaller role in the economic-, and democraticreform processes in the NIS, than in the last years of Perestroika and Glasnost. A new generationhas grown up, which is less aware of the key environmental issues of our times, and in the NIS,then the previous generation. The international community has done very little to convince the NISthat they are wrong in neglecting the dismal environmental heritage of soviet times.

4.2 The TIPE evaluation shows that the absence of a strong EU stand on environmental issues(i.e. see low funding priority and the absence of a clear NIS oriented environmental policy), isexperienced by environmental authorities and civil society in the NIS as a negative factor. Thisfactor has, according to them, contributed to the weakened influence of environmentalconsiderations in the NIS policy decisions4. Moreover this is seen as a barrier for focussing localenvironment initiatives. In various TIPE projects this has led to wasted efforts and frustration.

4.3 Weighing up the findings, the evaluation team concludes that the TIPE programme has onaverage provided low value for money, that better value was in practice achievable (see chapter4.8).

4.4 The above conclusion can be further qualified as follows:♦ The high relevance of the issues addressed by the programme is a good finding, however, this

has not been given much weight. In principle there is a shortage of TA support in environmentand in other sectors, and high relevance should be expected of the TIPE; The overall relevanceis weakened by shortcomings in programme design and design of individual projects (seechapter 2.3.2 and chapter 3).

♦ The low/medium potential for impact does count heavily in the final assessment, A majorproblem for impact has been the weak ability of project stakeholders to operate at interstatelevel. In most cases the necessary interstate organisations do not exist and the TIPE projectefforts to establish interstate structures based on national organisations have failed to evolvebeyond the level of activities funded by the Tacis projects. Thus the recipient structure andpartner commitment needed to utilise and follow up on the TA offered by the Commission werefound to be largely absent.

♦ The low/medium level effectiveness and efficiency levels imply that more could have beenachieved.

4.5 Several factors account for this difference between high issue relevance, and medium/lowperformance on the other programme evaluation criteria:♦ Adverse economic and political developments have had a negative effect on the commitment of

NIS partners to environment in general, and to interstate issues in particular (see the projectsynthesis reports and chapter 4.2);

♦ The absorption capacity in the NIS public sector for interstate TA in environment has declined,rather than increased in the period 1991-1999 (see chapter 4.2), in particular the previousknowledge infrastructure has been de-capitalised, and the new legislation and institutional setup are weakly enforced and under funded;

♦ The EU environment policy framework provides little specific guidance for TIPE programming(see section 2.2.2);

♦ Programme design has suffered from serious weaknesses, including overly ambitious generalobjectives and insufficiently specified target outputs and target impact. (these are on a projectby project basis discussed in detailed project synthesis reports);

♦ The Commission has lacked both good quality information on project implementation, and themanagement resources needed to adequately manage the TIPE project portfolio and fundinglevels (see chapter 4.6).

4 This finding is substantiated in annex F - interviews notes

TIPE evaluation


4.6 This last point confirms the, by now well known, and in several other evaluation studiesdocumented, limitations to the efficiency of the Commission as an aid provider to the NIS in the1990’s. In particular the evaluators find that the Commission has created insufficient capacity andinstruments for proper management of the TIPE5.

5 Recommendations for the period 2000-2006

Recommendations on the general policy setting for environmental initiatives5.1 The starting point for these recommendations is the undisputed fact that natureconservation and environmental protection in the NIS are essential for the ecological health of ourplanet and its people both in the NIS and in the Europe. At present structural conditions in the NISare still worsening instead of stabilising, let alone improving. It is in the long term interest of both theEU and the NIS that environmental considerations become fully integrated in the economic anddemocratic reform processes and resource allocation decisions by the main sectors in society:parliament, government, commerce and the public/civil society.

5.2 In this context the following priority actions/conditions can be identified to generate, in theperiod 2000-2006, more positive impact of the co-operation between the EU and the NIS partnerstates on the state of environment in the NIS.1. To clarify the policy context, an orientation paper (by DG XI) would be needed on NIS

environment and relevant EU/Commission policies.2. A joining of forces between the EU parliament, civil society and Ministries of Environment in the

EU and NIS would be needed to mobilise more support/constituency for EU-NIS co-operation inthe field of environment

3. The EU-NIS partners would need to formulate a common set of goals for co-operation in thefield of environment, and to monitor progress through PCA sub-committee meetings;

4. An EU/Commission decision would be needed to fully integrate environmental considerations inits co-operation activities including the Tacis programme project cycle

5. A Commission/Tacis programming decision would be needed to provide a fixed allocation ofnational programming resources for environmental management capacity building in the NISand policy harmonisation with the EU and WTO-related environment issues.

Recommendations on the programming context of TIPE5.3 To make an impact with the programme on the key (inter-)state environment issuesidentified in the period 1992-1999, the Commission should provide continuity, take a stronger lead,and make an effort to increase absorption capacity in the NIS. This will require substantially higherfunding levels for environment then in the past, including higher funding for the TIPE.The rationale for this recommendation is that the Tacis programme and in particular the TIPE areessential instruments for keeping interstate environment on the agenda in the NIS and forimplementing the international commitments towards environmental protection and sustainabledevelopment made by the EU and NIS. Moreover, especially in the field of natureconservation/biodiversity, some of the main problems on the former territory of the Soviet Union(e.g. the regional seas) require an interstate approach. To enhance NIS absorption capacity,institutional support (by means of resident EU experts and twinning arrangements) for environmentauthorities (Ministries) should be a priority for the coming decade. This agenda should includeharmonisation of policies, licensing and inspectorates, environmental funding etc. What is needed isa essentially a similar approach as in the Phare countries, without the procedural detail of theAcquis Communautaire.

5.4 The impact of TIPE can be increased, if the Commission further clarifies its environmentpolicies and strategies towards the NIS as a whole, and towards specific interstate issues inparticular.

5 This conclusion is reached in various previous Tacis programme evaluation studies commissioned by the

Evaluation unit since 1996. See also section 4.6 of the main report for a quotation from a memorandumfrom the EAP Task Force Secretariat for the House of Lords (UK), 1998.

TIPE evaluation


5.5 The effectiveness, and efficiency of TIPE projects can be increased through synergy withnational level actions funded by e.g. the Tacis national programme. This requires that theCommission integrate environment as a horizontal, cross-sector theme in the Tacis nationalprogramming framework.

5.6 The impact of the TIPE projects, and especially the support for the regional seasprogrammes, can be increased by adding an investment programme to the TIPE instruments.Phare has some experience in this matter amongst others with the MUDP project, in whichCommission funding makes it possible for municipalities to take soft loans from the EBRD. (seechapter 3.2.2)

Recommendations on TIPE programming aspects5.7 Introduce a separate Indicative Programme for the environment component of TIP, andprovide continuation in the programming by distinguishing seven sub-components with separateblock allocations for funding.♦ The Environment for Europe Process (EEP),♦ The Regional seas conventions,♦ Other international conventions,♦ NIS -internal - cross border programmes.♦ NREC annual workplans♦ Investment support♦ Institution/capacity building for national, regional, and municipal environment authorities.

5.8 Review the funding requirements with annual or two yearly Action Plans.

5.9 Identify programming and implementation partners for each of the components in theIndicative programming phase, and delete the overall consultation practice with the national co-ordinators.

Recommendations on programme cycle management aspects5.10 The main report, in chapter 7, contains a number of recommendations based on observedstrengths and weaknesses in the TIPE programme management cycle. They cover the following:♦ NIS wide project contracts are inefficient, this can be solved by identifying smaller regions for

TIPE programming♦ Task Manager(s) are overloaded, this can be overcome by allocating more management

resources to the TIPE in Brussels and by de-concentration of management functions.♦ Project selection/formulation criteria are good but are not strictly applied. Stricter application can

prevent mistakes: ”projects should only be funded if they include clearly identified counterpartsable to oversee the implementation of projects on an interstate basis”.

♦ Many projects lack clear focus. More attention should be given to designing for impact. Betterexperts, and/or deeper analysis, is needed in project formulation and TOR writing

♦ Interstate projects are complex, tougher contractor selection criteria are needed♦ Task managers need better quality information on project progress and problem analysis. This

can be done by organising Tacis monitoring resources on a project rather than on a regionalbasis, and by stricter enforcement of project internal monitoring, and progress reporting.

TIPE evaluation


1 Introduction

1.1 Background

The era of Soviet central planning left an appalling legacy of environmental problems, with whichthe NIS are only now beginning to come to terms. In the 1st decade of technical co-operationbetween the NIS and the EU, the Tacis Interstate programme has been the main vehicle forpromoting environment projects under Tacis. Over the period 1992-1997, on which this study isfocused, Tacis committed around 52,5 Million Euro to the TIPE. Other Tacis instruments, such asthe national programmes and the cross border programme have also committed funds forenvironment projects but on a smaller scale.

This report sets out the evaluation of the Tacis Interstate Programme (TIP) in Environment,hereinafter referred to as the TIPE. The TIPE is the environment component of the Tacis InterstateProgramme, which is one of instruments developed by the European Commission for providingTechnical Assistance (TA) to the NIS and Mongolia6.

The Evaluation Department in the SCR of the European Commission has commissioned theevaluation by means of a restricted tender. The Netherlands Economic Institute (NEI) hasundertaken the study with a team of experts from NEI, Mobilisation for Environment (MOB), andMilieuKontakt Oost Europa. The major part of the fieldwork was done in the period 2nd September1999 – 2nd December 1999. An interim report was submitted at the end of November, and the draftfinal report was completed in February 2000. The findings and recommendations of the interimreport and draft final report have provided the Commission with inputs for the TIP IndicativeProgramming process. The final report of the evaluation was completed at the end of June 2000and is updated to the end of March 2000.

1.2 Structure of the report

The report is divided in six chapters. The introductory chapter lists the key questions raised by theTerms of Reference, outlines the methodology used, and concludes with acknowledgements.Chapter two describes the programme, and presents an assessment of the TIPE programmedesign logic, with reference to the key questions. Chapter three summarises the findings of the casestudies for six projects, which together account for the major part of the TIPE funds utilised in theperiod covered by the evaluation. In chapter four, the analysis moves from the project level upwardsto questions on programme relevance, effectiveness, efficiency, sustainability and impact. Chapterfour, moreover, deals with project cycle management issues. Chapter five, subsequently, presentsthe conclusions and lessons learned. The report concludes with recommendations in chapter six.The recommendations cover issues of continuation, new priorities for programming, programmedesign issues, and project cycle management aspects. The report has annexes on methodology(annex A); experts who contributed to the evaluation, and prepared the study outputs (annex B); alisting of people met (annex C); and documentation used (annex D).

1.3 Evaluation objectives and key questions

The evaluation provides an analysis of relevance, effectiveness, efficiency and impact of thepolicies and strategies adopted by the European Commission in the framework of the TIPE 1992-

6 The two other components of the Tacis Interstate Programme, besides environment are: Networks

(TRACECA, and INOGATE), and Justice and home affairs.

TIPE evaluation


1997 Action Plans. The evaluation has a policy orientation and provides lessons learned, andorientations for programming under the New Tacis Regulations for the period 2000-2006.

The orientation and main aims of the evaluation are mentioned in the Terms of Reference of thestudy.

The objectives of the evaluation

“The evaluation will mainly focus on the analysis of relevance, effectiveness, efficiency and impactof the policies and strategies adopted in the framework of the Tacis 1992-1997 InterstateProgrammes for environment. Therefore the main aims of this evaluation exercise are as follows:♦ To provide recommendations to the Indicative Programme 2000 and next programming

exercise with regard to possible policy and strategic reorientation, resource allocation patternsand strategies. The consultant should also make recommendations on possible continuationfor on-going activities.

♦ To suggest possible improvement in the programme cycle to enable the adoption of a moreeffective decision making, planning, and management process ( however, clearly indicating ifthese improvements refer to Tacis overall programme cycle and needs to be dealt with in thelarger context or if it is improvements that can be undertaken only for the Tacis InterstateEnvironment)”

Source: Excerpt from the Terms of Reference for service contract.

With reference to the above objectives, the following key questions for the evaluation wereformulated after further discussions were held with Commission services on the aims of theevaluation, and after a review of programme and project documentation:

Key questions for the evaluation

Programme design analysis♦ To what extent has the TIPE a distinct identity, vis-à-vis other Tacis support instruments,

which is of operational guidance for programming?♦ What were the main features of the TIPE policies and strategies during the period 1993-1999,

and what were the dynamics in their evolution?♦ What have been the central assumptions and logic behind the environmental projects in the

TIPE annual Action Plans for the years 1993-1997; How valid did this TIPE programme logicprove to be, and why?

Programme relevance♦ To what extent does the TIPE implement EU and Commission environment policy towards the

NIS?♦ To what extent has TIPE succeeded in identifying, and selecting for support, those priority

environmental problems, which can be successfully addressed only on the basis of aninterstate co-operation approach?

♦ To what extent have TIPE projects addressed environmental problems which can beconsidered relevant in terms of their NIS environmental priority, and their political priority in theNIS and EU?

♦ To what extent has the TIPE succeeded in addressing problems for which the Tacis TAapproach and EU experience has value added to offer, compared with support from bilateral,or other multilateral aid channels?

♦ To what extent can one consider the TIPE a relevant programme, when weighing up theabove dimensions of relevance?

TIPE evaluation


Programme effectiveness♦ To what extent did the completed TIPE projects achieve their project objectives; and to what

extent can one, with reason, assume that projects, which were still ongoing during theevaluation (November 1999), will achieve their objectives?

Programme efficiency♦ How can TIPE project budget resources be assessed in relation to project results?♦ To what extent has the TIPE been an efficient channel for promoting solutions to

environmental problems?

Programme sustainability and impact♦ To what extent can project results be maintained or continued after the contractual ending of

Tacis funding?♦ To what extent has the TIPE succeeded in making lasting contributions, promoting solutions to

environmental problems, and what are these contributions?♦ What can be considered the impact of the TIPE programme. To what extent is this impact

bigger than the sum of project impacts?

Overall assessment of programme value added♦ Has the programme generated impact, to justify the use of Tacis funding, taken into account,

other funding priorities in the NIS and impact potential of other interstate and national levelprogrammes.

Programme cycle management♦ How effective have the TIPE programming procedures been for project selection?♦ Are project budgets generally generous or meagre for the required activities♦ How well are the projects designed for visible results♦ Are projects sufficiently designed for impact♦ To what extent have Commission procedures for transparency been applied in the tendering

practice for TIPE during the period covered by the evaluation.♦ Does the Commission have sufficient instruments to influence project implementation♦ Does the Commission have the sufficient good quality information on project implementation

to manage the projects and take responsibility for the results?♦ Does the Commission have the capacity to follow up on information from progress reports and

monitoring reports♦ To what extent does the commission use the feedback on project results for programming

purposes?

Future programming♦ Does the evolution of environmental policy development in the NIS offer a basis for an impact

reorientation of TIPE programming? If so, in what directions and to what extent?♦ What lessons does the evaluation provide for future TIPE programming and project design?♦ Which ongoing projects, or components of TIPE projects, warrant further support in the period

2000-2003?

1.4 Approach and other methodological aspects

A team of Dutch and NIS experts conducted the evaluation (see annex B). The study wasimplemented in four phases (see table 1.1)

In the course of the evaluation 16 outputs are produced by the EU and NIS experts of the team.These outputs, which are listed in annex B, served as background material for the project casestudies and synthesis reports. Six of the outputs are detailed project evaluation reports. Thesummary sections of those reports constitute chapters three of the main report.

TIPE evaluation


For the documentation review, the evaluation team contacted the Commission, the Tacis monitoringunits, and the contractors for the selected case studies.

Table 1.1 The time table and phases of the TIPE evaluation

Phase and activities DeadlinesPhase 1 – preparationsContract signature 2 September 1999First round of interviews with Commission services 10 October 1999Desk study 15 October 1999Start field work and NIS logistics 29 October 19991st phase report 1 November 1999Phase 2 – fieldwork and programming recommendationsCompletion field work by four evaluation teams 15 November 19991st drafts of conclusions on six project case studies 22 November 1999Interim report, presentation of preliminary conclusions andrecommendations

30 November 1999

Discussion on interim report with Commission services 2 December 1999Phase 3 – finalisation of project synthesis reportsFollow up interviews and data collection by NIS experts 1 February 2000Submission of six project synthesis reports 15 February 2000Submission draft final “synthesis” report 25 February 2000Phase 4 – dissemination and feedback of evaluation resultsFeedback from the Commission on the project synthesis reports 14 March 2000Follow up activities variousFinal report 30 June 2000

During the field work seven countries were visited including the Russian Federation, Ukraine,Moldova, Georgia, Azerbaijan, Uzbekistan and Kazakhstan. 188 people were interviewed ( the listof interview partners is provided in annex C). The following table gives details on the stakeholdergroups interviewed in the different countries. The lists includes Ministers of Environment, officialsfrom Environment Ministries, parliamentarians, journalists, NGO members, local TIPE project staff,EU experts and contractors, and staff of Commission services in Brussels and in the countryDelegations, and experts hired by the Commission for its network of Co-ordination units andMonitoring units.

TIPE evaluation


Table 1.2 Stakeholders and beneficiaries interviewedBrussels

RF

UK

R

MO

GE

O

AZ

UZ

B

KA

Z

AR

M

To

tal

Ministers of Environment/Tourisme 1 1 1 1 �Ministerial Staff 11 2 5 17 2 4 ��Regional Government Authorities 2 2 2 1 �Parliamentarians 1 1 �Media personnel 1 2 �NGOs members 8 3 2 12 ��Researchers 1 4 2 4 1 1 ��Commission staff /Delegations 7 3 1 1 1 3 ��Tacis co-ordination units 2 2 2 3 3 1 1 ��Tacis monitoring units 2 2 1 2 2 1 ��NIS TIPE project personnel 2 5 3 6 1 5 10 ��EU experts/contractors 1 1 3 �Donor Staff 1 5 7 3 ��

Total 7 35 22 15 52 11 20 25 1 188NB. Other details on methodology are provided in annex A.

1.5 Acknowledgements

The core team of the evaluation comprised: Max van der Sleen (team leader); Esther van der Meer,and Nick van der Lijn (all NEI BV); Imre Csikos and Johan Vollenbroek (both MOB); and Ton vanEck (MilieuKontakt). This team would like to thank all the NIS experts who made writtencontributions, for the various outputs produced, during the course of the evaluation, and whocontributed substantially to our understanding of the environment in which the TIPE is working.Special thanks go to Lyudmilla Ivanova of the NEI Moscow office, who managed to get us to theright place at the right time. Moreover, we wish to thank the Commission services in Brussels, andthe staff of the Tacis support network in the countries we visited, for their support, and for sharingtheir views on TIPE policies and strategies with us. Finally we would like to thank, the large numberof persons that we interviewed for their contributions. We hope that you will find your views on thepast performance reflected in this report, and that you can share our views on the future orientationof the TIPE.

TIPE evaluation


TIPE evaluation


2 Programme description and analysis

2.1 Introduction and the problems addressed by the TIPE

2.1.1 Introduction

The Tacis Interstate Programme for Environment (TIPE) is a component of the general TacisInterstate Programme. The environmental projects are programmed within the context of the TIP,and are found in the Action Plans (AP), which are annually prepared. These Action Plans are thebasis for the programme description and analysis presented in this chapter. The description in thischapter covers the programme objectives (section 2.2), as well as the programme logic (section2.3) embodied in the environmental policies and strategies that are mentioned in the Action Plans(AP). Moreover, the description summarises the portfolio content (section 2.4), and the mainorganisation and management features of TIP implementation (section 2.5). The chapter starts,however, with some background on the problems that are addressed with the TIPE.

2.1.2 The environmental problems in the Soviet legacy

The Soviet period in the former Byzantine Empire left a society with deep roots in an authoritariansystem of government, paternalism, civil obedience, and communal values. The Renaissance andthe civil code (contract law) of Napoleon which shaped West European civilisation, had little impactin Soviet society and this has left the NIS (with no tradition and thus) weak in terms of the rule oflaw and in terms of respect for individual civil liberties and rights.

Against this background the legacies of the Soviet Union in the field of environment can becharacterised as follows.♦ The vastness and diversity of the natural areas in the societies that have emerged from the

Soviet Union are of global significance and this makes nature conservation and environmentalprotection in these countries an issue of global significance.

♦ In the field of nature conservation the main challenge is to keep the inherited systems of naturereserves in tact; to keep the enforcement functioning during a period in which the integrity ofpolitical and administrative power is weakening and commercial opportunities for illegalexploitation of nature resources are booming.

♦ In the field of environmental protection the main challenge is to put a stop to the predatoryattitude of the Soviet type production, to halt the degradation of environmental conditions and toreduce pollution levels that endanger human health.

While these challenges are clear, the parties in society who are stakeholders in these issues arebadly equipped to address these problems effectively:

1.� The state systems of environmental protection are ineffectual in the NISDespite the fact that environmental protection became official policy in the Soviet Union in19727 at the same time as in western Europe, its elaborate protection system was flawed andhad little practical relevance because it put the authorities simultaneously in charge of using andprotecting a particular natural resource. Thus the system that has emerged is weak inperforming its main tasks: legislation, enforcement, dissemination and integration ofenvironmental considerations in decision making in society.

7 As part of the Soviet response to the UN Conference on Human Environment in Stockholm (which it

boycotted for political reasons).

TIPE evaluation


”The right hand was catching fish, and it task was to catch as much fish as possible, while the lefthand was supposed to watch the right had lest it should catch too much fish. But with an eternallyempty stomach, the head would normally order the left hand to stay behind the back.” (Zabelin,1998)

2. The political powers, presidential administrations and parliaments are dominated by interestgroups for whom environment is a low priority.

3. The new political parties that make up the Parliaments in the NIS have no tradition as civilsociety organisations, they have no fixed constituencies and thus are far removed from citizensand civil society organisations.

4. The private sector, is dominated by a business community which has its roots in former stateenterprises, who thrived on exploitation of natural resources and are now privatised in anbusiness environment in which they have virtually no competition and less social obligationsthen under Soviet times.

5. Citizens have emerged from the Glasnost and Perestroika period sensitive to environmentalissues and familiar with environmental action and disasters (Chernobyl etc.) but generally lackinformation on environmental issues and on health risks from pollution.

6. Civil society organisations and green NGO’s have weak roots in society, and in general havelittle to non influence on political decision making processes beyond the level of localadministrations. In Russia, the October revolution of 1917 initiated a period of fifty years inwhich citizens were not allowed to organise themselves outside the party and state apparatus.In the 1960’s dissidents and human rights movements began and in Russia the rock culture hadits influence, but only the Perestroika policies of Gorbatchev created the conditions for theemergence of civil society. Only since the middle of the 1980’s has civil society grown rapidlywith the human rights movement, the environmental movement and the press as the mainpillars. These organisations developed as adversaries to political and administrative powers andonly in the 1990 began to develop and use opportunities for co-operation with authorities andother segments in society.

7. The media are not fully independent and share with the citizens the lack of information onenvironmental issues.

8. Academic institutions and knowledge centres have lost funding, have weakened en in generallacked in the past the mandate to investigate and monitor pollution aspects

These are in a nutshell the issues, which are in one way or another addressed by the TacisInterstate Programme for Environment.

2.2 The objectives of the TIPE

The paragraphs 2.2.1 and 2.2.2 describe how the Commission’s thinking about the objectives forthe TIPE have evolved over time, and analyse to what extent these objectives reflect wider NIS andCommission policy towards environmental issues.

2.2.1 Description

The word environment was not mentioned in the first Tacis programme document “Action Plan”,signed on 2 August 1991 by the Commission and the USSR.

On November 1st,1992, the first Inter-State Action Plan was signed in Brussels. This documentrefers indirectly to support for environmental issues, in two places:♦ In Part II – central Asia and the Caucasus, section 3.4 - Government support, a project is

mentioned with the objective: “To provide institutional support in implementation of the

TIPE evaluation


requirements of the 1992 Bucharest Convention on the environmental protection of the Blacksea. This is an essential complement to action being funded by the Phare regional programmecovering Bulgaria and Romania, and, with Global Environmental Facility support, by theGovernment of Turkey (1 Meuro)”;

♦ In Part II, 3.6.3.1 which deals with “Water and Cotton”, a second action is mentioned with theobjective: “To determine the amount of sustainable irrigated land, for which, finally, availablewater resources must be determined, in competition with human and industrial users” (3Meuro). This project dealt with environmental issues in the Aral sea basin.

In the 2nd Inter-State Action Plan (1993), the concepts of sustainability and resource managementare mentioned, and for the first time a programme level objective is formulated:♦ To help strengthen the institutions that will manage the framework for sustainable development.

The Action Plan sets out two separate programme components for environmental issues: Under theheading “Environment and Water Resources” (section 3.3) the Commission provides 4.25 Meuro foran environmental support facility. Moreover, 6 Meuro is allocated for “support to Chernobyl affectedregions (Belarus, Ukraine, Russia)” (section 3.4).

The 1994 Inter-State Action Plan does not contain an environment component, presumablybecause of absorption capacity problems; the actions mentioned in AP (1993) had not yet beenstarted up.

In the 1995 Inter-state AP, environment has a prominent place. For the first time, it is mentioned asa concentration sector (section 3.2), and projects for an amount of 12 Meuro are outlined. Thedescription opens with the sentence that: “The era of Soviet planning has bequeathed an appallinglegacy of environmental problems in the NIS”. A number of key legacy problems are mentionedsuch as: nuclear contamination, airborne pollution, waterborne and marine pollution, and the needfor conservation of major natural ecosystems including the vast reserves of the Siberian Forest. Thedocument gives two arguments for making environment a concentration sector. One “within the NISsome steps are taken towards the creation of inter-governmental fora for environmental issues; andtwo the Dobris/Lucerne/Sofia initiative is helping to provide international focus through the adoptionof national environmental action plans”(AP 1995, section 3.2).

The broad objective of the 1995 TIPE is formulated as:♦ To support the development of capacity within the NIS, to address environmental problems

effectively.

In the AP (1996), the process of increased concentration and strategic focus is continued, andfurther arguments are given for support to environment (see below). The AP, however, does notspecify further programme level objectives for the concentration area Environment.In the strategy paper underlying the AP 1997, reference is made to three, main Tacis objectives. Inthe APs for 1998, and 1999, reference is made to the PCA and EU enlargement context of Tacis,and this is reflected in a further refinement of the objectives.

The evolution over time in the objectives mentioned in the TIP Action Plans for Environment isshown in the box below.

TIPE evaluation


Evolution in the Objectives for the environment component of the TIP

♦ To help strengthen the institutions that will manage the framework for sustainabledevelopment (1993)

♦ To support the development of capacity within the NIS, to address environmental problemseffectively (1995)

♦ To development institutional capacity in the participating countries, including an efficient legaland administrative framework (1997)

♦ To promote collaboration between the NIS and Mongolia in finding solutions to the most majorenvironmental challenges facing them (1998)

♦ To encourage the incorporation of environmental considerations into the process of economicreconstruction to ensure sustainable development (1997); and to international trade standards(1998, 1999)

♦ To establish immediate programmes to bring relief to regions where human health or naturalecosystems are severely jeopardised by environmental hazards (1997, 1998, 1999)

♦ To raise public awareness of environmental issues (1998,1999)♦ To promote investment local and international investment in environment projects through the

provision of technical assistance (1998, 1999)

From the above, four main objectives can be derived.1. Capacity building/institution building for integrating environment in policy decision making2.� Immediate relief programmes3.� Public awareness raising; and4.� Investment promotion

An indication on recent policy and strategies agreed on high political level can be obtained from aCFSP8 document from the European Council, Common Strategy of the European Union on Russia,Cologne 3-4 June 1999.

The document presents a Common Strategy for implementation. In the section Common challengeson the European Continent, it mentions that The European Union will co-operate with Russia in thefield of environment and health, including: Nuclear and chemical waste storage, precautions againstinfectious diseases, phytosanitarian controls, and:♦ By supporting the integration of environmental considerations in economic reform and by

assisting in the creation of effective systems for monitoring and ensuring compliance withmultilateral environmental agreements, and supporting Russian efforts to strengthen theenforcement of national environmental legislation;

♦ By working with Russia, especially in areas adjacent to the enlarging Union to reduce water andair pollution and to improve environmental protection and by co-operating on promotingsustainable use of natural recourses, in particular in the various fora for regional co-operation.

2.2.2 Analysis and findings

The strategy papers and APs for 1992-1997, are precise and clear, but they do not contain a cleardescription of the TIPE objectives and strategies pursued by the Commission. They thus give littleguidance for understanding the basis of TIPE programming. Policy and strategy areunderdeveloped elements in these documents, and very little detail is given on the programme levelobjectives, which the Commission wishes to pursue with the TIPE.

The TIP strategy and planning documents do not refer to the EU environmental policy documents,and TIPE programming is not based on them. In general the policy setting for the projects is weaklyformulated and the evaluation team has the strong impression that programme objectives havebeen formulated to cover the selected projects and not the other way around.

The Commission services (DGXI), have over the years not developed a communication ororientation paper for EU environment policy towards the NIS. One of the findings of the evaluation is

8 CFSP stands for Common Foreign and Security Policy.

TIPE evaluation


that this absence of orientation, is experienced by Tacis partners and environment stakeholders asa barrier for focussing local environment initiatives. This has in various projects led to wasted effortsand frustration.

Despite the to the absence of a clear NIS oriented environment policy, the Commission has playedan active role in international donor co-ordination, and especially in the Environment for EuropeProcess. TIPE Resources have been instrumental for funding projects initiated in the EEP and forfunding projects supporting international conventions

The PCA context provides little new guidance for the TIPE, apart from the link made between tradeand environmental standards, which is reflected in the TIPE objectives mentioned in the 1998, and1999 programmes.

The above cited environmental paragraph, in the CFSP document from the European Council(Common Strategy of the European Union on Russia, Cologne 3-4 June, 1999) gives a neworientation for environmental co-operation between the EU and the Russian Federation.The importance of integrating environmental considerations in economic reform policies isemphasised. Moreover, monitoring system capacity development and enforcement ofenvironmental legislation are new elements, which will require action at national level in addition tothe Inter-State level of the TIPE.

On the basis of this analysis of EU environment policy and TIPE strategies, and on the basis of thestakeholder interviews conducted in this evaluation, the evaluation team comes to the followingreflection on the evolution of NIS and EU environment policy development. In the period 1992-1997,political/policy interest in the NIS (and the EU) for the problem of environmental degradation in theNIS, has fallen to low levels. Lower then existed a decade ago, when the Soviet Union seized toexist as a legal entity, and the Tacis programme started. NIS wide, the environmental authoritieshave lost funding, staff and policy influence. Moreover, the knowledge infrastructure, which existedin academic institutions, has declined. The EU projects and other bilateral aid programmes have notsucceeded at reversing this form of decapitalisation. In this respect the EU policies towards the NIShave differed considerably from their policies towards the Phare/EU accession countries. In the co-operation between the NIS and the EU, the Tacis programme has given lip service only to the needfor integrating environmental considerations in the economic reform process.

On the basis of the above analysis the evaluation concludes that:♦ Despite statements in various EU-NIS cooperation documents and statements, the Commission

has not created the tools needed for effective support to the NIS in the area of national levelcapacity building and policy harmonisation in the field of environment.

♦ The TIPE has in the past focussed much more on immediate relief programmes (regional seasetc.), awareness raising and investment support, than on capacity building.

♦ The future prospects for TA funded by the Tacis programme for this key objective are not goodeither. Only in Kazakhstan is environment selected as one of the three concentration sectors forthe Tacis programme. In the other countries environment capacity building will have tocompete for funding with other sectors.

♦ Past experience shows that environment authorities in the NIS are weak players in suchcompetition and will require outside i.e. Commission support to obtain funding for their projectproposals.

2.3 Programme intervention logic

In paragraphs 2.3.1 and 2.3.2 the arguments given by the Commission for funding environmentprojects via the TIP, are summarised, and the soundness of the logic of these arguments isassessed against the evaluation findings of project performance.

TIPE evaluation


2.3.1 Description

“The Inter-State programme aims to assist the New Independent States (NIS) and Mongolia toidentify and pursue solutions to problems which are of an inter-state nature. As such, it operateson a basis analogous to the European Union's concept of subsidiarity since it only includesactions, which are best undertaken on a multi-country, rather than on a national, level. Projectsincluded in the Inter-State programme are designed to complement and reinforce thoseundertaken in the Tacis national programmes, Cross Border Co-operation and Small ProjectsProgrammes.”

Source: Quotation from the TIP Action Plan for budget year 1999.

The first attempt of the Commission to formulate a programme-logic for the environment componentin the TIP stems from the AP 1995. In that document the Commission states that, the Tacis Inter-State programme is an excellent vehicle for programmes to assist the NIS in the field ofenvironment. In support of this statement, the Commission gives three mutually supporting reasons:♦ The cross boundary dimension of the problems. “Most of the major environmental challenges

facing the NIS are of an interstate nature (have an interstate dimension), and requirecollaboration of the NIS for their solution”;

♦ Common causes of the problems: “ While there are obviously differences of emphasis betweenNIS, there is also an important commonality in the environmental issues stemming from thecommon former system of management of the Soviet economy; this gives scope for thedevelopment of common strategies and policy instruments; The Inter-State programme is anefficient means for promoting such common strategies”; and

♦ Common international interest in solving the problems. “For the major environmental issuesfacing Europe as a whole, a co-ordinated strategy encompassing the European Union, theemerging countries of Eastern and Central Europe, and the NIS together is required. The TacisInter-State programme could effectively help build such a unified approach9”.

The Commission, moreover, states that: “The programme will provide support to the NIS at both theinter-state level and the national level, with an emphasis on creating capacity in each participatingNIS to address the wide range of the most pressing problems. In this way it is intended that Tacisresources, which are obviously very limited in relation to the magnitude of the environmentalproblems faced, can have a maximum multiplier effect”.

The quote illustrates that Commission, at the time understood the complementarity of inter-statelevel and national level support, and was prepared to use TIPE for national level actions, given theabsence of alternative channels of Tacis funding. The evaluation team further observes that theCommission sees a link between capacity building and multiplication of effects, given a situation ofscarce resources. Probably maximum impact is meant and not multiplication of resources.

In the AP (1996), the policies and strategies for environment as a concentration area for the TIP arefurther detailed, and in particular three new reasons are given for support to environment:♦ “European expertise in environmental technologies and industrial processes justify the

involvement of the EC in this area”(section 5.5.3); and♦ The pan-European/NIS environment Ministers Conferences launched in the context of the

Environment for Europe process have provided the political commitment to environmentalactions in the NIS and have produced agreement on an Environment Action Programme to beimplemented throughout the NIS;

♦ “Previous Tacis activities, in this field have encouraged other donors to pledge their resources(e.g. co-financing of the Regional Environmental Centres by the US). Further actions will builtupon this collaborative approach” (section 5.5.6).

In the AP (1997), a number of shifts in policy situation in the NIS are mentioned which areconsidered unfavourable for Inter-State co-operation. In response to this the following policy isannounced: “The emphasis in the 1997 programme will be on initiatives, which are technicallyfeasible and well justified and thus more easily insulated from the shifting political relations amongstthe NIS. The emphasis will be on exploiting concrete co-operation between the NIS where it is 9 For environmentalist, Europe encompasses the European sub-Continent West of the Ural mountains, or

for bird migration purposes, West of the Lena river)

TIPE evaluation


technically possible rather than engaging in the politicised process of NIS wide institution and policybuilding”.

In the 1997 AP the argumentation for selecting the environment sector as the 2nd concentration areaof the TIP (Networks being the 1st one), remains more or less the same as in 1996.

The programme logic of the TIPE which is implied in the above, can be summarised as follows:1 The Soviet approach to industrialisation and agriculture has left a legacy of environmental

degradation, which is at a critical level;2 The critical level poses a long term threat not only to the NIS population but also to the

populations of neighbouring regions including the EC;3 Most of these problems, are of an interstate dimension and have causes which are of an

interstate nature. They require collaboration between the NIS for their solution (subsidiarityprinciple);

4 The European Continent dimension gives the EU an interest in helping the NIS find solutions tothese problems;

5 Europe has the expertise in environmental technology and industrial process needed to findlong term solutions;

6 The NIS political commitment to environmental action has increased as a result of theEnvironment for Europe process, and agreement has been reached on developing andimplementing a unified approach to environmental action programming, priority setting andimplementation;

7 Previous Tacis actions in this field have prompted other donors to pledge their own resourcessupport has been successful and this has prompted other donors to pledge their funds. Furtheractions will build upon this collaborative approach, and thus contribute to bring about theneeded solutions


The programme logic seems generally sound, but it has some notable weaknesses:♦ The subsidiarity principle presumes that Inter-State institutions exist on which project support

can be built. The evaluation shows that has not been the case, despite that fact that the TIPEprojects have extended substantial support to maintain or establish new consultation and co-ordination instruments;

♦ A second assumption, which has proven valid but is not realised with the TIPE, is that Inter-State-Actions needs to be complemented by national level action to become fully effective. Inthe case of environment, this national level action has been missing, because on the one handTacis does not consider environment a priority field for its Country Programmes, and becauseunder TIP the involvement of national level environment authorities has been insufficient;

♦ Donor co-ordination has worked well on policy and strategy level, but the evaluation shows thatthis has not worked well on project implementation level;

♦ The technology available in the EU is indeed relevant for the NIS, but without NIS access toinvestment sources and with insufficient project implementation capacity, the TA supportprovided via the TIP can have little impact.

2.4 Programme content

In section 2.4.1, the project portfolio of the TIP and its environment component is given in a tabularform. In section 2.4.2, the portfolio is analysed with the aim of answering three questions:♦ To what extent does the project portfolio reflect the objectives and programme logic as laid

down in the programme documents;♦ To what extent is there a consistency in the project portfolio in terms of themes compared with

ad hoc issues; and♦ To what extent does the project selection correspond with the selection criteria laid down in the

programme description in the TIP Action Plans.

TIPE evaluation


2.4.1 Description

The project portfolio of the TIPE is presented below. The data are taken from two sources:♦ The contract register from the Desiree (CRIS) data base, and♦ A document with the title Tacis Inter-State Programme, Status of projects1992-1999.The latter document summarises the projects and budget allocations published in the variousInterstate Action Plans.

The table illustrates that type of activities funded under the TIP has changed over time. The TIP hasserved as a nursery for many new programmes, which later became separate facilities (e.g. EES,Twinning, Lien etc). The current composition of the TIP started in 1996. As of that programmingyear the TIP has three components: Networks, Environment, and “Third pillar issues” i.e. Justiceand home affairs. Also, as of that year the environment component has received approximately onethird of the funds. The percentage and amounts fluctuate per year, which is an indication that theenvironment competes with the other components for Tacis funding. The Commission does not usea fixed distribution key for TPE components funding.

Table 2.1 Environment projects and budgets in the TIP Action Plans for 1992-1999Action Plans and Budget allocations in Million EuroEnvironment projects, and 7,3

FRPSRQHQWV 92 93 94 95 96 97 98 99 TotalRegional seas programme 5 5.00Black sea environment 1 2 3 6.00Danube river 1 1.00WARMAP10 3 1.75 2.5 7.25Environmental Support Facility 4.25 4.25Caspian sea 3 3.00Support to Chernobyl-affected regions 6 1.5 7.50CEPS and Environm. Action Plans 4 3 7.00Widening Environ. APs (WEAP) 3.7 3.70Raising public awareness (REAP 3 2.5 5.50Regional Enviro. Centres (NREC) 2.4 3.5 5.90Info. system for agro/env monitoring 2.5 2.50Joint environment programme (JEP) 5 5.00HACCP Food testing centre 2 2.00Bio-diversity in the west TienShen 1.5 1.50Trans-boundary rivers project 4 4.00Co-operation EEA – NIS 0.4 0.40Total Environment in TIP 4 12 0 12 12.5 12 10.5 8.5 71.50Total networks 11.5 20 10 14.5 27.5 22.5 26.8 19 151.80Total Justice & home affairs 0.5 3 3 3.5 10Sub-total TIP “new style” 15.5 32 10 26.5 30.5 37.5 40.3 31 233.30Enterprise restructuring 4 15 5 4 28.00Human resource and facilities 7,5 20 17 23 67.50Total others 1.63 5 15 12.5 1.5 2.5 1.7 39.80Total TIP Action Plans 28.6 72 47 66 42 40 42 31 368.60

Source: Tacis Inter-state Programme Actionplans 1992-1998


In this evaluation the composition of the TIPE portfolio has been analysed from three perspectives:♦ The policy cycle logic in the TIPE project portfolio♦ The main themes/objectives recurrent in the TIPE project portfolio♦ The extent to which the projects meet the project selection criteria

Perspective 1: The policy cycle logic in programmingThe programme cycle logic in the TIPE portfolio is shown in Table 2.2 below

10 WARMAP was classified under agriculture in 1992.

TIPE evaluation


Table 2.2 shows that the projects supported by the TIPE cover most elements important to policycycle, Notable is the conclusion that support for national level capacity building has been limited.This finding is all the more remarkable in view of the evolution in TIPE programme objectivesanalysed above (see section 2.2.1), which shows that capacity building is the main objective of theTIPE. This finding is also remarkable because in the Phare programme of the Commission, thisaspect was the main focus of the activities. It covers: Policy development, Implementation(instrument development. legal framework development, licensing), Monitoring & Enforcement, andPolicy Integration. The explanation for this finding is that TIPE is first an interstate programme andhas focussed on interstate issues. The fact that the solution for these issues is intimately linked tocapacity building at national level is recognised but has not been acted upon in TIPE programming.The project evaluations summarised in chapter three, show that this limited attention to nationalcapacity building has negatively influenced the effectiveness of the TIPE projects.

Table 2.2 Policy cycle logic in the TIPE portfolio

Policy cycle elements Status Relevant TIPE projects

Problem identification Done International Conventions, CEPs, and sea projectsStrategy formulation Done Studies under various projects since 1992Awareness Raising Ongoing Focus since 1995 with REAP and RECPolitical commitment building Done Environment for Europe process and Donor co-

ordinationCapacity building (ministries) Limited PMU Ukraine, NEAP II resident advisors,Investment promotion Ongoing CEP, WEAP, JEP, NEAP IIImplementation support Limited Project Implementation Units for sea projectsImmediate relief programmes Ongoing Seas projects, Chernobyl

Perspective 2: The main themes in the portfolioThe main themes in the TIPE portfolio are listed in Table 2.4. presented at the end of this chapter.The table provides details on which projects belong to which theme (objective); how much moneywas allocated to the theme, and what International policy dimensions are addressed.

The table shows that the Commission has implemented with the TIP its strategy of “fewer projectsfor more effect”. The table, moreover, shows that TIPE funding has provided continuation on mainthemes and projects.

The concentration on themes can be illustrated by aggregation of the data on the four mainobjectives mentioned for the TIPE in the APs .

Objectives Funding 92-97 Funding 98-99Public awareness raising 8.4 Million Euro 3.5 Million EuroInvestment promotion 3.7 Million Euro 8.0 Million EuroIntegration of environment 6.1 Million Euro 0.0 Million EuroImmediate relief programmes♦ Seas programmes 26.8 Million Euro 0.0 Million Euro♦ Chernobyl 7.5 Million Euro 2.0 Million Euro♦ Transboundary rivers 0.0 Million Euro 4.0 Million Euro♦ Biodiversity 0.0 Million Euro 1.5 Million EuroTotal 52.5 Million Euro 19.0 Million Euro

The tabulation makes the change in programming since 1997 evident. Noticeable is the absence offunding for the regional seas programme. The evaluation observes that this reflects real absorptioncapacity problems as well doubts in the Commission about the effectiveness of the assistance.Other changes are the increased emphasis on investment promotion and the absence of specificnational public sector focussed capacity building projects.

This evaluation deals with the projects funded in the programming period 1992-1997. The otherprogrammes have only just started operations.

TIPE evaluation


Perspective 3: The role of the selection criteriaThe APs have over the years mentioned the following selection criteria, which are in line with theprogramme logic developed for the TIP:1 Projects should relate to issues and problems at the Inter-State level for which solutions require

common approaches and collaborative actions by several NIS (the subsidiarity principle)2 Proposals are eligible only if forwarded/supported by two or more partner countries3 Projects should, where consistent with the orientations set out in the strategy papers and AP,

build on existing Tacis Inter-State initiatives started under previous programmes.4 Projects selected should, as far as is practical, reflect a certain degree of geographical balance;5 Projects should only be funded if they include clearly identified counterparts able to oversee the

implementation of the projects on an inter-state basis.

Table 2.3 illustrates the extent to which the TIPE programmes (sets of projects with the samegeneral aim) meet the selection criteria. The numbers of the selection criteria correspond to the listabove.

Table 2.3 The TIP project selection criteriaTIPE programme component Projects Selection criteria

1 2 3 4 5Public awareness raising REAP/REC L H M H LInvestment promotion WEAP/JEP L H H H LIntegration of environment CEP M H M H LImmediate relief programmes Seas/Chernobyl H H H H M

Legend: H = the extent is High; M= Medium; L = Low.

The table shows that the five selection criteria have not played a decisive role in the built up of theTIPE portfolio. The 1st criterium, common problems, that need co-operation for solutions atinterstate level, is valid only for the immediate relief category of projects, i.e. the regional seasprogrammes, Chernobyl affected areas, and Transboundary rivers. The projects also score very lowon the 5th criterium of inter-state bodies, which can supervise implementation.

This analysis leads to the conclusion that the TIPE has been used to fund projects, which do notfully correspond with the criteria laid down by the Commission for the programme.Given the funding for Investment promotion type projects, the evaluation team concludes that thispractice has increased in the budget years 1998 and 1999.

2.5 Programme organisation and management

2.5.1 Description

Separate organisation and management arrangements are not made for the TIPE. The Commissiondeals with the environment component, in the same way as the rest of the TIP. The generalarrangements are described in the APs under the heading Programme Implementation.

The TIP programming cycle differs from the national/country programming cycle with respect to thefact that no Multi-annual Indicative Programmes are made��. Programming is done on an annualbasis and starts with a TIP strategy paper, which is prepared by the Commission. This forms thebasis for the preparation and submission of project proposals. The proposals are discussed inprogramming Missions undertaken by Commission staff in the NIS and Mongolia. On that basis acomprehensive document is prepared and discussed in an (often stormy) wrap-up session with thenational co-ordinators (CUs), which results in agreement on the AP contents. On that basis thefinancing proposal is made for submission to the Tacis Committee Meetings in either June or July ofthe budget year.

The programming time table is tight and runs throughout the year.

11 A multi-annual AP was written for 1995-1996.

TIPE evaluation


The programming process of the TIP financing proposal (example AP 1997)Preparation of the TIP strategy paper: September – December 1996Strategy paper translated and distributed to Co-ordinating Units (CUs) January 1997Deadline for project proposals 21 MarchProgramming mission to the NIS March/AprilProject selection/Finalisation of Project descriptions 11 AprilInter-Service Consultation/Internal distribution 11 AprilTranslation and distribution to Co-ordinating Units 30 AprilInter-State Committee Meeting (in Brussels or NIS location) MayAP/ (Financial proposal) preparation JuneProgramme translation/distribution to Tacis Committee Early JuneTacis Committee meeting, endorsement of the AP July 1997

For the different aspects of programme implementation such as tendering, contracting, monitoringand audits, standard Tacis procedures apply to the TIP.


The evaluation team has the following observations to make on the programming and managementprocedures that are applicable to the TIP and its environment component.

The Commission itself, has over the years taken an active lead in the identification and formulationof the proposals. In fact very few of the environment projects in the portfolio have directly originatedfrom the NIS itself during the programming process.

The project selection criteria are good, and allow the NIS partner to come forward with goodproposal, nevertheless Commission support for a proposal is a precondition for success. In thiscontext an important factor is that, the TIP is not a subsidy channel for projects. It is an instrumentfor a policy dialogue and subsequent support aiming at common objectives.

The majority of the strategic themes addressed by the “TIPE” find their origin in donor co-ordination,and international conferences, and the Commission initiates projects on that basis. Some 80% ofthe environment projects funded in the period 1993-1997, are related to the Environment for EuropeProcess and other international programmes with large funding by international donors.Approximately 20 % of the TIPE funds are used for projects for which the NIS themselves havetaken the initiative.

The orientation of the TIPE on strategic themes is a result of pro-active programming and lobbyingby the Commission. It justifies this with reference to the fact that the NIS are partners in theEnvironment for Europe process and have participated in the other international conferences wherethe project concepts have emerged. This is a valid point, but it reduces the scope for new initiativesfrom regional groups in the NIS. Moreover, their negotiating power is limited, and over time hasbecome even more limited in parallel with the increasing political differences between the NISmembers. This has reduced their willingness to form a common front towards the Commission. Inthe project selection process and the final Inter-State Committee meeting (wrap-up session) wherethe final programme is approved, the Commission chairs the meeting, and more often than not theparticipants do not see eye to eye on the selection. Diplomacy, lobbying and some power playshave at different times in the past characterised this instrument for NIS co-operation.

The APs mention that the representatives of the NIS Governments have the overall co-ordinatingresponsibility for the implementation of the Inter-State programme, but this means nothing. Theinterviews conducted by the evaluation team with CU staff in the six countries visited, clearly pointout that the CUs feel completely unable to take this responsibility. They are ill informed, and theirco-ordinating responsibility is not specified in operational terms. For example they have no mandatetowards the contractors. The CUs themselves do not consider that they are in any way responsiblefor the TIPE actions in their countries.

TIPE evaluation


A, perhaps, specific problem for the TIPE is that the institutional setting of the projects in the NISand Mongolia is very weak. There is no functioning CIS body that could, at interstate level, takeresponsibility for implementation, and also at national level, clear partner organisations are lacking.Thus the subsidiarity principle which is featured so prominently in the programme design logic is notmatched at the programme implementation and management level.

Government agencies for environment, including the Ministries of environment have neither aformal, nor an informal role in the programming process for the environment component in the TIP.One Minister interviewed for this evaluation was quoted in saying: “No one knows what theseprogrammes are all about”

The evaluation team concludes that: In this setting, the responsibility for programming,implementation and management of the TIPE clearly lies with the Commission, and its contractors,and not with government representatives in the NIS and Mongolia. NIS authorities have very littleinfluence on programming and only limited influence on the implementation. This finding points at aserious weakness in the TIPE programme design. Moreover the evaluation shows that thisweakness has a negative influence on the ability of the project partners to solve implementationproblems.

2.6 Conclusions

1 The TIPE operates in a weak EU/Commission policy setting towards NIS environment2 In the TIP strategy papers and APs little attention is paid to elaborating an environment

programme strategy3 The programme logic and selection criteria formulated in the APs are not consistently applied

and appear to have plaid only a marginal role in project portfolio development.4 The proclaimed demand driven approach or dialogue based approach is not reflected in the

TIPE management structure5 The programme content has focussed on public awareness raising, immediate relief

programmes. and lately on investment promotion rather than on capacity building forenvironmental management.

TIPE evaluation


Table 2.4 The TIPE policy matrixTIP budget years

Policy setting Donors Themes/objectives Main TIPprojects

92 93 95 96 97 Sub-total

98 99 Projecttotal

Themetotal

A. The environment for Europe process

USEPA/AID Information & public participation NREC 2,4 2,4 3,5 5,9NGO support ESF 0,5 0,5 0,5

SOPHIA &ARHUSConvention Public awareness raising REAP 3 2,5 5,5 5,5 11,9

0 0USEPA/AID Environmental Action Planning REF/CEP/EEA cooperation 1,75 4 0,4 6,15 6,15 6,15

DOBRIS, EAP WB/EBRD Investment promotion WEAP/JEP/NEAP-II 3,7 3,7 5 3 11,7 11,70 0

B. The regional seas programme 0 00 0

WB Black sea environment programme RSP/BSEP 1 1,5 2 3 7,5 7,5 9,5BucharestConvention WB Danube river programme Danube river 1 1 2 2

0 00 0

Reg. Initiative GEF Aral basin environment programme RSP/WARMAP 3 1,75 1 2,5 8,25 8,25 12,25Info systems for Agro/env Monitoring Monitoring 2,5 2,5 2,5RIO, agenda

2000 Biodiversity Tsienshen Tsienshen 0 1,5 1,5

0 0Reg. Initiative UNDP/GEF Caspian sea programme CSEP 1 2,5 3 6,5 6,5 6,5

0 0C. Other "immediate" relief projects 0 0

0 0TIP Chernobyl affected areas Chernobyl/HACCP food 6 1,5 7,5 2 9,5 9,5TIP Transboundary rivers Transboundary rivers 0 4 4 4

Total TIP budget allocations for environment 4 12 12 12,5 12 52,5 10,5 8,5 71,5Source: Information compiled from TIP actionprogrammes 1992-1999

TIPE evaluation


TIPE evaluation


3 Analysis and findings at project level

The evaluation team undertook a detailed assessment of six projects, which together account for80% of the TIPE funding in the period 1992-1997. A report was prepared on each project and therespective findings and recommendations are summarised in this chapter. The six projects were:3.1 Aral Sea Programme (WARMAP)3.2 Caspian Sea Environment Programme (CSEP)3.3 Black Sea Environment Programme (BSEP)3.4 Development of Common Environment Policies in the NIS (DCEP)3.5 Raising Environment Awareness and Developing Environmental Media (REAP/TEAP)3.6 The project to establish New Regional Environmental Centres (NREC)

3.1 The “Aral Sea” Programme

The Aral Sea will die without change in policy

3.1.1 Introduction

The ongoing Tacis project WARMAP (Water Resources Management and Agricultural Production inthe Central Asian Republics) is designed to support the GEF sponsored Aral Sea BasinProgramme.

The problem. The drying out of the Aral sea, and the resulting destruction of the ecosystem,biodiversity and economy of the Aral sea Basin and its inhabitants, is one of the most glaringenvironmental disasters of our time. The root of these environmental problems can be traced to themassive expansion of cotton based irrigation in the period 1929-1991.

Regional initiatives. In 1992, Following independence, the Central Asian States involved(Kazakhstan, Kyrchistan, Tadjikistan, Turkmenistan, Uzbekistan) established an InterstateCommission for Water Co-ordination (ICWC), The ICWC was made responsible for management ofthe annual water allocations for each state and the schedules for the operation of the reservoirs.The River Basin Organisations (BVOs) were maintained and given the task of carrying out ICWCdecisions. The ICWC meets quarterly or whenever the need arises. Its decisions are binding on allfive states. In order to co-ordinate all regional water and environment related issues theInternational Fund for the Aral Sea (IFAS) was established in 1993 by the 5 Central Asian republics.

International donor support. The European Commission has allocated Tacis funds for ICWCsince 1992, and has supported the Aral Sea Basin Programme (ASBP) which was launched andsupported in 1994 by the international donor community from the beginning. The long-termobjectives of the ASBP have been defined as♦ stabilising the environment of the Aral Sea Basin,♦ rehabilitating the disaster zone around the sea,♦ improving the management of the international waters of the Basin, and♦ building the capacity of the regional institutions.

Strategy. The strategy chosen in the GEF sponsored programme builds on the belief that fullrestoration of the Aral Sea is impossible. At best what could be expected is a retardation of theSea’s decline. At basin level, the most important measure would be to arrive at joint actions toreduce the salinity and increase the flow of the rivers to the deltas. It is further assumed that thecurrent level of agricultural production will be maintained for the coming years and may evenincrease. Finally, the current strategy builds and relies heavily on the existing institutional watermanagement institutions in the region.

TIPE evaluation


Strategic Action Plan and funding. The results of the first three years of activities of the ASBPhave been summarised in a Transboundary Diagnostic Analysis. This document formed the basisfor a draft Strategic Action Plan (SAP), prepared by IFAS. The preparation of a SAP was made acondition for financing the Aral Sea Basin Programme by the international donors. On the basis ofthe SAP, the Global Environment Fund has approved a 21 million US $ project. The specificobjectives of this project have been defined as: (1) at least a 15% reduction of water use by the endof the project, and (2) to pave the way for increased investment in the water sector by the publicand private sectors as well as donors12. The GEF project experienced start up problems, but as of1999 it will address the following issues: Water and Salt Management; Water Conservationcompetition; Public Awareness; Dam Safety; Transboundary Water Monitoring; WetlandsRestoration and Programme Management.

3.1.2 The Tacis project

The Tacis contribution is generally known as the WARMAP project and it was designed to supportthe implementation of the GEF sponsored Aral Sea Basin Programme. Tacis has so far committed7,25 Million Euro in three budget allocations for WARMAP in the APs of 1992 (3 Million); 1993 (1,75Million) and 1996 (2,5 Million).

WARMAP contains the following modules:Module 1: Support and assist Interstate Legal Agreements;Module 2a: WARMIS regional water quantity management database development;Module 2b: Water use and farm management survey (WUFMAS-database);Module 3: GEF Project component F “Project management support”.

WARMAP 1 started in 1995. This was followed by WARMAP II, which started in February 1998 andruns to mid 2000. The key counterparts of the project are the Interstate Commission for Water Co-ordination (ICWC), the International Fund for the Aral Sea (IFAS), and SANIRI, a watermanagement institute in Tashkent (Uzbekistan).

In view of the major role which Tacis funding has played so far, the evaluation team has focussedon strategic issues of the ASBP as well as on the ongoing Tacis project activities. The findingspresented below are based on project documentation and on meetings with more than 20representatives from different stakeholders in Almaty and Tashkent.

3.1.3 Conclusions

Relevance of ASBP design♦ The environmental heritage of the Aral Sea will be lost unless a major change in ICWS and

international donor policy is made. In the implementation of the Aral Sea Basin Programme it iswidely assumed by all experts that the Aral Sea is lost. However, this assumption is not basedon any solid substantiation or justification in a diagnostic study on the Aral Sea crisis. Theabove assumption is in our opinion only valid in the case that no substantial changes in thecurrent water management practices (especially agricultural production and irrigation) will takeplace. How the Aral Sea would be affected if radical changes in water management are to beimplemented in the basin is not assessed thoroughly in any document. In our opinion, the factthat no serious efforts have been undertaken to study the feasibility of different watermanagement scenario’s is in our view a sign that from the beginning environmentalconsiderations were surrendered to agricultural considerations.

♦ The current strategy and action plan is not geared to saving the environmental heritage of theAral Sea. The actions should be targeted at water consumption reduction in primarily agricultureand population through the introduction of tariffs and awareness building. However there is littleevidence of concerted action towards common aims. Current measures are apparently basedon a draft SAP that has been prepared by the EC-IFAS. This document, however, is recognisedby World bank and other stakeholders as inadequate in terms of priority and long-term andshort term target setting and is not based on a widely accepted diagnostic study of theproblems. Despite these shortcomings this document is the governing document for the currentAral Sea Basin Programme

12 Estimates range from US $ 10-20 Billion needed for reconstruction and improvement of the water

infrastructure.

TIPE evaluation


♦ As a consequence, the objectives of the Aral Sea Basin Programme are quite modest and donot target at substantial required changes in water management and agricultural practices. Interms of environmental objectives, the Aral Sea Basin Programme has been defined as“stabilisation of the environment in the Aral Sea basin” (whatever this may be).

♦ The reality is that there is enough current water to fill the Aral Sea. However, current watermanagement practices (inefficient irrigation and flood protection measures, i.e. dams) andexcessive water use in agriculture and cities are causing the Aral Sea to dry up. In fact, excesswater because of bad management is discharged into desert pans. Large new desert lakes (onein Uzbekistan and one in Turkmenistan) have been formed the last years in expense of the AralSea. This is something NGOs should go after and with help of the Commission put pressure onthe water management lobby (in fact ICWC) to change practices.

♦ Under the current institutional structure the Aral Sea has to be considered as lost. The currentprojects are not able to solve this problem, because they focus mainly on technical solutionsinstead of institutional problems. The water managers in charge of water management in formerSoviet Union (causing the environmental catastrophe) are now the key players (i.e. ICWC) inthe Aral Sea Basin Programme and are as such financed by the donors.

♦ The institutional setting of the Aral Sea Basin Programme (and thus of WARMAP) contributes tolow effectiveness of the ASBP. The regional water management institutions (IFAS and ICWC)have been given a substantial say in the programming and implementation of the Aral SeaBasin Programme. However, both institutions are weak, monopolise information in the regionand appear to be incapable of defining and managing an integrated and complex programmewithout substantial professional assistance. The fact that the programme is implementedexclusively with the above mentioned regional institutions is one of the major causes of the lowexposure of the results (and therefore low effectiveness) in the programme.

ASBP Impact♦ The above mentioned factors make the Aral Sea Basin Programme to perform as misguided. It

needs new target setting. The Tacis contribution in the form of WARMAP does not address realpriorities and its impact on achieving any of the wider goals of the Aral Sea Programme areexpected to be very limited. Furthermore, the institutional constraints in the programme as awhole (and consequently in WARMAP) makes the visibility of the programme low.

Effectiveness and efficiency of the Tacis project♦ The technical quality of the results produced by WARMAP I and II is high. The databases

produced are very useful as background information for future implementation of remedialprogrammes affecting water management in the Basin. However, the programme remains weakin its institutional and environmental aspects. For instance, it is unlikely that the programme willbe able to come up with the required interstate legal agreements and there are problemsrelated to the future ownership and use of the Tacis financed databases. Also, the integration ofenvironmental aspects into the design of the programmes was limited. (The WARMAPcontractor was not aware that WARMAP is part of the Tacis Interstate EnvironmentProgramme, instead he thought it was an agriculture programme).

Project cycle management aspects♦ Project progress reports, monitoring reports and interviews indicate that the Commission is not

managing the implementation of the project adequately. Contracting procedures are slow andthe long periods in contracting and revision of contracts seriously hamper progress of a project.Lack of leadership as well as well as ineffectual consultation between Tacis and other donorswas mentioned as a serious problem by almost all parties involved. In this respect the delays inthe start up of the GEF project have played a negative role. In addition the TOR for thecontractors lacks quality and current contractors appear to have made the TOR used for theircontract extension themselves. This process bears in itself a potential conflict of interest. Thesystem of (progress) report approval is not clear. Contractors simply have to assume that theirreports are approved, since they do not receive this in writing.

3.1.4 Recommendations

1 Continuation of Tacis support is recommended. Taking into account the relevance of theproblems addressed for EU environmental policy, Tacis should continue its intervention in thearea. In case of a continued presence of the EU in the Aral Sea Basin Programme, the EU

TIPE evaluation


should however consider to take a more pro-active and leadership approach in furtherprogramming of the measures in the Aral Sea Basin Programme.

2 Continuation should be linked to conditionalityIt is recommended that any follow-up by Tacis should address re-targeting the Aral Sea BasinProgramme as a whole and subsequent targeting of the WARMAP programme itself.Furthermore, it is recommended that any follow-up of Tacis’s contribution is made conditionalon the following terms:♦ New target setting of the Aral Sea Programme, including the drafting of a Strategic Action

Plan, based on an Aral Sea Basin integrated environmental study. This would also meanstrengthening of the environmental component in the programme, which is at presentlargely focusing on water quantity management;

♦ Substantial strengthening of the institutional development component (includingstrengthening of communication and involvement of other institutions in the region) andintroduction of new water management concepts, taking strongly into accountenvironmental and cultural considerations, instead of economic development only;

♦ Completion of the interstate agreements on water management issues;♦ Strengthening public awareness and NGOs to support the process.

3 It is recommended that Tacis plays an active role in the systematic mobilisation of regional andinternational NGOs and other players in support for consistent remedial action in the Aral Seacrisis. This will be of paramount importance to generate sufficient know-how, and platform formobilising public and state support for adopting and implementing a new and better targetedstrategic action plan.

3.2 The “Black Sea” Programme

The Black Sea environment programme has lost its initial momentum

3.2.1 Introduction

The problem. The Black Sea is a virtually landlocked sea with very limited exchange of water withthe Mediterranean. Six countries surround it: Bulgaria and Romania along its western rim, Ukraineand the Russian Federation along its northern rim, Georgia along its eastern rim and Turkey alongits southern rim. The Black Sea ecosystem has been seriously damaged in the last decades. Itcontinues to be threatened due to a number of factors including fertilisers, pesticides, heavy metals,oil spills, solid waste dumps, sewage discharges, the introduction of exotic fish species, over-fishingetc. (for details see the Black Sea evaluation report). These inflows and practices constitute aserious risk of losing valuable habitats and landscape and ultimately, the biodiversity andproductivity of the Black Sea ecosystem. Moreover, there are also economic losses related toforegone opportunities for tourism and the fishery sector. The consequences have not beenassessed in detail, but indicative calculations show that, in case of beach tourism alone, actionsleading to a 20% improvement in Black Sea water quality could generate $550 million in annualeconomic benefits to coastal economies. This estimate does not include expected benefits tohuman health and fisheries. The challenge, which the region is facing at the moment is to secure ahealthy Black Sea environment at a time when economic recovery and further development arebeing pursued.

Regional initiatives. In the beginning of the nineties there was a growing concern among the sixBlack Sea countries on the condition of the Black Sea and a general agreement emerged on aconcerted multi-country action to save its resources. In 1992 the six countries signed theConvention for the Protection of the Black Sea against Pollution (Bucharest Convention), which wassubsequently ratified and entered into force in 1994.

International donor support. The Black Sea Environmental Programme (BSEP) was initiated inJune 1993 at the request of the governments of the above mentioned Black Sea countries. TheBSEP, funded by the Global Environment Facility (GEF) and a number of donors, is managed by theUnited Nations Development Programme (UNDP) in close co-operation with the World Bank, the

TIPE evaluation


United Nations Environment Programme (UNEO), the European Commission, and other donors. TheBlack Sea Programme Implementation Unit (PIU) has its office in Istanbul, Turkey.

Strategy and action plan. The aim of the BSEP is to provide a sustainable basis formanaging the Black Sea through capacity building, environmental assessment, development andharmonisation of policy and legislation, and by facilitating environmental investments. It wasinstrumental in assisting the Black Sea governments to develop the Black Sea Strategic ActionPlan, signed by all six Ministers of Environment of the six Black Sea countries on 6 October 1996.This Action Plan sets out a strategy for rehabilitation and protecting the Black Sea in the nextdecades.

3.2.2 The Tacis support

The European Commission has been on of the first donors for the Black sea programme. The firstallocation was made in the TIPE AP of 1992, other funds were committed in 1995, 1996 and 1997.In total some ten projects were implemented under the Tacis Interstate Programme or as Tacisfunded components of the Danube Programme, which is also directly related to the BSEP. Theassessment of the Programme in this evaluation is restricted to two of these projects, with a totalbudget of 6,1 Meuro representing the bulk of total project cost of the Tacis intervention in this area:♦ The Tacis part of the Tacis/Phare 1995 Funds for the Black Sea Environmental Programme. .♦ Tacis 1996 and 1997 Funds for the Black Sea Environmental Programme.

The objective of the project funded through the 1995 budget year (as included in the ToR) is: “toassist Bulgaria, Romania, Russia, Ukraine and Georgia to strengthen their individual and sharedcapacities to tackle the Black Sea’s environmental problems.” This objective should be achieved byreaching a number of results in five different components.1 To improve environmental education and public awareness of Black Sea environmental issues

in the Tacis and Phare partner countries;2 To assist local groups in the five countries to design, set up and implement demonstration,

environmentally-sound, entrepreneurial aquaculture projects to help develop theenvironmentally sustainable economic potential of the Black Sea, in all five countries, andSupport to the Black Sea Environmental Fund;

3 To transfer EU knowledge and experience of licensing to Regional Inspectorates in the partnercountries (Pollution assessment and control);

4 Contribute to the development of effective regional coastal zone management regimes with afocus on tourism. This includes training, and legislation development;

5 To restore the commercial, educational and scientific potential of the Batumi Dolphinarium (thiscomponent was cancelled at a later stage).

The objective of the project financed under the Tacis 1996 and 1997 funds are described in the ToRas: “to support Georgia, Russia and the Ukraine in their participation in the implementation of theBucharest Convention and the Black Sea Strategic Action Plan. The BSEP Programme Co-ordination Unit (PCU), located in Istanbul, plays a very important part in this support, and so theproject also allows for the provision of technical assistance to the PIU.” (The PCU was laterrenamed PIUs). Once again the project included five components:1 Support to the Regional Activity Centre for Pollution Monitoring and Assessment (Odessa,

Ukraine);2 Support to the Regional Centre for the Development of Common Methodologies for Integrated

Coastal Zone Management (Krasnodar, Russia);3 Support to the Regional Activity Centre for the Conservation of Biological Diversity (Batumi,

Georgia);4 Support to the implementation of Georgian, Russian and Ukrainian national contributions to the

Bucharest Convention and the Black Sea Strategic Action Plan;5 Provision of technical assistance to the BSEP Programme Co-ordination Unit (PCU).

The conclusions and recommendations presented below are based on project documentation andinterviews with over 30 stakeholders in Georgia (Tlblisi), Russia (Rostov, Krasnodar), Turkey(Istanbul) and Ukraine (Kiev)..

TIPE evaluation


3.2.3 Conclusions and lessons learned

Relevance of the BSEP♦ The ecological problems of the Black Sea aggravate the present economic problems of the

coastal areas of the countries in question. Income from tourism and fisheries are very muchdepending on a sufficiently high ecological quality of the Black Sea. Present pollution ishampering tourism development. Many beaches are closed in summer because of health riskscasing major losses in this locally important sector.

♦ The ecological problem of the Black Sea can in principle be solved. Technical solutions areavailable. However, implementation is depending on adequate funding and sufficientinstitutional support in each of the Black Sea countries to take action by executing a number ofwell justified projects.

Quality of Tacis project design♦ The BSEP is focused on actions and solutions at an inter-state level. The need for supporting

the BSEP with national level measures is insufficiently recognised in the BSEP, and this is aserious fault also in the Tacis project design. A firm conclusion from the Black Sea projectsimplemented so far is that the effectiveness and efficiency of the Interstate Programme islimited by amongst others lack of national follow-up.

♦ It is too early to assess the effectiveness and sustainability of the support for the three thematiccentres i.e. pollution monitoring (Odessa); coastal management methodologies (Krasnodar),biodiversity (Batumi). One observation can be made however, the TORs for the EU staff focusthe centres on national issues rather than on co-ordination of the activities in and amongst thesix countries. The latter is needed and the establishment of such regional centres was theintention of the relevant paragraphs in the Bucharest convention.

Effectiveness and efficiency♦ The majority of the project components are implemented, however the TORs of the projects are

in general not very specific about targets and specific results, and progress reports give limitedinformation on tangible achievements.

♦ The environmental awareness component is one of the most successful components of theTacis Black Sea programme. One out of many examples is: “In the past I thought that wetlandsare there to be drained and converted into agricultural land. This is what we learned in Soviettimes. Now I have learnt and understood that in addition to their ecological value they do have asubstantial economic value in a number of aspects including a cheap but effective way ofnutrient removal” (Civil servant in Batumi).

♦ The evaluation team did not fully succeed in getting a comprehensive picture of the outputs andachievements of the Tacis efforts. There are, however, strong indications that the BSEPprogramme has lost focus and momentum over the last few years. Delays in funding by theGEF have played a role in the uncertainty amongst the state stakeholders about furtherimplementation.

Project cycle management issues♦ The set up of the Tacis Black sea projects is complicated, too complicated for the limited

management resources available for the TIPE. Complications arise form the different fundingcycles of Tacis and Phare, the involvement of 6 countries, the ten components and theinvolvement of more then five sub-contractors. Co-ordination has been facilitated somewhat bythe (for Tacis) unusual practice of awarding all four Tacis and Phare contracts supporting theBSEP (since 1995) to one and the same contractor.

♦ The Tacis reporting and monitoring systems are not organised on an interstate basis and theevaluation team has observed that the reports do not succeed in giving a comprehensiveinformation picture on the progress and problems of project implementation. As a result theCommission services (Task Manager) do not have the tools to take appropriate managementdecisions.

Lessons learned♦ Countries around the Black Sea coast are more than a coastline. Municipalities and regions

along the coast have to work within the law for the whole country. Furthermore, most of thepollutants originate from in-land source. Insufficient attention has been paid to this fact.

TIPE evaluation


♦ Licensing and enforcement does not make much sense in case the tools to reduce emissionsare not provided. Therefore, it may be considered to include a substantial component on cleantechnologies into the assistance scheme.

♦ There is an urgent need to assist a number of countries (especially Georgia, RussianFederation and Ukraine) with their financial obligation to support the Black Sea Secretariat. Thisis a very critical issues to make the 70 MUS$ GEF programme go. Turkey has alreadyconfirmed that it will finance 40% of the total costs in the first years.

♦ Tacis Interstate environmental projects are generally not well targeted. The outputs are notconcrete enough and seem to remain often on the level of workshops and study trips, which assuch may be useful. A spokesperson of the Ministry of Environment of Ukraine formulated thisas follows: “The output of a project on environmental legislation should be a new draft law. Aseries of workshops and discussions is certainly useful, but not enough”. This may also belinked to the lack of a comprehensive strategy behind the Tacis intervention through theIntestate programme for environment.


The evaluation team recommends to continue with TIPE support for the BSEP but to rethink the roleof Tacis in consultation with on the one hand the UNDP/GEF and on the other hand with the PIU inInstanbul and the BSEP country co-ordinators. The aim of such consultation would be to set cleargoals within the BSEP for future contributions of the Commission. A number of ideas are listedbelow under two headings: support for Inter-state level actions, and support for national levelactions.

Inter-state level actions♦ Support to the BSEP riparian countries to meet their financial obligations towards funding the

work of the PIU is needed to prompt GEF to release its funds for the BSEP. Some financialengineering with national ministries and Ministries of Finance may be needed, The work of thePIU is essential for re-establishing momentum and concerted action of the riparian countries.

♦ The preparation and finalisation of the following key documents and actions are a priority♦ Integrated water quality and biodiversity monitoring programme♦ Start up of implementation of the water monitoring system♦ Update of the inventory of past and present pollution of the Black sea including a proper

priority setting for pollution parameters (so far only indicate data on different sources ofpollution exist)

♦ Pollution reduction strategy, specifying how to start up reduction of pollution for prioritypollutants in a cost effective manner by each of the riparian countries

♦ Biodiversity recovery strategy♦ Ammend the current contract and TORs for EU and NIS staff working with the three thematic

centres and refocus on the regional support and co-ordination function.

National level actions♦ Provide investment support for priority projects, which can serve as pilots for replication. The

projects can be selected on the basis of the Black sea action plan and the NEAPS, togetherwith the partner countries and in consultation with the PIU in Istanbul.

♦ The set-up of an investment programme of Tacis together with for instance EBRD or EIB couldbe considered. In Romania there is already some experience with the so-called MUDP projectwhere the contribution of Phare makes it possible for cities to take soft loans. The total amountof funds involved in that programme is 155 Meuro.

TIPE evaluation


3.3 The “Caspian Sea” programme

The ecological threats to the Caspian Seafrom oil exploration and transport are enormous and growing

3.3.1 Introduction

The problem. The Caspian Sea is seriously threatened by pollution. It is a multi-national problemand the Volga contributes considerably to the pollution load of especially nutrients. Much of thecoast is considered to be moderately polluted, but parts of the Caspian adjacent to Azerbaijan areextremely polluted, particularly Baku Bay and the waters of Sumgayit.

Regional initiatives. Although the Caspian littoral states and FSU have been involved for manyyears in scientific investigation of the environmental problems of the Caspian Sea, there have beenfew positive actions to ameliorate the degradation of the region’s environment. The lack of anyformal regional framework within which to determine and carry out positive actions is currentlyhampering efforts. Agreement is further hampered by the presence of large oil reserves below thesea, and the disputes over the status of the Caspian as a sea or lake, which will affect ownership ofthe resources. However, in 1994 a Regional Committee for the Rational Exploitation andConservation of the Biological Resources of the Caspian (the Regional Committee) wasestablished.

International donor support. Shortly afterwards, in April 1995, agreements were reachedbetween the Caspian littoral states and a joint World Bank and United Nations mission, whichdefined a first phase of a Caspian Sea Environment Programme (CSEP). A meeting held in Istanbulin May 1997 introduced the Tacis project and the potential Global Environmental Facility (GEF)project to the riparian countries. Since then UNDP has been preparing the GEF project, which wassupposed to run in parallel to the Tacis project to form a complete Caspian EnvironmentProgramme. This Tacis project is financed within the framework of the European Union’s Tacisinter-state environmental action programme, and is based on the initial CSEP agreements.

Strategy and action plan. The 1st phase of the Caspian Sea Environment ProgrammmeCSEP has the objective to promote sustainable development and management. The specificobjectives have been defined as:1 To foster the collaborative spirit between the countries in the region, which has weakened in

recent years, as a result of the collapse of the USSR and the relative isolation of the IslamicRepublic of Iran;

2 To reinforce institutional capacity, strengthen national, regional, and local environmentagencies, improve pollution monitoring capabilities, establish networks for regional emergencyresponse contingency planning, and public awareness;

3 To develop common approaches to national environmental reviews, status and trends reports,the preparation of legislation and harmonisation of standards, strategies for the conservation ofnatural resources and the coastal zone management. This should be encouraged through thepreparation of a Caspian Sea environmental priorities study, which will be a comprehensive andshort to medium-term analysis of necessary investment and other actions; and

4 To define on the basis of this study urgent investment possibilities and implement feasibilitystudies on these investment possibilities

An important element in the strategy is to get international agreement on a Framework Conventionfor the marine environment of the Caspian. Since 1995, the UNDP has been working towards thisgoal.


The Tacis project and the UNDP/GEF project have similar aims and modalities, and they areintended to form two complementary components of the Caspian Sea Environment Programme(CSEP). Work on preparation of the CSEP was initiated by UNDP under its project developmentfacility stage (PDF). However, preparation of the GEF component was considerably delayed and is

TIPE evaluation


expected to be in full swing only after completion of the current Tacis contract in the beginning of2000. This delay caused the project to suffer considerably in its effectiveness and efficiency.

Due to the above mentioned delays with the GEF component, the main expected results from theTacis project, set out in the TOR presented presented in May 1997 in Instanbul, were amended,and down tuned after the inception phase of the project as follows:♦ Establishment of the Programme co-ordinating unit (PCU);♦ Establishment of four thematic centres (dealing with desertification, sea level fluctuation,

fisheries, data gathering);♦ Work towards the preparation of the Strategic Action Plan (rather than a prepared Strategic

Action Plan);♦ Work towards the preparation of an urgent investment portfolio (rather than a prepared urgent

investment portfolio).

The following organisations/institutions take part in the project: Institute for Fisheries (Astrakhan),Caspian Sea Inspectorate (Baku), Caspian Institute for data information and management (Baku),KazNIIMOSK (Almaty), Institute for Desert Flora and Fauna (Ashkhabad), Ministries/Committees forthe Environment in five countries of the Caspian Sea, and the PCU.

The conclusions, learned and recommendations presented below are based on file and dossierstudies, background papers prepared by NIS experts for this evaluation and interviews by theevaluation team with some 20 representatives of different stakeholders in Moscow, Baku andAlmaty.

3.3.3 Conclusions

RelevanceThe Caspian Sea Environment Programme is the single biggest environmental project in the region.Unlike the Aral Sea crisis, the environmental problems are severe, but the solutions seem to bemore manageable. In terms of EU environmental policy the programme is very relevant.

Effectiveness and efficiencyThe effectiveness of the interventions appear to have been quite limited so far. Because of co-ordination problems between the donors in the Programme, the initially defined outputs in theprogramme will not be reached at the end of the project. The fact that the UNDP contribution in theprogramme started very late was the principal cause for the downgrading of the initial expectedoutputs from “submitting a Strategic Action Plan and Project Investment Programme” to “workingtowards the SAP and PIP”.

The project is implemented well by the contractor and the “revised workplan” is likely to beimplemented as planned.

A serious problem is the lack of (reliable) data and poor exchange of data. This prevents making aproper inventory of problems.

The maximum fee level for NIS experts applied for Tacis contracts has been mentioned as a barrierfor utilising top national experts on the project team , thus lowering efficiency.

Impact of the Tacis assistance:The Tacis assistance can be credited with two positive developments:♦ Countries and relevant institutions involved are talking again and mistrust between them is

reducing;♦ For the first time, environmental issues of the Caspian Sea are being tackled in an integrated

way and in a joint effort.

Project cycle managementThe project cycle management from the side of the Commission is considered to be inadequate.♦ Contracting procedures are slow and the long periods in contracting and revision of contracts

seriously hamper the efficiency in project progress.

TIPE evaluation


♦ The Commission does not show clear leadership in this project, and the Commission services(Task Manager) have insufficient capacity to adequately respond to project implementationproblems.

These and other weaknesses in the project management capacity of the Commission are raised inprogress reports, monitoring reports and are mentioned by almost all parties interviewed. A tellingsign for the inadequate communication between the Commission, the Contractor and the Tacismonitors is the disagreement between the contractor and the monitoring team on a formal issue i.e.should the monitors take the original objectives and planned outputs of the TOR, or should theytake the inception phase workplan as the target setting framework for monitoring projectimplementation and progress?

Lessons learned♦ There is to limited involvement of -and poor communication between -other relevant line

Ministries at the on-set of the Programme (especially in relation to the Convention of theCaspian Sea). Internal communication (within institutions involved) needs considerableimprovement;

♦ Geo-political/economical discussions and interests hamper the drafting of commonenvironmental action (Caspian Environmental Convention still not completed and agreed upon);

♦ Environmental issues are low on the political agenda of the litoral countries of the Caspian Sea.The aimed for framework convention needs agreement on clean methods of oil exploration, andshipping. the Gurev nuclear power plant, the Volga river pollution, coastal management etc.Given the fact that these are the drivers of the economy of the litoral states, it may prove verydifficult to get agreement on an environment convention. More difficult then in the Black Seawhere tourism and fishing are two strong win-win sectors.


On-going Tacis support for the CSEP is fully justified. The project is implemented well, and has highvisibility. At this stage in time, the Tacis project and the other international assistance programmesare the only guarantee that the Caspian environment will (at least somehow) remain on the politicalagenda of the countries involved.

Any further follow-up activity will need to anticipate on the identified institutional difficulties andstress the importance of drafting a high quality SAP, which is to be agreed upon by all relevantinstitutions and littoral states. The process of drafting this SAP would need to systematically involvethe research institutions currently involved in the thematic centres, the international oil industry,NGOs the littoral Governments and the international donor community, in order to raise itsacceptability.

Integration of the Volga river into the Caspian Strategic Action Plan is desirable and needs seriousconsideration in TIPE programming.

3.4 The “Development of Common Environmental Policies”programme

A positive, be it small, contribution to NIS capacity building

3.4.1 Introduction

The problem. This project addressed a problem of central importance for the NIS i.e. the weakpolicy making and management capacity of the NIS environmental public authorities. Environmentalproblems in the NIS are of an extremely severe scale and complexity. They therefore need to betackled in an integrated and co-ordinated way, at national and regional level. This calls for thedevelopment of specific structures and necessary capacity for making environmental policy, and forthe integration of environmental considerations in other fields of policy making. This is a complexand difficult task wch has to be tackled in an unfavourable external environment: the state systemsof environmental protection are ineffective in the NIS; Moreover the political powers, presidential

TIPE evaluation


administrations and parliaments are dominated by interest groups for whom environment is a lowpriority; NIS government, furthermore have no tradition in involving the public and civil societyorganisations such as NGOs and media in environmental policy making and decision making.

National and international initiatives: The Environment for Europe process and the EAP�The “Environment for Europe” process (EEP) aims at co-operation on environment protection in theECE-region. The EEP is the main international co-ordinated framework for development ofenvironmental policy in European arena. The process started in 1991 and has thus far included anumber of agenda-setting conferences attended by the environmental ministers of the EU and theNIS, and other stakeholders. Starting in the Czech Republic in 1991, a series of pan-European/NISEnvironment Ministers Conferences was launched (Dobris-1991, Lucerne-April 1993, Sofia-October1995, and the most recent in Aarhus 1998).

A main instrument of the EEP, has become the Environmental Action Plan (EAP) , and the EAPTask Force , the secretariat of which is handled by the OECD. The EAP represents a consensusbased on three elements:1 The integration of environmental considerations in the process of economic restructuring;2 The development of institutional capacity in Central and Eastern Europe (CEE);3 Immediate programmes to bring relief to regions where human health or natural ecosystems are

severely jeopardised by environmental hazards.

The structure of the EAP is essentially that of a blueprint for East-West environmental co-operation,focusing on policy reform, institutional strengthening and investment-related activities. It does notcontain targets, timetables or specific activities, but provides a framework for actions required fromcountries to identify the degree of priority of problems and to develop cost-effective solutions. Theaim of the EAP is to solve environmental problems by ensuring that clear priorities and goals areestablished and by determining the most efficient way to achieve these results.

Strategy, action plan and funding. The EAP called for the development of country-specificNational Environmental Action Plans (NEAPs). The NEAPs are environmental strategy papersincluding both a methodological part about problem identification, prioritisation, ecological andeconomic analysis and a concrete list of problems and solutions, targets, implementation means,timetable, needed resources.

NEAPs are not an end in themselves, but rather tools that countries can use to strengthenenvironmental policies and institutions, and ultimately, to improve environmental conditions. Theprocess of elaborating NEAPs is just as important as the product.Various donors including the US have provided support to the NEAPs and the various NIS were atdifferent stages of the development of their NEAPs at the time that the Tacis DCEP project started.


The Commission has from the start been an initiator and supporter of the Environment for EuropeProcess (EEP). The Commission, moreover, has with Tacis (TIPE) funding been the largest donorfor projects initiated in the EEP. Tacis first supported the NIS involvement in EAP through the 1993‘Regional Environment Facility’. This provided support for editing and disseminating the EAP andother key documents throughout the NIS.

The project ‘Development of Common Environmental Policies’ was designed to further support theimplementation of the EAP, and specifically to support the completion of the NEAPs and to followup on the NEAPs with training and investment preparation activities. Furthermore, the project wouldsupport common or co-ordinated responses to environmental problems, strengthen of interregionalor international institutions dealing with environment and support the co-ordination of actions by theNIS to deal with regional environmental problems (incl. conventions, transboundary water and airpollution).

The target group of this project consisted of the partner organisations. These were, at internationallevel, the Interstate Ecological Committee, and at national level the responsible ministries or StateCommittees for environment of the various NIS.

TIPE evaluation


The project had a multi-country set-up. The contractor was to provide support to all NIS countries(except Tadjikistan) in the development of environmental policy. The design of the project was fairlysimple. The main component consisted of the provision of mainly long-term and also short-termassistance at the national level in the development of NEAPs. Additional components envisagedprovision of documentation, training in the drafting of environmental legislation, support for activitiesat the interstate level, strengthening of project preparation capacities within the relevant ministries inthe NIS. The project also organised a number of meetings and conferences: one national EAPconference in each of the NIS as a first stage capacity evaluation, a number of seminars andworkshops on specific topics, and two meetings at interstate level (one inception meeting and onefinal meeting).

The conclusions, lessons learned and recommendations set out below are based on an review ofproject documentation, interviews with over 50 stakeholders in, Georgia, Kazakhstan, Moldova,Russian Federation, Ukraine, Uzbekistan, and six country studies on the current status of theNEAPs by six national experts. (see the 1st progress report of th evaluation and annex C).

3.4.3 Conclusions

RelevanceThe importance of the NEAPs themselves and also the importance of the participatory approachthat characterises the NEAP formulation cannot be overemphasised. In NIS, environmentalmanagement is a low priority that has to compete with scarce resources. In this context, it is ofutmost importance that the authorities and other stakeholders involved in good environmentalmanagement work on the basis of a clear strategy and have an action plan (with clear priorities) attheir disposal. Moreover, such a plan needs to be endorsed by government to gain real value as aninstrument for environmental action.

More important than the final quality of the different NEAPs is the progress achieved from eachcountry’s previous stage of environmental policy making and the present NEAP. For example,although the latest Russian NEAP still includes many outdated characteristics (lists instead ofprinciples), it also introduces some measure of analysis and concepts for environmental policyimprovement. This represents a big step forward and should be appreciated as a new stageallowing further progress in future. In general, there are indications that all counterparts havesubstantially altered their thinking during the period of this project.

Quality of project design♦ A strong aspect of the project design was the use of long-term resident EU advisors attached to

the NIS Ministries of Environment (or State Committees). This proved successful for completingand furtherance of the NEAP as a product.

♦ A number of weak aspects limited the ambition level and potential impact of the project:♦ The TOR does not sufficiently define indicators for measuring the implementation success

of the planned outputs and insufficient conceptual distinction is made between the NEAPproduct and the NEAP formulation and endorsement process.

♦ The design did not sufficiently cater for clear need in the NIS for strengthening the NEAPimplementation capacity of the Ministries involved.

Effectiveness and efficiency��

♦ The contractor has mainly worked with the national ministries. The inter-state committee forenvironment has not played a role. It existed on paper rather then as a concrete partnerorganisation.

♦ Each country in the NIS does have a NEAP at the moment. In most countries this NEAP isrecognised as the central policy framework document and in a number of counties the NEAP isendorsed by parliament and/or by the presidential decrees. The Tacis project has in somecountries contributed to this, in others the Tacis support has focussed on follow up actions.

♦ The project has supported a range of activities related to the NEAP, and in most countries theplanned outputs were achieved. The contractor was responsible for the division of theresources amongst the countries and activities and in general has met quality standards

13 Details are provided in the project evaluation report submitted to the Commission on 25 February 2000.

TIPE evaluation


♦ The level of commitment and support from the national partner organisations has variedbetween the countries.

♦ The DCEP achievements are at the national level. Little was achieved with respect to theobjective of: “developing a common and co-ordinated response to environmental problems of atransboundary nature”. This finding can be explained by (a) the different pace of progress of thevarious countries towards the implementation of environmental policies and plans; (b) theattitude of various countries towards Inter-NIS co-operation; and (c) the different priorities of thecountries.

Project cycle management♦ The findings are similar to those for the regional seas projects.

Impact♦ There is a clear risk that the impact and sustainability of the project will remain low. In most NIS,

the NEAP are documents rather then policy, and the implementation capacity of the Ministriesand/or State committees for environment is low, which also limits the absorption capacity fordonor funded TA projects. In most NIS the new legislation is ineffective within the remains of theold soviet regulatory framework, and funding levels for the apparatus of environmentalmanagement have fallen to dramatic levels in some countries. For example in Georgia, the MoEdoes not have the monies to maintain its telephone bills and operate its web-site.

♦ The NEAPs, nevertheless are an essential achievement, and they form a foundation on whichto built further on policy formulation. legislation, implementation etc. the NEAPs moreoverprovide a basis for further donor support. In this respect the Commission has funded a numberof TIPE projects that support capacity building including the WEAP and JEP which focus oninvestment preparation and training.

♦ A notable weakness of the NEAPs is that the interstate environment issues are largely absent.For example the Black Sea, and Caspian Sea Environmental Action programmes are notmentioned Moreover there is little evidence that the POs and the EU contractor have utilised theopportunities for public awareness raising and involvement of NGO offered by the NEAPprocess and endorsed in the justification for the DCEP project.

Lessons learnedThe value added of the project in terms of the principles which lie at the heart of the Commission’sFifth Programme for Environment, has been limited. Considering the limited scale of the project, nomajor impact in these terms could have been expected, nevertheless some lessons can be drawnfor the design of future technical assistance:♦ Integration of environmental considerations into other policy areas can only be achieved

through political will. The NIS environment authorities have in general, too little political clout forthis, and would need support from the Commission. A clearer commitment from theCommission to environmental considerations would be needed to bring about a real change.This commitment would need to be demonstrated in the PCAs and in conditionalities for tradeand aid agreements. Although hard, this is an inescapable conclusion if this goal is to bepursued;

♦ Closer co-operation is needed with the OECD EAP Task Force. The EAP Task Force runs itsown agenda of meetings and inter-sate training sessions with very little co-ordination with theTacis projects in the field. More co-ordination not only in the design phase but also in theimplementation phase can enhance the relevance and effectiveness of Tacis projects and theTask Force actions.

♦ To develop the NIS societies to the point where environment protection is perceived as ashared responsibility between the public sector, private sector and civil society, will requiredecades. It will require a number of steps including legislation and enforcement mechanisms,which will provide concrete incentives for environmental sound behaviour by all key actors andthe public at large. Thus far the EEP and TIPE have focussed on policy formulation andenvironment awareness raising (Arhus Convention). In the period 2000-2006 this needs to befollowed by policies and projects give priority to building the right incentive system in legislationand behaviour.

♦ The strongest point of the project was the use of long-term in-country advisors. This is proved tobe a effective way to cover gaps in specific skills and to support in-country processes. For aproject of this size (in terms of financial resources), it is also a more efficient solution than touse short-term experts. The recipients also liked this structure because they only had to deal

TIPE evaluation


with a limited number of people and were always well-informed about the status of the projectthrough their own advisor. The system is also beneficial to implementing activities that meet thelocal priorities and in responding quickly to changes in these priorities. Finally, it enables thedevelopment of better working relations with the counterparts.

♦ Due to the differences between the countries and the absence of a functioning inter-statePartner Organisation, the inter-state aspect in project design has not added value to the project.On the contrary, the inter-state set up increased the complexity of implementation and hasreduced the transparency of the project for the POs.


1 It is recommended that the Indicative programme for TIP(E) for the period 2000-2006 givesspecial attention to the Institution building process of public sector authorities in environment.

2 The evaluation of this and other TIPE projects highlight the need for comprehensive TA supportproject for institution building of the public sector authorities, dealing with environmentalmanagement in the NIS. TA support with resident and short term experts is needed to transferenvironmental management know how and built local capacity for harmonising policies andinstruments in the NIS with EU best practices.

3 The TA needs to focus on the basic tasks of environmental ministries i.e. policy making,preparation of legislation and implementation capacity development, implementation monitoringand enforcement, and integration of environmental policies and concerns in the overall policydecision making on economic reform processes in the NIS.

4 In addition to the standard elements of the TA programmes, which the Commission has fundedin Phare countries, the assistance should also focus on basic administrative and governanceskills such as management training, the organisation of meetings, decision making processes,interaction and mobilisation of interest groups, media skills etc. These elements are needed toempower the National, regional and local authorities, and are needed as a foundation forintegrating the NIS authorities in the EEP and to foster the interstate collaboration and “commonapproaches”, propagated by the DCEP project.

3.5 The programme for “Raising Public EnvironmentalAwareness”

Plenty of visible activity and a wide civil society focus

3.5.1 Introduction

The problem. The purpose of the REAP project was to promote public awareness ofenvironmental problems in the NIS and Mongolia and to encourage public action, which will helpovercome these problems and ensure that they are taken into account during the “transitionprocess”. In particular the project aims to address five related problems that are a legacy of SovietUnion times:♦ A low level of awareness about environment problems and the risk for health and ecosystems♦ A low level of understanding of environmental issues, their causes and effects♦ A superficial perception of priorities for action♦ A lack of knowledge about what can be done to solve environmental problems♦ A lack of organising ability to encourage public action

The initiative and donor support.The project originates from the Environment for Europe Process,and is linked to the EEP objective of developing societies in Europe and the CEEC which are basedon market-oriented economic reforms while at the same time improving the state of theenvironment. Involvement of NGOs, business and other stakeholders in (environmental) policymaking is one of the themes in the EEP. The initiative for the REAP project was launched during theAarhus conference of the EEP. This conference emphasised the importance of information as thebasis for mobilising public participation on environmental issues. In the Aarhus Declaration, 23/25-06-1998-Preamble, it is stated that the EEP should focus more of its resources on the countries inCEE and NIS to ‘promote a convergence in environmental policies and conditions within theEuropean region, as a step forwards towards sustainable development’. In this context, the Aarhus

TIPE evaluation


convention was adopted on “Access to Information, Public Participation in Decision Making andAccess to Justice in Environmental Matters”.

3.5.2 The Tacis project

The project “Raising Environmental Awareness of the Public and Developing Environmental Mediain the NIS and Mongolia” (hereinafter referred to as REAP) was implemented through two maincontracts (REAP1, and REAP2). Two budget allocations were made from the Tacis InterstateProgramme for Environment (budget years 1995 and 1997) for a total of 5.7 Million Euro.

The project has been important in the TIPE on account of its theme, budget allocation, and becauseit has prepared the ground for follow up activities in the period 2000-2006. The theme, awarenessraising, is a basic element in the TIPE programme, and almost every TIPE project has anawareness-raising component. The REAP, however, stands out amongst the TIPE projects,because it aims to raise the general level of environmental awareness in the public of the NIS; it isnot focused on specific issues. The project has initiated a large number of awareness raisingactivities focused on different target groups. REAP has received 15% of the total TIPE budget, and25 % of the budget allocations in 1995 and 1997.

Project strategy and activities.Three objectives were formulated:♦ To raise the general level of awareness in NIS societies about solutions to environmental

problems;♦ To influence Governmental policy, and NGO activity;♦ To build the specific capacity of the selected target groups for tackling environmental problems.

Project design distinguished six key components with the following main activities:1 Benchmark/Impact survey. Two NIS wide surveys on environmental awareness were organised

and the results were published in two voluminous reports.2 Promotion of the Local Agenda 21 process, and the Aarhus convention. Among others

International Conferences on LA 21 related topics were organised. In this context a small grantfund was established by each of the four regional offices, with which local activities of NGOswere supported.

3 Video Libraries and Public Events component. In the partner states environmental video-productions were selected, and these have been widely spread, mainly through a network ofNGOs.

4 Media Work. In the partner states the broadcasting of environmental productions were to bepromoted. Besides an environmental media competition was organised for environmentaljournalists and regional training for environmental journalists was organised.

5 NGO development. This comprised training for NGOs, production of Tool Kits for NGOs, andthe small grant fund mentioned under component 2.

6 Parliamentary Capacity Building. Open parliamentary meetings (OPMs) were the main activitieshere. They focused on key environmental issue on the international (EEP) agenda (seecomponent 2) and key national issues such as energy saving in Georgia.

The project was set up on a decentralised basis with four regional co-ordination offices:♦ A Regional co-ordination office in Kiev for Moldova, Ukraine and Belarus;♦ An office in Moscow for Russia and Mongolia;♦ An office in Tbilisi for the Caucasus Region;♦ An office in Almaty for the Central Asian Republics.

The evaluation is based on desk-research and interviews in Moscow, Kiev, Chisinau, Tbilisi, Almaty,and Brussels. At the time of the evaluation field visits, REAP2 was about to finish, but a final reportwas not yet available at the time of writing.

The project partner at Interstate level was the Inter-Parliamentary Assembly of the CIS. In the NIScountries, the contractors(s) worked with various civil society organisations and individuals.

TIPE evaluation


3.5.3 Conclusions

Relevance♦ The REAP project fitted well into the overall strategy of Tacis to invest in sound environmental

policy making. It is complementary to other TIPE projects including the development ofCommon Environmental Policy, establishment of the New Regional Environmental Centres(NRECs) and Widening of the EAP project.

♦ The REAP project aimed to raise environmental awareness in the NIS. This is a key issue forimproving the environmental situation in the NIS in the future, and highly relevant as a subject forsupport from the Commission.

Quality of project designA weakness of the project has been the rather general approach formulated in the TOR, and thegeneral rather than focused project design. Objectives were formulated only in general terms andwere not differentiated by subject, target group or country. The consequence of this “weak projectintervention logic” has been that the project has focused on activities rather than on capacitybuilding and impact.

Effectiveness and efficiency♦ REAP can be counted as a successful project in the TIPE portfolio. Relevance, effectiveness

and efficiency are evaluated as high.♦ The project can be characterised with the phrase: “Plenty of visible activity and a wide civil

society focus. One person interviewed referred to it as “Activities galore and smiling faces allaround”. The Contractor has initiated a large number of activities, and a number of them provedso successful, that there has been a follow up, or continuation, some with other donor support(eco-house in Russia) and some by the beneficiaries in the target groups (video-libraries, NGOtoolkits).

ImpactThe impact of the project has been difficult to assess, partly because only very general targets wereformulated, and partly because of weakness in evaluation methodology14. Still a number ofconclusions can be drawn. The activities themselves have clearly served the purpose of raisingenvironmental awareness in the target groups. The participants of workshops, training events andOPMs were well selected and have obtained skills and insights which are important for theirprofessions. In addition the project has set up a network of video-libraries which can be counted asconcrete improvements of public access to information on environmental issues. The impact interms of policy influence has been limited, likewise the project objective of NGO capacitydevelopment has not been pursued systematically and the impact is assessed as small. In thiscontext a negative factor has also been the delays with the NRECs. These centres would havebeen suitable as inheritors of the REAP know-how, equipment, stocks and staff, and they couldhave played a main role in continuation of the activities and strengthening the sustainability of theoutcomes of the REAP initiatives. However, as the NRECs were not established in time, they couldnot fulfil this role.

Lessons learned♦ Due to the legacy of Soviet times, and the troubles since that time, the level of public

understanding of environmental issues is still very low in the NIS. Most members of theParliament lack a clear understanding of the type of actions needed, which is due to lack ofadequate training in the effective use of their position to support the process of environmentalreform. In addition, many special interest groups such as NGOs and individuals need to haveaccess to high-quality, up-to-date information and support. Likewise, the government sector andthe private sector need to be better informed about problems, causes, solution directions, andbest EU/CEEC/NIS practices for “Aarhus Convention type” developments.

♦ Interventions on environmental awareness are very relevant and needed. However, buildingenvironmental awareness is a complex issue, involving all main sectors in society: government,politics, civil society and the private business sector. Furthermore it is a long and on-goingprocess, which requires a systematic, and expertise based approach, in which the differentorganisations have different roles.

14 This is not a specific problem of this evaluation, but a general problem in evaluation studies.

TIPE evaluation


♦ The positive response from the beneficiaries to the results of the projects confirms that there isa real demand for raising the awareness of environmental issues. This demand exists in allsectors of NIS societies: the civil society sector including the media, the parliament/politicalsector, the government sector as well as in the business sector.

♦ The TIPE programme of the EU can be a valuable channel for mobilising relevant EU expertisein this field and for delivering the required services. However, the Commission does not havenor pursues an awareness raising strategy for environment in the NIS. This lack of strategycomplicates TIPE project planning.

♦ The TIPE project programming can be improved. Future intervention needs to be more specificand more impact oriented. This can be achieved by involving national and regional partners atan early stage in the project design, and by clearly specifying the objectives per target group.

♦ The overall efficiency and impact of the TIPE actions can be improved by strengthening theconnection between the environmental awareness raising interventions and other TacisInterstate projects. This requires amongst other things, involvement of relevant governmentauthorities in TIPE programming, and strengthening the co-ordination between Tacis and otherdonors, as well as strengthening the co-ordination within the TIPE portfolio. A communicationinstrument, like a common data-base or newsletter could be valuable in this context.

♦ Interventions as the creation of video libraries and open parliamentary meetings are veryeffective. A shortage of resources limits the translation of relevant materials in NIS languages.Aid projects can help removing this bottleneck by providing funds for this to localNGOs/foundations, which are developing as documentation centres.

♦ The demand for support from the Small Grant Funds established by the project was high and ingeneral this activity can be judged as successful. The management of the SGF, however wascomplicated and time-consuming. To make SGFs an efficient instrument, the Commission hasto make arrangements, which allow for decision making and accountability at counterpart levelor contractor level.

♦ A general point arising from the REAP evaluation is that in the TIPE programme and projectcycle, the role of the national government agencies responsible for aid co-ordination, i.e. therole of the Tacis Co-ordination Units should be clarified.


The starting point for the recommendations, are five findings of the evaluation team:♦ This TIPE evaluation indicates that, by and large, the past decade can be counted as lost, from

an environment perspective. In the past decade very little of practical value has been done toovercome the abysmal environmental legacy of the Soviet times. The past decade, moreover,has brought very few positive changes of a structural character. Pollution loads have gonedown because of the collapse in industrial and agricultural production, rather than as a result ofhigher awareness, and better management of environmental issues. In this context it is aserious cause for worry that the environment, at the turn of the century, is a low political priorityfor NIS governments as well as for the EU, its main trading partner.

♦ Environmental awareness and knowledge about solution directions to problems, is still very lowin the NIS and Mongolia, despite the fact that some of the most pressing environmentalproblems in the world are found in these regions.

♦ A positive development is the Aarhus convention and its ratification process in the NIS. TheAarhus convention acknowledges the importance of environmental awareness, as a mainprerequisite for building a constituency for environmental policy making and policyimplementation.

♦ The REAP project has contributed to the ratification process, and has initiated a large numberof activities which have proven useful for further increasing environmental awareness of thepublic and for strengthening the capacities of NGOs, media partners and parliamentarians.

♦ REAP has built a good record and name for Tacis amongst the REAP target groups.

Continuation is recommendedThe evaluation team recommends that:1 The Commission adopts Environmental Awareness Raising as one of the main themes for its

PCA Agenda with NIS countries;2 The Commission incorporates support for the ratification and implementation of the Aarhus

Convention as one of the objectives in the Indicative Programme for the TIPE (2000-2006).

TIPE evaluation


Recommendations on project design3 For future “Aarhus Convention” implementation oriented projects, an approach would be

suitable, which is differentiated by country, and which makes the targets explicit with regard tothe range of topics and target groups. Amongst other things, an awareness raising strategy,should make the following distinctions in project design:Objectives for the general public, NGOs, and media (possibly further specified by education,age):♦ Provision with better information on the state of the environment and current environmental

problems;♦ Information and initiation of discussions on governmental (local, national or international)

environmental policies;♦ Fostering of changed ecological behaviour of consumers;♦ Initiation of discussions on better (technical) solutions for environmental policies;♦ Prevention oriented actions.Objectives for policy-makers, legislators, officials♦ Introduction of best practices in legislative systems;♦ Modification of monitoring and control systems;♦ Introduction of clean technologies;♦ Introduction of liability for ecological damage;♦ Improving public relations policies;♦ Activating public participation in decision making.Objectives for the private enterprise sector♦ Elements of the above (possibly differentiated by sector or pollutants)Others♦ Promote that the NRECs adopt the ratification and implementation of the Aarhus

Convention as a theme for their work plans.

3.6 The programme for “Establishing Regional EnvironmentCentres”

The long road to registration: how a means has become an end in, and of itself

3.6.1 Introduction

This section summarises the evaluation of the NRECs programme, a high priority for theCommission’s TIPE in the period 1995-1999, but also a highly problematic project.

The problem. The aim of the NRECs programme is to establish a network of environmentalcentres in the NIS, recognised by its clients as independent, non partisan, non government, andnon commercial. The specific problems, which the project sets out to tackle, are presented aschallenges/ objectives for the NRECs♦ co-operation amongst the apparatus of the state, civil society organisations and the business

enterprises;♦ free access to information;♦ public participation in environmental decision making.

In addition the network concept of NRECs, is intended to promote:♦ co-operation amongst stakeholders across borders for environmental problems with a regional

or international dimension.

Regional initiatives and international donor coordination. The history of the NREC programme datesback to 1989, when the US president agreed with the Hungarian Government to jointly establish aRegional Environment Centre (REC) in Budapest. Within six months after the visit of the president,the REC was registered and operational with premises granted by the Hungarian Government andaround six Meuro in budget pledges from its other founders (USA, EU, and Japan). This REC hasproven its worth for the Phare countries, and in 1995, in the preparation trajectory for the third

TIPE evaluation


ministerial conference on Environment for Europe in Sofia, Bulgaria (October 1995), the RECHungary was requested to study the feasibility of establishing likewise organisations in the CIS.Strategy. The feasibility study presented the following results and strategy to the NIS and EUEnvironmental Ministers assembled in Sofia, and to the NGO conference, which was held inparallel15:♦ The study was carried out in Russia, Moldova and Ukraine, and identified the need for a

NewREC, i.e. an environmental centre to serve NGOs, central and local governments, and thebusiness community;

♦ The NewREC should consist of a network of national offices, co-ordinated by a head office;♦ The objectives of the NewREC should be to (a) foster the participation of all sectors of society in

environmental decision making, (b) support regional co-operation, and (c) offer services ininformation exchange, training and capacity building, and grants for NGOs and local authorities.

Action plan and funding. This presentation had the desired effect. The Declaration of theSofia Conference welcomed the initiatives to establish additional regional environmental centres forthe NIS and encouraged interested donors as well as governments of beneficiary countries to assistin creating a network of such independent centres. A Statement of intend to co-operate in takingfurther steps to establish NewREC was signed at Sofia by Russia, Ukraine, Moldova, the US,Switzerland, Finland, Norway the European Commission and the REC/Budapest.

In the aftermath of the conference, the concept was further endorsed by the European Parliament inits resolution on the Sofia Conference. Moreover, an understanding was reached between theinterested donors, that a NEWREC implementation budget would require some Euro 12 Million forthe period 1996-2000. In this context the Commission took the lead and indicated that it wouldmatch co-financing from other donors with equal contributions from its Tacis Interstate Programmefor Environment


The Commission has taken the lead in this multi-donor supported programme and has allocatedsome 6 Meuro from TIPE -Action Plan budgets of 1993,1995 and 1997. The funds were intendedfor full use in the period 1996-1999, but due to absorption problems only 2,4 Meuro has used so far.

The specific objective of the programme was formulated, in 1996, as follows: To create and supportthe development of NewREC as an international organisation in the period 1996-2000. NewRECwould have a central headquarters (ICIO) with central funding and national offices in the NIS. Theorganisation should be founded by the participating NIS governments and by interested donorcountries. It would have a management board in which NGOs are strongly represented and alsoAcademics are members.

In 1997, when it turned out that the project stakeholders could not come to agreement on thecountry location of the central headquarters (Russia or Ukraine), the concept was changed. Insteadof one central NewREC, it was decided by the donor community and stakeholders in the NIS tosupport the creation of four National RECs (the NRECs) and, to link them in an internationalnetwork (phase 1 & 2), and to expand this network of NRECs to Central Asia (phase 3).

A large number of organisations take part in the NREC programme, including NIS governmentalorganisations (mainly Ministries of Environment), NGOs, academic institutions, and obviously theNRECs themselves. From the Western side, the stakeholders include the European Commission,EU Governments (e.g. Norway), the USA (US EPA, USAID) and NGOs (EU, USA).

NREC has a high political profile and the recommendations on its future are relevant for the overallpolicy orientation of TIPE in the period 2000-2006.

The conclusions, lessons learned and recommendations presented below are based on review of alproject progress reports, dossier studies in Russia, Georgia and Brussels and interviews with some50 involved representatives of stakeholders in the capitals of Armenia, Georgia, Kazakhstan,Moldova, Russian Federation Ukraine, and Uzbekistan.

15 Source: 1st ISB proceedings, status report, May 7, 1996.

TIPE evaluation


3.6.3 Conclusions

RelevanceThe programme to establish regional environmental centres in the NIS was well intended, itaddressed a number of key issues and the Commission gave it high priority in the period 1995-1999.

Quality of project designIn the set up, it was assumed that the new charter of REC Budapest, established in 1990, would bea good model for establishing a NewREC in the NIS in 1996. This has turned out to be a wrongassumption for a number of reasons:♦ The political trend in the NIS in 1995 was to move away from Russian government inspired CIS

wide organisations towards national or problem focused co-operation;♦ The weakness of NIS environmental authorities has prompted them, as intended co-founders to

have serious, and partly justified, hesitations about a REC concept which would furtherstrengthens the NGO sector, a sector with a history of being an adversary of the governmentrather than a partner;

♦ The strength of the existing NGO network and their dependence on external donor finance hasmade key actors in the NGO community hesitant about the REC concept. In particular, inRussia, a part of the NGO sector had misgivings over the fact that governments were to be co-founders of an organisation which could become a “competing” channel for internationalcommunity support targeted to civil society’s involvement in strengthening the environmentalcause.

Effectiveness and efficiencyFour years after the Sofia conference, in November 1999, when the evaluation team made its fieldvisits, only the REC Moldova was functioning. The Commission has little to show for the rest of itsefforts in support of the NRECs programme, and, in the last two years, very little progress has beenmade. In the 1st quarter, the Commission, together with other founders finally succeeded to breakthe deadlock for the establishment of the NRECs in the Russian Federation and in the Caucasus. Atthe same time USAID support for NREC in Ukraine helped to overcome the last registration hurdlesthere.

Project cycle managementMost of the problems that have arisen, find their origin in one element of project design, i.e. the ideathat the national governments in the NIS should be co-founders with the Commission in the NRECs.This created foundation and registration problems, which could not be solved, despite considerableefforts by all stakeholders. Despite being well informed about these registration problems, andabout the delays, which this caused, the Commission did not take decisive action to break thedeadlock. One option would have been to change the design and support registration of the NRECsas foundations without government participation or as NGOs. The Commission did consider this,but did not pursue this course, because in their view (DG XI), this would diminish the non-partisanappeal of the NREC. The price of this course of action can be considered high; the project hasperformed very poorly in terms of effectiveness and efficiency, in several countries. (In March 2000,the NREC in Russia was finally registered without the government as co-founder).

ImpactDue to the delays in project execution, the evaluation team did look at realised impact, butassessed the potential impact of the national NREC linked in an international network. Theevaluation team observes that the pragmatic switch in concept from one central NewRECorganisation to several national based NRECs, may weaken the appeal of the concept for donors.In particular the NIS wide information exchange function seems lost or will at best be much moredifficult and costly to organise. On the other hand the ownership of the national NRECs may behigher but at what costs? A new feasibility study was not made on the sustainability of nationalRECs, and moreover the NREC establishment process has been donor driven instead of demanddriven.

Lessons learned♦ Replication of a complex programme, in different surrounding, needs careful and critical

analysis of the complexities of the new setting. This requires not only expertise of the subject

TIPE evaluation


matter, but, also a devils-advocate approach. The latter has been lacking in the originalfeasibility study, undertaken by REC/Budapest.

♦ Ownership building is of paramount importance in establishing new organisations. It is also aprocess, which has its own dynamics, which requires a flexible and nurturing response from thestakeholders. The Russia experience shows the dangers of starting up such a process,discontinuity in the TA and then trying to override and ignore a ownership building processtaken in hand by a group of NGOs. The result has been a disinterest in a large segment of theNGO sector in the NREC concept.

♦ The project management arrangements made by the Commission for the TIPE are insufficientto deal adequately with complex projects. The NREC is just one example of the manyambitious, complex and highly relevant projects that constitute the TIPE portfolio. Theinformation flows towards the Commission are sufficient to alert the management aboutimplementation problems in a timely manner. However, this “monitoring” information is notdesigned to analyse the deeper, underlying problems, and such analysis would be needed toprompt a Task Manager to take decisions to change project design with a view to regaining lostmomentum.

♦ In parallel with the NRECs programme, support programmes are needed for Environmentalministries and other state authorities with environmental management tasks. A key problem forNREC development and operations has become the fact that central and local governmentagencies have remained very weak and ineffectual with respect to their main tasks: legislation,enforcement, communication and integration of environmental considerations in decisionmaking by the different stakeholders in society. In this setting, the practical effectiveness of themain tasks of the REC organisations may be doubtful. A NREC aims to provide information, toraise awareness, to help develop local expertise and organise horizontal exchanges ofinformation to support the integration of environmentally sound behaviour at local governmentlevel. It cannot, however, replace the role of the state in these fields. As long as the stateauthorities on environment are weak and ineffectual themselves, the potential for the NRECs todevelop as a constructive and non-partisan organisation will be limited. Moreover, without thestate as potential partner, a strong NGO sector can only play a adversarial /political role and nota partnership role.

♦ The Commission’s powers to solve problems at a higher political level in the NIS are verylimited. This factor should be taken into account when supporting ambitious projects. The fouryears of struggle, with the founders and registration issue, are a testimony of the weakcommitment of the beneficiary governments towards the NREC concept, and the severe limitsof the aid programmes (Tacis, and US-EPA) in influencing decision making processes in thepartner countries. In Russia, the Commission made a political issue out of the NREC, both withletters to the Government and by discussing it in the PCA sub-committee dealing withenvironment. On the NREC Ukraine, US-EPA enlisted the support of the vice president. Theseefforts have, perhaps, not been in vain, yet, so far, the desired registration has not taken placeand deadlines have passed without comment (written in November 1999, by the end of the 1st

quarter of 2000 all four NREC are registered).


The starting point for the recommendations is the finding of the evaluation team that the levels ofcommitment to the NREC programme vary substantially per country and region.♦ In Moldova, there is a strong commitment to the NREC from all stakeholders. The challenges

are “3rd phase development problems”. They concern the orientation and content of its function,the development of environmental expertise and sustainability for the period 2000-2006.

♦ Also in the Caucasus, there is a strong commitment to the REC concept, despite the problems,delays, disappointments and irritation that have arisen between almost all involved parties.Moreover, the completion of the founding and registration phase seems near, with or withoutAzerbaijan as co-founder.

♦ In Central Asia, the first steps towards establishing a NREC in Almaty have been set. Variousstakeholders have verbally expressed their commitment, and the co-founding governmentshave agreed on Almaty as the location. The NGOs, however, are generally weak, ethnicdifferences are an issue, and the styles of government are autocratic. Thus, the commitmentsmade should be treated with caution. Furthermore, the hesitations towards cross border co-operation are strong in Central Asia which justifies a more closer analysis of the feasibility ofexpanding the NRECs network into the region.

TIPE evaluation


♦ In the Russian Federation, large parts of the NGO community and existing independentfoundations operating in the field of environment have lost their faith in what they consider aTacis driven rather then a Russia needs driven project. Moreover, the relevance of operatingthe NREC in Russia, in the year 2000 remains to be seen. Other co-operation channels exist,and the NGO sector already provides most of the services envisaged for the NREC.

Recommendations on continuation of activities in 2000-2006Despite the problems experienced in the past, the evaluation team recommends to:1. Continue Commission support for the NRECs programme in Moldova and Caucasus from the

TIPE;2. Closely monitor the usefulness of NREC in Russia and the Ukraine before committing further

funds for operations in 2001;3. Expand in Central Asia, with caution.

Recommendations on programme design1. Incorporate in the TIPE Indicative Programme for 2000-2006 the principle that the individual

NRECs can have three roles:♦ NIS counterpart organisations for TIPE programming;♦ Recipients of Commission grants on the basis of work plans endorsed by the NREC boards;♦ Contractors for implementing Commission projects in the field of environment.

2. Create within the TIPE Indicative Programme for 2000-2006, and in the supporting AnnualAction Plans, a blocked budget allocation for the NRECs.

3. Incorporate in the TIPE Indicative Programme the principle that through TIPE, Tacis can makegrant allocations to individual NREC on the basis of their annual and/or multi-annual work plans.

4. Define for each NREC a geographical area of operations, which can be considered interstate,for the purposes of the TIPE programme funding for activities. For example Georgia –concerted action in at least two of the three Caucasian states; Moldova - concerted action withUkraine or Rumania.

5. Undertake a new feasibility study for the existing NRECs and for establishing NRECs in othercountries in the NIS. Amongst other things the sustainability and value added should be lookedat, as well as the modus of co-operation for NIS wide information exchanges based on ICTtechnologies and central web-site administration.

TIPE evaluation


4 Analysis and findings at programme level:Part I: issues of content

4.1 Some figures

Of the total Tacis Programme Funds, over the period 1993-1997, slightly under 2.5 percent hasbeen allocated for specific environment projects. Over time this percentage has increased withsome percentage points after 1997. In that year pressure from the European Parliament and EUMember States put environment on the Tacis agenda and it became for the first time an option forthe Tacis partner states in the NIS to incorporate environmental projects in the National IndicativeProgrammes. The Russian Federation and Kazakhstan have made use of this with a small numberof projects in the 1997, and 1998 national programmes.

The Tacis interstate programme is by far the largest channel for environmental projects in Tacis. Inthe budget years 1992-1997, 52.5 Million Euro was allocated to environment projects. Thisrepresents some 70% of the total allocations for environment in the various Tacis programmes. In1998 and 1999, the allocations for the environment component of the TIP amounted to 10.5, and8.5 Million, respectively. Details on funding are provided in Tables 2.1 and 2.4 presented in chaptertwo.

4.2 The political economy of environment

In the NIS, a new generation has grown up since the time when public attention for environmentalissues was a motivating factor for political action during the Glasnost and perestroika period in the1980’s. Now, after more than a decade of unparalleled economic decline in living standards, ahesitant revival can be observed in this public interest for environmental issues, but the awarenesslevels have substantially dropped over the decade. At local level, NGO’s, local governments andother civil society organisations, including some media, are becoming visible as serious actors inthis context. For the other sectors in society: enterprises, parliaments and central government,environment has become and still remains a non-issue. Two examples may illustrate this.

In 1989, in the Russian Federation, the Minister for Environment was a member of the SecurityCouncil. Ten years later the Ministry has been split up and has been downgraded to a StateCommittee. In Georgia, in November 1999, the Ministry of Environment does not have enoughfunds to pay its telephone bills and its Web Site is not available any more. In this situation acondition put by the WB/GEF, that the littoral states fund together US $ 120,000 for the Black SeaSecretariat, becomes a major barrier for releasing a US $ 70 million fund for the Black SeaProgramme.

The budget allocations for Environment and for TIPE illustrate better than words that environmenthas low priority not only for the NIS states but also for the Tacis programme. They are indicative forthe extent to which environmental concerns have lost political attention in the EU member statessince the end of the ‘80s. One may or one may not regret this situation, but an elaborate system ofenvironmental management is already in place, and the basic infrastructure in the fields of air, waterand waste, health is functioning well. This management and infrastructure capacity is not in place inthe NIS, and it will take, in the best of circumstances, decades to build. It is clear that the economicdire straits in the NIS severely restrict environmental investments of any kind by NIS governments.In the ministerial conferences -organised in the context of the Environment for Europe Process-these issues are discussed at length. Conference papers and documents draw attention to the needfor the donor community and particularly the European Union, to actively promote thatenvironmental concerns become a fully integrated aspect of economic decision making and

TIPE evaluation


management in the NIS. Such decisions will shape the future and thus determine the framework forthe evolution in the state of the environment in the NIS. Moreover, an important consideration foraction is that the size of NIS ecosystems and natural resources, makes environmental managementand active protection of ecosystems of world-wide significance. These are tasks which westerncountries should set themselves, if they correctly understand (i.e. price) the implications of thecurrent environmental issues, and if they are seriously committed to concepts of sustainabledevelopment and policy targets as expressed in the resolutions of the Rio conference.

Economic decisions that entail long term environmental degrading of environmental conditions,threaten to hamper the development of civilisation in the next millennium just as much asnationalism threatens human rights. These threats need to be contained at the earliest possiblestage, and this requires intervention. In the human rights area, key politicians voice the opinion thatthe UN should be given the mandate to interfere with military means, luckily in the field ofenvironment this can be done with trade and aid measures. The large majority of the interlocutors ofthe evaluation team in the NIS, have on this issue spoken with a common voice. They expressedthe view that their only hope is that the European Union takes the lead in this field, and puts its“Tacis” monies where its mouth is.

The funding figures mentioned above are an indication for the finding in this evaluation that over theperiod 1991-1999, the EU has mainly paid lip service to the environment. In this respect, theevaluation team has found that stakeholders in the EU and NIS are apprehensive about the new(draft) regulations of Tacis. They fear that in the first seven years of the next millennium,environment will remain a low EU priority for co-operation with the countries on the other side of thefuture EU borders.

The conclusion which can be drawn from the above is that TIPE has operated and is likely tocontinue to operate in very adverse conditions, both in the NIS and in the EU. Moreover, one of thefindings of the evaluation is that these adverse conditions have had a negative effect on overallprogramme performance.

4.3 Validity of the programme intervention logic

This section deals with a number of key questions regarding the validity of the TIPE programme logic:♦ To what extent has the TIPE a distinct identity, vis-à-vis other Tacis support instruments, which is

of operational guidance for programming;♦ What were the main features of the TIPE policies and strategies during the period 1993-1999,

and what were the dynamics in their evolution?♦ What have been the central assumptions and logic behind the environmental projects in the TIPE

annual Action Plans for the years 1993-1997; How valid did this TIPE programme logic prove tobe, and why?

These questions have partly been discussed in chapter two. Some further comments are madebelow.

4.3.1 Think global, plan interstate, act local?

“The Inter-State programme aims to assist the New Independent States (NIS) and Mongolia toidentify and pursue solutions to problems, which are of an inter-state nature. As such, it operates ona basis analogous to the European Union’s concept of subsidiarity since it only includes actions,which are best undertaken on a multi-country, rather then on a national level. Projects included inthe Inter-State programme are designed to complement and reinforce those undertaken in the Tacisnational programmes, Cross Border Co-operation and Small Projects Programmes”Source: TIP Action Plan 1997.

The above description contains a number of key words on the identity of the TIP: problems are ofinterstate nature, multi-country action, subsidiarity, and complementarity. The evaluation shows thatthe first two aspects have clearly guided TIPE programming and project selection. The last twoaspects, however, have not featured strongly in the field of environment.

TIPE evaluation


The principle of subsidiarity stands for tackling problems at the most appropriate administrativelevel. In the Interstate Programme, this is assumed to be the interstate administrative level.However, it is precisely at that level that the administrative organisations at CIS level haveweakened over time. In operational practice, the TIPE projects have had virtually no support fromCIS co-ordinating bodies, and most projects have operated without a clear counterpart organisationin place. Furthermore, the case studies show that project efforts to support interstate type co-ordination have generated low value for the money used.

Complementarity between interstate and national programmes has been largely absent and theevaluation team finds that this has weakened the validity of the interstate approach. On the basis ofreview of the history of Commission support for the Environment, the evaluation team finds that, bydefault rather than by design, the Tacis Interstate Programme has become the main Tacis channelfor co-operation with the NIS in the field of environment. Due to the non-eligibility of the environmentas a priority sector for the Tacis national programme, complementarity has not taken place. Theevaluation cases studies, especially the Black Sea programme and the CEP project, clearly showthat TIPE, to be successful, needs follow up and concerted action with programmes implemented atnational level. The TIPE interventions take place both at international level and at local level, wherespecific activities are being developed. These activities, particularly in the seas programmes, arelike islands, not integrated at an intermediary national level, e.g. institutional setting, legaldevelopment strategy or policy priorities.

“It is certainly true that policy and strategy making for improvement of the condition of the Black Seashould be a combined and concerted action of the six littoral countries. However, if this is notfollowed up by activities on the national level in each of the countries, then no achievements will bemade” (Black Sea synthesis report).

Thus the evaluation team concludes that the Commission has not sufficiently recognised that in theenvironment component of the TIP, the intended synergy/complementarity with the nationalprogramme actions could not materialise (because environment was not an eligible concentrationsector for Tacis prior to 1997).. To compensate for this, it would have been more transparent if, forthe TIPE, a separate set of project selection criteria had been developed and the differences inintervention logic had been spelled out in a separate section in the TIP strategy papers and ActionPlans. The practical outcome of this omission has been the neglect of national level legislation inthe design of the regional-seas programme, and this is one of the factors explaining the limitedresults and low potential for impact found by the evaluation team.

4.3.2 The SWOT of the TIPE

The case studies presented in summarised form in chapter three, generated a number of findingswhich are listed below in a SWOT (Strengths, Weaknesses, Opportunities and Threats) framework.

Strong points♦ TIPE Programming is based on priorities arising from the Environment for Europe Process and

from donor-co-ordination♦ The focus on specific themes has allowed Tacis to make substantial contributions to the

regional seas programmes and to play a leading role (even if the Commission has beenreluctant to make use of this).

Weak points♦ Absence of national programmes to complement and embed TIPE actions in national level

measures♦ The lack of a clear NIS counterpart for programming♦ The absence of (inter-state) partner organisations (PO) in some of the projects♦ Unsatisfactory procedures for involvement of “owners” in the programming♦ Insufficient environmental expertise inputs in TOR writing♦ Absence of a built in structured exchange of views on TOR with experts inside the Commission

Services (DG XI), and with expert organisations outside the Commission (OECD, WB, UNDP,NGO’s etc.)

TIPE evaluation


♦ Totally inadequate staffing levels for Interstate Project Cycle Management in Brussels.♦ Tacis monitoring resources not appropriately organised for interstate type projects

Opportunities♦ The economic crisis is effectively reducing pollution loads (less nitrogen in agriculture etc.)♦ Technical solutions are available for win-win situations, especially in combination with

Investments in the restructuring and reorganisation of agricultural production, industrialproduction, oil and gas extraction and transportation.

Threats♦ Funding shortages from NIS stakeholders disrupt TIPE Action Plan implementation♦ Donor procedures are inflexible which makes concerted actions unreliable♦ The absence of strong Partner Organisations make TIPE projects more dependent on the

quality of the Tacis Contractor and vulnerable to weak performing contractors. Thus, the qualityand commitment of experts becomes a key factor in effectiveness and efficiency.

♦ High level political commitment and economic sanctioning is a pre-condition for real impact.Environment issues do not at this stage of post communist capitalism have the political cloudrequired

♦ Riverbasin and seas pollution from regional conflicts (Chechenya, Kosovo, etc.)♦ Regional disintegration, particularly in the Caucasus and Central Asian States.

4.4 Programme relevance issues

This section discusses some key questions regarding programme relevance:♦ To what extent does the TIPE implement EU and Commission environment policy towards the

NIS?♦ To what extent has TIPE succeeded in identifying, and selecting for support, those priority

environmental problems, which can be successfully addressed only on the basis of an interstateco-operation approach;

♦ To what extent have TIPE projects addressed environmental problems which can be consideredrelevant in terms of their NIS environmental priority, and their political priority in the NIS and EU;

♦ To what extent has the TIPE succeeded in addressing problems for which the Tacis TA approachand EU experience has value added to offer, compared with support from bilateral, or othermultilateral aid channels;

♦ To what extent can one consider the TIPE a relevant programme, when weighing up the abovedimensions of relevance?

The above questions have been dealt with extensively in chapter two. The comments given belowlook at the relevance issue from a slightly different perspective.

4.4.1 The relevance of different types of projects

The evaluation team has observed that TIPE portfolio contains three types of projects:♦ “Risk” type projects, which address real environmental problems with interstate dimensions.

These fit in with the “immediate relief objective of the TIPE.♦ “General” type projects which focus on awareness raising, EAP, training, investment

preparation etc.♦ “Environmental Agency” building type projects indicated with the name NEWREC’s, which are

intended to become non-partisan champions of environmental causes and recipients of donorsupport for their work programmes.

The “risk” type projects (e.g. regional seas, Chernobyl) can be considered highly relevant. Theseproblems are recognised world-wide as requiring international co-ordinated attention. As such theyfit perfectly with the TIPE identity (derived from its objectives). They can be considered as highpriority projects of world wide significance and as such have also with great visibility and publicawareness raising potential. In these projects the EU provides support in parallel to UNDP (GEF)and World Bank.

TIPE evaluation


The “general” type projects are focused on public awareness raising, EAP, EAP related training andinvestment preparation. They are largely linked with the Environment for Europe process and theOECD EAP Task Force. They address important issues but they do not fully fit in with the TIPEidentity. From a TIPE perspective, in which the first selection criteria (see section 2.4) is consideredto be the main one for shaping the identity of the TIPE these general projects are less relevant.

The New RECs show a mixed performance on the relevance criteria. The non-partisan conceptproved a difficult base to communicate clearly and the different levels of NGO capacities in the NISmake the concept of uneven relevance across the NIS.

4.5 Programme effectiveness

This section deals with the following question:♦ To what extent did the completed TIPE projects achieve their project objectives; and to what

extent can one, with reason, assume that projects, which were still ongoing during theevaluation (November 1999), will achieve their objectives?

To assess programme effectiveness, the evaluation team selected a number of key projects fordetailed evaluation. In total these projects (presented in chapter 3) account for approximately 60percent of the Total portfolio. The budgets of the six projects/programmes amount to 30 millionEuro. This compares with 52.5 of the total TIPE budget for the period 1992-1997. This selectionwas made in such a way that it gives insight in the performance of the programme as a whole.However two main projects have not been looked at in same detail: WEAP, and the projects linkedwith the Chernobyl affected areas. This limitation also limits the extent to which the findings can beconsidered fully representative. The projects showed a mixed performance on effectiveness. On asimple scale of High, Medium and Low, the evaluation findings substantiate a classification ofmedium level effectiveness. Details are provided in chapter 3, and in the 3d progress report, whichcontains the synthesis reports for the six case studies.

4.6 Programme efficiency

This section summarises the findings on the following questions♦ How can TIPE project budget resources be assessed in relation to project results?♦ To what extent has the TIPE been an efficient channel for promoting solutions to

environmental problems?

The two questions deal with different dimensions of efficiency. The answer to the first question isfound in the project performance, summarised in chapter 3; the answer is that the overall efficiencyfluctuates from low to medium, when measured against the high, medium, and low scale.

The second question looks at the causes for this score on efficiency, and asks whether designaspects have a structural influence on efficiency.

The findings of the evaluation team in this respect are as follows:

It is a misconception that that inter-state contracts are efficient instruments. This is a key point in theexplanation of the poor performance. The evaluation shows that in most cases they are not efficient.Efficiency gains in dissemination of common approaches, training materials development anddistribution etc., are more then offset by the different needs of the recipients and by themanagement complexities for the contractors and also for the Task Manager in Brussels. Mostproblems arise at the local level instead of the general project level. Therefore one Interstate projectcan generate the same amount of work as 13 distinct projects. Thus the workload of the (a) TIPETask Manager(s) with ten contracts should be compared with a national Task Manager with 130projects. This is of course a mission impossible, and that is what the evaluation shows.

TIPE evaluation


The Commission appears to have not realised that the workload involved would be so high, despitethe fact that other evaluations have pointed this out earlier.

On this issue of efficiency the OECD/EAP Task Force Secretariat has made a number of commentsin its Memorandum to the House of Lords (UK), 15 June 1998 which we reproduce in the boxbelow.

“Tacis has been working to satisfy the operational requirements of the EC, on the one hand and tocope with unstable and sometimes obscure institutional arrangements in the NIS on the otherhand. Lack of staff, and an unclear division of tasks in some Tacis environmental projects havehindered project management.”

“NIS participants to the EAP Task Force have often provided the following comments on thesubject of efficient resource management in Tacis:♦ Too few staff:♦ Confusion within the NIS about who is doing what in Tacis and lack of co-ordination between

programmes, e.g. interstate vs. country environment budgets;♦ Overlap of management and planning of projects;♦ Unclear role of Tacis in-country offices; there are no or very few staff in the country offices

which have the mandate or expertise to deal with environment.”Source: OECD, 1998

The combination of the high workload of the TIPE NIS wide projects, and to few Task Managers(one fulltime and one parttime) explains why the TIPE projects are not sufficiently managed in thesense of guidance, steering, problem solving etc. The TIPE Task Managers simply do not have thetime to go beyond programming and contract budget issues. At times in the past not even progressreports were formally approved. The above observations indicate that the TIPE is not efficient aschannel for TA, and management shortages have in some projects contributed to low efficiency.

4.7 Programme sustainability and impact

The following questions have been answered in chapter three for the six project cases studies:♦ To what extent can project results be maintained or continued after the contractual ending of

Tacis funding?♦ To what extent has the TIPE succeeded in making lasting contributions, promoting solutions to

environmental problems, and what are these contributions?♦ What can be considered the impact of the TIPE programme. To what extent is this impact

bigger than the sum of project impacts?

The findings are summarised in Table 4.1.

The table shows for instance that the REC project (in Moldova) has had some impact (low, but notzero) on the identification of environmental problems in Moldova, on policy formulation, on strategydevelopment and on integration on environmental policy in economic reform policies of theGovernment policy. The evaluation did not observe impact of the REC Moldova on other policyaspects. Reading horizontally, the table shows amongst other things, that all six projects had someimpact on strategy formulation concerning the problems they address, the extent of this impactvaried.

TIPE evaluation


Table 4.1 Programme Impact indicationsCase studiesR

EC

RE

AP

CE

P

BS

EP

WA

RM

AP

Caspian

TotalPolicy life cycle phases, anddimensions for impact measurement

Problem identification L H M H M M MIncreased availability of environment informationEnvironmental impact assessmentPublic awareness raisingPolitical awareness raisingPolicy formulation L L M M L L LMRecognised need for interstate combined actionEnvironmental priority settingPolitical priority settingPolicy target formulation and BudgetingStrategy development L M M M L M LMTarget group participation in strategy developmentTarget group policy developmentAction plan developmentImplementation instruments developmentManagement capacity developmentAction Plan implementation - - L M L L LLegal framework developmentExpertise development and research priority settingSpecific target group actionsCommunication policy implementationMonitoring and enforcement - L - L M L LMonitoring capacity developmentFeedback and enforcementImpact evaluation and feedback - M - L L L LTarget group attitudes and behaviour monitoringCapacity building amongst stakeholdersRelations and co-operation between stakeholdersImpact measurementEnvironmental situation monitoringPolicy and strategy fine-tuningPolicy integration L M ML ML L L LPolicy integration with other walks of life

Overall Impact L M M ML L ML ML

The table should be read as follows: the first column gives (in bold type face) the different phases inthe lifecycle of a policy, which aims to solve or bring under control an environmental problem. Undereach phase the different dimensions for impact measurement of a policy are mentioned. The righthand side of the table lists the six projects taken as case studies in this evaluation. The impactvalues found during the study are given as High, Medium or Low.

An important conclusion of the study is that there is evidence that four of the projects have actuallymade a contribution to changing attitudes, and/or creating new institutional structures, which havethe potential to positively effect the behaviour of the public and target groups in an environmentalfriendly direction. The projects concerned are the three regional seas projects and the awarenessraising project.

TIPE evaluation


4.8 Overall assessment of programme value added

This section deals with the main question of the ex-post part of this evaluation:♦ Has the programme generated sufficient impact, to justify the use of Tacis funding, taking into

account other funding priorities in the NIS and impact potential of other interstate and nationallevel programmes?

Weighing up the findings, the evaluation team concludes that the TIPE programme has on averageprovided low value for money, and better value was in practice achievable.

The conclusion can be further qualified as follows:♦ The high relevance of the programme is good finding, however this has not been given much

weight. In principle there is a shortage of TA support in environment and in other sectors, andhigh relevance should be the standard. Moreover design weaknesses have reduced thepotential impact of the projects.

♦ The low impact does count heavily in the final assessment, A major problem for impact hasbeen the weak institutional structures at interstate and national levels for utilisation and followup of the TA offered by the Commission (low absorption capacity).

♦ Medium level effectiveness and low efficiency levels imply that more could have been achieved.Part of the problem lies clearly in the unfavourable political, and economic situation in which theTIPE operates. Another factor is that the Commission has not given its programmeimplementation departments (now SCR) the manpower to effectively manage the projectcontracts. An third factor is that the Commission has little reliable feedback on projectperformance and impact. The monitoring system is not fully geared for providing acomprehensive assessment of NIS wide inter-state projects in the field of environment.Moreover, the projects themselves are not designed to provide information on effectiveness andimpact. Most projects lack in clarity as far as their performance targets are concerned.

TIPE evaluation


5 Analysis and findings at programme levelPart II: issues of management

5.1 Project cycle management issues

The methodology for the study lists the following questions as subject for the evaluation ofprogramme cycle management aspects:

♦ How effective have the TIPE programming procedures been for project selection?♦ Are project budgets generally generous or meagre for the required activities?♦ How well are the projects designed for visible results?♦ Are projects sufficiently designed for impact?♦ To what extent have Commission procedures for transparency been applied in the tendering

practice for TIPE during the period covered by the evaluation?♦ Does the Commission have sufficient instruments to influence project implementation?♦ Does the Commission have the sufficient good quality information on project implementation to

manage the projects and take responsibility for the results?♦ Does the Commission have the capacity to follow up on information from progress reports and

monitoring reports?♦ To what extent does the Commission use the feedback on project results for programming

purposes?

Most of these questions have been dealt with already in chapters 3 and four of this report. Thefollowing table shows, in bullet point style, strong and weak points of the TIPE project managementcycle. For programming issues the reader is referred to chapter 2.

PCM phases Strong and weak points

Programming – project identificationStrong♦ TIPE APs are approved on time by the Tacis Committee♦ Co-operation with UNDP and WB on regional seas♦ TIPE funding linked to the Environment for Europe process

Weak♦ Unclear decision making process for partner states♦ Insufficient communication with NIS ministries and National co-ordinators

From identification to formulationStrong♦ Good initial co-ordination in –house and with interested donors

Weak♦ Unclear institutional arrangements♦ ToR lack focus and are weak on environmental best practices

From formulation to contractingWeak♦ General slowness in Tacis/EC contracting procedures♦ Selection process has not always succeeded in screening on insufficiently

qualified contractors and weakly designed proposals

TIPE evaluation


Contract implementationStrong♦ Some excellent contractorsWeak♦ Most contractors have not been pro-active enough to add value through

networking and donor co-ordination♦ Dispersion of activities and aims limits the achievement of sustainable results

Implementation monitoring and contract managementStrong♦ Active, concerned, professional staff as task ManagerWeak♦ Environmental expertise in monitoring teams generally too low to signal

problems with project design and content; This problem is compounded by thecountry by country approach through which the interstate overview is lost

♦ Task Managers swamped with work, and ineffective in follow up on signalsfrom contractors and monitors

Evaluation and feedback to programmingWeak♦ Systematic data collection on performance and impact are lacking♦ No lessons learned♦ design mistakes repeated♦ contracts extended and renewed also with low performing contractors

Programme fundingWeak♦ Funding levels fixed arbitrarily, without clear link to funding requirements of the

portfolio♦ 2nd allocations in APs have been made to projects which had not yet started

under the 1st allocation

TIPE evaluation


6 Conclusions and Lessons learned

6.1 General conclusions

With due reference to the information provided in chapters three and four, the conclusions of theevaluation team can be presented without further nuances as follows.

TIPE performance criteria Indicator value:Relevance♦ Seas projects:♦ General projects:♦ NRECEffectivenessEfficiencyImpact and SustainabilityOverall assessment

HighHighHighMediumMedium/LowMedium/LowMedium/LowMedium/Low

The evaluation shows that TIPE has selected important problems, which can be addressed andsolved by pooling the resources of the EU and other international donors. That this happens isessential from a world ecological heritage point of view. The evaluation also makes clear that highrelevance does not imply success. Several factors account for this difference between highrelevance and low/medium impact. These include: adverse external circumstances, weaknesses inprogramme design, and the by now well-known limitations to the efficiency of the Commission as anaid provider to the NIS in the 1990’s. These aspects are reflected in the findings of the evaluationthat TIPE performance scores on most criteria averages as between medium and low. In theexperience of the evaluation team these scores rank below average in comparison with theoutcomes of other programme evaluations.

See paragraph 6.2 overleaf.

TIPE evaluation


6.2 Supporting conclusions at programme level

Evaluation criteria Main performance indicatorsKey questions and/or evaluation hypotheses

Scores

1. Programmerelevance

1.1 TIPE has a clear identity as an aidchannel?

1.2 TIPE is oriented towards NIS needs?1.3 TIPE fits in with EU environmental policies

and international priorities ?

H

HM

Seasprojects

GeneralProjects

1.4 NIS states are committed to the TIPEprojects?

1.5 EU member states are committed to TIPE?1.6 Programme portfolio is in line with

programme aims? (NIS common approachrequired)

1.7 EU/Tacis expertise can make a difference?1.8 EU funding is adequate to make a

difference?

M

M

H

M

M

L

M

M

M

L2. Quality of

programme design2.1 Programming procedures keep TIPE

focussed on its identity?2.2 Programming is NIS demand driven?2.3 Ownership is built into programming?2.4 Sustainability is built into project design?2.5 Donor co-ordination and EU and RF

environmental expertise target projects forefficiency, tangible results and impact?

2.6 Synergy with other programmes?2.7 Programming capacity2.8 Project cycle management capacity2.9 Monitoring set up for TIPE2.10 Contractor selection criteria

H

MMLL

MLLLM

L

LLLL

MMLMM

3 Efficiency 3.1 Cost-effectiveness M/L M/L4 Effectiveness 4.1 Actual versus planned project

achievementsM/L M/L

5 Sustainability andImpact

5.1 Evidence of problem stabilisation/control5.2 Evidence of problem abatement

ML

ML

6 Overallassessment

M/L M/L

TIPE evaluation


7 Recommendations for futureprogramming

The main recommendations for future programming of the TIPE are presented in the executivesummary. In this chapter 7, the rationale for some of the recommendations is elaborated on.

7.1 The rational for continuation of TIPE

Despite the low impact and low efficiency observed for TIPE by the evaluation team, the relevanceof the TIPE justifies continuation and even substantially higher funding levels. The rationale for thisrecommendation is as follows. Recommendation is that the Tacis programme and in particular theTIPE are essential instruments for keeping environment on the agenda in the NIS and forimplementing the commitments towards environmental protection and sustainable developmentmade by the EU in its PCAs with the NIS.

7.2 Mobilise (member state) support for environment

Staff members of environmental ministries in the Members States, members of the EuropeanParliament and staff members in the Commission services who have knowledge of …..♦ NIS environmental degradation;♦ the risks involved for public health;♦ the fragility of NIS ecosystems which are of vital importance for life on this planet and its bio-

diversity;

…..should be mobilised to fight to get environment back on the agenda in the Commission and inthe NIS partner states.

The EU more than any other trading and political partner for the NIS has the geo-political clout andarguably the moral obligation to put environment back on the agenda. Moreover, the issues that canbe addressed by TIPE fit in well with EU international policies on environment, and the EU has therelevant expertise to make its programmes a success. Such action would give a clear signal to theNIS partner states and give hope to the public and concerned organisations.

In this context, the planned EC communication on co-operation in the field of environment with theNIS would be a useful policy instrument guiding the environmental content of the Tacis programme.

7.3 Considerations for Tacis programming (2000-2006)

Environment should be incorporated as an obligatory sector in the Tacis national programmes. Formore impact, TIPE projects need synergy with national programme environmental projects.To have impact in the field of environment in the NIS, the European Union should puts its moneywere its mouth is. This means that the Commission should show more commitment to therealisation of its policy statements and declarations as contained in the PCAs, ministerialconferences, etc. This includes two aspects.

First, this evaluation shows that effective environmental action on international or regional issuesrequires instruments, which cover the range between THINK GLOBALLY, and ACT LOCALLY. Aninterstate programme is by itself not enough. A properly designed interstate project requires projectactions at national levels which places the actions at local level in the proper national levelframework.

TIPE evaluation


Second, a large share of environmental action needs to take place at micro level. A much closerintegration with the SME sector is needed. In this respect, the USAID reorientation of strategy isillustrative: environment is no longer a separate sector, but entirely channelled through theenterprise support sector.

Fully integrate environment in Tacis indicative programming for the national programmes and inproject design. For impact, environment considerations should be fully integrated in the Tacisproject portfolio.To become a mature programme, in line with internationally proclaimed EU environmental policies,environmental considerations should become an integral part of the Indicative Programmes underthe Tacis national programmes. In this respect, the concept of putting the environment sector as achoice option in the national programme menu of six options is inconsistent with effective action inthe sphere of environment. It is to be feared that this in effect will mean a setback in environmentalprogramming compared to the period previous to 1997. Faced with the Tacis guideline thatcountries can pick only three of the six options, it is reasonable to expect that very few countries, ifany, will opt for environment. Whatever the rationale for this restriction has been, it clearlyoverlooked the synergy links that should be developed between national and interstate programmesfor environment projects. Moreover, this menu approach is in deviation with the basic principle inEU environmental policy, which defines environmental considerations as a sector cross cuttingtheme, to be included as horizontal tasks rather then as vertical (sector) tasks in the Tacis indicativeprogramming exercise and in general project design.

Advertise more the economic benefits of environmental investments.The main obstacle to pollution abatement measures remains the need for investments. However thesize of the investments needed to achieve significant results, as well as the time horizon beyondwhich the net benefits become positive are often overestimated. More attention could be given topricing of environmental changes, cost-benefit analysis of investments in clean technologies, etc.Examples encountered in this evaluation are bio farming and decreases in tourism because ofpollution. The main motivation behind policy choices are economic, and only a clear perception ofnot too distant benefits can bring about real commitment of national authorities.

Specific recommendations on possibilities for various actions are listed below.

7.4 Continuation of ongoing actions

♦ The main international programmes focused on saving the regional seas are highly relevant anddeserve further support. These include: Aral Sea, Black Sea and the Caspian Sea.

♦ The NRECS have the potential to develop further as relevant instruments for promotingenvironmental aims at “regional” levels. Most of the activities (as under REAP, WEAP)supported with TIPE projects in the past could be continued under work-plans of the individualNRECs.

7.5 Improving relevance in programming

♦ Base the indicative programming approach on donor co-ordination rather than on NIS partnerco-ordination. The absence of clearly identified recipient agents (RAs) makes co-ordination withthe Tacis co-ordinators less important in terms of relevance.

♦ Limit the number of interstate projects and give blocked budget allocations on the basis of fouryears, to be triggered annually on the basis of progress reports and annual work plans

♦ Introduce allocations for the Aral Sea, Caspian Sea, Black Sea, (including relevant river basins)and a NREC allocation with subdivisions per regional centre.

♦ Provide block funding for REC annual work plans,♦ Reduce the gap between programme identity as described in the TIP APs and the practice of

project selection. Widen the eligibility criteria for TIPE to facilitate supporting projects endorsedby the Environment for Europe Process. Accept national level focused activities under TIPEprovided they are designed to provide specific synergy with the central interstate issues.

TIPE evaluation


7.6 Recommendations for future programme management

The scope for improving efficiency might be found in the following directions:♦ Introduce smaller sized regions♦ De-concentrate the TIPE management structure to Delegations or new programme

management units (see also below)♦ Support the NRECs as POs and recipients for awareness raising projects”♦ Refocus the monitoring approach on TIPE compatibility♦ Solicit comments on ToR from outside experts and OECD EAP task force etc.♦ Strengthen donor co-ordination in programming and management♦ Allocate sufficient resources for TIPE management.

For improving effectiveness♦ Bring more expertise into project formulation

♦ Utilise external environmental expertise to assess priorities for Tacis interventions andleadership and introduce project formulation workshops attended by problem owners andEU, NIS and donor experts with the aim of agreeing on project strategies, and the the TORfor contractors

♦ Make DGXI a stakeholder in TIPE programming, and tendering♦ Design for concrete results and dissemination

♦ Identify and contract a dissemination agent for each project.

For improving sustainability and impact♦ Design for impact

♦ Identify the networking requirements for political support♦ Ensure that the PO is committed to the project objectives and has capacity to play an active role

in implementation and follow up.♦ Toughen the criteria for contractor selection.

A number of the above recommendations are further specified below.

Introduce smaller sized regionsConsideration should be given to replacing NIS-wide projects with projects designed for smallerregions relevant to the key TIP projects. Seven such areas could be 1) Russia REC – RF regions;2) Ukraine REC; 3) Moldova-Ukraine-Romania (cross border programme); 4) Caucasus. 5) Aralbasin; 6) Caspian basin; 7)Black Sea basin 8) Central Asian Republics. The rationale is that insmaller sized regions projects can be more needs oriented. Replication and dissemination ofspecific outputs can be arranged via NREC work-plan funding.

De-concentrationTo improve the quality of Tacis project management, the first requirement is to limit the workload ofTask Managers in Brussels to manageable proportions. This can be done by more Task Managerposts or by de-concentration. We propose that programming and contract budget issues, includingpayment authorisations remain the prerogative of the Task Manager in Brussels. Implementationsupport and trouble shooting could be made the responsibility of the EU Delegations, the NRECs orproject specific PIUs.

Refocus the monitoring approachTo maintain a good overview of the progress of projects with interstate-wide activities requires thatthe Tacis monitoring systems change from a country specific to a regional approach. Thecommission can consider to agree with the Tacis monitoring contractors that they appoint onemonitor with appropriate environmental expertise and strategic vision for each larger TIPE project.

Design for impactImpact in interstate projects requires political commitment and concerted action by differentstakeholders in different countries. This is not a technical know-how transfer issue. It is essentially apublic relations task with political overtones. In the NIS, where decision making is still very muchconcentrated in sometimes a few competing hands, networking and support mobilisation is crucialto success. This should be recognised in the project design, and networking should be a key

TIPE evaluation


element in the ToR. In this connection high level political consultations are also an instrument butthis needs to be well arranged and pre-cooked for visible success. Contractors should be madeclear that this is perhaps one of their most important tasks.

Toughening the criteria for contractor selectionIn view of the above mentioned networking and impact dimension, the contractor should be able todemonstrate that it can field a team leader and project director with substantial networking skills.The evaluation shows that Project director and team leader quality has been a main factor in tracingthe changes project momentum.

Introduce a separate Indicative Programme and funding for the TIPE.The analysis in chapter two has shown that the general TIP programme logic and project selectioncriteria are not tailor-made for the inter-state environment programme. It is recommended toacknowledge this and create a separate Indicative Programme for the environment component,which is reviewed on an annual basis

Introduce a number of sub-components (windows) for support, including:♦ The Environment for Europe Process (EEP),♦ The Regional seas conventions,♦ Other international conventions,♦ NIS -internal - cross border programmes,♦ NREC annual work-plans,♦ Investment support,♦ Institution/capacity building for national, regional, and municipal environment authorities.

TIPE evaluation


Evaluation of theTacis InterstateProgramme inEnvironment

Annexes

TIPE evaluation


TIPE evaluation


Annex A Guideline to the methodologyused

A.1 The content of annex A

The main elements in the study approach, which are of methodological interest. are indicated withitalics in the box below (the figures in the brackets, point at the relevant sections of the Annex).

The study approach

Specification of the focus of the evaluationTOR analysisConsultations with the client (A2)Key questions, and specific issues for the evaluation (A3)

Data collectionDocumentation reviewInformation sources, barriers, and limitations (A4)Commissioning of sub-studies, and ToRs (A5)Verification of information through project case studiesThe case study format

Data analysisLogical framework analysisProgramme logic analysisImpact measurement and impact indicators (A6)Application of evaluation criteria, and performance indicators (A7)

Formulation and verification of findings and conclusionsProject logical frameworks, and evaluation matrices (A8)Team assessment of the findingsCommunication, and feedback on the findings with the stakeholders

The methodological aspects of the above identified elements in the study approach, are brieflydescribed below. Further details are available in the study proposal, the 1st progress report, and theinterim report.

A.2 Consultation with the Client

The Terms of Reference focussed the study on lessons learned, and recommendations for thefuture of the TIPE, rather than on ex post evaluation. In the kick of meeting between the Contractorand the Commission (SCR and DG1a), the orientation given by the ToR was verified. Specifically,the question was raised, what the main purpose of the evaluation was. Was the purpose to providean ex-post evaluation of results and outcomes, or was the main purpose to give an input forprogramming? The Client confirmed that the latter was the case, and even moved forward, thedeadline for interim results from the study, to better fit in with the Commission agenda forpreparation of the TIPE Indicative Programme for the period 2000-2006.

TIPE evaluation


For the methodology this implied that verification of past performance was not a major issue, and itwas decided to mainly utilise secondary dat sources for that purpose, such as Contractor progressreports, and reports from the Tacis Monitoring units in the NIS. In particular this meant that asystematic and representative interview schedule for stakeholders, to assess past performance wasnot needed. Instead the methodology focussed on tools for:♦ Assessing policy issues, programming relevance, general impact; and♦ Lessons learned for further improving relevance, and impact, as well as for, effective, and

efficient project implementation.

A.3 Key questions, and specific issues for the evaluation

The Terms of Reference are specific about the evaluation approach, and the issues to beaddressed, and they are grouped with reference to the main evaluation criteria: relevance,effectiveness, efficiency, impact and sustainability. While the listed issues listed are consideredrelevant, we have singled out four questions for the evaluation:

1. The extent to which TIPE has a distinct identity, vis-à-vis other Tacis support instruments, whichis of operational guidance for programming;

2. The extent to which TIPE has succeeded in identifying, and selecting for support, those priorityenvironmental problems, which can be successfully addressed only on the basis of an interstateco-operation approach;

3. The extent to which the resources used by TIPE can be justified, in relation to the impact of theprogramme, and in relation to other priorities, and delivery mechanisms for Tacis funding;

4. The extent to which, and in what directions, the evolution of environmental policy developmentin the NIS offers a basis for an impact reorientation of TIPE programming and TIPEmanagement.

In addition to the issues listed in the TOR, the preliminary conclusions presented in the 1st progressreport, were used to assess the following specific issues in the 2nd phase of the project:

Programme relevance♦ To what extent has the TIPE succeeded in addressing priority environmental problems. Has

relevance changed over time?♦ To what extent is the absence of an EC communication on NIS related environmental policy

initiatives a factor effecting (presumably demonising) TIPE relevance and impact.♦ To what extent is the assumption valid that the broad focus of TIPE has negatively effected

programme impact.♦ To what extent gives the New Tacis regulation directions for increasing the relevance and

impact of TIPE?

TIPE management♦ To what extent has Tacis developed tailor made programme management procedures for the

specific co-ordination requirements of the interstate programme;♦ To what extent has co-ordination with the Commission services taken place, and in what

phases of the programme and project cycle specifically;♦ To what extent has donor co-ordination been effective, and in what phases of the programme

and project cycles.

REAP - awareness♦ To what extent has the project (approach) been an efficient instrument for REAP?♦ Is the approach suitable for follow up after November 2000?

Common environmental policies♦ Relevance. To what extent were the stake-holders involved, and committed to the work plan

(contents) of the programme.♦ To what extent are the NEAPS suitable for NIS wide priority setting, and provide guidance for

future TIPE programming.

TIPE evaluation


Widening the NEAPs♦ To what extent is there an “interstate” consensus on the priorities for Tacis funded investments;♦ To what extent are these priorities relevant from an wider EU policy perspective on international

co-operation;♦ To what extent contributes the EU approach to parallel financing rather than co-financing with

other donors to efficiency and impact;♦ What are the main considerations for assessing the priority of a further WEAP orientation of

TIPE, in the next indicative planning period, vis-à-vis, other TIPE objectives (risk management,visibility of results, future impact, etc.).

New RECs♦ To what extent has the non-partisan concept for the REC’s promoted by the Tacis programme

been appropriate for the NIS;♦ To what has the assistance been successful in tackling the ownership and sustainability

aspects of the non partisan REC approach in different NIS;♦ To what extent is future funding relevant for TIPE.

Aral Sea and WARMAP♦ To what extent has the international Aral sea programme had an impact;♦ Have the EU funds been used efficiently and effectively in this context;♦ To what extent is further funding relevant;♦ Is it reasonable to expect concrete, sustainable results from any further support in the period

2000-2003.

Caspian Sea programme and Black Sea programmes♦ What are the main environmental problems involved;♦ What have been the concrete results of these international programmes;♦ To what extent is it reasonable to assume that these programmes will have a sustainable

impact;♦ What are the main assumptions behind this expectation;♦ To what extent has the Tacis support been relevant in these programmes;♦ Is Tacis recognised, credited publicly for its contributions;♦ Does the Tacis support have value from a REAP perspective;♦ To what extent is future TIPE funding relevant.

The above list of questions were addressed in 2nd report (Interim findings), and 3rd progress report(six project synthesis reports).

A.4 Information sources, barriers, limitation to the scope of thestudy��

A.4.1 Information sources

Written documentationA list of recommended documentation is included in the TOR. In addition we made use of recentliterature on environmental management in the EU and NIS, and of published papers on Aral Sea,Caspian Sea and Black sea environmental issues available through internet and the network of theSEU (Socio-Ecological Union) a Russian based internal NGO.

Open ended interviews with programme level stakeholders and resource persons:Interviews were held with more than hundred people including:♦ Commission services staff (DG XI, DGIa, SCR);

16 This section was first published as section 7.2 of the 1st progress report.

TIPE evaluation


♦ EU contractors;♦ EU Delegations in Russia, Ukraine, Georgia and Kazakhstan;♦ Tacis Co-ordinating Units: as above plus Moldova, Azerbaijan and Uzbekistan;♦ Partner Organisations (Ministries, State Committees in seven countries).Of key interviews written notes were made and kept in the internal project file.

Structured interviews with project level target groupsInterviews were held with representatives of the main TIPE project target groups:♦ Parliamentarian committee experts in Russia, Georgia, Azerbaijan and Kazakhstan;♦ Central and local government officials;♦ Producers/employer organisations;♦ NGO’s;♦ Scientists;♦ Media representatives.The interview schedules and interview notes are kept in Annex F of the final report. Annex E,however, is an internal project report and is not publicly available.

A.4.2 Barriers

Logistics formed a barrier for the data collection in the field work phase of the evaluation. Choiceshad to be made on where to go, and whom to see. Thus the evaluation coverage of actual activitiesinitiated by the TIPE and of people involved in the implementation was fairly small, considering thelarge number of people effected by the TIPE projects. By using NIS experts as part of theevaluation team, we tried to mitigate the importance of this barrier. They increased our flexibility,and coverage, and they have provided us with access to people and information, whichsubstantially increased the efficiency of our field work.

A.4.3 Limitations

Regarding impact, evaluations generally lack the resources to collect and analyse scientificallyrelevant data. The information sources available are limited to the ones listed above. While theinformation from the interviews was valuable to get an impression from the target groups onprogramme results and benefits, they have only a limited value for assessing programme impact.Especially for the impact of the REAP and NEAP, a representative sample survey would have beenneeded to come to more reliable conclusions. Such a survey, however was beyond the scope of thestudy.

A.5 Commissioning of sub-studies, and ToRs

A large part of the data collection and analysis work has been done by local experts recruited formNGO networks in the NIS. In particular the NEI BV commissioned eight papers. Six papers giveviews on the status of the NEAPs in Russia, Ukraine, Moldova, Georgia, Azerbaijan andKazakhstan. The seventh paper deals with the Black Sea programme activities in Krasnodar (RF)and Rostov; the 8th paper deals with the preparation process of the NEAP in Georgia. The papersand the authors are listed in Annex B.

The purpose of the papers, was to provide the evaluation team with a deeper level of insight in tothe subject matter than can be obtained from documentation review and the short field visits, whichare the standard tools of evaluators. It has served this purpose, however, the quality of thecontributions has fluctuated between good and fairly poor. The papers are available from NEI onrequest.

TIPE evaluation


A.6 Impact measurement and indicators

A.6.1 The model

The main theoretical model of transition/societal reform considered applicable is derived fromDouglas North. He has argued convincingly that historically speaking, major social and economictransformations in society are the result of basic changes in the “belief system” of society. Thismeans changes in norms and values, in beliefs and attitudes, i.e. in the cognitive system. Suchchanges are in due course reflected in the institutions, which society has set up to regulate theinteractions between civil society, parliament, the government and the commercial sector. Theseinstitutional changes include the constitution, property rights, legal system, the role for the state andtaxation, market organisation & social security arrangements, and civil society development Suchinstitutional change implies changes in the opportunities, and incentives for behaviour. The outcomeof this is changed behaviour patterns for all key actors in society. In the context of TIPE, thechanges in behaviour of target groups is the condition for positive environmental impact.

A.6.2 The term impact, and impact classification

The term impact stands for a sustainable, and valuable contribution towards solving anenvironmental problem or minimising the risks ensuing from an environmental problem.In line with the model of Douglas North for societal change, the following classification of impactwas used:♦ A sustainable change in attitudes of target groups, which leads to behavioural change

favourable from environmental point of view;♦ A change in the institutional setting, which provides opportunities and incentives for target

groups to change their behaviour in a way favourable from an environmental point of view;♦ Direct environmental abatement actions, which improve environmental conditions.

Impact was thus defined, in this study, as the positive of negative environmental effect of specificproject achievements. For example, a new law is in itself not considered impact, until it is enforced,and this enforcement has a positive environmental effect.

In this TIPE study, no attempt has been made to measure impact directly, instead projectdocumentation and academic papers and media coverage and interviews were used to get an ideaabout the extent to which the various TIPE projects have contributed to sustainable changes inattitudes, institutional setting and concrete actions which are linked to sustainable improvement inenvironmental conditions. As a guide for the study, the following fields for observing impact wereidentified.

TIPE evaluation


A.6.3 Impact dimensions, and indicators

Policy life cycle phases Dimensions for impact measurementProblem identification Increased availability of environment information

Environmental impact assessmentPublic awareness raisingPolitical awareness raising

Policy formulation Recognised need for interstate combined actionEnvironmental priority settingPolitical priority settingPolicy target formulation and Budgeting

Strategy development Target group participation in strategy developmentTarget group policy developmentAction plan developmentImplementation instruments developmentManagement capacity development

Action Plan implementation Legal framework developmentExpertise development and research priority settingSpecific target group actionsCommunication policy implementation

Monitoring and enforcement Monitoring capacity developmentFeedback and enforcement

Evaluation and feedback Target group attitudes and behaviour monitoringCapacity building amongst stakeholdersRelations and co-operation between stakeholdersEnvironmental situation monitoringPolicy and strategy fine-tuning

Policy integration Policy integration with other walks of life

A.7 Evaluation criteria and performance indicators

The evaluation team started its work with six criteria and a ranking system with five qualifications.The qualification system was designed to illustrate the relative performance of sampled TIPEprojects in countries selected for the evaluation. The evaluation criteria and the correspondingperformance indicators are tabulated below. The five qualifications that were initially are:1 = Excellent Å HIGH2 = Good3 = Sufficient Å MEDIUM4 = Poor5 = Bad Å LOW

At a latter stage of the study, when discussing the results of the project case studies, theCommission made the suggestion to use three rather than five qualifications for clarity purposes.This suggestion was excepted.

TIPE evaluation


Evaluation criteria Main dimensions for performancemeasurement

Relevance of the problems addressedby the programme

1.1 Environmental priority (environmental riskdamage; and population size effected)

1.2 Political priority1.3 Tacis priority / value added potential

Quality of programme design 2.1 Environmental relevance of the Tacisintervention/project focus;

2.2 Contractor guidance quality of the ToR2.3 Appropriateness of the selected partners and

target groupsEfficiency of programme implementation 3.1 Implementation on time

3.2 Contractor performance3.3 Monitoring performance3.4 TM management follow up3.5 Cost-effectiveness

Effectiveness of programmeimplementation

4.1 Actual versus planned project achievements4.2 Quality of the achievements

Impact and sustainability of programmeactions

5.1 Changes in environmental awareness5.2 Changes in environmental legislation5.3 Changes in planning capacity5.4 Changes in implementation capacity5.5 Changes in attitudes and behavioural of key

actors and the public5.6 Evidence of problem abatement5.7 Evidence of problem stabilisation/control

Early on in the study, the evaluation team came to the conclusion that it would not find muchevidence of impact, as defined above, in terms of environmental problem abatement, or problemstabilisation. For this conclusion the team had two reasons:♦ The problems addressed by TIPE are of considerable complexity and size compared with the

inputs provided under the TIPE. To bring these problems under control, let alone to achieve realimprovements in environmental conditions, substantially more efforts than provided thus far willbe required from all stakeholders.

♦ The data available, generated during the programme implementation are not suitable formeasuring impact. For the alternative, i.e. primary impact data collection, the time andresources at the disposal of the evaluation team are not sufficient to undertake a systematicimpact assessment.

Having realised this, we have combined the criteria of sustainability and impact. The rationale fordoing so is that the indicators for sustainable changes are preconditions for impact and theirrealisation will in due course result in real impact.

A.8 Project logical frameworks and evaluation matrices

For five projects of the six case study projects a logical framework evaluation matrix was prepared:♦ WARMAP♦ Caspian Sea♦ CEPS♦ REAP♦ NRECs

The matrices summarise the intervention logic of the projects, as well as the relevant indicatorvalues and specific comments from the evaluation team on project performance. They thus serve asa summarising tool for project evaluations. They are presented on the next page.

TIPE evaluation


Intervention logic WARMAP Projectachievements(Performanceindicators)

Comments(assumptions)

Wider ObjectiveStrengthening national and regional planningcapabilities in the area of land and watermanagement (building on and extending thework done in WARMAP1 through training ofstaff, development and application ofplanning systems)Promote an economic approach to land andwater management in the CAR both throughco-ordination of efforts of Governments andexternal aid and through provision of short-term economic and legal policy advise andrealistic database on related subjects.

1. No reported structured impact. Fragmented impact onthose persons directly involved in the project. Noevidence of structured training.2.1 No evidence of progress in economic approachesand structured interstate co-operation2.2 Database established but not (yet) applied in policydevelopment

1. WARMAP is linked to the GEF sponsored Aral Sea BasinProgramme. Therefore the evaluation also took this programme intoaccount to some extent. This programme is assessed by us as veryweak in its environment and institutional components. Described as“misguided missile which needs new target setting. This affects theeffectiveness and efficiency of the WARMAP programme in a negativeway. Follow-up activities are recommended to be made conditionalupon new target setting of Aral Sea Basin Programme2. The process of project design was not transparent (no evidence oflocal involvement). The ToR is weak and its internal logic leaves roomfor multiple interpretations of objectives, as well as required level ofoutputs and inputs.

Immediate objective1. Preparation of water usage, water

planning and other legal interstateagreements

2. Creation of Management InformationSystems

3. Development of planning capacities ofrecipient staff

1. No legal interstate agreements achieved so far2. Two databases established (WARMIS ad

WUFMAS)3. No evidence of structural improvement

1. WARMAP was assessed by us as institutionally weak2. Useful but exposure remains weak3. No impact achieved on decision making level

OutputsModule 1: Interstate Legal AgreementsModule 2a: WARMIS regional water quantitymanagement databaseModule 2b: Water use and farmmanagement survey (WUFMAS-database)Module 3: GEF Project component F “Projectmanagement support”.

The project will not succeed in achieving to reach any ofthe required Interstate Legal Agreements (Module 1).WARMIS and WUFMAS database (Module 2) is veryprofessional and gives valuable background informationfor further implementation of the ASBP. However theexposure of the databases are still weak.Accessibility of the databases in the future are notensured (because of ownership problems)So far impact of WARMAP was low. Although it hadsome impact on the water management sector in the

The institutional setting of the project as well as the overall unclearobjectives of the Aral Sea Basin Programme continue to pose aserious risk to the sustainability of the project.

TIPE evaluation


countries involved (i.e. ICWC) the subsequent impact onthe decision-makers higher up remains low.

Intervention logic WARMAPcontinued

Projectachievements(Performanceindicators)

Comments(assumptions)

InputsModule 1: Support and assist InterstateLegal AgreementsModule 2a: WARMIS regional water quantitymanagement database developmentModule 2b: Water use and farmmanagement survey (WUFMAS-database)Module 3: GEF Project component F “Projectmanagement support”.

Problematic for reasons which are partly due to the weakinstitutional capacity building in WARMAP2 and partlybecause of the extremely difficult institutionalenvironment, that the contractor has to operate. It isunlikely that any interstate legal agreement will bereached at the end of WARMAP2.Done (see comments)Done (see comments)Done (see comments)

WARMAP2 project was largely a technical project with focus onprofessional implementation of especially Module 2 and with too littleconcern over institutional problems.The legacy of poor institutional arrangements and poor overallenvironmental planning will remain a risk for the effectiveness of anyintervention in the Aral Sea Basin Programme.

TIPE evaluation


Intervention logic, Caspian Sea Project achievements (Performance indicators) Comments (assumptions)Wider ObjectivePromote sustainable development andmanagement

No reported structured impact so far. Positive aspect isthat states start to collaborate in the Programme

Immediate objective1. Fostering a collaborative spirit

between the countries in the region;2. reinforce institutional capacity,

improve pollution monitoringcapabilities, establish networks forregional emergency responsecontingency planning, and publicawareness;

3. develop common approaches tonational environmental reviews,status and trends reports, thepreparation of legislation andharmonisation of standards,strategies for the conservation ofnatural resources and the coastalzone management;

4. urgent investment possibilitiesand the implementation feasibilitystudies related to them.

1. Although with great difficulties, collaborationbetween countries started

2. No evidence of strengthened capacity so far3. No evidence of achievements (Nov 99)4. No evidence of achievements (Nov 99)

1. Good achievement so far2. Possibly too early to make an assessment3. Possibly too early to make an assessment4. Possibly too early to make an assessment

Outputs1. Establishment of the PCU2. Establishment of thematic centres

dealing with:- Desertification;- Sea level fluctuation;- Fisheries;- Data gathering.

3. Work towards the preparation of theStrategic Action Plan;

4. Work towards the preparation of anurgent investment portfolio

1. PCU established in Baku2. Thematic Centres established3. No evidence of SAP preparation so far4. No evidence of achievements so far

The original ToR included the actual drafting of the SA and the urgentinvestment portfolio. In the Inception phase this has been downgradedto “work towards” the preparation of these documents. The reason forthis was the lack of concerted action between donors (TACIS andUNDP) to come to complementary results.Strong points of the Programme are:♦ Countries and relevant institutions involved are talking again and

mistrust between them is reducing;♦ For the first time, environmental issues of the Caspian Sea are

being tackled in an integrated way and in a joint effort.

TIPE evaluation


Intervention logic Caspian seacontinued

Project achievements (Performance indicators) Comments (assumptions)

Inputs1. Establishment of the PCU2. Establishment of 4 Caspian thematic

centres3. Working programmes for the 4 thematic

centres;4. Thematic inputs to the TDA on

4 themes;5.� The TDA (prepared jointly with the GEF

project);6. Analysis of the pollution of the Caspian

and its effects, and options forabatement;

7. Brief standardised project descriptionsfor PIPs identified by rapid appraisal;

8. The work programme for developing theSAP and implementing PriorityInvestment Projects (PIP);

9. a draft structure for the SAP (preparedjointly with the GEF project);

10. workshops and seminars which providea focal point for CEP activities;

11. initial work on the development of aregional information and datamanagement system (to be completedby the GEF project);

12. networking facilities, including goodtelephone communications, initial dataexchange links, the CEP (World Bankprepared) web page,

13. training courses; and14. general and specific publications to

heighten awareness of Caspianenvironmental issues.

1. Done2. Done3. Done4. Under implementation5. Not implemented (yet?)6. Not implemented (yet?)7. Not implemented (yet?)8. Not implemented (yet?)9. Not implemented (yet?)10. Some seminars/workshops implemented11. Under implementation12. For Baku ok. For rest could not be assessed.

Reportedly ok13. Not implemented14. Sporadically activities implemented (not systematic.

During the assessment it appeared that the project would becompleted in January 2000. However, many activities are still notimplemented. Project implementation faces several major difficultiesbecause of following reasons:♦ Lack of leadership and co-ordination from the side of the

Commission/TACIS;♦ Lack of co-ordination and appropriate communication between

donors of the Programme;♦ Lack of (reliable) data and poor exchange of data to allow for

proper inventory of problems;♦ Lack of involvement of and poor communication between other

relevant line Ministries at the on-set of the Programme (especiallyin relation to the Convention of the Caspian Sea). Internalcommunication (within institutions involved) also needsconsiderable improvement;

♦ Geo-political/economical discussions and interests hamper thedrafting of common environmental action. (Caspian EnvironmentalConvention still not completed and agreed upon);

♦ Environmental issues are low on the political agenda of countriesinvolved in the Programme. However, also the TACIS programmeapparently gives environment a low priority (No or limited nationalTACIS environment programmes exist. Environment is onlytackled through the TACIS Interstate Programme);

♦ Current acceptable fees for experts in the Programme are too lowto allow for professional needed senior environmental experts tobe involved in the Programme.

TIPE evaluation


Intervention logic, CEPS Project achievements (Performance indicators) Comments (assumptions)Wider Objective♦ Develop capacity in NIS to address

serious environmental problems♦ Improve project identification and

preparation capacity♦ Facilitate common co-ordinated regional

responses to transboundaryenvironmental problems

Successful in the sense that NEAPs have beenpromoted and thus the AP promoted in the individualNIS. It has been unsuccessful in the sense of inter-NISco-ordination and tackling transboundary problems.Although some indications exist that project promotedpositive change of approaches at recipient, institutionalcomponent (i.e. fostering political will and skills toimplement environmental policies) will needconsiderable strengthening.

The process of project design was not transparent (no evidence of localinvolvement). The ToR is weak and its internal logic leaves room formultiple interpretations of objectives, as well as required level of outputsand inputs.

Immediate objective♦ Deploy advisers to support

establishment or further development ofthe National Environmental Action Plans

♦ Strengthen interstate institutions dealingwith the environment; support interstatedialogue

♦ Co-ordinate actions that deal withregional environmental problems, incl.conventions, transnational water and airpollution

Deployment of local long-term advisors ensured continuityand coherence of national actions. As indicated before,project did not result in strengthening interstate co-operation and addressing transboundary environmentalproblems.

A weak point is that the design is not clear about the targets to bereached by the contractor. For instance, as mentioned before, no clearimmediate objectives have been defined and no distinction betweenthe NEAP product and process is made. As a result the approachtaken by the contractor appears to be non-coherent. Focus is either onprocess (e.g. Russia) or on product (e.g. Georgia). Also it remainsunclear how the additional national seminars/trainings will contribute toa translation of the NEAPs into further development of environmentalpolicies.

OutputsFurtherance of NEAP process, and:Provide knowledge and expertise in NEAPprocessImproved capacity to identify and prepareprojectsProvide knowledge to develop andimplement environmental legislationImprove counterpart offices

Outputs as planned in the Inception Report have beenlargely achieved in terms of quantity. Quality of outputsis generally satisfactorily, but impact remainsquestionable. This has to do with the fact that at the endof the project the absorption capacity of ministriesappears still to be low, as evidenced by the slow pace ofreforms and as underwritten in many of the monitoringreports of this and other projects. Also the necessarylegislative changes in most countries are not evidencedyet. The old soviet regulatory framework is still in placeand major efforts will still be necessary to change this.Remaining obstacles for implementation of NEAPs relateto uncertainty of available resources and commitment ofGovernment to develop an environmental strategy.

Remaining obstacles for implementation of NEAPs relate to uncertaintyof available resources and commitment of Government to develop anenvironmental strategy.

TIPE evaluation


Intervention logic, CEPS continued Project achievements (Performance indicators) Comments (assumptions)InputsLong-term TA at national level to assistNEAPs developmentShort term TA for training and ad-hoc input toNEAPs processTwo interstate conferencesNational workshopSpecific training coursesEnvironmental legal support ServiceGeneral support (incl. study tours)Fund IEC chairman and office

The implementation through long-term advisors forMinistries of Environment was a valuable approach, butmight have been more effective if guided with somemore short-term expertise input and/or superregionallyorganised activities (seminars). Fine-tuning of the projectto local needs and conditions was only partly successful(depending on capacity and commitment of recipientorganisations).

-

TIPE evaluation


Intervention logic, REAP Project achievements (Performance indicators) Comments (assumptions)Wider ObjectiveRaise status of environmental issues in NIS Unknown for most components. Positive indications on certain elements Difficult to assessImmediate objectiveRaise awareness of solutions toenvironmental problems by influencingenvironmental policy, NGO activity andgeneral public awarenessTo build the specific capacity of the selectedtarget groups in tackling environmentalproblems

Seen from the size of the NIS and the scarce means within the project itcould not be expected that major improvement in raising environmentalawareness and capacity building could be made. As complementary toother TACIS Interstate projects the intervention is logical and useful.

The effectiveness of the project could behigher when much more clear involvement ofthe partner states would have beenguaranteed. The lack of knowledge about theproject activities and objectives in Ministries ofEnvironment for example causes that theinterrelation with other TACIS projects is weak.

OutputsMedia component implementedNGO component implementedParliamentary component implemented

Media Component:A large public has been reached through broadcasting of environmentalproductions on a.o. Russian, Moldovan, Ukrainian television. In countriesas Azerbaijan and Georgia negotiations for broadcasting are going on.Through media training and seminars for journalist in all project countriesthe level of environmental journalism is reportedly improved.Within the media component a huge number of environmental videolibraries is opened. It is hard to measure now but the impact especially forNGOs (out of school environmental education) and teachers (in-schoolenvironmental education) may be substantial.

NGO Component:NGOs are involved by the project in processes as EEP. Especially thepreparation towards the Aarhus conference held in 1998 reportedlyresulted in a higher visibility and understanding of NGOs towardspolicy makers and national governments. The small grantsprogramme in relation to the Kura Araks river programme had verylikely the highest positive impact.

Parliamentary Component:The organisation of Open Parliamentary Meetings by the project resulted indiscussions on a high level on issues as environmental legislation andnational environmental policy making. It is mentioned by interlocutors andmembers of monitoring teams that the impact of the OPM especially on theratification (and the process to ratify the convention) of the AarhusConvention was high.

-

TIPE evaluation


Intervention logic, REAP continued Project achievements (Performance indicators) Comments (assumptions)InputsBenchmark/impact survey onenvironmental awarenessLocal Agenda 21:Small grant funds andseminarsVideo libraries and public eventsMedia work: environmental productionsNGO development: training, tool kits andsmall grants fund establishmentParliamentary capacity building activities

For Caucasus region ok. For Central Asia low impact (seecomments). For other regions could not be assessed thoroughly.Final report of contractor indicates that results are satisfactory

In interviews with beneficiaries of the project the ParliamentarianComponent was seen as the component with the biggest impact.Especially in two partner states (Ukraine and Moldova) interventionresulted in a high commitment towards environmental policymaking. Both countries are among the first states which ratified theAarhus Convention on: Access to Information, Public participation indecision making and access to Justice in Environmental Matters.Also in Georgia and Armenia thanks to successful OpenParliamentary meetings which were organised the need for theratification of the Aarhus convention was made clear.

Elements found to hamper the effectiveness ofthe programme in CA are:

Less effective NGOs as compared toCaucasus. No networks exist;No (financial) management skills at the NGOs;Region too big (loose time with travelling);Low profile of NGOs in region (with exceptionwith Kyrgizia);No interest from parliamentarians;NGOs in some regions (NGOs) moreinterested than in regions where Governmentis strong;Political unrest disrupts programme;Underpaid expatriate experts.

TIPE evaluation


Intervention logic, NRECS Project achievements (Performance indicators) Comments (assumptions)Wider objectiveTo contribute to solving environmentalproblems

To foster the participation of all sectors ofsociety in environmental decision makingand

To support efforts at promoting co-operationamongst stakeholders across borders”

Zero reported impact,Future impact perhaps from the small grants programmein MoldovaNo evidence of REC induced participation.The establishment issue has put REC on the politicalagenda as an embarrassmentThe NRECs have not adopted a common work plan forcommon problemsIn the Caucasus the set up process has started crossborder co-operation

Specific objectivesOriginal : To create and support NEWRECas international organisation in the period1996-2000Amended. To create and support fourNational RECS and to link them in aninternational network (phase 1 & 2): and toexpand the network of national RECs toCentral Asia (phase 3)

NEWREC given upOnly Moldova REC operational (Jan 2000)USEPA supports some activities in UkraineNo network

Legal form alien Å registration struggle in each country

OutputsISB established Done Useful butICIO functioning Not done, idea abandoned no agreementISB replaced by NEWREC Board No ISB nor Board present on ICIONational RECs established Moldova established and registered

Georgia last minute problemsUkraine imminentRussia search for pragmatic solution

A slow process of consensus building on REC concept

New REC staff trained Executive directors trained Behind scheduleWork Programme of activities developed Not done Not startedSmall grants programme operational Moldova OK Pilot proceduresWork Programme implementation Moldova started

TIPE evaluation


Intervention logic, NRECS continued Project achievements (Performance indicators) Comments (assumptions)Activities – phase 1Establishing and running ISB Completed and ended UsefulEstablishing NEWREC Headquarters Failed, idea abandoned No consensusEstablishing NEWREC Board of Directors rescheduled for phase 2 DelayedEstablishing four National OfficesRussia RECUkraine RECMoldova RECCaucasus REC

Process blocked (established as NGO in March 2000)Registration imminent (established in 1st quarter 2000)CompletedAzeri approval withheld (established in 1st quarter 2000)

Establishing National Board of Directors Moldova, Georgia OK politics issueActivities – phase 2 Started in Moldova onlyRunning the four National Offices Moldova operational strategy neededActivities – phase 3 Not started yetReplication to the remaining NIS Budget allocation made by the CommissionInputs – phase 1Funding from local governmentsFunding from REC BudapestFunding from donor governments

Problems with premises and refurbishmentProvided as and when requestedAmple funding from the Commission

Inputs –phase 2 DG XI and US-EPA support MoldovaInputs – phase 3 TIPE 97 allocated 1.6 Meuro

TIPE evaluation


Annex B Experts involved in the evaluationand outputs

This annex lists the evaluation teams involved in the various outputs and reports producedduring the study. The evaluation core team comprised Max van der Sleen, team leader, NEIBV; Esther van der Meer, (NEI); Nick van der Lijn, (NEI); Johan Vollenbroek, MOBilisation forEnvironment; Imre Csikos, (MOB) Ton van Eck, MilieuKontakt.

Reports prepared by the core team for the Commission

♦ 1st Progress report, NEI BV, November 1st, 1999, Max van der Sleen, Esther van der Meer, Nickvan der Lijn

♦ Interim Report, preliminary conclusions and recommendations, NEI BV, November 30th,1999,Max van der Sleen, Esther van der Meer

♦ 3d Progress report, six project evaluation synthesis reports, NEI BV, February 18th, 2000, Maxvan der Sleen

♦ Draft final report, Evaluation of the Tacis Interstate Programme in Environment, Feb.25th, 2000,Max van der Sleen, Nick van der Lijn

♦ Final report, May 29, 2000, Max van der Sleen

Study outputs, expert contributions, submitted to the Commission

Output 1 Brief presentation of EU policy and objectives for the EnvironmentImre Csikos

Output 2 Presentation of the evolution of environmental policies in the NISJohan Vollenbroek

Output 3 Expert paper on the NEAP Azerbaijan, Rena KuliyevaOutput 4 Expert paper on the NEAP Georgia, Manana KochladzeOutput 5 Expert paper on the NEAP Kazakhstan, Sergey SoyanikOutput 6 Expert paper on the NEAP Moldova, Piotr GorbunenkoOutput 7 Expert paper on the NEAP Russian Federation, Tatiana Guseva,Output 8 Expert paper on the NEAP Ukraine, Sergey FedorynchykOutput 9 Comparative analysis of GEO-NEAP, Georgi DarsimeliaOutput 10 BSEP, evaluation mission report to Krasnodar and Rostov on Don, S. Balashova,Output 11 WARMAP, evaluation synthesis report

Imre Csikos, Alisa Klima, Leila Yermekbayeva, Igor GlukhovtsevOutput 12 Black Sea (BSEP), evaluation synthesis report

Johan Vollenbroek, Georgi Dartsimelia, S. BalashovaOutput 13 Caspian Sea, evaluation synthesis report

Imre Csikos, Eldar Salahov, Chingiz MamedovOutput 14 CEPs, evaluation synthesis report

Esther van der Meer, Imre Csikos, Nick van der Lijn, Georgi Dartsimelia. TatianaGuseva

Output 15 REAP, evaluation synthesis reportTon van Eck, Imre Csikos, Max van der Sleen

Output 16 NRECs, evaluation synthesis reportMax van der Sleen, Tatiana Guseva, Ton van Eck, Leonid Roshka

TIPE evaluation


TIPE evaluation


Annex C People met and interview notes

During the course of its work, the evaluation team interviewed a large number of people. Most ofthem are listed below, and we would like to thank them once more for their co-operation. The listalso indicates the interviews for which file notes or transcripts were made. Those notes arecollected in a separate annex (E). The interview notes, however, are treated as confidentialinformation, and as such will not be published. NEI BV will retain them for a period of five years.They will be available for consultation at NEI only, if substantiation of certain findings of this study isneeded.

Name Organisation Function Date NotesRussian Federation,MoscowV.I. Danilov-Danilian Goscomecologia Chairman 01-11-99 1+5Tatjana V. Guseva Env. non-govt org.

Ecoline / MendeleevTech Univ.

Advisor/Professor

01-11-99-03-11-99

51

S. Zabelin Socio-EcologicalUnion

Co-chairman 01-11-99

Victoria Kolesnikova Socio-EcologicalUnion

Press Secretary 01-11-99

V.S. Gavrilov Goscomecologia Deputy Head 02-11-99 2Thomas Wiley EU Delegation Second

Secretary02-11-99 3+54

Vladimir M Korneyev EU Delegation Env. Expert 02-11-99 3Ilya Belov SEU (CCI) staff member 02-11-99 4AlexanderGeorgievsky

SEU (CCI) staff member 02-11-99 4

Ecoline Director 02-11-99Elena A. Bogdanova State Committee of RF

for Env. Protection, Int.Coop. Dept.

Senior Expert 02-11-99 53

Yuri A. Platonov State Committee of RFfor Env. Protection, Int.Coop. Dept.

Deputy Head 02-11-99

Sergey B. Tveritinov State Committee of RFfor Env. Protection, Int.Coop. Dept.

Deputy Head 02-11-99

Yevgeny S. Belkin State Committee of RFfor Env. Protection, Int.Coop. Dept.

Deputy Head ofEnv. AnalysisDept

02-11-99

Irina Limonova State Committee of RFfor Env. Protection, Int.Coop. Dept.

Deputy Head ofDiv.

02-11-99

Tatjana I. Stepanova State Committee of RFfor Env. Protection, Int.Coop. Dept.

Senior Expert 02-11-99

Dania Checheneva State Committee forEnv. Protec.

Div of Analysis 02-11-99 50

Alexey Knizhnikov ISAR local expert 02-11-99Peter Welch Tacis Monit. Unit monitor 03-11-99Valery Churbanov Tacis Monit. Unit monitor 03-11-99Boris Kalyakin Tacis CU staff member 03-11-99Natalia Kosheleva WEAP Coord. Office

for Russia & MongoliaProject Manager 03-11-99

Mary Hall Tacis CU Progr. Manager 03-11-99

TIPE evaluation


Name Organisation Function Date NotesNatalia I. Mironova Member of High Env.

Council at RF StateDuma

Olga Ponizova Eco Accord 03-11-99Michael Kozeltsev Eco Accord Executive

Director electRussian REC

05-11-99 5 + 56

Giovanni Cremonini EC Delegation SecondSecretary

05-11-99 55

Y. E. Kazakov USAID Policy Advisor 10-11-99 6Mr Vladimir V.Khlobystov

Center for Prep. &Impl. of Proj. on TA,North CaucasusBranch (NCB CPPI)

Director 16-11-99

Mr Peter P. Ulianov Com. Of Natural Res.Of Rostov-on-Donregion

Chairman 17-11-99

Mr Vadim A.Nikanorov

Com. Of Natural Res.Of Rostov-on-Donregion

Chief Specialist 17-11-99

Mrs Maria V.Paraschenko

Public Rel. & Env. Info.Of State Com. ForEnv. Protect. OfRostov region

Chief of theDept.

17-11-99

Mr Leonid P. Yarmak State Com. On Env.Protect. Of Krasnodarregion

Director ofInternationalCentre

18-11-99

Mrs Rieks Bosch ICZM of the TacisBlack Sea Env. Prog.

Team Leader 18-11-99

Mrs Ekaterina I.Antonidze

ICZM of the TacisBlack Sea Env. Prog.

Deputy Directorof InternationalCentre

18-11-99

Mr Fyodor G. Kiyashko State Com. On Env.Protect. Of Krasnodarregion

Head of SpecialMarineInspection

18-11-99

The interview

s are covered in a separate visit report

EU Delegation/Brussels BelgiumHelen Holm Prog. Officer for

Env. InterstateProg.

X

Boris Iarochevich EU Delegation Adm. Unit,Russia

02-11-99 X

Guy Mustard EU Delegation Relations w/Belarus,Moldova &Ukraine

02-11-99 X

Werner Langhals EU Delegation DetachedNational Expert

02-11-99 X

Dino Sinigallia EU Delegation Resp. forCaucasus &Central Asia

02-11-99 X

Robin Liddell EU Delegation Principle Adm. 02-11-99 XSeamus Jefferson EU Delegation AdministraTor 02-11-99 XMr Tim Turner Tacis Adv. To the

Black Sea Prog. Impl.Unit

Advisor 04/05-11-99

31

TIPE evaluation


Name Organisation Function Date NotesGeorgia, TbilisiMs Nino Chkhobadze Ministry of

EnvironmentMinister 06-11-99 7+32

Nato Kirvalidze Caucasus REC ExecutiveDirecTor

06-11-99 8

Merab Barbakadze Ecolawclub REC NGOboard member

06-11-99 8

Jan de Voogd Tacis Monitoring MoniTor 06-11-99Tbilisi Municipal Committee on environment – Saturday meeting 06-11-99David Nikoleishvili POSEIDON NGOs & Media 06-11-99Kakha Tamarashvili EBA NGOs & Media 06-11-99Manana Kochladze GREENS NGOs & Media 06-11-99Jimi Dabrundashvili GREENS NGOs & Media 06-11-99Levan Tavartkiladze GREENS NGOs & Media 06-11-99Gia Todua GAIA NGOs & Media 06-11-99Giorgi Dzamukashvili VARKETILI NGOs & Media 06-11-99Tamaz Dundua ELKANA NGOs & Media 06-11-99Marika Tarasashvili GAEC NGOs & Media 06-11-99Gocha Kakabadze Little TOWN NGOs & Media 06-11-99Nino Chelidze STATE RADIO NGOs & Media 06-11-99Maka Jakhua GREEN WAVE Radio NGOs & Media 06-11-99Eka Khvedelidze Tacis BSEP ICWS local

coord.06-11-99

Mamuka Gvivala ICZM Centre DirecTor 06-11-99Vano Vashakmadze CauClub DirecTor 06-11-99Gia Abramia CER DirecTor 06-11-99

One note on the m

eeting was m

ade

Zaal Lomtadze Ministry of Env. Head of Env.Policy

06-11-99

Maia Kapanadze Ministry of Env. Dept. Head ofPR

06-11-99

Maka Tsereteli Ministry of Env. Deputy head ofEnv. PolicyDept.

06-11-99

Malkhaz Adeishvili Ministry of Env. Deputy head ofEnv. Econ. Dept

06-11-99 X

Mr Jeromme Cassiers EU Delegation Deputy 08-11-99 37

Ms Sima Kanaan UNDP Deputy ResidentRepresentative

08-11-99 34

Tamuna Jibladze UNDP Expert 08-11-99

Olivier Breteche Tacis CU Project Manager 08-11-99 35Mr Soso JosefTsiskarishvili

Tacis CU ExecutiveDirecTor

08-11-99 35

Tamuna Tsulukidze Tacis CU Expert 08-11-99Ms Darejan Kapanadze World Bank Project Officer 08-11-99 38

Mr Vazha Shubladze State Dept. of Tourism Chairman 08-11-99 36Keti Chachibaia REAP project Regional coord.

for Caucasus09-11-99 9

Gia Sopadze Security Council Env. Advisor 09-11-99 XTemur Basilia Advisor to the

President09-11-99 X

Lika Todua Ministry of Env. Head of Divisionof Air protection

09-11-99 X

Avto Budagashvili Ministry of Env. Air Department 09-11-99 XMtia Gvilova Ministry of Env. Air Department 09-11-99 XTamaz Cholokava Ministry of Env. Head of Water

Department09-11-99 X

TIPE evaluation


Name Organisation Function Date NotesKrzystof Michalak OECD, non-member

Countries Branch Env.DirecTorate X

Alexander Arobelidze WEAP Project Country Rep. 09-11-99 XZaal Japaridze Municipal Dev. Fund of

GeorgiaExecutiveDirecTor

09-11-99 X

Nata Sabanadze US Embassy Ass. to the FirstSec

09-11-99

Marina Makarova Min. of Env. Head, WaterDept.

09-11-99

Lia Zaridze Min. of Env. Trainees 09-11-99Nino Chanturia Min. of Env. Trainees 09-11-99Ilo Mtskvedadze Min. of Env. Trainees 09-11-99Tiniko Tetvadze Min. of Env. Trainees 09-11-99Janri Karchava Min. of Env. Trainees 09-11-99Marika Uchaneishvili Min. of Env. Trainees 09-11-99Paata Bolashvili Min. of Env. Trainees 09-11-99Mr Akaki Komakhidze Black Sea Biodiversity

Protect. Act.Centre/Marine Ecology& Fisheries Res. Inst.

Director 09-11-99 39

Armenia, YerevanAnna Gevorkian Tacis Monitoring 05-12-99 10

Moldova, ChichinauMr Arcadie Capcelea Minister of

Environment, Moldova03-11-99 11

Mr AndreiDumbraveanu

Ave Natura NGO 03-11-99 12

Mr. Vasile Ernu Ave Natura NGO 03-11-99 12Mrs Ala Rotaru Ministry of Env. Executive officer,

Department for regulation of ecologicalimpact and bio-diversity

03-11-99

Mrs. Angela Bularga LINKS local coordinator 03-11-99 13Mr. Vasile Cotruta REC Moldova Executive

Director04-11-99 14

Mr. A. Jolondcovschi Ministry for Env Deputy Minister 04-11-99Mr. Sergiu Magdil Ministry for Env Dep Head

Policy04-11-99

Mr. Ilya Trombitsky MP/BIOTICA MP 04-11-99 15Mr. Nicu Vrednic Ministry of Env PR department 05-11-99Mr. Andrei Isac Ministry of Env Head policy

division.05-11-99

Mr. Valeriu Moshanu former NEAP local expert 05-11-99 16Mr. Luke Becker Tacis CU Team leader 05-11-99 17Mr. VeaceslavScobiola

Tacis CU local economist 05-11-99 17

Mrs Diana Shcirca UNDP Moldova Participanttraining seminar

Ukraine, KievMr B. Drozdowskij Tacis CU 08-11-99 18Ms Svitlana Kaltygina Tacis CU ?? 18+47Mrs Marie Mozahajski TEAP Coordinator 08-11-99 XTamara Mal’kova Coordinator XOlexi Pasyuk Coordinator X

TIPE evaluation


Name Organisation Function Date NotesMrs Virginia Blanku EU Delegation Prog. Officer for

Env.09-11-99

Ms SvetlanaKravchenko

TEAP Dir.Parliamentarycomponent

09-11-99

Mrs KatherinaPirozhenko

Tacis Mon. Team 10-11-99

Mr Potapov VitalyIvanovich

Min. of Env. Protection& Nuclear Safety

Head of Dept 10-11-99 45

Mr Radu Mihnea Black Sea Prog Co-ordinator 10-11-99 40Mrs Larisa Homik Center for Media

Initiative11-11-99 49

Mr Vitaly Pantintsev 11-11-99 49Mr Sergey Fedorinchik Zeleny Svit 11-11-99 49Mrs Anna Onisimova-Golubovskaja

MAMA86 11-11-99 49

Mr Boris V.Bourkinskiy

Inst. Of Market Econ.& Ecological Res.

Head 11-11-99 41

Mr Stepanov Inst. Of Market Econ.& Ecological Res.

Dep. Director 11-11-99 41

Mr Nikolai O.Bogoyavlenksy

Odessa Reg. StateAdm.

Dep. Chairman 11-11-99 42

Mr Valerij Mikhailov Integrated CoastalZone Mgt. (RAC)

Director 11-11-99 43

Mr Richard J.Lisovksky

Integrated CoastalZone Mgt. (RAC)

Head ofLaboratory

11-11-99 43

Mr Vyacheslaw I.Oleshchenko

Legal Dept. of theAdm. Of the Pres. OfUkraine

Deputy Chief 12-11-99 46

Mr Michel Nadon Tacis monitoring unit Team leader 12-11-99 48

Almaty, KazachstanWaltraud Gehrig REAP Reg. Co-

ordinaTor08-11-99 X

Zere Mukitanova Tacis Monitoring MoniTor 08-11-99 57Simon Paul Tacis Monitoring Team Leader 08-11-99 57Murat Musataev Min. of Env. Vice Minister 08-11-99 58Dr Karibaeva Kuralay Nat. Env. Centre for

Sustainable Dev.08-11-99 58

Bulat Esekin NEAP Co-ordinator 08-11-99 58Vladimir BogachevCrispin Meelboom Tacis Coord. Unit 09-11-99 59NarimanKipshakbayev

Interstate Coord.Water Comm. ofCentral Asia (ICWC)

09-11-99 60

Olga Palagina WEAP Local Rep.Zharas Takenov UNDP 09-11-99 61Aliya Satubaldina EU Delegation TA Section

Prog. Officers09-11-99 19+62

Charlotte Adriaen EU Delegation TA SectionProg. Officers

09-11-99 19+62

Michael Humphries EU Delegation Ambassador 09-11-99 20+63Pieter van den Hoven WARMAP-2 Team Leader 10-11-99 64Michael Yoder US Embassy Env. Officer for

Central Asia &the Caspian Sea

10-11-99 21+65

Tatiana Lim US Embassy Advisor XRobert den Haan WARMIS database EU expert 11-11-99 66

TIPE evaluation


Name Organisation Function Date NotesRim AbdulovichGiniyatullin

national expert 11-11-99 67

Viktor Dukhovniy 12-11-99 68Galina Stulina WUFMAS office national expert 13-11-99 70Julia Shironova WUFMAS office national expert 13-11-99 70Sergej Nirosin WUFMAS office national expert 13-11-99 70Mikhail Horst WUFMAS office national expert 13-11-99 70V.I Sokolov WARMAS office Teamleader 13-11-99 71

AzerbadjanStuart Gunn PCU Office, Gov.

HouseProject Manager 04-11-99 23

Bahtiyar Muradov PCU Office, Gov.House

Regional Co-ordinaTor

04-11-99 23

Imran Abdullov State Committee forthe Env.

Dept. of Env.projects

04-11-99 24

Fuad Akhundzade State Committee forthe Env.

Acting Chairman 04-11-99 24

Budag Budagov Geographical Inst. Professor 04-11-99 25Dr Jahangir Kasimov Tacis CU Exec. DirecTor 04-11-99 26Isabella Hadjieva Tacis CU national expert 04-11-99 26Irene Lucius Tacis CU Proj. Manager 04-11-99 27Zulfugar M. Kuliyev Fisheries Ind.

Research Inst.DirecTor 05-11-99 28

Abdul Kasymov Inst. of Zoology Head of Dept. 05-11-99 29Latifa Guseynova Thematic Centre for

Pollution ControlRegional Co-ordinaTor

05-11-99 30

UzbekistanMr. GhinyatulinMr. Ofarinov EBRD XMr. Khabirov VODGEO Director XMr. Mironenkov Ministry OfficialMr. Dukhovny SIC – ICWCMr Bryan Morgan Tacis monitoring team

in TashkentEU monitor 12-11-99

Rahilia Dadabayeva Tacis monitoring teamin Tashkent

National monitor 12-11-99

Misc. WUFMAS team project staff 12-11-99Mr Peter Reddish CU Tashkent Team Leader 15-11-99 22+72Mr. Khabibullaev SCNP Chairman 13-11-99Mr. Samoilov SCNP Head Dept. of

Econ. & Mgt. ofNatureResources Use

13-11-99

Mrs. Petrova SCNP, NEAP Co-ordinator 13-11-99Mr Ton Lennaerts World Bank GEF Expert 15-11-99 73Mr Masood Ahmad World Bank GEF Advisor 15-11-99 73

The notes indicated with a number or cross are compiled in an internal dossier.

TIPE evaluation


Annex D Documentation and literature

Interstate action programs and strategy papers♦ Tacis inter-state programme, Status of projects 1992-1999♦ 1999 Tacis Inter-state action programme, May 1999♦ 1999 Tacis Inter-state action programme, February 1999♦ 1998 Tacis Inter-state action programme♦ 1998 Tacis Inter-state action programme, Strategy paper♦ 1997 Tacis Inter-state programme, June 1997♦ 1998 Tacis Inter-state action programme, Strategy paper♦ 1997 Tacis Inter-state programme, Strategy paper♦ 1996 Tacis Inter-state programme, July 1996♦ 1995 Tacis Inter-state programme♦ 1993 Programme, Regional (inter-state) actions♦ 1992 Technical assistance programme, regional (inter-state) actions, draft financing proposal♦ 1991 Technical assistance for the USSR indicative programme♦ Tacis 1996 Action Programme Russian Federation

National Environmental Action Plan (NEAP)♦ Azerbaijan Republic National Environmental Action Plan, draft for discussion, August 1997♦ Azerbaijan Republic National Environmental Action Plan, Baku 1998♦ Republic of Moldova National Environmental Action Plan, August 4, 1995♦ The national program of environment protection and rational use of the natural resources of

Ukraine♦ Environmental Action Programme (EAP) for the Republic of Armenia♦ The National Environmental Action Plan of the Russian Federation for 1999-2001♦ National Environmental Action Plan for Sustainable development of the Republic of Kazakhstan

Project ReportsTacis ENVREG 9602♦ Technical Proposal (English & Russian), SPAN Consultants in communication support♦ Addressing the Clean-Up and secondary Medical Effects of the Chernobyl Disaster, Inception

Report, 15 July 1999

Tacis Funds for the Black Sea Environmental Programme♦ Tacis 1995 Funds for the Black Sea Environmental Programme Tenderdossier by ICWS, July

15, 1996♦ Tacis 1995 Funds for the Black Sea Environmental Programme, Progress Report 5, April-June

1999♦ Tacis/Phare 1995 Funds for the Black Sea Environmental Programme, Inception Report, 30

May 1997♦ Tacis 1996 and 1997 Funds for the Black Sea Environmental Programme Phase 1 & 2,

Technical proposal, Phase 1♦ Tacis 1996 and 1997 Funds for the Black Sea Environmental Programme Phase 1 & 2,

Technical proposal, Phase 2♦ Tacis 1996/1997 Funds for the Black Sea Environmental Programme - Phase 2,

Implementation Report, June 1999♦ Tacis 1996/1997 Funds for the Black Sea Environmental Programme Phase 1 & 2, Progress

Report 4, March-May 1999♦ Tacis 1996 and 1997 Funds for the Black Sea Environmental Programme Phase 1, Draft

Inception Report, April 1998

TIPE evaluation


♦ Tacis Black Sea environmental programme, Integrated coastal zone management, Evaluationreport

Project to establish new regional environmental centres in Russia, Moldova, Georgia and theUkraine (Phase 2)♦ Addendum No 1 to TA contract 98-0230, Appendix1: Complement to and Annex B Organisation

and methods♦ Project to establish new regional environmental centres in Russia, Moldova, Georgia and the

Ukraine (Phase 2), Project Progress report 1, July-December 1998♦ Project to establish new regional environmental centres in Russia, Moldova, Georgia and the

Ukraine (Phase 2), Progress report 2, January-February 1999♦ Project to establish new regional environmental centres in Russia, Moldova, Georgia and the

Ukraine (Phase 2), Progress report 3, March-May 1999♦ Project to establish new regional environmental centres in Russia, Moldova, Georgia and the

Ukraine (Phase 2), Progress report 4, June-July 1999♦ Project to establish new regional environmental centres in Russia, Moldova, Georgia and the

Ukraine (Phase 2), Progress report 5, August-October 1999♦ NEWREC international supervisory body, Minutes of first ISB meeting and associated

documents♦ Minutes of third meeting of the NEWREC international supervisory body, ISB, Moscow, Russia,

30 June-1 July 1997♦ Annex B: Organisation and Methods♦ Letter from Mr Danilov-Danilyan, European Commission, subject: Establishment of Russian

Regional Environmental Center Reference♦ Establishment of the new regional environmental center in Russia, Moscow, 1998

Widening of the EAP to the NIS and Mongolia, Environmental project identification and preparationprogramme♦ Widening of the EAP to the NIS and Mongolia, Environmental project identification and

preparation programme, Terms of reference♦ Widening of the EAP to the NIS and Mongolia, Technical proposal, October 1997♦ Widening of the EAP to the NIS and Mongolia, Progress report No 3, June 1999♦ Widening of the EAP to the NIS and Mongolia, Telavi water supply improvement feasibility

study report, Draft final, August 1999

Development of common environment policies in the NIS and Mongolia♦ Development of common environment policies in the NIS and Mongolia, Relevant sections from

original proposal by ERM♦ Development of common environment policies in the NIS and Mongolia, Inception report, 22

May 1997♦ Development of common environment policies in the NIS and Mongolia, Final report, 30 April

1999♦ Development of common environment policies in the NIS and Mongolia, End of project

assessment report No 8, 23 March 1999♦ Development of common environment policies in the NIS and Mongolia, Conference on

regulatory reform and sustainable development, 4 and 5 March 1999♦ Development of common environment policies in the NIS and Mongolia, Terms of reference

Background papers♦ Priorities for environmental expenditures in industry, Eastern Europe and the former Soviet

Union, by Mark Ambler and John Marrow♦ Environmental action programme for Central and eastern Europe, Setting priorities♦ Foreign donor assistance for environmental protection in Russia and evaluation of its

effectiveness, 1992-1997♦ The European Unions Tacis programme, Reflecting on the future, Brussels, 26.05.98♦ Integrated Coastal Zone Management (ICZM) provided by Rieks Bosch (file BSEP-ICZM-

revision.doc♦ Pollution hot spots on Black Sea coast in Russia (file 3a32dm.gif)

TIPE evaluation


♦ Guide on sustainable development of Black Sea and Azov Sea coastal zones, 2nd version,Krasnodar, 1999, in Russian

♦ Lorence Mee, How to save the Black Sea (booklet in Russian), disseminated among NGO’sand students under ‘Public Awareness and Participation’ subcomponent

♦ How to save the Black Sea, Strategic action plan for children (booklet in Russian), disseminatedamong youth under ‘Public Awareness and Participation’ subcomponent

♦ Environmental problems of Kazakhstan, Almaty 1997 (booklet in Russian)♦ Republic of Kazakhstan Ecology and natural resources, Long term strategy up to the year 2030♦ Sergey Kuratov, Reform of Ecological Legislation in Kazakhstan♦ Documents prepared by EAP Task Force for the ministerial Conference “Environment for

Europe” which was held in June 1998 in Aarhus, Denmark♦ Current Status of the environmental policy development, prioritisation and implementation of the

national and regional environmental action plans♦ Background papers on the National Environmental Action Plans (NEAPS) for Ukraine, Moldova,

Russia, Georgia 1, Georgia 2, Azerbaijan, Kazakhstan♦ Project Report of NGO seminar about NEAP, 1997♦ Svyatoslav Zabelin, Background paper on environment in Russia 1998, Social transformation in

environment protection and nature conservation: case of Russia♦ Towards Sustainability Volume II, A European community programme of policy and action in

relation to the environment and sustainable development, 27 march 1992♦ Agreement on partnership and co-operation, Fulltext signed in Corfu on 24 June 1994 by the

European Union and the Russian Federation♦ Memorandum Views on the environmental aspects of the European Union Tacis programme♦ Common strategy of the European Union on Russia, European Council, Cologne, 3 and 4 June

1999♦ Environment for Europe, report on the fourth ministerial conference held 23 till 25 June 1998 in

Aarhus, Denmark♦ Ecoline, center for co-ordination and information the Socio-ecological union♦ The SEU times, May/June 1999♦ Russian Conservation News, No 18 Winter 1999♦ Evaluation of Tacis country programme in Russian Federation, 2nd Draft Final report Synthesis

REAPReports♦ Tacis Environment, section I, Research report, May 1997♦ Tacis Environment, section II-IV, Research report, May 1997

Action programmes, Indicative programmes and PCA’s over the period 1996-1999

TIPE evaluation


TIPE evaluation


Annex E Terms of Reference

TIPE evaluation


Date post:	11-Apr-2018
Category:	Documents
Upload:	truongthuan
View:	217 times
Download:	4 times