Lot 2 Impact evaluation of ROP 2007-2013 interventions · 2020. 4. 28. · Strategy&Consulting S.A,...

1

Lot 2 – Impact evaluation of ROP 2007-2013 interventions

Evaluation Report

KAI 1.1 Urban development integrated plans

August 2019

Liliana Lucaciu Coordinator

Vasile Strat Evaluator

Șerban Totoescu Evaluator

Alina Boșoi Evaluator

Eugen Perianu Evaluator

Contact person (on behalf of the consortium):

Alina David – Project Director

Authors:

2

TABLE OF CONTENTS

1. EXECUTIVE SUMMARY .................................................................................................................. 6

2. CURRENT SITUATION .................................................................................................................. 11

3. STAGES OF THE STUDY ..................................................................................................................... 13

3.1. DESCRIPTION OF THE EVALUATION METHODOLOGY ..................................................... 13

3.2. ANALYSIS OF THE SPECIFIC LITERATURE......................................................................... 15

3.3. DATA COLLECTION: APPLICATION OF THE EVALUATION METHOLODOGY ................. 19

3.4. METHODOLOGICAL CHALLENGES AND LIMITATIONS ...................................................... 25

4. ANALYSIS AND INTERPRETATION ................................................................................................ 27

4.1. EVALUATION QUESTION 1: Care este efectul net al intervenţiei fondurilor pentru DMI 1.1 şi care sunt factorii care au determinat acest efect? .......................................................................... 27

4.2. EVALUATION QUESTION 2: What are the best performing interventions, to whom were they addressed and under what circumstances were these implemented? ........................................... 59

5. CONCLUSIONS, RECOMMENDATIONS AND LESSONS LEARNED ..................................................... 71

List of Graphics

Graphic 1 Portfolio of projects KAI 1.1 on the 3 financed areas ...................................................... 27

Graphic 2 Perceptions of beneficiaries regarding the effects of the interventions of KAI 1.1 ........ 40

Graphic 3 Perceptions of the beneficiaries regarding the development of local capacity for urban development planning ..................................................................................................................... 41

Graphic 4 Perceptions of beneficiaries regarding the development of local coordination mechanisms...................................................................................................................................... 41

Graphic 5 Perceptions of the beneficiaries related to the relevance of the projects financed under PIDU .................................................................................................................................................. 51

Graphic 6 Perceptions of the beneficiaries regarding the involvement of local partners in the development and implementation of PIDU ..................................................................................... 53

Graphic 7 Perceptions of beneficiaries regarding the purpose of projects implemented at metropolitan level ............................................................................................................................ 54

Graphic 8 Perceptions of the beneficiaries regarding the sustainability of the obtained effects and the continuity of the PIDU................................................................................................................ 57

Graphic 9 Perceptions of beneficiaries regarding the taking over of 2007-2013 projects by the subsequent planning exercises ........................................................................................................ 57

ANNEXES (separate document)

Annex A – Specific literature

Annex B – Instruments applied

3

Annex C – Case studies

Annex D – Primary data collected

4

ABBREVIATIONS LIST

RDA Regional Development Agency

MA ROP Management Authority of the Regional Operational Program

SB State budget

BI Bucharest Ilfov region

C Center region

DID Difference in Difference (Difference in Difference)

EU European Union

ERDF European Regional Development Fund

ESPON European Spatial Planning and Observation Network

FID Framework Implementation Document

GD Government Decision

GP Growth Pole

IB Intermediate Body

KAI Key Area of Intervention

MC Monitoring Committee

NE North East region

NIS National Institute of Statistics

NW North West Region

ONG Non-governmental organization

PEST Political, Economic, Social, Technological Factors

PSM Propensity Score Matching

RAI Annual Implementation Report

ROP Regional Operational Program

SE South East Region

SM South Muntenia region

SMIS Single Management Information System

SW South West Oltenia Region

SWOT Strengths, Weaknesses, Opportunities, Threats

UC Urban Center

UDP Urban Development Pole

W West region

5

PREAMBLE

This document represents the Evaluation Report for the project “Lot 2 – Impact evaluation of ROP 2007-2013 interventions”, evaluation topic: KAI 1.1 Integrated urban development plans, concluded between the Ministry of Regional Development and Public Administration through ROP Evaluation Unit within ROP Management Authority („Contracting Authority” and „Beneficiary”) and the Consortium made up of Civitta Strategy&Consulting S.A, Archidata, NTSN Conect and Structural Consulting Group.

SYNTHETIC DATA RELATED TO THE PROJECT

PROJECT TITLE ”LOT 2 – IMPACT EVALUATION OF ROP 2007-2013” INTERVENTIONS

Contract number 12 of 14.01.2019

Contract/ Financing Value 1.044.000 lei

Date when the Administrative Order was signed/ Date when the contract started

21 January 2019

Duration 6 months

Contracting Authority/ Beneficiary

Ministry of Regional Development and Public Administration

ROP Evaluation Unit

Provider

Contact persons

Consortium made up of Civitta Strategy&Consulting S.A, Archidata, NTSN Conect and Structural Consulting Group.

Alina David

Project Director

Project team Liliana LUCACIU – Coordinator

Vasile STRAT – Evaluation Expert

Șerban TOTOESCU – Evaluation Expert

Alina BOȘOI - Evaluation Expert

Eugen PERIANU - Evaluation Expert

6

1. EXECUTIVE SUMMARY

The overall objective of the KAI 1.1 is to increase the quality of life and create new jobs at the level of the supported areas. In order to achieve this goal, investments were made to rehabilitate urban infrastructure and improve urban services (including urban transport), develop business and entrepreneurship support structures, and investments to rehabilitate social infrastructure (including social housing) as well as to improve social services. A total number of 90 cities have been financed under KA1.1 by means of 505 projects that absorbed over € 976 million, contributing to increasing the quality of life for residents and the creation of 5294 jobs. The financing has been structured into three categories of cities: Growth Poles, Urban Development Poles and Urban Centres, reflecting the polycentric urban development approach. The evaluation addresses two evaluation questions: (I) What is the net effect of KAI intervention 1.1. and the factors that determined this effect and (ii) What are the best performing interventions, for whom and under what circumstances? Findings related to KAI 1.1 intervention The Integrated Urban Development Plan (PIDU) was a new tool in the practice of urban development, the experience of using this instrument in the context of KAI 1.1. being acknowledged by stakeholders as a first useful step for the subsequent strategic planning process as of 2014-2020. The experience acquired during the 2007-2013 programming period has contributed to increasing the capacity of cities to manage urban development in an integrated approach. This aspect is emphasized by the extension of the integrated planning concept, the beneficiaries considering that the documents and the planning process for 2014-2020 are of an enhanced quality level. However, PIDU represented a challenge, the beneficiaries indicating the difficulty of developing thematically balanced plans covering the whole range of eligible areas and sub-areas. This is also confirmed by the analysis of the portfolio of projects indicating that the sub-urban public space (street, pedestrian areas, passages, parking lots, green spaces, lighting systems, etc.) dominated 65% of the total number of KAI 1.1 projects 91% of the amount financed by the KAI 1.1. The effects of the KAI 1.1 intervention were analysed at project level to capture the thematic, immediate, and city-wide effects for the effects of the integrated approach. The analysis of impact indicators at the level of cities that received financing show similar developments for urban centres and development poles and a differentiation in some aspects for growth poles. Thus, the population of urban centres and urban development poles fell in 2017 compared to the reference year 2008 with rates of 17 and 26 per 1000 inhabitants, except for growth poles which recorded an increase of 1.9 per 1000 inhabitants. Employment has improved in all categories of cities, both from the perspective of the decrease of unemployment rate and from the perspective of the increase of the rate of employees. Negative variation trends, similar to the three categories, are recorded for the school population at the level of all pre-university cycles, except for the primary one, which

7

has recorded an ascending trend. The attractiveness of cities measured by the rate of change in the number of dwellings had similar increases in the three types of financed cities (Growth Poles, Urban Development Poles, Urban Centres), between 29 and 32 houses per 1000 inhabitants. At the level of the projects, urban infrastructure investments have generated positive effects in terms of fluidization and traffic safety, with travel time reductions averaging between 20% and 30% reported, the impact on KAI 1.1 being amplified by the fact that approximately 53% of the projects included investments in urban public space spaces including pedestrianized walkways, passages and car parks. At the same time, investments made in parking lots, pedestrian crossings, lighting systems have increased the comfort of the inhabitants and visitors, an aspect highlighted during the field visits and supported by the qualitative information collected and by the analysis of the sustainability reports. The rehabilitation and creation of new green spaces, municipal and urban parks, investments made in alternative modes of transport such as cycling, have had an effect on increasing the quality of life of the inhabitants through an active and healthy lifestyle. The evaluation highlighted the important effects of investment in heritage buildings, as evidenced by the visually improved appearance, but also mentioned during the focus groups and interviews. The effects also aimed at increasing the number of tourists, thus noticing a project that registered 23,000 tourists per year after the restoration of a municipal museum. Another effect is represented by the increase of tourist attractiveness of the cities by reintroducing heritage buildings into the cultural circuit, facilitating the increase of cultural events and implicitly the development of the cultural life of the city, but also through the promenade and recreation. Investments in social infrastructure have helped to increase the quality of services and increase the number of beneficiaries reflected in improved housing and care in the elderly, disability centres, expanding the capacity to provide services including new services such as: personal development, sports and arts for young people, in some cases focusing on disadvantaged groups. The projects analysed indicate an increase in the total number of beneficiaries, with values ranging from 30 to 320 persons. Video surveillance systems have been included in almost half of investment projects, in urban centres even in 69% of projects, with all projects reporting a decrease in crime rates, 30% of which show declines between 7% and 40%. Business infrastructure area was the least attractive, with the exception of growth poles where 70% of the projects financed under this operation can be identified. The least was invested in the business infrastructure, both as number of projects (3.5% of the total number of projects financed by DMI 1.1) and as value financed (1.4% of the total value financed by DMI 1.1), due to the high co-financing, the lack of project proposals at the stage of elaborating the integrated plans, the lack of clarity regarding the role of the local administration in the development of the business infrastructure. However, 800 jobs were created and over 930 were maintained through investments in business infrastructure, succeeding in attracting 69 economic operators within the newly created business structures. In general, the funded projects aimed to create clusters of

8

companies by attracting in the financed infrastructures of companies from certain fields or by attracting new companies that need support in the start-up phase, such desires being partially achieved. The counterfactual analysis shows a net impact of urban intervention on four of the six analysed dimensions, namely: natural population movement, migratory movement of population, evolution of the resident population and economic dimension from the perspective of employees and the unemployed. There was no confirmed net impact of intervention on the attractiveness of cities as a housing option and on education from the perspective of the school population, other factors that determine the general urban trend having a stronger influence than the interventions under KAI 1.1. The production of the results and the effects of the intervention has been strongly influenced, by intensity and scale, by the involvement of local stakeholders and the persistence of municipalities throughout the intervention cycle, from the preparation of integrated plans and the formulation of projects to project implementation and the analysis of the effects obtained. A major influence was set by MA ROP by the adoption of a new integrated development approach, perseverance and the ability to support its transposition into an investment financing mechanism. The complementarity of other projects from other sources of funding contributed to the amplification of the effects of the investments financed under KAI 1.1. Among the factors identified as negatively affecting the implementation of KAI 1.1. and the production of effects, the evaluation team has mentioned "the public procurement process" that generated delays in the implementation or application of financial corrections. Another factor that affected the implementation was the lack of correlation with other infrastructure projects funded by other operational programs, which has created delays in the implementation. The competitive project selection process for the urban centres was, on the one hand, a quality-enhancing factor; on the other hand, project preparation periods proved to be insufficient for the ability of authorities and service providers to prepare projects of good quality. Last but not least, PIDU's focus on urban infrastructure has limited the coverage of the various investment needs needed to increase the quality of life and create jobs at the city level. The comparison of the effects produced for the three types of beneficiary cities (Growth Pole, Urban Development Policy, Urban Centres) shows that keeping the proportions, the urban infrastructure effects are similar in all categories. Thus, priority issues have been addressed, thinking has been integrated, the effects are obvious and significant. In terms of social infrastructure, other than surveillance systems, in cities where projects have been implemented in this area, the effects are also similar as a type of benefit - rehabilitated infrastructure, better services, new services, in some cases increasing capacity and thus addressing a larger number of beneficiaries. In the case of business environment infrastructure, growth poles dominate as a number of projects, and implicitly as amplitude of the effects, compared to the 5 projects implemented by the Urban Development Poles and the Urban centres. Growth poles have managed to configure metropolitan areas and thus support the polycentric approach to urban development. The results obtained through DMI 1.1 interventions were concentrated at the level of the localities in which they were implemented, benefiting mainly the respective communities, but there were also projects implemented of metropolitan interest.

9

The evaluation found, based on the quantitative and qualitative analyses, that the interventions that gave the best results are those related to the urban infrastructure in general, where the results indicate a certain contribution to the increase of the quality of life and the attractiveness of the cities, as well as to the creation of jobs. . A particular example refers to the rehabilitated heritage objectives, better results being obtained in the situations in which the beneficiaries have succeeded in developing the socio-cultural activity of these institutions, thus highlighting their role and contributing more to the revitalization of the cultural life of the city, respectively. The interventions in the social infrastructure also had results that contributed to the increase of the quality of life, but being much smaller in number compared to the investments in video surveillance systems, these being rather associated with the urban infrastructure than the social one. The investments in the infrastructure of the business environment had better results in the situations where they were based on an integrated thinking, of the economic cluster type, as well as in the situations in which the combination of the public and the private investments was successful. Conclusions KAI 1.1 has reached its goal, consistently promoted the concept of integrated urban development and created the framework for beneficiary cities to develop their capacity to develop and implement PIDU, thus contributing to the increase of quality of life and job creation. The implementation of the integrated approach towards urban development was limited by the focus of investments on urban infrastructure, to the detriment of social and business infrastructure. DMI 1.1 produced the expected effects, contributing to the substantial improvement of the aspect of the beneficiary localities, the stimulation of some economic and social activities, the increase of the comfort level of the citizens, the fluidization and the increase of the traffic safety, the increase of the number of visitors or the revitalization of the cultural life. In the social field, investments have contributed to increasing the quality of social services, building new locations and developing new social services. Although the share of investments in business infrastructures was low compared to the other areas, they contributed to the creation and maintenance of jobs in cities through the business structures benefited by the attracted companies. DMI 1.1 had a significant impact on the cities through the 4,491 newly created jobs and 4,133 jobs maintained. The exercise of the implementation of DMI 1.1 2007-2013 also had indirect effects, such as the increase of the management capacity of the integrated urban development plans of the public administration and of the local partners. Increasing management capacity at the local level and continuing to support the development of metropolitan areas remain priority needs for further integrated urban development. As a whole, cities are at the beginning of the road in terms of an integrated approach to urban development. The 2007-2013 programming period and the exercise of elaborating integrated plans represented a good start and an important opportunity for cities to start thinking and planning integrated urban development, respectively to develop their management capacity for urban development.

Recommendations

10

1. Reconsider the economic development in the prioritization of investments, increasing its role and importance, by stimulating cities to invest in this area, in parallel with the development of skills at local level. In the long term, the only feasible way for the development of a city or metropolitan area is economic development

2. In order to support the elaboration of quality plans and projects on integrated urban development, the MA ROR must support the development of cities' capacity to manage the urban development process by financing training, studies, exchanges of experience between Romanian cities and cities in the European Union, technical assistance on the topic of urban development, including financing, within the framework of THE REGIONAL DEVELOPMENT AGENCY, support activity for all cities in the region.

3. Increased ROP requirements can influence the quality of integrated development plans. Criteria regarding the balanced approach of the areas of intervention, the orientation towards results and impact, the involvement of local partners in the whole process, including with the assumption of targets for the selected indicators, can be included in the evaluation of the plans.

4. The financing of the interventions must be done in a flexible, non-limiting way in order to allow each beneficiary the freedom to choose the most relevant investments, remaining within the general framework prescribed by the operational program.

5. ROP should also finance complementary actions for infrastructure investments to support and stimulate the production of targeted effects, such as studies, training, technical assistance and other capacity development measures for infrastructure operation.

6. MA ROP should consider the possibility of financing integrated urban development plans through combinations between the form of financing through non-reimbursable funds and the form of financing through financial instruments (loans, guarantee loans) in accordance with the guidelines in this regard in the proposal of EU Regulation 375 on structural and investment funds (e.g. Title 5 - Financial support).

7. It is recommended that the requirements regarding the format of the urban development plans be flexible, the acceptance of any format provided the technical and eligibility criteria are met.

8. AM ROP must support the development of the local governance system appropriate to the integrated urban development, through specific measures to develop the capacity of the stakeholders, to promote the use of ITI and DLRC (Local Development under Community Responsibility) tools.

9. Providing submission deadlines that allow an adequate period of preparation of urban development plans (8-12 months) can facilitate the improvement of the quality of the support plans and documentation.

10. Metropolitan areas are under development and should be further supported, and their explicit identification as beneficiaries of the future urban development priority should be explicit.

11. It is recommended to adapt the system of indicators to meet the needs of evaluation, ensuring: (i) covering each specific objective or theme with indicators of outcome and achievement, (ii) ensuring consistent reporting to allow aggregation and processing, (iii) ensuring the necessary data through collaboration protocols with the National Institute of Statistics or other data holders.

12. It is recommended to use additional indicators than those used in the current evaluation, in order to capture the dimensions of the impact produced.

Lessons learned

11

The integrated urban approach is a complex, ever evolving concept that makes its implementation, transposition into plans, programs and projects a challenge. The experience acquired from the implementation of KAI 1.1. demonstrated that the cities learned practically during the elaboration of the first PIDUs, realized the limitations in that phase and used the experience in the next programming period. Capacity development needs continue to be sustained both by the concept itself, but also by the changes in the urban environment, the sophistication of planning, financing and operating methods and tools in the multitude of areas that interventions can cover. The main challenges are thus expanding the understanding of the concept in the broader environment of local authority partnerships, developing the capacity to manage urban development beyond managing project portfolios of integrated urban development strategies.

2. CURRENT SITUATION The interventions that received support under this KAI, namely the elaboration and implementation of the integrated urban plans, represent the strategic response of the ROP 2007-2013 for solving the problems generated by the economic changes that occurred in Romania after 1990, as a consequence of the restructuring of the industry and the reduction of the economic activity in many between the cities of the country. According to the analyses made at the time when ROP (2006-2007) was elaborated, many regional urban centres were in a state of socio-economic decline, thus having a decreasing role in the development of adjacent areas and regions having the reduction of public investments in basic infrastructure, general deterioration of the social situation and decrease in population incomes as a background. According to ROP 2007-2013 programming documents, KAI 1.1 had the objective of increasing the quality of life and creating new jobs in urban areas. In order to achieve this objective, investments were made to rehabilitate the infrastructure and improve the urban services (including urban transport), to develop the business and entrepreneurship support structures, as well as investments for the rehabilitation of social infrastructure (including social housing) and for the improvement of social services. Interventions under 1.1. were structured based on a vision of polycentric and balanced urban development of the country, on three categories of urban localities:

Growth poles - large urban centres (7 municipalities designated by GD no. 998/2008, with subsequent modifications and completions) and their areas of influence;

Urban development poles (13 municipalities designated by GD no. 998/2008, with subsequent modifications and completions);

Urban centres - cities and municipalities with more than 10,000 inhabitants. In this category were included the 6 sectors of Bucharest.

KAI 1.1. financed a total number of 90 urban localities, of which 7 growth poles, 13 urban development poles and 70 urban centres through 505 projects completed by the cut-off date of this evaluation, namely December 31, 2018. Localities were financed on the basis of the integrated urban development plans (PIDU), strategic planning tools introduced in the urban development in Romania for financing KAI 1.1., allowing an integrated approach towards the development of the interventions on several social and economic areas.

12

The importance of KAI 1.1. is reflected by the over 967 million Euro with which it contributed to the development of cities through 505 implemented projects. On the other hand KAI 1.1. represented a challenge from the perspective of the implementation, the amount of 1,320 million Euro allocated not being fully used, 25 contracted projects being cancelled and 16 projects being still under implementation at the cut-off evaluation date.

13

3. STAGES OF THE STUDY

3.1. DESCRIPTION OF THE EVALUATION METHODOLOGY

The evaluation methodology was built on two evaluation questions: ((i) What is the net

effect of the interventions under KAI 1.1. and the factors that determined this effect

and (ii) What are the best performing interventions, to whom were they addressed and

under what circumstances were these implemented?, and 15 assumptions that focus on

evaluating the key aspects of the impact and the factors of influence.

The methodological approach was based on the optimal combination of quantitative and

qualitative methods, adapted to the particularities of the intervention, namely:

- Financing the three types of financed urban localities: Growth Poles, Urban

Development Poles and Urban Centres in all eight regions of the country;

- The two dimensions of the effects of the intervention:

- Effects at city level1 as an entity

- Effects at thematic level corresponding to the three financed areas

(urban infrastructure, business environment infrastructure, social

infrastructure).

- Comparative approach of the three types of urban localities (and the eight

regions) in order to identify possible differences regarding the effects of the

intervention generated by their characteristics.

The evaluation aimed at both the gross effect of the intervention and at the net effect, representing the program's contribution to the observable gross effect, isolated from other influencing factors. The net effect of the intervention was analysed by quantitative counterfactual methods only for investments in urban centres, only these meeting the minimum conditions for applying these methods, namely homogeneity, number of interventions and beneficiaries, allowing the construction of a control group. Counterfactual analysis was included as evaluation method for estimating the net effect of KAI 1.1 interventions. The impact variables (established in the conceptualization phase of the analysis) were analysed and compared both for the cities that received financing (the treated group) and for the cities that did not receive financing (the control group) in order to estimate the net effect of investment. Counterfactual analysis methods are complex and are presented in detail in Annex B8. In order to capture the specific thematic effects and also the overall effects on the urban development, according to the integrated approach of the KAI, the analysis was carried out on two levels: project level, capturing specific thematic effects and city level presenting the aggregate effects of the integrated interventions. This approach is in line with the recommendations regarding the evaluation of the results of the integrated interventions aimed at the urban development described in the European

1 It refers to the urban locality and is used as a generic term for both cities and municipalities as a form of

territorial administrative organization.

14

Union document "WORKING PAPER Indicators for integrated territorial and urban development (ESPON)"; Quantitative impact evaluation methods have been supplemented with qualitative

methods, all of which are based on the theory of change. The methods applied are

briefly presented in the three tables in this chapter: Collection methods and

techniques, Data collection methods and techniques and Complementary methods and

techniques of qualitative analysis.

For qualitative analyses, the case study at the city level was a key element of the analysis, and was supplemented by other methods of collection and analysis. The involvement of the stakeholders was planned both at central and local level, and we mention here the Association of Municipalities in Romania, the Association of Romanian Cities, the Federation of Metropolitan Areas and the Urban Agglomerations of Romania, representatives of the business environment, etc. The methodological approach towards evaluation has identified a wide range of data collection methods and techniques as described below. Data collection methods and techniques:

The desk research aimed at understanding the program and the KAI, both regarding

the implementation and the effects obtained;

The specific literature was studied in order to formulate a conceptual context

necessary for the interpretation of the information obtained and for formulating the

most relevant findings and conclusions;

Interviews with the main authorities involved in the management and

implementation of KAI 1.1 (MA ROP, IBs RDA) and with key actors at regional level

were planned;

The case studies were aimed at collecting information from the field and at

elaborating in-depth analyses of the interventions in the 3 fields;

Regional focus groups were planned, with the involvement of RDA representatives,

representatives of local authorities, project managers to collect qualitative

information on the implementation and effects of integrated plans;

The survey aimed at collecting data from the beneficiaries of financing (Growth

Poles, Urban Development Poles, Urban Centres);

The Delphi method aimed at interrogating practitioners in the field of urban

development at the level of the beneficiary cities (local public administrations) to

collect the opinions regarding the preliminary recommendations resulting from the

evaluation process;

The nominal group was included in the methodology for analysing the results of the

operation "Growth poles" and aimed at integrating the views of the local authorities

regarding the preliminary results of the evaluation and completing the analysis with

new information;

15

The panel of experts aimed at supporting the analyses carried out, by debating the

results of the evaluation process, in a context that goes beyond the national

framework;

The use of cross data was aimed at supporting the data analysis process (time series,

cross-sectional and longitudinal data) and applying counterfactual methods, by

correlating the relevant data on the beneficiaries and the control group.

Methods and techniques applied for analysing and interpreting the data and the

information

The SWOT and PEST analyses sought to highlight the external and internal factors

that influenced the implementation of the intervention and the production of

effects, both at project level (part of the case studies) but also at the city level and

evaluation area;

The stakeholder analysis aimed to identify and map all the stakeholders from various

levels (central, regional, local);

The analysis of the primary and secondary data aimed at creating a clear image

regarding the project portfolios at PC, PDU, CU, as well as at regions level;

The elaboration of the visual diagram followed on the one hand the synthesis of the

information on the identified needs, the objectives and the strategy of the KAI, and

on the other hand the situation of the implemented projects, through a GIS map;

The theory of Change and the log frame represent the method and the instrument

that allow the analysis of the causality between the phenomena identified inside and

outside the intervention;

The territorial representation / analysis of the results sought to visualize the results

recorded after the implementation of the ROP projects, at the level of each

development region;

The benchmarking analysis aimed at comparing, through qualitative methods,

certain aspects of the integrated urban development approach in the cities that have

been the subject of the case studies with a set of good practice elements identified

by the specialized literature;

Counterfactual analysis was included in the evaluation methodology as a quantitative

method for assessing the net impact of KAI 1.1. Due to the specificity of the three

financed areas, this method was only applicable for interventions in at the level of

the Urban Centres.

3.2. ANALYSIS OF THE SPECIFIC LITERATURE

Theories and approaches in the field Theoretical aspects regarding urban development (how to define the problems, defining the desirable) are constantly dynamic, being influenced by the dynamics of thinking in the field, the results of practical experiences, the dynamics of needs and desires on a global or European level in the socio-economic field in general. Urban development is a discipline that is difficult to define, given the multitude of aspects it deals with. The essence of this discipline can be defined by the aspects it addresses rather than by a

16

general paradigm or a universal prescriptive approach. Contemporary urban planning aims to strike a balance between conflicting requirements regarding economic development, social equity, environmental protection, sustainable development and aesthetic aspects. The current framework of thinking on urban development at European Union level EU-level thinking on urban development is constantly dynamic, with a series of periodic studies underpinning the various EU policy documents. Cities of tomorrow: Challenges, visions, ways forward (European Union, Regional Policy, 2011) is summarising a series of desires, directions of action, as well as a series of principles regarding the governance model necessary to achieve the proposed decisions regarding European cities. Objectives and values that describe the European city of tomorrow:

a place with advanced social progress, with a high degree of social cohesion, a socially balanced housing space policy, as well as with social, health and "education for all" services;

platform for democracy, cultural dialogue and diversity;

a place of ecological regeneration;

an attractive place and an engine of economic growth. The key principles of future urban and territorial development at European level,

which should:

to be based on a balanced economic growth and a territorial organization of activities, with a polycentric urban structure;

be based on the development of strong metropolitan regions and urban areas that can provide adequate access to services of general economic interest;

be characterized by a compact structure, with a limited urban area;

to offer a high level of environmental protection and environmental quality in cities and their neighbourhoods.

Necessary directions of action:

Competitiveness in the global economy must be combined with sustainable local economies;

Creating a resilient and inclusive economy;

The potential of socio-economic, cultural, generational and ethnic diversity should be further exploited as a source of innovation;

Combating spatial exclusion and the problem of energy resources through better housing;

A holistic approach to environmental and energy issues is needed;

Small and medium-sized cities can play an important role in the well-being of not only their inhabitants, but also of the surrounding rural populations. These cities are essential for avoiding rural depopulation and for promoting balanced territorial development;

A city must have attractive open public spaces and promote sustainable, inclusive and healthy mobility.

EU policy in the urban development area An EU Urban Agenda has been elaborated, several political agendas in this area having been developed in the past. The Leipzig Charter (2007) and the Toledo Declaration (2010) are continued through the EU Urban Agenda. The urban agenda for the EU

17

represents an integrated and coordinated approach to the urban dimension within EU and national policies and legislation. By focusing on concrete priority themes within partnerships dedicated to each theme, the Urban Agenda aims at improving the quality of life in urban areas. In 2016, the Amsterdam Pact, agreed by EU ministers responsible for urban issues, created the EU Urban Agenda. EU policy in the field of urban development is constantly developing. The main elements highlighted in this policy have been maintained since 2007 and until now, as briefly described below. The novelty elements of the Amsterdam Pact refer to the launch of 12 partnerships on key issues of urban development and to the establishment of directions of actions related to: a better reflection of urban development in EU law, identifying the best financing modes of urban development, the development of the knowledge base on urban development. Main elements highlighted in the EU policy in the period 2007-present: The importance of an integrated approach to urban development; The importance of the balance between the desires related to the different fields:

economic, social, cultural, environmental, etc.; The need to involve the administration from different levels (city, county, region,

national, EU) in urban development issues; The need to improve the coordination of sectoral policies; Need for appropriate coordination mechanisms: cross-cutting (cross-sectoral);

verticals (various levels of administration); horizontal (all stakeholders). Experiences in the urban development area The experiences in the field of urban development were analysed mainly at European level. The information was extracted from two studies carried out on a number of 50, respectively 10 projects from different EU member countries. In the studies, the method of analysis used is the case study. The effects of the projects are analysed bottom-up, that is from the perspective of the objectives defined at the project level, not from the point of view of the objectives defined top-down at the operational program level. The main findings and recommendations of the studies are:

Cities are trying to reposition themselves from an economic point of view;

At the level of the analysed projects, infrastructure investments predominate;

The success of the projects is dependent on a set of other measures and activities, besides the rehabilitation of the infrastructures;

Some projects are part of a much larger development plan, such plans can have an extended time horizon, even decades. It is recommended for all projects to be aligned with the city's strategic development plans;

The involvement of stakeholders is an essential element for success and this involvement can go beyond the triple helix model (administration, academic, business environment) and may involve various other categories of beneficiaries of the implemented projects. For example, in the case of social projects, it is important to include NGOs as well in the partnership;

Careful analysis of the level of needs (demand) is needed in order to avoid the development of infrastructures that will be under-utilized due to lack of demand;

An inclusive approach to economic development is needed, which takes into account underdeveloped areas and disadvantaged target groups or which encounter major difficulties in integrating into the labour market;

18

Monitoring and evaluation of urban development, as a basis for substantiating policies in the field, is a practice rarely encountered. Policies are predominantly developed based on empirical knowledge and based on funding opportunities and what is politically appropriate;

The notion of integrated approach can have different meanings for different stakeholders and between different policy documents. The best practical examples of integration are those that have succeeded in integration both vertically (different levels of administration) and horizontally (between stakeholders);

Studies have shown that cities can produce good results by implementing certain projects at one point. In order to develop a permanent practice in this respect with constant results, it is necessary to collaborate both vertically (with different levels of administration) and horizontally (inter-sectoral policies, stakeholder involvement), as well as to exchange experience with others cities.

Evaluations in the urban development field The main aspects learned from the experience of the evaluation exercises of the interventions in the field of urban development are: Emphasis is placed on the use of case studies in urban development projects as the

main method of evaluation;

The effects of the projects are often difficult to surprise or summary described, being based mainly on qualitative analyses;

For the quantitative analysis of the results, the evaluation has difficulties due to the fact that most of the result indicators defined at the operational programs level are, in fact, reformulated performance indicators;

There is a small number of indicators defined at the level of operational programs regarding integrated urban development.

Main issues and implications for the current evaluation The main aspects identified during the study of the specialized literature, together with the elements outlined during the consultations organised in the inception phase with MA ROP, have made a major contribution to the better understanding of the intervention logic of DMI 1.1 and to the definition of the hypotheses related to Theory of change.

The importance of an integrated approach to urban development;

The need for a common vision for the future of the city and a balance between long, medium and short term vision;

The importance of the balance between the decisions related to the different fields: economic, social, cultural, environmental, etc.;

Urban development should be based on the development of strong metropolitan regions and urban areas that can provide adequate access to services of general economic interest;

Need for a system of monitoring and evaluation of the implementation and results of urban development plans and projects;

Consistency / continuity is needed in the implementation of urban development plans and projects;

Small and medium-sized cities can play an important role in the well-being of not only their inhabitants, but also of the surrounding rural populations. These cities are

19

essential for avoiding rural depopulation and for promoting balanced territorial development

The need for appropriate mechanisms of involvement and coordination: o Transversal (cross-sectoral policies); o o Vertical (involving the various levels of the administration - city, county,

region, national, EU - in urban development issues; o Horizontal (involving all stakeholders - specialists, NGOs, business environment,

citizens, etc.). Involvement of various categories of stakeholders and in the financing (financial and non-financial) of urban development projects.

Projects should include both infrastructure activities and complementary activities (e.g. training, development of management plans, efficiency of activities, development of monitoring and evaluation systems etc.)

3.3. DATA COLLECTION: APPLICATION OF THE EVALUATION METHOLODOGY

Data collection methodology and techniques

No Method Details

1. Desk research

Program documents (ROP, Framework Implementation

Document, Monitoring Committee minutes, applicant guides,

project databases and achieved program indicators) were

analysed, documents related to the implemented projects

(PIDU, project descriptions, sustainability reports, other

relevant information), various other documents identified during

the evaluation.

The specialized literature was analysed (studies, strategies,

analyses, etc.);

Databases and relevant statistical data sources (NIS), data

sources at local / regional level were processed and analysed.

2.

Semi-

structured

interviews

Interviews were conducted with the main authorities involved

in the management and implementation of KAI 1.1 (MA ROP, IB

RDA) and with key actors at central level (Association of

Romanian Cities). The Romanian Municipalities Association was

invited to the panel of experts, as it did not show availability for

the interview. The Federation of Metropolitan Areas has no

representatives, the contact with the metropolitan areas taking

place within the Nominal Group;

Both individual and group interviews were organised;

The organisation of the interviews has allowed to obtain

relevant information for all the operations financed, on the 3

areas of interventions;

Number of interviewees: 25.

3. Case studies

8 case studies were elaborated at the level of the 8 beneficiary

cities: 4 urban centres, 2 urban development poles, 2 growth

poles, in 8 regions.

20

The selection of the case studies was made in such a way that

they are representative both for the types of interventions

funded (Growth Poles, Urban Development Poles, Urban Centres)

and at regional level. The selection criteria for the 8 case

studies are presented in Annex C Case studies.

The case studies included interviews with PIDU coordinators

(from the local administration) and with representatives of the

beneficiary institutions of the implemented projects.

During the research for the case studies, site visits were

organised for several projects, the number of projects visited

varying according to the portfolio of implemented projects of

the visited city (min. 2-3 projects, max. 8 projects, in total

being visited 29 days. projects).

4. Focus-groups

8 focus groups were organised at regional level, one per region,

with the involvement of representatives of local authorities and

RDAs. Within the focus groups, the aspects regarding the

different types of interventions supported (growth poles, urban

development poles and urban centres), respectively the 3

horizontal themes - urban infrastructure, the business

environment, social), as well as aspects regarding the

complementarity of these interventions and the potential areas

for creating synergies were pursued from different perspectives.

5. Survey/

opinion poll

The survey was applied to the entire population of beneficiaries

(90 cities), in an online format, receiving 28 responses (31%

response rate).

6. Delphi

The questionnaire was applied to the entire population of

beneficiaries (90 cities), being addressed to the mayors, in Word

document format and 15 replies were received. Due to the

consensus reached after the first round of consultation, the

second round was not necessary.

7. Nominal group

It was organized on July 9, 2019, as part of a larger event (the

meeting of metropolitan areas), which brought together

representatives of metropolitan areas and county-resident

municipalities. This method provided the opportunity to discuss

conclusions with PC representatives, generating ideas for future

financing of urban development of metropolitan areas.

8. Experts panel

It has brought together specialists from different areas of

activity: academic environment, policy making, ROP

management system representatives, and practitioners. Details

on the methodology for the expert panel are included in Annex B

Instruments applied.

9.

Use of cross data, time series, longitudinal

This method was mainly used as part of the counterfactual

analysis. The counterfactual analysis is presented in its entirety

in Appendix B Instruments applied.

21

data

10. Observation techniques

This analysis sought the systematic collection of information

targeting the most relevant aspects for various types of projects

in the case studies, according to the observation grids (Annex B

Instruments applied).

Methods and techniques for analysing and interpreting data and information needed for

the evaluation of the impact of KAI 1.1.

No

.

Proposed

analysis

methods and

techniques

Methodology

1. SWOT analysis SWOT and PEST analyses were performed at the level of the

evaluation field, the urban development, taking over information

from the analyses performed in the case studies. The conclusions of

the SWOT and PEST analyses were used in the case studies to

understand the context at the city level at the time of PIDU

elaboration and the evolution of this context.

2. PEST analysis

3. Stakeholder

analysis

This analysis synthesized all stakeholders from various levels

(central, regional, local, ROP system, specialists, etc.) and is

included in Annex B Final Instruments applied.

4.

Analysis of

primary and

secondary

data

As part of this analysis, aspects regarding the number of projects

contracted / completed, the structure of the project portfolios by

types of cities and regions, values financed against allocation,

values reached for the indicators related to the effects related to

the different types of projects, were followed. At the same time,

the values reached for the impact indicators at the city level were

analysed.

All qualitative information collected through the applied methods

was also analysed: documentary research, interviews, focus groups,

case studies, etc.

Table, matrix and graphical representations were used to highlight

the recorded results, in a comparative way on the following levels

of analysis:

o types of beneficiaries (Growth poles, Urban development poles,

Urban centres)

o regional level, with comparative analysis of the project portfolios

5.

Elaboration of

the visual

diagram

Has synthesized the key elements of the DMI 1.1 intervention - it is

included in Annex B Used tools.

6. Log frame The log frame was used during the initial phase in order to

contribute to the understanding of the intervention logic of KAI 1.1

22

and to the formulation of the theory of change hypotheses. The log

frame developed by the evaluators is presented in the Appendix B

Instruments applied.

7. Theory of

Change

The Theory of change for KAI 1.1 was reconstructed on the basis of

the programming documents, the analysis of the specific literature,

the discussions with the actors involved in the programming and

implementation process to identify the implicit elements (not

expressed in the programming documents but implicitly taken into

consideration. in the planning process) and based on the experience

of the evaluators who can detect certain missing elements. The

theory of change was thus reconstructed (it is made more explicit

and completed) and validated by the members of the Evaluation

Coordination Committee. The formulated assumptions were then

interrogated during the evaluation process, through various methods

- interviews, case studies, focus groups, documentary research,

survey - to explore to what extent these are confirmed in the

implementation practice..

8.

Territorial

representation

/ analysis of

the results

A matrix was developed, included in the evaluation report, the

result of the analysis of the project portfolios financed at the level

of each development region.

The analysis of the project portfolio took into account the

heterogeneity of results and indicators reported at project level and

was limited to 86 projects out of the 505 completed. The

elaborated matrix is presented, for illustrative purposes, in Annex D

Primary data collected. The analysis of these indicators, even on a

limited sample, supported the evaluation on two levels: it provides

an indication of the types of effects of the various projects and how

to quantify these effects, indicates the need to develop the system

of indicators and the system of collecting indicators, aspects

expressed as findings, respectively recommendations of the

evaluation.

Complementary quality analysis methods and techniques for impact evaluation in the

context of KAI 1.1.

Complementary

quality analysis

methods and

techniques proposed

Methodology

Benchmarking analysis The analysis targeted, by means of the case studies, to present a

comparative analysis of the extent to which, at the level of the

analysed cities, some elements of best practice identified in the

specific literature were present.

How was triangulation ensured?

23

Information sources can be triangulated by extending the data sources, by involving all

stakeholders in the evaluation process: management structures (ROP MA and IB),

beneficiaries, specialists in the field. The triangulation of the evaluation methods was

ensured by combining the methods for the analysed effects pursued in order to

consolidate the findings and conclusions.

Counterfactual analysis

Counterfactual analysis for determining the net effect of KAI 1.1 intervention was

possible only at the level of the urban centres because only in this case there is a

possibility of comparing the beneficiary cities with cities in a control group. The analysis

cannot be applied in the case of the Urban Development Poles or the Growth Poles

because in these cases, the entire universe - all 20 units - received assistance under KAI

1.1, consequently there is practically no control group.

Counterfactual analysis was carried out in three main stages:

selection of the urban centres treated according to the selection criteria - the

treatment group,

selection of the counterfactual group from the untreated urban centres - the

counterfactual group,

quantification of the effect (impact) by comparative analysis of the two groups,

based on the evolution of the impact indicators at the city level selected by the

evaluation.

The difference-in-difference technique (DID) and propensity score method (MSP) were

used in order to quantify the effect of KAI 1.1 intervention (for urban centres). DID is

calculated in two stages. In the initial stage, the difference between the two groups is

evaluated, at the time before the intervention (T0) and also at the time after the

intervention (T1). In the second stage the difference between the two differences,

calculated in the previous stage for the two time moments, is evaluated. The matching

score method was applied to ensure similarity from the perspective of the observable

characteristics of the two samples of beneficiaries (treated and untreated). The

methods are presented in detail in Annex B8.

In order to analyse the effects of the intervention at city level, the evaluation selected

a set of statistical indicators that can best reflect the effects of integrated urban

development interventions. The main source for choosing these indicators is the

European Union document WORKING PAPER Indicators for integrated territorial and

urban development (ESPON). The chosen indicators are relevant to indicate the

evolution of the city's attractiveness, quality of life and economic development, i.e. the

effects pursued through the intervention of KAI 1.1.

Indicator recommended by ESPON

Indicator used during the evaluation

Observations

Net migration rate The number of immigrants

Both indicators were used to highlight the evolution of both factors that make

24

Rate of number of emigrants

up the net migration.

Population within an area of 50 km

The domiciled population

The data for this indicator could not be identified for Romania, at the level of the local territorial administrative unit (city and municipality) and another proxy indicator was identified.

Population natural movement

Number of deceased persons

Both indicators were used to highlight the evolution of both factors that make up the natural movement of the population.

Number of live births

New private houses as percentage of the total number of houses

Existing houses at the end of the year


The structure on passenger modes of transport


Access to public services (schools and hospitals)

School population (primary level) School population (high school level) School population (high school level)


Long-term unemployed as a percentage of total unemployed

Number of the unemployed


Average number of employees

This is an additional indicator used to evaluate the evolution of the economic dimension.

The selected variables, corresponding to the six dimensions of the impact of urban

development interventions, used to analyse the impact of KAI 1.1 at the city level, are

as follows:

- Dimension related to the natural movement of the population (indicators:

number of deceased persons, number of live births)

- Dimension related to the migratory movement of the population (indicators:

number of immigrants, number of emigrants)

- Dimension related to the general evolution of the population (indicator:

domiciled population)

- Economic dimension (indicators: number of unemployed, average number of

employees)

- Educational dimension (indicators: school population (primary, secondary and

high school level)

25

- Dimension related to the attractiveness of the city (indicator: houses existing

at the end of the year)

Period covered by the collected data

The data collected covered the entire period of programming and implementation of

KAI 1.1 interventions, including all the completed projects, as well as the period after

the completion of the interventions in order to capture the impact. The analysis period

is 2007-2017.

3.4. METHODOLOGICAL CHALLENGES AND LIMITATIONS

The application of the methodological tools required the contact with a large number of stakeholders and the correlation of the evaluation activities with their working agenda. Also, the evaluation methodology followed a sequential application of the methods, so that the implementation period extended for a longer period than initially planned. This difficulty was mitigated by the allocation of additional resources to the evaluation.

During the aggregation of the values of the result indicators at project level, by analysing the most recent sustainability reports of all 505 completed projects, the heterogeneity of these indicators and the units of measurement represented a limit to the approach. The matrix thus constructed includes a number of 86 completed projects for which such representation was made and reflects the mentioned heterogeneity and is included, as an illustration, in the Annex D Primary data collected.

The only indicators that can be aggregated at national level for the entire portfolio of projects related to KAI 1.1 are the program indicators, their values being recorded by ROP monitoring system. However, the evaluators found that the values reported and aggregated in the ROP system are strongly distorted for these indicators as well. For example, in the case of a certain urban infrastructure project, temporary jobs (during the execution of the project) were taken over as jobs created in the ROP monitoring system, inducing the idea that new jobs were created. Another example concerns the maintenance of jobs: another analysed urban infrastructure project reports existing jobs during the project (within the contractor team) as jobs maintained, inducing the idea that these jobs were maintained as a result of the project for an indefinite period.

In order to overcome these limitations, an analysis based on several sources of information was made, focused on the sustainability reports of the 86 projects supplemented with qualitative information collected through case studies and focus groups at regional level, information quantified later on by the survey applied to the level of the entire population of beneficiaries (the 90 beneficiary cities).

For the counterfactual analysis, initially the indicators identified in the specific literature were considered as being the most suitable for measuring the effects at the level of the urban development. As detailed in Chapter 3.3, the analysis of data availability in the context of KAI 1.1 interventions also required the use of proxy indicators. This aspect did not affect the quality of the evaluation results and is a good reference for other future evaluations for which it is recommended to use additional indicators.

The response rates to the online survey and the response to the invitations to participate in focus groups was relatively low, primarily due to the significant period since the completion of the interventions, but also to the overload of the persons

26

currently implementing ROP 2014-2020 projects. This factor was mitigated by the allocation of additional resources for contacting stakeholders.

27

4. ANALYSIS AND INTERPRETATION

4.1. EVALUATION QUESTION 1: CARE ESTE EFECTUL NET AL INTERVENŢIEI FONDURILOR PENTRU DMI 1.1 ŞI CARE SUNT FACTORII CARE AU DETERMINAT ACEST EFECT?

Analysis of the portfolio of projects financed under KAI 1.1 PIDU is the key instrument for implementing the integrated approach to urban development through KAI 1.1. PIDU analysis focused primarily on the quality of the matrix of objectives and projects, highlighting the following situations: - Plans that include a complete and balanced matrix: include all 3 areas in a

balanced manner: urban, social infrastructure, business environment at all levels (strategic objectives, priorities / policies / etc., Projects) - the case of growth poles and some cases from categories Urban development centres, Urban centres.

- Plans that include a partial and unbalanced matrix: cover all 3 areas at the level of strategic objectives, but at project level there is a lack of balance – there are mainly projects covering the field of urban infrastructure, to a lesser extent the field of social infrastructure and to a very limited extent or not at all the field of business infrastructure. Most of the plans at the level of urban centres and some of the plans at the level of urban development poles identify only the projects for which financing applications have been submitted under DMI 1.1, and there are plans that identify both the projects submitted under DMI 1.1 as well as other projects.

At project level, the analysis of the project portfolios financed under KAI 1.1 is summarized in the graph below and further analysed at the level of the three areas.

Source: data processed during the evaluation based on the information obtained from MA ROP

Urban infrastructure area dominates from the points of view of the number of projects and financed value (65% of the total number of projects and 91% of the total value

0%

20%

40%

60%

80%

100%

Urban infrastructure Social infrastructure Business environmentinfrastructure

Percentage no of projects out of the total KAI 1.1

Percentage of the financed value out of the total KAI 1.1

Graphic 1 Portfolio of projects KAI 1.1 on the 3 financed areas

28

financed under KAI 1.1), the beneficiary cities identifying numerous development needs in this area. Within the field of urban infrastructure, projects targeting the sub-domain of urban public spaces (city streets, sidewalks, squares, pedestrian areas, bridges, overpasses and underground passages, walkways, parking lots, infrastructure related to urban public utilities, green spaces, etc.) – represent 53% out of the total number of projects financed under KAI 1.1, namely 72% of the total value financed by KAI 1.1. Social infrastructure area: approx. 30% of the total number of projects financed under KAI 1.1 belong to the field of social infrastructure, amounting to approx. 6% of the total value financed by KAI 1.1. Of these, 47% were concerned with the installation of video surveillance systems (in the case of urban centres, 69% of the total number of projects in the social field implemented by the Urban Centres concerned the installation of surveillance systems). Indeed, such systems have a social purpose, but no more than the rehabilitation of a street or a park, or the construction of a parking lot. From this point of view, the evaluators' opinion is that the surveillance systems are rather part of the urban infrastructure category. Cities have only invested in surveillance systems in the field of social infrastructure, primarily because they did not identify social projects at the time of developing PIDU, or because they focused on financing them from sources other than KAI 1.1.

Area related to business environment infrastructure: except for the growth poles, very little investment has been made in this area, for the following reasons, presented in descending order of importance paid by the beneficiaries: no project proposals in this area were formulated by any of the partners at the time of the elaboration of the integrated plans, the level of co-financing of 50% exceeded the local budgetary possibilities, the needs in this area were not perceived as acute neither by the local administration nor by the local partners, it is not clear what attributions the public administration has in this area, projects were financed on the business environment from sources other than KAI 1.1. Approx. 5% of the total number of projects under KAI 1.1 and approx. 3% of the total value financed under KAI 1.1 belong to this area, including the projects related to the leisure tourism infrastructure. The projects strictly related to the business environment reach 1.4% of the total value financed by KAI 1.1. Most have invested in the business infrastructure. Growth poles, where 13 of the total 18 projects are found nationwide, the growth poles financing investments in the business environment and from sources other than KAI 1.1.

29

Analysis of the gross effect of the intervention The effects of the interventions financed under KAI 1.1 were analysed on two levels: overall effects at the city level, respectively immediate, thematic effects of the implemented projects, at the level of the three funded areas (urban infrastructure, social infrastructure, business environment infrastructure). Effects of the intervention at city level In order to analyse the effects of the intervention at the city level, the evaluation chose a set of statistical indicators that can best reflect the effects of integrated urban development interventions. The main source for choosing these indicators is the document of the European Union WORKING PAPER Indicators for integrated territorial and urban development (ESPON). The chosen indicators are relevant to indicate the evolution of the city's attractiveness, quality of life and economic development, i.e. the effects pursued by KAI 1.1. An analysis of the evolution of the impact indicators (summarized in the table below), shows that there are both indicators with positive evolution and indicators with negative evolution, mainly reflecting the macroeconomic and social situation at national level (external migration, population aging, lower unemployment rate). The evolution of the values of the impact indicators at the level of the different types of beneficiary cities, in the period 2007-2017, calculated as the difference between the value after the intervention (the average of the years 2016 and 2017) and the value before the

30

intervention (the average of the years 2007-2011). The rate represents the value of the initial indicator relative to 1,000 inhabitants

Impact at city level TEMPO NSI table

Urban Centres

Urban Development Poles

Growth Poles

National level

Dimension related to the natural movement of the population

Rate for the number of deceased persons POP206D 0.76 1.06 0.16 0.37

Rate of live births POP201D -0.54 -0.32 0.19 -0.28

Dimension related to the migratory movement of the population

Rate related to the number of immigrants POP310E 1.90 2.51 1.1 1.30

Rate of number of emigrants POP309E 0.72 0.64 0.87 0.55

Dimension related to the general evolution of the population

Modification rate of the domiciled population POP108D -17.37 -26.54 1.91 -12.75

Economic dimension

Rate related to the number of unemployed SOM101E -10.53 -9.88 -9.80 -6.86

Rate related to the average number of the unemployed persons

FOM104D 3.16 17.66 6.63 10.18

Educational dimension

Rate related to school population (primary level)

SCL103D 4.77 10.59 12.81 4.82

Rate related to school population (secondary level)

SCL103D -5.69 -1.62 -0.87 -5.17

Rate related to school population (high school level)

SCL103D -12.80 -16.46 -13.55 -8.05

Dimension related to city attractiveness

Rate of modification of the number of houses LOC101B 29.50 29.8 32.05 28.51

Source: data processed by the evaluation team based on the data extracted from the National Institute of Statistics Positive evolution; Negative evolution

Dimension related to the ”natural movement of the population”. The rate of death toll is increasing, reflecting an aging process. The rate of live births is declining due to decreased fertility and aging population. Other causes are the decline of the population caused by the emigration flow, due to the standard of living (limited income) that does not stimulate a family's desire to have more children. The exception is the growth poles, large cities that are more attractive options than other cities and which continue to attract immigrants from other small cities, rural areas or less developed areas, an aspect illustrated by both the increase in the number of new-borns and by the increase of the population domiciled, respectively the increase of the number of houses. Dimension related to the ”migratory movement of the population”. The rate of the number of immigrants is increasing for all types of cities. However, it is important to note that the indicator also does not capture internal immigration (at the country level). Given that cities continue to attract immigrants from rural areas, it may be possible to measure the attractiveness of certain areas by measuring this phenomenon. The fact that although the number of immigrants is increasing has not led to the increase of the rate of new-borns can be explained by the small number, in absolute values of immigrants or other factors that influence the birth rate. The rate of the number of emigrants is also increasing, due to the external emigration (in other

31

countries) which is one of the important socio-economic problems throughout the country. Dimension related to the ”general evolution of the population”. The decrease of the domiciled population is due both to the natural movement of the population (increase of the rate of deceased persons, decrease of the rate of live births) and to the flow of emigrants. The influx of immigrants could offset these factors, but many of the immigrants (the domestic ones) do not change their domicile in acts only after a certain period, which means that although the resident population may be increasing due to immigrants, the domiciled population does not reflect the same situation. The exception is the growth poles that register an increase in the population of the population, in particular due to a lower rate of deceased persons and due to the increasing rate of the number of new-borns, in combination with the flow of immigrants. Thus it becomes obvious that large urban agglomerations increase their attractiveness due to economic opportunities, and due to the existence of more diversified and probably higher quality services. Economic dimension. The increase of the level of economic activity, at national level, can be described as a macro trend that is due to the end of the crisis period and is evident both by the decrease of the unemployment rate and by the increase of the average number of employees. The rate of the number of unemployed is decreasing, and the rate of the average number of employees is increasing, in line with the national trend of decreasing the unemployment rate. Educational dimension. The rate of primary school population is increasing, although the number of new-borns is decreasing, however the rates of the school population at secondary and high school levels are decreasing. The increase in the number of new-borns will be reflected in the future growth of the school population as long as families do not decide to migrate to the socio-economic opportunities offered by the western European states. The high school level has the highest decrease, due to the tendency of the students not to continue the studies at this level. Dimension related to ”attractiveness of the city”. The number of houses is increasing, indicating both an increase of investments in the real estate field by the construction of housing complexes, this tendency being accentuated by the exit from the crisis and the reinvigoration of the flow of bank loans for the population. The existence of both positive and negative evolutions of the impact indicators at the city level indicates that the investments made (both from KAI1.1 and from other sources) had effects on some dimensions (for example, the economic dimension, which records an evolution strongly positive, or the natural migratory movement that indicates an increase in the attractiveness of cities) but less so on others (for example, the natural movement of the population, or the domiciled population). This is explained by the fact that in some dimensions the impact is observable faster, whereas in others the effect requires a longer period to occur. Effects of the intervention at the level of the metropolitan areas The integrated plans of all the Growth Poles include the strategic direction of development of the entire metropolitan area, but the number of projects actually implemented under KAI 1.1 in other localities of the metropolitan area differs between

32

the Growth Poles. Most of these are identified in the metropolitan areas belonging to the Growth Poles Constanta (South-East), Brașov (Center), Ploiești (South-Muntenia) and the least in the metropolitan areas belonging to the Growth Poles Iasi (North-East), Cluj (North-West), Timisoara (West), Craiova (South-West). The growth poles managed to attract many other sources of financing besides KAI 1.1, the most important being other axes of the ROP 2007-2013 and other operational programs, but there were also cases (for example, Ploiesti) in which they were attracted. private sources for financing some business structures. The Constanta growth pole, for example, financed the 2007-2013 integrated plan of the metropolitan area with a total amount of over EUR 1 billion. Although it is difficult to make a distinction between the effects obtained at the local level and those at the metropolitan area level, as all the localities are part of a whole, the projects implemented by the Growth Poles have nevertheless largely addressed the needs of the local communities (for example, urban and social infrastructure of the localities in the metropolitan area) and to a small extent the needs of the whole metropolitan area (for example, infrastructure projects of interest for the whole metropolitan area). From this point of view, the projects implemented in the metropolitan areas have contributed to a small extent to obtaining effects at the metropolitan level. However, there are also projects (for example, social centres for people with disabilities, or rehabilitation of county roads) that serve several localities, so in such cases the benefits obtained through the implementation of the project are felt at the metropolitan level.. Effects of the intervention at the level of the financed projects In order to analyse the effects of the intervention at the level of the financed projects, the evaluators identified a set of effects of the nature of the impact that reflects the logic of the intervention on KAI 1.1. These effects come from the programming documents, to which the evaluators' proposals are added and are presented below. The effects thus defined were analysed on the basis of sustainability reports and of the information collected through qualitative methods (case studies, focus groups, interviews, etc.). The effects reported in the sustainability reports for each of the projects are quantified by a variety of indicators and a variety of units of measurement, each city choosing which indicators it considered the most appropriate and which is the most relevant way to measure it. For this reason, the aggregation of the obtained values was not possible. Consequently, the evaluation team elaborated an analysis based on the information extracted from the sustainability reports of the 86 projects and on the qualitative information collected through the case studies and focus groups at the regional level, information quantified by the survey applied to the entire population of beneficiaries (the 90 beneficiary cities). The only indicators that can be aggregated at national level for the entire portfolio of projects under KAI 1.1 are the program indicators, their values being recorded by ROP monitoring system. However, the evaluators found that the values reported and aggregated in ROP system are strongly distorted for these indicators as well. For example, in the case of a particular urban infrastructure project, temporary jobs (during project execution) were taken over in the ROP monitoring system as created jobs, inducing the idea that new jobs were created. Another example concerns the maintenance of jobs: another urban infrastructure project reports the existing jobs during the project (within the contractor team) as jobs maintained, inducing the idea that these jobs were maintained as a result of the project for an indefinite duration.

33

Urban infrastructure Effects targeted by the intervention on KAI 1.1

Quality analysis of the effects obtained by the intervention on KAI 1.1

Examples of achieved values of the indicators presented in the sustainability reports subject to analysis

The emergence of new socio-economic activities (as a result of the rehabilitation of abandoned buildings or contaminated or unused sites)

Only one of the analysed projects could be included in this category: the multifunctional centre in Pitesti. The rehabilitated multifunctional centre, a former downstairs cinema as an activity, now hosts a series of activities: film screening, symphonic and other concerts, various shows, plays, seminars, within the centre being created 7 new permanent jobs.

A number of 5 projects at national level could be included in this category, for example: realization of a multifunctional centre in Pitești, rehabilitation of the summer theatre in the Romanescu park in Craiova, Rehabilitation of the Casa Armatei building and its preparation for the Lipova Recreational Center. There are also social centres that have been built in abandoned buildings (ex: former thermal power stations) but they have been classified in the social infrastructure category.

Increasing number of visitors to heritage objectives

The analysed projects report increases in the number of visitors between 684 and 23,800 visitors per year.

Ex: Increasing the number of visitors from 37591 to 38275 (+684) as a result of the project Restoration and consolidation of the Nicolae Simache Clock Museum in Ploiești, from zero to 23,800 visitors / year in the case of the project that aimed at establishing the Iași municipal museum

Increasing or maintaining transport capacity

A percentage of 30% of the analysed projects indicates an increase of the transport capacity with percentage values between 5-80%.

For e.g.: growth of the number of travels by 20% due to the project Rehabilitation, modernization and extension of public transport infrastructure with the creation of an intermodal terminal in Vaslui, increase no. people using public transport from 20% to 35% due to the project Increasing the quality of the road infrastructure in Curtea de Argeș municipality.

Reducing traffic congestion and increasing traffic safety

A percentage of 78% of the analysed projects indicates a reduction of traffic congestion, as follows: with percentage values between 15-140% or by qualitative assessment (yes, there has been a fluidization of traffic). A percentage of 14% of the analysed projects indicates an increase in traffic safety, as follows: with a percentage value of 20% or by qualitative assessment (yes, there was an increase in traffic safety / a decrease in the number of accidents).

Ex: reduction of travel time by car by 30% in Mioveni, due to the project of modernizing the IC Bratianu street by creating two additional lanes, increased speed from 32 to 41 km / h in Botosani, due to the project Rehabilitation and modernization of the main ring roads, a 15% reduction in travel time between localities in the Constanța metropolitan area, due to the DJ Modernization project 228A between Ovidiu-Poarta Albă.

Development of alternative forms of transport (bicycle)

A percentage of 25% of the analysed projects indicates an increase of the users of the bicycle as a form of transport, as follows: with percentage values between 15-140%, or by qualitative assessment (yes,

Ex: increase of bicycle users from 20% to 32% in Curtea de Argeș, due to the project Increasing the quality of road infrastructure in Curtea de Argeș municipality, increase of bicycle users by 10% in Suceava, due to the project Rehabilitation of the central area by

34

there has been a fluidization of the traffic). A percentage of 14% of the analysed projects indicates an increase in traffic safety, as follows: with percentage values between 10-50% or by qualitative assessment (yes, there was an increase in the number of bicycle users).

creating underground parking lots, pedestrian rehabilitation and streets.

Increasing the access of citizens to information of public interest

Only 4 projects were financed at national level aimed at this objective, and the portfolio of analysed projects does not include such projects.

Only 4 projects were financed at national level aimed at this objective, and the portfolio of analysed projects does not include such projects.

Creating or maintaining jobs Number of jobs created at national level: 3,257

Number of jobs maintained at national level: 2,879

Social infrastructure

Effects targeted by the intervention on KAI 1.1



Development of new services None of the analysed projects report the development of new services at the level of the supported institutions. However, at the level of the entire KAI 1.1, a number of financed projects have created previously non-existent services (e.g. the youth centres in the metropolitan area of Constanța, the community centre "Grandparents and grandchildren" in Brăila, the day centre for the elderly in Negrești Oaș), all these services being actually new. The case studies conducted as part of the evaluation also indicate the existence of various modalities of development of the services offered, in particular through the new equipment procured within the projects (for example, within the medical offices within the homes for the elderly).

Ex: increase in the no. of beneficiaries served by the Craiova elderly home, from 190 to 320, due to the rehabilitation of the centre and the increase of the possibility to use previously unusable accommodation spaces.

Increasing the number of beneficiaries served

A percentage of 20% of the analysed projects indicates a growth in the number of beneficiaries, with values between 28 and 600 beneficiaries. In fact, a number of financed projects have created previously non-existent services (e.g. Youth Centres in the metropolitan area of Constanța), all beneficiaries of such locations being actually newly-served beneficiaries.

Ex: The 100 beneficiaries of the home for the elderly in Alba Iulia, the 121 beneficiaries of the Center for the recovery and rehabilitation of persons with disabilities Tuicani in Moreni, the 104 beneficiaries of the Emergency Reception Center "Cireșarii" in Ploiești.

Increasing the quality of the All the financed projects have enhanced the quality of the services, Ex: 20% crime reduction, increased security

35

services offered due to the fact that the upgrades and rehabilitations of the beneficiary institutions created conditions much improved for the beneficiaries of the services provided. 85% of the analysed projects indicate the total number of beneficiaries, with values between 30 and 320 persons. Some projects indicate the no. of potential beneficiaries (for example, cultural homes, where the population of the locality is indicated, for example 4,305 beneficiaries, in Breasta commune in the Craiova metropolitan area).

near schools by 20%, in Alexandria.

Increasing security (reducing crime)

All the analysed projects (which had as object the installation of video surveillance systems) indicate the decrease of the number of crimes, 30% of them indicate decreasing percentages between 7% -40%, the rest of 70% offer qualitative evaluation regarding the results (ex: the decrease of the number of offenses is affirmed or the decrease with a high value of the number of offenses is appreciated).

Ex: growth no. of beneficiaries served by the Craiova elderly home, from 190 to 320, due to the rehabilitation of the centre and a better the possibility to use previously unusable accommodation spaces.

Creating or maintaining jobs No. of jobs created at national level: 1,234

No. of jobs maintained at national level: 1,254

Business environment infrastructure Effects targeted by the intervention on KAI 1.1



Development of the entrepreneurial environment Boosting economic activity Launch of new economic activities

75% of the analysed projects report no. of companies attracted to the financed business structures, with values between 1 and 25. In total, 69 companies were attracted at the national level in the newly created business structures through the funded projects. 40% of the analysed projects report the employment rate of the financed business structures, with values between 30% -100%.%.

For e.g.: Lumina Verde Ploiești multifunctional business centre, with 11 companies attracted 92% employment rate, Iași Regional Technology Center, with 1 attracted company (large company, with 270 operational jobs in the technological centre).

Creating or maintaining jobs All the analysed projects report no. of jobs created or maintained in the financed business structures, with values between 7-270 (created jobs), respectively between 10-108

For e.g.: Business Center South Craiova with 34 jobs created, Center Suceava for the support of the traditions of Bucovina, with 7 jobs created

36

(maintained jobs). In total, 803 jobs were created at national level and 938 jobs were maintained through the financed projects.

and 10 maintained, Multifunctional Business Center Lumina Verde Ploiesti, with 18 jobs created and 108 jobs maintained.

37

The qualitative analysis of the effects obtained through the interventions under KAI 1.1, at the level of the financed projects, indicates the following:

Urban infrastructure: substantial improvement of the aspect of the beneficiary localities (ex: by rehabilitation of streets, sidewalks, historic centres), stimulation of economic (ex: commercial and service activities) and social activities in the rehabilitated areas, a classic example being the historical centres (where the activity of restaurants or cafes have been developed) and some roads have been refurbished, improving connectivity between areas of cities or with economic zones, increasing the comfort level of citizens (e.g. by building parking lots, rehabilitating lighting systems, rehabilitating or building parks and green areas, replacement of street furniture, improvement of access for people with disabilities), fluidization and increase of traffic safety (ex: by creating road passages, pedestrian crossings, widening of arteries, installing traffic management systems), decongesting the central areas, increasing or maintaining head transport activity (e.g. by widening the roads or rehabilitating the road), increasing the number of visitors or revitalizing the cultural life to the rehabilitated heritage objectives, increasing the use of the bicycle as a form of transport (by creating bike paths), creating or maintaining places for work in the operating phase of some of the investments (3,257 newly created jobs, 2,879 jobs maintained).

Examples from various cities regarding the effects of interventions under KAI 1.1 in the field of urban infrastructure are presented below, other examples being provided in case studies (Annex C Case studies). • Rehabilitation of a previously unused historical monument building in Zalău

(Transylvania building), which now hosts various events (theatre performances, concerts, conferences);

• Increasing the attractiveness of the northern area rehabilitated from the point of view of the road infrastructure, both as a residential location and for the business environment, in the city of Marghita;

• In Baia Sprie, the rehabilitation of 3 residential areas neighbourhoods, former blocks of miners' neighbourhoods: rehabilitation of road infrastructure, creation of green spaces, playground, diversion of the flow of traffic outside these neighbourhoods, leading to the increase of the attractiveness of these areas, demonstrated by the price increase housing;

• Intelligent traffic management systems, in Zalău, Galați; • Revitalization of the historical area (the lower part of the city) in Sibiu, by

rehabilitating the road infrastructure; • In Brașov, the development of an entire area of the Coresi mall (development as a

commercial area, location of company headquarters, construction of residential blocks, facilitated the access of other localities to this area) by rehabilitating the respective access artery, Coresi road;

• Revitalization and enhancement of the historical centre in Piatra Neamț through road infrastructure works, creating a parking lot, contributing to the development of the social, cultural and economic role of this area;

• Increasing the comfort of the citizens and reducing the crime rate by installing a new public lighting system and a video surveillance system in Alexandria;

• Reduction of maintenance and repair costs by rehabilitating some tram lines in Ploiești;

38

• Development of the ring zone of Târgu Jiu by rehabilitating the urban infrastructure, contributing to the increased connectivity of this area, increasing the level of economic activity in the area;

• Restoration and consolidation of the historical building of the Maria Filotti theatre in Brăila, leading to an increase in the number of visitors, an increase in the number of cultural events, the organization of courses on various topics of interest to children;

• Construction of a new parking lot on the Galati ring road, as part of a larger project to modernize the ring road, creating a connection with the port of the New Galati Basin, thus contributing to the connection with the intermodal centre that will be developed in the harbour;

• Revitalization of Adjud Culture House area through street rehabilitation (market, parking lot) creating a pedestrian area and increasing the number of socio-cultural events in the rehabilitated area;

• Installation of intelligent system of public lighting in Adjud, with light sensors and lamps with high energy efficiency.

Social infrastructure: 75 institutions providing social services were supported, with the following results: increase of the quality of the services provided (ex: the rehabilitation of the infrastructure of the social centres led to much improved conditions for the beneficiaries), the construction of new locations and the development of new services (e.g.: day, youth centres, specialized centres) for various target groups (young people, children, elderly, disadvantaged people, people with disabilities, etc.), increasing the number of beneficiaries (e.g. by rehabilitating previously unusable spaces and thus increasing capacity); through the 73 projects aimed at the installation of surveillance systems in 73 cities, the increase of the safety degree of the population was obtained by reducing the number of crimes, identifying the perpetrators of crimes, contributing to the reduction of traffic congestion. Another effect of the projects in the field of social infrastructure refers to the creation or maintenance of some jobs during the operating phase of some of the investments made (1,234 newly created jobs, 1,154 jobs maintained).

Examples from different cities regarding the effects of KAI 1.1 interventions in the field of social infrastructure are presented below, other examples being presented in the case studies (Annex C Case studies). • The rehabilitation of an unused building and the launching of new social and cultural

activities, within the new community centre "Grandparents and grandchildren" in Braila, having as beneficiaries elderly persons and children, these meeting to create a bridge between generations;

• Creation of a new headquarters for the community centre for the elderly in Adjud providing social, medical, cultural services, by rehabilitating the building of a former thermal power station, a centre that can now serve 300 beneficiaries compared to 25 previously;

• Rehabilitation of the social services centre for young people in difficulty in Târgu Lăpuș: rehabilitation of the building infrastructure, extension of the capacity through the attic of the bridge to expand the accommodation capacity, development of new services for children with autism;

• Rehabilitation of a building (multifunctional centre of social services) to be used as a headquarters for NGOs from the social field in Brasov, now fully equipped, 100% occupied, creating a new public transport line to this location, contributing to the increase in the number of activities economic on the route;

39

• Creation of a new day centre for the elderly in Negrești Oaș, which offers facilities for spending time and meals at home, by rehabilitating a former thermal power station;

• Contribution of video surveillance system to the investigation of road accidents, in Zalău;

• Decreased crime rate in schools due to video surveillance systems in Marghita; • Rehabilitation, modernization, capacity building of the centre for persons with

disabilities in Brașov, setting up a mobile team for home visits, thus supporting the identification of cases that were likely to remain unknown;

• Increasing the comfort of the citizens and reducing the crime rate by installing a new public lighting system and a video surveillance system in Alexandria.

Business environment infrastructure: In the case of the few projects that targeted this field (18 projects completed out of 505 projects completed nationally under KAI 1.1), the main effects were: attracting companies in the newly created structures, creating or maintaining jobs within the attracted companies. In general, the financed projects aimed at creating clusters of companies by attracting in the financed infrastructures of companies from certain fields or attracting new companies that need support at the initial stage, such desires being partially achieved (according to the quantitative analysis presented above).

Examples from different cities regarding the effects of KAI 1.1 interventions in the field of business environment infrastructure: • Creation of a business centre for SMEs newly established in Negrești Oaș, which

offers office spaces at discounted prices for a maximum of 6 SMEs; • Hosting 13 SMEs in a new headquarters in Ploiești, in the multifunctional business

centre Lumina Verde, under the conditions in which the supply of office spaces at the city level is deficient;

• Creating a set of spaces for business environment in Craiova, in the southern area of the city, which combines exhibition spaces with office spaces designed to stimulate clusters in the IT and communications fields, respectively business support services, spaces that attracted 22 profile firms.

Overall, the projects financed under KAI 1.1 mainly contributed to the increase of the quality of life of the inhabitants, especially through investments in urban and social infrastructure. Investments in urban infrastructure have a potential effect also in the sense of increasing the attractiveness of the city in general, for both emigrants and potential investors (either local or from other localities) in this sense, investments in urban infrastructure also contribute to economic development. However, although it is a necessary condition, the development of urban infrastructure is not sufficient for the development of the business environment. The direct investments in the business environment made through KAI 1.1 had limited effects due to the very small number of projects and the difficulty of the city setting some objectives in this field. The approach adopted by the cities that have invested in the business environment reflects a stage in which the most effective solutions are still being sought. Compared to the urban or social infrastructure where the needs were clearly outlined and the solutions could be more easily identified, in the case of the infrastructure of the business environment neither the need for support was as clear nor the solutions. The main reasons indicated by the local authorities for the low level of investments in the infrastructure of the business environment are:

40

No projects were proposed in this area at the time when the integrated plans have been elaborated;

The co-financing required exceeded the local budgetary possibilities; Projects in the field were financed from other sources; The responsibilities of the public administration regarding the development of the

business environment are not clear.

Source: Survey applied at the level of local authorities in the 90 beneficiary cities (Annex D Instruments collected)

Other effects identified by the quality analysis (desk research, case studies, focus groups) and quantified in the opinion poll applied to the 90 beneficiary cities:

The local management capacity of integrated development plans has improved, also at the level of local stakeholders (local partners). The partnership process was focused on the elaboration stage of integrated plans, within this stage the level of involvement of local partners was high. The involvement of local partners was reduced in the implementation phase of PIDU, being limited to ad-hoc situations that required consultations or information regarding the implementation of certain projects, especially with local public institutions for projects that required such collaboration, such as street rehabilitation projects and water-canal systems. With a few exceptions at the level of growth poles, the involvement of local partners in the implementation phase did not formally materialize, as members of the PIDU

Graphic 2 Perceptions of beneficiaries regarding the effects of the interventions of KAI 1.1

41

implementation coordination committees. This implies a smaller opportunity for capacity development for local partners, as they formally participate only in one part of the urban development process, that is, the planning process. In other words, permanent governance models that include local partners at all stages of the urban development process have not yet been developed. The charts presented below illustrate both the extent to which the management capacity has been developed, as well as a number of aspects of urban development planning that have developed as a result of the experience of developing PIDU within DMI 1.1. Of the aspects illustrated in the graph, the most evolved compared to the previous plans the structured approach, the prioritization and the selection of the funded projects, and the least the involvement of the local partners.

Source: Survey applied at the level of local authorities in the 90 beneficiary cities

Specialized literature in the field of urban development clearly indicates the need for appropriate coordination mechanisms for the success of integrated interventions: cross-cutting (cross-sectoral policies); vertical (involving the various levels of the administration - e.g.: city, county, region); horizontal (involving local partners). The desk research, the case studies and the information collected during the focus groups show that at the level of the beneficiary cities a mechanism of coordination between sectoral policies is developing, that is a mechanism that ensures the correlation of the objectives and investments related to different sectors (e.g.: roads, utilities, social services, economic, education, health, etc.) with each other and with the strategic objectives of the integrated plans.

Graphic 3 Perceptions of the beneficiaries regarding the development of local capacity for urban development planning

Graphic 4 Perceptions of beneficiaries regarding the development of local coordination mechanisms

42

Source: Survey applied at the level of local authorities in the 90 beneficiary cities

Basically, the entire process of involving local partners in the development of PIDU is the basis of the inter-sectoral coordination mechanism. In the situations when working groups have been used to develop integrated plans, especially for growth poles, this mechanism has been better supported. Similarly, a mechanism of collaboration between the local authorities and the authorities at the county level (the county council) in the process of planning and implementing urban development is ongoing. In the case of growth poles, the involvement of the county councils in the various stages of the process of preparation and implementation of the integrated plans was at a much higher level than in the other cities, especially due to the nature of the metropolitan approach that involves several administrative-territorial units. The graph presented below illustrates these important aspects for the quality of the planning and implementation process of urban development plans. An important aspect related to the local capacity to implement the concept of integrated urban development is the typology of investments, the complementary, synergistic balanced approach of the multitude of fields of urban development. The fact that the investments in this field were directed by the applicant's guides to physical infrastructure constituted a factor limiting the type of investments made. Investments in infrastructure is not the only way to stimulate the business environment and certainly the beneficiary cities have a wide range of specific situations that require specific solutions chosen from a wider palette, not just infrastructure investments. The box presented below includes examples of the role of the administration in the field of urban economic development and the types of actions that the administration can take.

43

Examples regarding the role of the administration in the field of urban economic development

Analysis of the net effect of the intervention, at the level of the financed urban centres In order to determine the extent to which the obtained change can be attributed to the intervention (the net effect of the intervention), two methods were used in tandem: Counterfactual evaluation and Theory-based evaluation (using the Theory of change)2). The counterfactual analysis performed in this evaluation compared the urban centres that benefited from financing under KAI 1.1 with the urban centres that did not benefit from the financing under KAI 1.1 The analysis was based on the set of impact indicators at the city level presented previously and grouped by dimensions and applied 2 working scenarios, both leading to the same results. Scenario 1 compared all 70 urban centres that benefited from financing with a number of 114 other urban centres (all eligible urban centres in Romania that did not receive funding) by Difference in Difference (DID) methods. Scenario 2 compared 54 urban centres that received funding with a number of 104 other urban centres that did not receive financing, using the Propensity Score Matching method (similar cities were compared, thus selected according to a series of matching indicators) and the Difference in Difference (DID) method. Both scenarios applied the Difference in Differences (DID) method, taking into account two variants of values recorded after the intervention: at the level of 2017, respectively an average of the years 2016-2017. Each of these variants has advantages: taking into

2 Ghidul Evalsed (2013), Carol Weiss (1995)

The role of municipalities at the level of the regions can be classified into two broad categories of actions aimed at promoting employment. The first category is the promotion and exploitation of the basic factors of production, such as natural resources, infrastructure, labour force, etc. These actions consider the location as a source of "comparative advantage" for companies, compared to other locations. The second category of actions involves the development of a support system that supports a friendly and functional business environment, the emphasis being placed on the location as a source of "competitive advantage" for companies. In this case, the local administration can invest in a business support system, the development of clusters and partnerships, often based on the so-called quadruple spiral (administration, academia, business environment, civil society). As a result, local authorities have a crucial role to play in improving the conditions for creating jobs, as they can act to facilitate a favourable business ecosystem, promote the modernization of the local economy, train the workforce, promote apprenticeship programs, support entrepreneurship. , providing adequate infrastructure and mobility, providing quality public services, controlling urban development and land use, reducing time and procedures for obtaining building permits, finding ways to stimulate the creation of local jobs, promoting financial instruments and so on It is also important to be aware of the need to involve the different levels of the administration in this process, as the local authorities do not act in isolation.

EU Urban Agenda Partnership JOBS AND COMPETENCIES IN THE C0NTEXT OF THE LOCAL ECONOMY, ACTION PLAN

24 July 2018

44

account the values of 2017 has the advantage of allowing more time for the effects to manifest. Taking into account the average of 2016-2017 values has the advantage of reducing certain peaks or momentum (fluctuations). Counterfactual analysis could not be applied in the case of Growth Poles and Urban Development Poles, as these are unique cases, with no comparable control group. The results of the counterfactual analysis, scenario 1, values expressed as rates per 1,000 inhabitants

Variable (at UC level)) TEMPO table

Sample DID (16/17) – (07/11)

DID 2017 – (07/11)

Rate related to the number of deceased persons

POP206D 70/114 0,0362 -0,4870**

Rate related to the number of live births

POP201D 70/114 0,0489* 0,1520**

Rate related to the number of immigrants

POP310E 70/114 1,344** 1,938**

Rate related to the number of emigrants

POP309E 70/114 0,0717 0,1188

Rate related to the modification of the domiciled population

POP108D 70/114 10,4264** 11,9128**

Rate related to the number of unemployed

SOM101E 70/114 0,9071 0,9054

Rate related to the average number of employees

FOM104D 70/114 16,83** 17,33**

Rate of school population (primary level)

SCL103D 70/114 -0,1307 -0,2644

Rate of school population (secondary level)

SCL103D 70/114 0,1299 0,1532

Rate of school population (high school level)

SCL103D 70/114 -1,5893 -1,6998

Rate related to the modification of the number of houses

LOC101B 70/114 -2,22 -2,36

** Difference of over 10% between treatment change and control change (in the desired direction), * 5% to 10% difference between treatment change and control change (in the desired direction) DID = The difference of the differences, indicates the difference between the evolution of the group of urban centres that received financing and the evolution of a control group of urban centres that did not receive financing, for each of the indicators subject to analysis, in the period 2007-2017 16/17 - The value recorded after the intervention was calculated as an average of the years 2016-2017 07/11 - The value recorded before the intervention was calculated as an average of the years 2007-2011, considering that in 2007-2011 there was still no effect of the intervention.

The results of the counterfactual analysis, scenario 2, values expressed as rates per 1,000 inhabitants

Variable (at UC level) TEMPO table

Sample DID (16/17) – (07/11)

DID 2017 – (07/11)

Rate related to the number of deceased persons

POP206D 54/104 0,0016 -0,6719**

Rate related to the number of live births

POP201D 54/104 0,0250* 0,1363

Rate related to the number of immigrants

POP310E 54/104 0,6772** 1,2973**

45

Rate related to the number of emigrants

POP309E 54/104 0,0493 0,0764

Rate related to the modification of the domiciled population

POP108D 54/104 3,0209** 3,3899**

Rate related to the number of unemployed

SOM101E 54/104 0,2462 0,2033

Rate related to the average number of employees

FOM104D 54/104 14,3055** 14,4399**

Rate of school population (primary level)

SCL103D 54/104 -0,9696 -1,1696

Rate of school population (secondary level)

SCL103D 54/104 -0,1107 -0,2126

Rate of school population (high school level)

SCL103D 54/104 -0,7876 -0,8332

Rate related to the modification of the number of houses

LOC101B 54/104

-0,6200 -0,4874

** Difference of over 10% between treatment change and control change (in the desired direction), * 5% to 10% difference between treatment change and control change (in the desired direction) DID = The difference of the differences, indicates the difference between the evolution of the group of urban centres that received financing and the evolution of a control group of urban centres that did not receive financing, for each of the indicators subject to analysis, in the period 2007-2017 16/17 - The value recorded after the intervention was calculated as an average of the years 2016-2017 07/11 - The value recorded before the intervention was calculated as an average of the years 2007-2011, considering that in 2007-2011 there was still no effect of the intervention.

Net effects of the interventions under KAI 1.1 in the case of the urban centres

Dimension Effects obtained through the intervention on KAI 1.1, in the period 2007-2017

(urban centres that received funding compared to urban centres that did not receive financing)

Natural movement of the population (difference between the births and the deceased)

By decreasing the rate of the number of deaths from the rate of the number of live births, the rate of natural movement of the population results. The urban centres that have benefited from financing have a higher evolution than those that have not benefited from financing. There is an impact of KAI 1.1 on this dimension.

Migratory movement of the population (difference between immigrants and emigrants)

By decreasing the rate of the number of immigrants from the rate of the number of emigrants, the rate of the migratory movement of the population results. The urban centres that have benefited from financing have a higher evolution than those that have not benefited from financing. There is an impact of KAI 1.1 on this dimension.

General evolution of the domiciled population

The rate of change of the domiciled population shows a lower decrease in the case of the urban centres that have benefited from financing. There is an impact of KAI 1.1 on this dimension.

Economic dimension (level of employment expressed as difference between the employees and the unemployed)

The increase of the employment rate is higher in the case of urban centres that have benefited from financing. There is an impact of KAI 1.1 on this dimension.

46

Educational dimension (evolution of school population)

The drop in the school population rate is higher for urban centres that have received funding. There is no impact of KAI 1.1 on this dimension.

Dimension related to the attractiveness of the city (evolution of the number of houses)

The rate of change in the number of dwellings shows a lower increase in the case of urban centres that have received funding. There is no impact of KAI 1.1 on this dimension.

We analyse and highlight further on the net effect at the city level, by correlating the gross effects measured from the perspective of the macroeconomic indicators with the effects at project level. The fact that the counterfactual analysis indicates the existence of an impact on 4 of the 6 analysed dimensions represents a positive aspect. The calculation of an aggregate indicator of the impact indicators, which compares the assisted urban centres with the unassisted ones, also indicates the existence of an impact of the KAI 1.1 intervention (for details, see Appendix B Instruments applied, section Counterfactual Analysis). The lack of an impact on the dimensions of city and educational attractiveness, given that there is an impact on the migratory movement of the population, can be explained in various ways, for example: immigrants have not yet been able to build a home or establish a family with children at school age or do not want to stay in the city for the long term, which means that they do not invest in building a home and possibly do not set up a family in that city. At the same time, the lack of an impact on 2 of the 6 dimensions analysed indicates the need for continuous and higher investments in volume in order to be able to achieve an impact on all levels, these dimensions being some where the manifestation of the effect takes more time. Regarding the population area (natural movement, migration, population growth) the evaluation analyses indicate, based on the information collected through interviews, focus groups, case studies and surveys, that the complementary interventions on urban, social and business infrastructure have represented, by the cumulative effect, a catalyst for urban development, by increasing the quality of life in general, by indirectly supporting economic development through investments in urban infrastructure (e.g., rehabilitation of urban areas or arteries that led to the revitalization of economic activity in the areas those). These aspects contributed to increasing the attractiveness of the cities, a result reflected in the fact that compared to the cities that did not receive financing under KAI 1.1, the tendency of population decrease was lower in the case of urban centres that benefited from financing. In other words, KAI 1.1 had a contribution to mitigating the population decline phenomenon. Regarding the economic aspects, even if the urban centres did not invest directly through the integrated plans for supporting the business environment, the investments made in the urban and social infrastructure contributed, according to the sustainability reports and the information collected through focus groups and case studies, to economic revitalization of some urban areas. For example, the rehabilitation of historical centres or other urban pedestrian areas has had the effect of attracting HORECA-type economic operators (hotels, restaurants, cafes) or developing the activity of economic operators already located in the respective areas, fluidizing the circulation due to the rehabilitation of some arteries. the facilitation of the transport activity for economic purposes, the rehabilitation of some heritage buildings has contributed, in some cases, to the development of the cultural activity of the respective objectives and consequently to the increase of the volume of financial receipts, certain rehabilitated

47

social infrastructure objectives that offer performances to the general public have benefited also by increasing the volume of receipts, the rehabilitated recreational infrastructure contributed to the development of the economic activity of the respective objectives or in the respective areas. All these developments have contributed to the creation of new jobs, both directly as a result of the projects financed by KAI 1.1 (for the operating phase of some of the investments made, new jobs were created) and indirectly by stimulating the economic activity in the rehabilitated areas. Validation of the evaluation assumptions The evaluation assumptions were validated through the quantitative and qualitative analyses, ensuring the triangulation of the data, the verification of the findings and conclusions with the stakeholders. In order to present the conclusions regarding the validation of the assumptions, the following scale was used:

Validated assumption: qualitative arguments or quantitative information were identified indicating that the respective assumption was confirmed by the implementation practice;

Partially validated assumption: qualitative arguments or quantitative information collected indicate a lesser or only certain fulfilment of the respective assumption;

Invalidated assumption: qualitative arguments or quantitative information collected indicate that the assumption has not been confirmed in practice or that this information does not provide an adequate basis to consider that the assumption has been fulfilled.

As it can be seen, more than half of the assumptions have been validated. The hypotheses that have been partially validated refer to: the quality of the log frame within the integrated plans (aspect described above), the involvement of local partners in the elaboration and implementation of the plans, the extent to which the growth poles have succeeded in a metropolitan approach, supporting the investments in infrastructure. through complementary measures, the existence and functioning of systems for monitoring and evaluating the integrated plans. Considering the overall results of the assumptions presented in the theory of change, it is found that, as it was natural for a first programming exercise on the integrated approach, a good part of the assumptions are validated and a part of the assumptions still require development for the future exercises for planning and implementing urban development. In this regard, the information collected indicates significant improvements in the planning of urban development in the period 2014-2020. Overall, the counterfactual analysis, the interrogation of the theory of change and the information collected through the qualitative analysis indicates the existence of a contribution of the intervention financed under KAI 1.1 to the results referred to in the programming document: increasing the quality of life, increasing the attractiveness of cities and creating new jobs at the level beneficiary cities. The extent to which the assumptions of change theory have been validated for KAI interventions 1.1

Assumptions presented in the Theory of Change for KAI Invalidated Partially Validated

48

1.1 validated

Effects obtained

Investments in infrastructure in the 3 areas (urban infrastructure, business environment, social) have contributed to increasing the quality of life, increasing the attractiveness of cities and creating new jobs.

By implementing the projects within the integrated plans, the Growth Poles obtain effects including at the metropolitan level

By restoring urban public spaces, by contributing to increasing urban mobility, by rehabilitating heritage buildings, by developing alternative forms of transport, by improving and developing social services, investments in urban and social infrastructure have contributed in a complementary manner to the increase of the quality of life, of the attractiveness of cities and the creation of new jobs, both directly (through jobs created during the operation phase of the rehabilitated or newly built objectives) and indirectly, by stimulating various economic activities as a beneficial side effect of the rehabilitated urban infrastructure and job creation as a consequence. Numerous examples of the results of these investments and the contribution to improving the quality of life have already been provided previously and are based on the information collected through interviews, focus groups, on-site visits, as well as on the analysis of sustainability reports of the implemented projects. If the investments in the social infrastructure had been more extensive, especially for those cities that only invested in surveillance systems and if the investments to support the economic development of the cities would have been more numerous, the investments made could have contributed to the objective of increasing the quality of life and creating jobs. In the case of the Growth Poles, although it is difficult to distinguish between the effects obtained at the local level and those at the metropolitan area level, since all the localities are part of a whole, the projects implemented by the Growth Poles have nevertheless largely addressed the needs of local communities (e.g. urban and social infrastructure of metropolitan areas) and to a small extent the needs of the entire metropolitan area (e.g. infrastructure projects of interest for the entire metropolitan area). From this point of view, the projects implemented in the metropolitan areas have contributed to a small extent to obtaining effects at the metropolitan level.

Integrated approach towards urban development

Integrated urban development plans include a medium and long term strategic vision

PIDU addresses the different development directions in a balanced way: infrastructure, economic, social, environment

The pyramid structure (vision, strategic objectives, priorities, projects) is adequate: coherent, effective, feasible, balanced

The plans of the Growth Poles are based on the strategy of developing a metropolitan area

All integrated plans analysed in the desk research include formulations of strategic visions on the medium and long term, most of these visions being focused on natural desires for any city: quality of life, city attractiveness, economic development, etc. The development directions and the strategic objectives formulated generally cover the areas of urban, social infrastructure, business environment. At the level of projects, however, with the exception of Growth Poles and isolated cases of Urban Development Poles and Urban Centres, a much greater emphasis is placed on urban infrastructure than on other areas of development (social, economic), leading to an internal imbalance of the integrated plans. All integrated plans of the Growth Poles include as a strategic objective the development of the metropolitan area, however the extent to which the effective implementation of the plans has transposed this strategic objective into practice differs from the growth pole to the growth pole,

49

this aspect being described in the related analyses of the evaluation question no. 2.

Identification of the intervention priorities

The integrated urban development plans have been elaborated in collaboration with the various stakeholders at the local level

The financed projects are formulated on the basis of real needs at local level, not based on the existence of the financing source.

The financed projects have a high priority for the city

All the information collected through interviews, focus groups and case studies indicates that the local partners have been to a large extent involved in the elaboration of integrated plans. The modalities of involvement varied according to the type of partners and from one city to another, the usual methods being the public debate (where a number of local partners were involved - business environment, NGOs, academia, sometimes representatives of citizens), working groups (which usually involved decentralized local institutions, various other local public services), public consultation (addressed to citizens). Compared with previous experiences in the field, the level of consultation of local partners in the process of preparing integrated plans was higher, according to the information collected in the focus groups and through a survey. The involvement of the local partners in the elaboration of the integrated plans, by the mentioned methods, has contributed to the formulation of projects relevant to the needs of the cities and with a high degree of priority. Even though the applicant's guidelines indicated in detail the types of eligible investments, thus stimulating alignment to what could be financed rather than local needs, integrated plans identified real priorities, and if these priorities were not identified, financing applications were not submitted (the example of social infrastructure, more than half of the urban centres chose to invest only in video surveillance systems, not formulating financing requests for other social projects, or the example of the business environment). The only area where the financed projects have been strongly influenced by the existence of the financing source and the provisions of the applicant's guides is that of the supervisory systems, which have been chosen by the cities mainly to satisfy the requirement to formulate at least one project in the social field. Even in this case, once installed, these systems have proven their usefulness, an aspect confirmed without exception during all interviews, focus groups and case studies.

Projects synergy

The financed projects support each other, generating synergistic effects at the city level

In order to finance the projects within the PIDU, the local authorities managed to attract other sources of financing, apart from KAI 1.1, ensuring complementarity and synergies with the interventions of DMI 1.1.

The investments in infrastructure were supported by the beneficiaries through other complementary measures necessary to achieve the effects envisaged (e.g.: training, modifications of working procedures, elaboration of development plans, promotion, creation of new jobs, etc.).

According to the information collected during the on-site visits and interviews and verified within the focus groups, the funded projects related to the urban infrastructure field were, in all cases analysed, complementary projects, part of an integrated thinking that sought to solve acute problems of the cities, under the multiple aspects of these problems. For example, the rehabilitated and modernized arteries were located on the main axes of mobility of the city, adjacent structures such as the parking lots were placed in crowded areas, the rehabilitated heritage objectives were part of an urban ensemble usually located in central areas and having a major contribution to the revitalization of these centres. All the rehabilitated objectives thus contribute complementarily to the increase of the quality of life of the inhabitants. The rehabilitated social infrastructure objectives were also a priority for cities, contributing, like urban

50

infrastructure, to improving the quality of life of the inhabitants. Both urban and social infrastructure are part of a whole, contributing together to increase the quality of life. Regarding the infrastructure for the business environment, only 12 of the 90 beneficiary cities have implemented projects in this area, thus having a more limited contribution than the other 2 areas to urban development. Less than half of the integrated plans also included projects other than those funded by DMI 1.1, therefore, attracting other sources of funding for other projects complementary to those funded was not possible in these cases. In the case of integrated plans that included an extensive portfolio of projects, other sources of funding, mainly consisting of other ROP 2007-2013 axes and other operational programs, were largely accessed. Only in isolated cases was it possible to attract other types of financing sources, such as the private sector, EEA and Norwegian funds, the capital market. Complementary measures to support investments in infrastructure to maximize the effects have been implemented by the beneficiary cities on average. More precisely, the main measure was the creation of new jobs needed in the operating phase of the investments made, another example being the measures of development of socio-cultural activities in the case of rehabilitated heritage objectives. In contrast, other possible actions, such as campaigns to promote the use of the bicycle as a form of transport, campaigns to reduce the crime rate, actions to facilitate and stimulate the business environment were not generally implemented.

Plans and projects implementation management

The beneficiaries have the management capacity necessary for the implementation of the financed projects, and after their completion, the operating and maintenance costs are ensured.

Integrated plans and funded projects have been implemented in collaboration with various stakeholders at the local level

There is, at the city administration level, a functional system for monitoring and evaluating the implementation of urban development plans and financed projects.

In the case of urban development poles and urban centres, the management system for the implementation of integrated plans was based on teams made up of local government personnel,

and this team included, on average, other local partners such as those involved in the process of elaborating integrated plans. . In the case of the Growth Poles, the management structures of the implementation of the integrated plans were complex, including representatives of the UAT from the metropolitan area, the county council, decentralized institutions, other local partners representing NGOs, academia, business environment. For the implementation of the projects, all

the project teams have been set up, and there is a close collaboration with the relevant local

institutions for each project. The costs of operating and maintaining the rehabilitated objectives were taken over by the local administrations, and there is a guarantee period for the executed works, during which the costs of any necessary repairs are borne by the companies providing the works. For certain purposes for

which taxable services are provided (e.g. car parks, theaters, nursing homes) the operating and maintenance costs are partially covered by these taxes, which were generally applied after exiting the ex-post monitoring period. The monitoring system of the implemented projects worked properly, being stimulated by the practice of reporting to the Intermediate Bodies, both through the progress reports and through the sustainability reports. In contrast, the system for monitoring and evaluating integrated plans is in its infancy, and no formal evaluation of the results of the implementation of integrated plans has been encountered. Basically, it has not yet reached the level of elaborating monitoring or evaluation reports of integrated plans that analyse the results obtained compared to the targets in the plan, the factors that influenced the implementation and the results, to propose corrective

51

measures, to formulate lessons learned, etc.

Source: Survey applied at the level of the local authorities in the 90 beneficiary cities

(Annex D Instruments collected)

Factors that generated or influenced the effects obtained The identification and analysis of the factors that generated or influenced the obtained effects was based on the following elements, using both qualitative and quantitative methods: Verification of the assumptions that constitute the theory of change related to KAI

1.1. These assumptions, in themselves, represent the causes of the effects produced;

Analysis of other factors, internal or external to the intervention, which have influenced in a positive or negative manner the effects produced on two levels: as a degree of diffusion at the program level (extent) and as an intensity of the manifestation.

Factors with positive or negative influence on the implementation and effects of KAI intervention 1.1

Positive influence

Graphic 5 Perceptions of the beneficiaries related to the relevance of the projects financed under PIDU

52

MA ROP approach on integrated city development - persistence in promoting this type of approach;

Perseverance and capacity of cities in the whole process: preparation of integrated plans and formulation of projects, implementation of projects, analysis of effects obtained at the level of implemented projects;

Involvement of local stakeholders - a big plus for the quality of the integrated approach and for prioritizing the needs and projects formulated;

Availability and access to other sources of financing by those cities that have formulated other projects within the integrated plans, besides those financed by KAI 1.1;

In some cases, the elaboration and implementation of the integrated plans contributed to the development of the collaboration between the local and the county level of the administration;

The monitoring and evaluation system started to be used, even if at a rather limited level. For example, the evaluation of projects was focused on collecting and reporting the values of the indicators in the sustainability reports.

Negative influence

The urban development approach is still focused on urban infrastructure, and urban spaces, too few projects for business environment;

The effects of the interventions were limited by the fact that under KAI 1.1 only infrastructure investments were financed, as there is no possibility of financing other types of activities as well, for example exchange of experience, studies, capacity development of the beneficiaries;

The slow start of the process of planning the integrated plans in the case of Growth Poles and Urban Development Poles, inherently in the case of any new action with a high level of uncertainty;

Too short time allocated by the Urban Centres for the preparation of the integrated plans, due to the calls with deadline submission;

The quality of the technical projects proved to be in some situations deficient, leading to problems inherent in the execution period. One of the main causes reported is the diminution of the level of technical expertise necessary for the preparation of feasibility studies and quality technical projects, available on the market. This aspect was amplified by the limited technical capacity at the level of the staff within the local authorities, leading to difficulties in the evaluation of the quality of the technical solutions offered by the designers;

The procurement process continues to be a significant difficulty for all local authorities, both from the point of view of the legislation which is difficult to understand and can easily create the possibility of non-conformities leading to the application of financial corrections (most often subject of disputes for the beneficiaries), as well as in terms of the duration of the trial itself;

Lack of synchronisation between the projects financed from different operational programs, leading in certain situations to problems in execution.

Legend Widely spread factors Highly influential factors Widely spread and highly influential factors

The involvement of local partners was indeed an important additional factor for the success of urban development plans, but there was room for improvement. For example, the involvement of local stakeholders mainly took place during the elaboration stage of integrated plans and much less during the management of the

53

implementation of integrated plans (except for growth poles, where the involvement of local partners continued at a high level and during the implementation of the plan integrated), aspect illustrated in the following graph.


(Annex D Collected instruments) Regional differences An important aspect of the integrated approach towards urban development is represented by the structure of the financed project portfolios, an aspect analysed before and at the level of KAI 1.1 as a whole. In this section, the portfolios of projects financed at the level of the 8 regions are analysed comparatively in order to illustrate the way in which the regions approached urban development (which were more or less taken into account) as an illustration of the extent to which a balanced approach to the financed areas and sub-areas has been achieved. Structure of regional project portfolios: percent values financed by sub-areas, out of total value financed by region

Subareas V SV NV C SE NE SM BI

Urban public spaces 65.4% 86.7% 53.0% 64.6% 69.5% 79.1% 68.7% 93.5%

Rehabilitation of buildings or land 0.3% 0.5% 0.0% 3.2% 0.0% 0.0% 0.3% 0.0%

Cultural heritage objectives 0.9% 0.2% 13.3% 14.3% 4.3% 3.9% 0.6% 0.0%

Transport and population mobility 21.0% 7.1% 22.4% 6.5% 5.6% 12.2% 19.7% 0.0%

Development of alternative forms of transport (bicycle lanes)

2.4% 2.1% 0.9% 0.0% 0.0% 0.0% 0.0% 0.0%

Increasing the access of citizens to information of public interest

0.7% 0.0% 0.0% 0.0% 0.2% 0.1% 0.0% 0.7%

Social services infrastructure 3.0% 1.5% 5.5% 3.7% 8.6% 2.3% 2.9% 0.0%

Social houses 0.0% 0.0% 0.9% 0.4% 0.7% 0.1% 0.0% 0.0%

Increasing population safety 2.3% 1.2% 2.8% 3.5% 2.2% 1.3% 1.4% 5.8%

Development of the business environment 2.6% 0.7% 0.4% 2.1% 1.0% 1.0% 3.2% 0.0%

Leisure tourism infrastructure 1.3% 0.0% 0.8% 1.7% 7.9% 0.0% 3.2% 0.0%

Graphic 6 Perceptions of the beneficiaries regarding the involvement of local partners in the development and implementation of PIDU

54

The highest percentages invested in the sub-domain of Urban Public Spaces are identified in the regions of Bucharest Ilfov, South West and North East, and the lowest percentages in the North West, Center and West regions. The regions that have chosen to invest the most in transport infrastructure are North West, West and South Muntenia, and the least Bucharest Ilfov, South East, Center and South West. The highest percentages allocated to investments in social services infrastructure (excluding video surveillance systems) belong to the South-East and North-West regions, and the lowest to the Bucharest-Ilfov and South-West regions. The highest percentages allocated to investments in the business environment are found in the South-Muntenia, West and Center regions, and the lowest in the Bucharest-Ilfov, North-West and South-West regions. As we have previously mentioned, there are differences which are more or less significant in the investment portfolio, with the dominant sub-area of Urban Public Spaces, while the infrastructure of social services (excluding the Sub-area of Increasing population safety) and the infrastructure related to the environment businesses benefiting from much lower percentages of investments. We present below the situation of the program indicators regarding the jobs created or maintained by the financed projects per regions, with the mention made previously that these values are strongly distorted by the way of reporting by the beneficiaries or by the way in which these values are introduced in ROP monitoring system. The structure of the jobs created or maintained by the projects under KAI 1.1, per regions

Total jobs created and maintained

Urban infrastructure Social infrastructure Business environment infrastructure

Region Created Maintained Created Maintained Created Maintained

BI 239 59 153 5 22 - -

C 2063 657 657 114 199 218 218

NE 2808 1102 1194 66 68 16 362

NW 1711 69 12 503 442 405 280

SE 1474 706 254 313 165 30 6

SM 951 271 271 171 162 38 38

SW 505 262 137 19 19 34 34

W 514 131 201 43 77 62 0

Source: data processed during the evaluation obtained from MA ROP

The integrated plans of all the Growth Poles include the strategic direction of development of the entire metropolitan area, but the number of projects actually implemented under KAI 1.1 in other localities of the metropolitan area differs between the Growth Poles. Most are identified in the metropolitan areas belonging to the Growth Poles Constanta (South-East), Brașov (Center), Ploiești (South-Muntenia) and the least in the metropolitan areas belonging to the Growth Poles Iasi (North-East), Cluj ( North-West), Timișoara (West), Craiova (South-West). No notable differences were identified between the regions from the perspective of the type of effects obtained in the 3 financed areas, with the exception of the differences between the more or less metropolitan approaches of the Growth Poles. Graphic 7 Perceptions of beneficiaries regarding the purpose of projects implemented at metropolitan level

55


(Annex D Collected instruments)

Level of absorption KAI 1.1 The financial allocation for KAI 1.1 represented by ERDF funds and state budget (BS) was EUR 1,322,621,336. The total value financed at the level of KAI 1.1 (ERDF + BS) was approx. EUR 967,716,120, which represents an absorption rate of approx. 73%. The values are approximate, as the available financial data do not reflect the payments made during the period 2017-2019 for the projects that continued to be implemented during this period. Due mainly to the savings resulting from the procurement processes and to a small extent to the 25 cancelled projects, significant amounts remained unabsorbed, reducing both the effects achieved at the city level and the degree of absorption of ROP as a whole. Indicators used for KAI 1.1 A high number of immediate and result performance indicators were defined for KAI 1.1, both program and additional indicators, which is explainable if we consider the different typology of the financed investments. The beneficiaries chose the types of indicators that they considered the most relevant and reported these indicators using sometimes the same units of measure (e.g. number of persons benefiting from ......) sometimes different units of measure (ex: percentage increase number of ......, or absolute values increase number of ......., reduction of traffic congestion expressed either by deducting minutes of travel, either in percentages or in ratings etc.). These issues created insurmountable difficulties in the aggregation of these indicators, both by the ROP IB and MA, as well as by the evaluation team. As a result, only a relatively small proportion of the indicators reported by the beneficiaries were aggregated by MA ROP. For the purpose of this evaluation, values were introduced in a

56

database for a sample of projects, mainly in order to illustrate the type of results obtained, to analyse these results from a qualitative perspective and to illustrate the difficulty of aggregating the values of the indicators under the given conditions (Annex D Data collected). Sustainability of the effects obtained The sustainability of the effects obtained on the 3 levels represented by the 3 financed areas was analysed based on the qualitative information collected during the evaluation process. URBAN INFRASTRUCTURE For the most part, the sustainability of the effects obtained

through the interventions aimed at the urban infrastructure depends on the availability of budgets at the level of the local public administration for the maintenance of these objectives. In some cases, the guarantee of the works executed requested by the manufacturers covers a longer period than usual (ex: 5-10 years), this aspect contributing to the durability of the effects. In the case of certain objectives, for example heritage objects or public parking, the maintenance of the objectives can be supported, at least partially, by applying tariffs to visitors / users.

SOCIAL INFRASTRUCTURE Similar to the situation of the urban infrastructure, the sustainability of the effects obtained through the interventions aimed at the social infrastructure depends to a large extent on the availability of budgets at the level of the local public administration for the maintenance of the rehabilitated objectives and the financing of the operating expenses. The existence of sources of income at the level of some of the institutions (ex: tariffs for the homes for the elderly, or rates charged after the ex-post period in the case of activities for the public carried out by various beneficiary institutions) represents, of course, a contribution, lower or greater, to the sustainability of the effects.

BUSINESS ENVIRONMENT

INFRASTRUCTURE There are high chances that the results obtained through the interventions designed to support the infrastructure of the business environment will be maintained or even developed, provided that the economic development in general at urban level is supported by the administrations of the cities (or metropolitan areas). Only in such a favourable economic environment can the companies benefiting from the new infrastructure continue to develop, only thus can the clusters of companies targeted by intervention prosper. Even though, at least theoretically, the newly developed clusters can export and thus be less dependent on the local economic environment, even in this situation the local economy must remain competitive in order to be able to adequately provide the goods and services that these companies need.

57

The sustainability of the effects obtained is also dependent on the extent to which there is a continuity of urban development plans, a consequence in reaching certain objectives. From the point of view of the typology of the financed cities and of the approach promoted for each type of city, the ROP 2014-2020 did not stimulate the continuity of the approach of the urban development on the principles of the ROP 2007-2013 (Growth poles, Urban development poles, Urban centres). The priorities of the 2014-2020 ROP, however, have generally stimulated the consistency of the urban development approach, thus contributing to the sustainability of the effects obtained through DMI 1.1, aspects illustrated in the following graphs.



Graphic 8 Perceptions of the beneficiaries regarding the sustainability of the obtained effects and the continuity of the PIDU

Graphic 9 Perceptions of beneficiaries regarding the taking over of 2007-2013 projects by the subsequent planning exercises

58



0,00%

10,00%

20,00%

30,00%

40,00%

Average Not at all To a greatextent

Totally To a low extent

Projects within the integrated plan that were not funded (from any source) in the period 2007-2013 were taken over in the subsequent

planning exercises

59

4.2. EVALUATION QUESTION 2: WHAT ARE THE BEST PERFORMING INTERVENTIONS, TO WHOM WERE THEY ADDRESSED AND UNDER WHAT CIRCUMSTANCES WERE THESE IMPLEMENTED?

Analysis elaborated from the perspective of the type of city (Growth Pole, Urban Development Pole, Urban Centre)

Sub-areas Growth Poles Urban Development Poles

Urban Centres

No. of projects

% out of the total financed

value

No. of projects

% out of the total financed

value

No. of projects

% out of the

total financed

value

Urban public spaces 48 54.6% 68 79.5% 150 84.3%

Rehabilitation of buildings or land 3 1.1% 1 0.2% 1 0.1%

Cultural heritage objectives 5 3.4% 6 6.8% 9 5.1%

Transport and population mobility 18 25.7% 4 5.7% 5 3.6%

Development of alternative forms of transport (bicycle lanes) 2 1.0% 2 1.3% 0 0.0%

Increasing the access of citizens to information of public interest 1 0.2% 0 0.0% 3 0.2%

Social services infrastructure 34 5.3% 13 2.7% 28 2.4%

Social houses 3 0.4% 3 0.6% 0 0.0%

Increasing population safety 4 0.9% 7 1.4% 62 3.9%

Development of the business environment

13 3.1% 2 0.4% 3 0.4%

Leisure tourism infrastructure 5 4.4% 2 1.2% 0 0.0%

Total 136

108

261

Integrated approach: Growth poles have more elaborate integrated plans, with a complete log frame, which includes both projects financed under KAI 1.1 and other projects planned to be financed from other sources. The urban development centres and urban centres have integrated plans that can differ significantly in volume and degree of detail, over half of these plans including only the projects planned to be financed under KAI 1.1. As for the types of projects implemented, notable differences are observed regarding the proportion of investments in the transport infrastructure sub-field, where the Growth Poles have invested 5-6 times more, as a percentage of the total value financed, than the Urban Development Poles or Urban Centres , the main reason for the local authorities that invested little or no investment in this sub-area is that other urban infrastructure projects had a higher priority and that the projects targeting the transport infrastructure are expensive. Similarly, in terms of the percentage invested in social infrastructure other than supervisory systems, the Growth Poles have invested approximately twice as much as the Urban Development Poles or the Urban Centres, and in terms of the infrastructure of the business environment (excluding the leisure infrastructure) the Growth Poles have invested about 8 times more, as a percentage of the financed value, than the Urban Development Poles or the Urban Centres. In the field of social infrastructure,

60

more than half of the urban centres chose to invest only in surveillance systems. If we associate this type of investment with the urban infrastructure area, it turns out that over half of the Urban Centres did not choose to invest in the social infrastructure field under KAI 1.1, some of them implementing such projects under KAI 3.2. The portfolios of projects, as a percentage absorbed by investment areas and sub-areas, are close between the Urban Development Poles and the Urban Centres. Notable differences can be identified in the transport infrastructure sub-area, where Urban Development Poles have invested with approx. 50% more than the Urban Centres and the surveillance systems, where the Urban Centres invested approx. 3 times more than Urban Development Poles. Effects: Keeping the proportions, the effects at the urban infrastructure level are similar in all categories of cities - priority issues have been addressed, thinking has been integrated, the effects are obvious and significant. In terms of social infrastructure other than surveillance systems, in the cities where projects have been implemented in this area, the effects are also similar in type of benefits - rehabilitated infrastructure, better services, new services, in some cases increasing the capacity of service of a larger number of beneficiaries. In the case of the infrastructure of the business environment, the growth poles dominate as a number of projects, a comparison with the effects obtained through the 5 projects implemented by the Urban Development Poles and Urban Centres being difficult due to the lack of representativeness of this small number of projects. As shown in the table "Evolution of the values of the impact indicators at the level of the types of beneficiary cities, in the period 2007-2017" under chapter 4.1., the biggest differences between the 3 types of cities are identified as follows: the rate of change of the domiciled population: at the level of the Urban

Development Poles and the Urban Centres there is a significant decrease, the Growth Poles showing an increase, explained by the greater attractiveness of the big cities;

the rate of employees in the case of Urban Development Poles has the highest increase, which can be explained by the increase of economic investments in the respective cities;

the rate of school population at primary level has a significantly higher increase at the level of the Growth Poles and Urban Development Poles than in the case of the Urban Centres, and the decrease of schools at the secondary level is the most pronounced in the case of Urban Centres, the two aspects indicating a decrease of the young population (children) at the level of urban centres.

From the point of view of ROP intervention, these differences indicate high needs for further support at the urban centres level. Growth poles and Urban development poles are more attractive for both immigrants and investors, instead of urban centres, the major challenge is to identify local economic engines that will act for socio-economic development, increasing the attractiveness of these cities for young people and for potential other investors that could be attracted as a result of a domino effect, if the administration succeeds in initiating viable investments in local economic development. Best practice analysis Best practice identified at the level of urban development interventions was analysed both from the perspective of the approaches applied by the cities (by the identification

61

of the types of approaches that are highlighted for certain reasons) and from the perspective of the results obtained as a consequence of these approaches. Examples of best practice are not only identified at the level of those cases that do not follow the regular patterns. Regarding some of the aspects described below, for example the integrated approach in the field of urban infrastructure, the best practice has been manifested even at the level of the typical approach of the cities in this area. The diversified nature of the project portfolio under KAI 1.1 means that the best practices identified by the evaluation have multiple validities. For example, best practices may refer to certain preparatory activities for the integrated plans, to the solutions chosen to address the various local needs, to the synergy between projects, to the approach of certain areas of urban development, etc. Interventions for the development of the business environment In general, the formulation of projects within the PIDU that support the development of the business environment has raised more challenges for city administrations than the interventions in the fields of urban and social infrastructure, for reasons already mentioned above. The most important reason for this is that the local authorities do not have as clear a picture as to the possible directions of action they can take for the economic development of the city, compared to the clarity regarding the interventions in the urban or social infrastructure. In such a context, good practice approaches in this area are all the more evident. The examples presented below are not the only ones in the field, but the ones extracted by the evaluators from the situations with which they came into contact. Craiova growth pole adopted an integrated approach towards the projects intended to support the business environment, concentrating in a certain area of the city the 3 projects financed under KAI 1.1 and a project financed under KAI 4.1 ROP 2007-2013. These projects were conceived as a whole, and the area chosen for implementation was a marginalized area (both as a location, being located on the outskirts of the city, and from the point of view of the socio-economic situation, the area being the location of a former fair). There was a need to intervene to revitalize the area, the City Hall already having plans in this regard. The concept of intervention in the respective area provided: the construction of a multifunctional centre that includes office spaces and a large capacity exhibition space (similar to Romexpo Bucharest), this multifunctional centre being financed under KAI 4.1; development of an adjacent office infrastructure, through the 3 projects destined to the business environment financed within under KAI 1.1, which aimed to stimulate the creation of clusters by activity areas. The 3 office buildings house companies from the following fields: IT and communications (business support structure of innovative cluster type); cluster of business support services (e.g.: human resources, marketing, market research, management consulting, legal consulting, etc.); a security and protection company. Even if to achieve the initial desires suggested by the titles of the 3 projects there are still steps to be taken, overall the 3 projects, together with the multifunctional centre financed under KAI 4.1, have created an important platform to stimulate and support the business environment, while also succeeding in revitalization. a declining area of the city. Ploiești growth pole has formulated and implemented several projects aimed at supporting economic development, managing to combine financing both from public funds (KAI 1.1 and other sources) and from private sources. In this area, Ploiești Growth Pole has as its strategic objective the support of sustainable economic development integrated by new energies and technologies. 2 projects were financed under KAI 1.1

62

aiming at creating a multifunctional business centre and a business excellence centre for young entrepreneurs. Apart from these projects, the Ploiești Growth Pole has implemented 12 projects, financed by the Environment Fund, SOP CCE and private investments, respectively. Consultations with the citizens Consultation of citizens in the process of elaborating integrated urban development plans was carried out at the level of all the cities that benefited from financing under KAI 1.1. However, the way of consulting differed greatly from one city to another, from making the plan proposal available and inviting citizens to submit opinions and proposals, to citizen inquiries or involving citizen representatives in debates on the proposed plan. Comănești city had an approach that involved citizens in the elaboration of the integrated development plan at a very high level. Basically, the representatives of the City Hall went to all the neighbourhoods of the city and collected information on the development needs of the city and the respective areas from the perspective of the inhabitants, opinions, suggestions, proposals for solutions from them, through face-to-face interaction, in a process lasting with a significant investment of resources from the local public administration. As a result, the City Hall could have a detailed image regarding the needs of the city through the perception of its inhabitants, and the city's development plan thus reflected to a great extent the directions of action resulting from this needs analysis and following the proposals thus collected. Development of the metropolitan area Constanta was one of the pioneers of the development process of a metropolitan area, being an active promoter of the concept since the launch of ROP 2007-2013. The idea of metropolitan development lays at the base of the integrated development plan 2007-2013, Constanța municipality investing significant resources in the process of convincing the localities in the metropolitan area to adhere to this idea and to become members of the Association of Inter-Community Development Metropolitan Area Constanța. In this endeavour and at the same time during the process of implementing the integrated plan, Constanța County Council played a key partner role, through the support granted and through the active and constant involvement, including as a contractor of some of the projects implemented under KAI 1.1. Of all the metropolitan areas financed under DMI 1.1, the metropolitan area of Constanța has the largest number of projects dedicated to the localities in the metropolitan area financed by KAI 1.1: 9 projects in the field of urban infrastructure, 8 projects in the field of social infrastructure, 1 project in the field of leisure infrastructure, in total 18 projects out of a total of 30 financed by DMI 1.1 in the Constanta Growth Pole. For Constanța, the development of the metropolitan area represents a natural and necessary direction of urban development, which contributes to the decongestion of a large city through a polycentric approach. The projects designed for the development of the Constanta metropolitan area targeted both the needs of the local communities (e.g.: urban infrastructure rehabilitation, social infrastructure development) and the needs of the metropolitan area, in particular the need for mobility (e.g. the widening of the Mamaia-Năvodari boulevard, the modernization of the county road 228A Poarta 228A White - Ovidiu). Social infrastructure

63

All the cities that implemented projects and aimed at developing the social infrastructure based their intervention on a thorough analysis of the local needs in the field and of the target groups with the greatest need for support, the target groups most often targeted by the implemented projects being elderly people, children, young people, people with disabilities. From the point of view of approaching these types of interventions, all the projects with which this evaluation came into contact can be considered good practice, from all points of view: the analysis of the local needs in the field was relevant, the immediate results of the projects consist of rehabilitations and upgrades of buildings and spaces that are visibly leading to an increase in the quality of the services offered or to the provision of new services for the target groups targeted. The examples presented below were chosen because they illustrate best practice in this area. Growth pole Constanța has formulated and implemented a number of 9 projects in the field of social infrastructure under KAI 1.1, i.e. about 12% of the total number of projects implemented in this field at national level, disregarding the projects aimed at installing surveillance systems. Constanta growth pole is highlighted as an example of best practice in the field of social infrastructure through 3 aspects: the high number of projects implemented, the fact that 8 of the 9 projects were implemented in other localities of the metropolitan area than the municipality of Constanța thus addressing needs of the area in overall and the fact that 8 of the 9 projects target the young target group, thus concentrating the assistance for achieving greater effects. The 8 newly created youth centres aim to provide a framework and diverse possibilities for spending time for young people, through a wide range of cultural, sporting, artistic and recreational activities, thus aiming to offer viable and constructive leisure options, free time, socializing, and creating an attractive environment, an attractive metropolitan area that will contribute to the decision of the young generation to stay and build a future in this area. One of the 2 projects implemented in the social field by the municipality of Braila aimed at the creation of the Community Center "Grandparents and grandchildren". Turned, by rehabilitation, from a building in an advanced state of degradation into a heritage building, the centre offers children multiple possibilities: learn how to paint, dance, master various things, sing, do their homework in the after-school program, in while interested adults can benefit from vocational training courses. The novelty of the project, which also gave its name, consists in the fact that all these courses are held by elderly people, generically called "grandparents", specialized in various fields, who have retired, but who wish to act as volunteers within this Center. Moreover, the idea of the project was given by a group of retired educators, who started various projects with disadvantaged children in the central area of the city. The best practice in this project is to create a new social service, based on an idea from within the community, which addresses a real and acute need, all in a special framework achieved by rehabilitating a heritage building. Investments attracting other investments Although this intention was not explicitly included in the integrated urban development plans, the fact that there were situations in which it materialized represents a positive aspect. Investments that attract other investments have a beneficial effect on the development of the areas in which they are located, while also being potential examples to be followed by other investments. The growth pole of Brașov represents an example in this regard, through the project Accessing the economic area of Coresi. The

64

project aimed to achieve a new axis of road transport and increase the mobility of the inhabitants between the residential areas and those with economic and business destination, and thus managed to attract other investments for the economic development of a new area through new commercial spaces, companies headquarters, real estate investments, also opening up a new way of access to this area of visitors from neighbouring cities. In Târgu Jiu, the ring zone of the city developed as a result of the urban infrastructure projects that aimed at the modernization of streets, attracting real estate investments and opening new commercial spaces. Innovating approach Alba Iulia is noted as an example of best practice on urban development through a number of aspects. Understanding the importance of developing the internal capacity of the local administration to manage the urban development, especially from the point of view of understanding the concept of urban development and the good practices in the field, the City Hall appealed to experts in the field from other Member States, inviting them to Alba Iulia to hold seminars on this topic. It is the only example of such a practice with which this assessment came into contact. Continuing efforts to develop internal capacity, PIDU 2007-2013 was elaborated with internal resources of the City Hall, without the support of the consultants. This choice was motivated both by the desire to develop the internal capacity for urban development planning, and by the need to create a fully-fledged document for those who were to bear the responsibility of implementing the integrated plan, i.e. the Alba Iulia City Hall. The urban development approach included a polycentric thinking, in the sense of planning the urban development, having as its territorial framework a larger area than the city's territory, including neighbouring administrative-territorial units. The polycentric approach to urban development was supported, for example, by actions on a key aspect of development, namely labour mobility. In this sense, the municipality decided to intervene by creating a system of inter-locality transport through a single operator, the service thus offered being considered a municipal service. As in the case of the elaboration process of the PIDU, the Municipality of Alba Iulia had a special approach towards other cities in terms of accessing other sources of financing for the implementation of PIDU, besides DMI 1.1. Thus, in addition to the typical access to other operational programs 2007-2013 (e.g. SOPC, ECOS, SOPHR, PODCA) grants were accessed from other non-EU countries (e.g. Norway, Japan), community programs such as URBACT, as well as bank loans or the capital market by issuing municipal bonds. This approach demonstrates that the financing of urban development should not be limited to accessing the usual sources of non-reimbursable funds, but should be viewed as a proactive activity seeking financing solutions, not as a reactive activity, which accesses funds only if they are made available. Integrated approach of the interventions in the urban infrastructure area All the cities or metropolitan areas with which the evaluation came into contact have demonstrated an integrated approach to investments in urban infrastructure, these investments pursuing the central objective of improving the quality of life and attractiveness of cities through the multiple aspects that support it: improving urban appearance, traffic flow, rehabilitation of heritage objectives, rehabilitation of public transport infrastructure, etc. The investments targeted the areas with the highest needs, usually such areas being part of a certain flow of movement of the city, and many of the projects implemented were mutually supportive. For example, the

65

rehabilitated arteries of Bacău are component sections of the axes with the largest flow of vehicles and thus together they contribute to the fluidization of the traffic, the newly constructed parking in Mamaia is adjacent to the modernized promenade, the road crossings in the central area of Craiova fluidize the circulation in the area, and the newly built adjacent parking lot allows an increased flow of vehicles in the area without blocking traffic, while ensuring easy access for visitors to the rehabilitated historic centre, the rehabilitated arteries of Alba Iulia represent component sections of the city's entry-exit axes, on these sections being arranged and bicycle paths to encourage alternative transport, the multifunctional centre of Pitești is located in the central street area that has benefited from rehabilitation, thus contributing together to increase the attractiveness of the area. Buildings rehabilitation and reuse Under KAI 1.1, especially in the field of social infrastructure, there were a number of projects whose rehabilitation works consisted in the restoration of abandoned buildings, some in a strong state of degradation, to give them new functions. For example, in Negrești Oaș and Adjud, former thermal power stations were rehabilitated and destined for social services. In Negrești Oaș, a new day centre was created for the elderly, offering activities for leisure and meal at home to 50 beneficiaries. In Adjud, the rehabilitation of the thermal power plant created the possibility that the community centre that now works in this location, previously operating in a smaller location, will increase its number of beneficiaries from 25 to 300. In Constanta, the rehabilitation of a heavily degraded building, has whose infrastructure was only partially existing and located in a central area of the city, led to the creation of the Tomis Center of Excellence in Tourism and Services and at the same time to the preservation of a historic building.. Rehabilitation of heritage buildings and revitalization of their cultural-artistic activity 1280/5000 The rehabilitation of heritage buildings was one of the types of activities eligible in the field of urban infrastructure. The projects that targeted this sub-domain were aimed at highlighting important buildings for the socio-cultural life of cities, two such examples being illustrative in this regard. In Zalău, the rehabilitation of the Transylvania building that was not used before the project brought a much improved aspect to the central area of the city, the building being part of the whole of the Iuliu Maniu Square and, at the same time, it contributed to the revival of the socio-cultural life, here various activities are taking place now. events: shows, conferences, contests, etc. The rehabilitation of the Maria Filotti theatre building in Brăila contributed to the implementation of the vision of the City Hall for the restoration of the historical central area of the city, having as a result the possibility of continuing the socio-cultural activity of this building, the rehabilitation contributing decisively to the building's bringing to the standards required for operating authorization. The number of events and spectators or visitors increased after the completion of the project, the building hosting both theatre shows and other types of shows or socio-cultural events. Benchmarking analysis

66

The benchmarking analysis followed the extent to which certain elements of best practice regarding the integrated approach of urban development, formulated in the evaluation based on the analysis of the specific literature and the consultations from the initial stage with the MA ROP, were found in the case of the 8 cities that have constituted the subject of case studies. The elements of best practice thus analysed were as follows:

Best practice elements identified by means of the case studies

A mechanism for coordinating inter-sectoral policies at city level has been developed

A mechanism of collaboration and coordination between the local authorities and the authorities at the county level was developed

A mechanism of collaboration with local partners was developed: the academic environment, the business environment, NGOs, citizens

PIDU remained a document of integrated approach of urban development and after the completion of the implementation of the projects financed under the ROP 2007-2013

Medium business or social infrastructure projects included an analysis of the level of demand, to avoid financing structures for which there is not sufficient demand

The mechanism for coordinating inter-sectoral policies has been developed by involving local partners in multiple forms in the process of elaborating integrated plans and has worked properly in all the analysed cases, being supported either by working groups with the participation of the main decentralized institutions at local level. , or through public debates with the participation of these institutions, and in the implementation stage being supported by ad-hoc communication according to the situations arising during the implementation of the funded projects. A particular situation was reported in Bacău, where the municipality managed to create a permanent structure, the Intersectoral Partnership for Strategic Development of the Bacau Municipality, since 2009. The development of the inter-sectoral coordination mechanism was better supported in situations where they organized working groups related to the fields included in PIDU, with the meeting of local partners (Craiova, Constanța cases), these working groups allowing more in-depth analyses than organizing a debate as a single event. Also, this mechanism was better supported in the case of the Growth Poles Craiova and Constanța, where the existence of the Inter-Community Development Associations creates a permanent platform of dialogue both between the localities that are members of the metropolitan area and between the other local partners, representatives of the various social sectors. -economic. At the same time, the 2 Growth Poles Craiova and Constanța have included local partners in the management structure of the PIDU implementation, not only in the PIDU elaboration structures, thus further facilitating the inter-sectoral coordination. All in all, the case study cities have succeeded in creating a mechanism for coordinating inter-sectoral policies, mainly due to the appropriate involvement of local partners in the elaboration of integrated plans and due to the collaboration during the implementation of projects between the local administration and the institutions, whose area of responsibility included some of the financed projects.

The most significant differences between the 8 cities subject to analysis are manifested in the mechanism of collaboration between the authorities at the local level and those

67

at the county level. In general, the collaboration with the county council was not a key aspect for the elaboration and implementation of integrated plans, with the exception of the Growth Poles. In the case of Constanta Growth Pole, for example, the county council has had and plays a key role in the elaboration and implementation of the integrated development plan, implementing projects as a direct beneficiary and being a member of Constanta Metropolitan Area Association. In the case of Craiova Growth Pole, the County Council was involved both in the process of elaborating the integrated plan and in the management structure of the implementation of the integrated plan. In contrast, in the case of the other cities analysed, the collaboration with the county council represented a secondary, short-term aspect, no clear examples being identified in the information collected on how the process of elaboration and implementation of integrated plans led to the development of a mechanism of collaboration with the respective county councils.

Regarding the mechanism of collaboration with local partners, all the cities subject to analysis have a high level of collaboration and consultation with these partners in the preparation stage of integrated plans. The forms of consultation were diverse - public debates, public consultations, focus groups, working groups, etc., opinion poll among citizens. Each analysed city has chosen a mix of methods of involvement of the specific local partners, not differentiating a better or less good mix than others. Examples of methods used per city: Pitesti (public debate, citizen survey), Bacău (focus groups, citizen consultation), Craiova (working groups, correspondence consultation, debates), Constanta (working groups, consultative groups, debates) public).

Regarding the extent to which PIDU remained a document of integrated approach of urban development and after the completion of the implementation of the projects financed under KAI 1.1 ROP 2007-2013, there are practically 2 situations. In the first situation, PIDU 2007-2013 ceased its function with the completion of the projects financed under KAI 1.1, as these were the only projects included in the respective integrated plans, this being the case for the urban development centre Pitești, respectively the urban centres Lipova, Bistrita. In the case of the Craiova and Constanta growth poles, the Bacau urban development pole and the Alba Iulia urban centre, the project portfolios of the 2007-2013 integrated plans were larger (or much larger, for Craiova and Constanta) than the KAI-funded project portfolio. 1.1. For these cities, the planning exercise represented by the elaboration of the integrated strategies of urban development 2014-2020 represented an opportunity to continue the integrated plans 2007-2013, taking over the projects not implemented in the new strategies. In these situations, although the PIDU document 2007-2013 formally ceased its planning document function, the ideas and projects within the PIDU were taken over by the next programming period.

Regarding the extent to which the medium-sized business or social infrastructure projects include an analysis of the level of demand, in order to avoid financing some structures for which there is not sufficient demand, the evaluation found similarities in the cities that have been the subject of case studies. Specifically, with regard to the rehabilitated social infrastructure, the level of demand was already known based on the previous functioning of these centres, therefore the investments made were responding to an existing and known demand. In the case of the social infrastructure objectives newly built or offering new services, the level of demand was estimated based on the qualitative information collected during the elaboration phase of the integrated plans, from the local stakeholders. In terms of business infrastructure, only 2 of the 8 cities analysed through case studies have invested in this field. And in these cases, the demand level was estimated based on the qualitative information collected during the elaboration phase of the integrated plans, from the local stakeholders.

68

Analysis of the target groups benefiting of investments This analysis will seek to identify the target groups that have benefited most from the results recorded, at the city level as a whole and at the level of the 3 areas financed under KAI 1.1. Urban infrastructure: The main groups that benefited from the interventions carried out in this area are represented by: the population of the city, tourists, visitors, people who transit the city. For example: the rehabilitation of some road arteries, the construction of parking lots serve both the population of the city as well as visitors, tourists or persons passing through the city; the rehabilitation of some heritage objectives, the arrangement of the green spaces bring benefits to the population of the city as well as to the visitors or tourists; the restoration of the historical or civic centres contributes to the stimulation of the development of services in the area, the beneficiaries being the economic agents, the population, the visitors and the tourists. The creation of bike lanes stimulates alternative, environmentally friendly transport and benefits both direct users and the entire population by reducing the emission of pollution caused by motorized transport. Social infrastructure: The groups that benefited from the interventions carried out in this field are diverse: the elderly, the disadvantaged, the disabled, children, young people, the general population. In the case of the Growth Poles, the groups benefiting from investments are found both in the urban environment and in localities in the rural area that are part of the metropolitan area. The investments made in the metropolitan areas are mainly felt by the local community (the population of the respective locality) but there are also situations in which the rehabilitated objective is of interest for the metropolitan area as a whole (e.g. centres for people with disabilities). Business environment infrastructure: The beneficiaries of the intervention in this field are first of all the companies hosted in the created business structures, then the employees of these companies. Indirect beneficiaries are other economic agents suppliers or clients of the beneficiary companies directly, as well as the local communities in general that benefit indirectly from the economic development due to the funded projects. The target groups that benefited from the interventions under KAI 1.1 are therefore very diverse, and the type of benefits created by the projects funded for these target groups are also very different, so compare them to identify the target groups that have had the greatest benefits, it can only be done at the level of qualitative assessments. Overall, it is evident from all the information collected through interviews, focus groups, on-site visits that the population of cities or metropolitan areas, taken as a target group as a whole, registered the greatest benefits as a result of investments in urban infrastructure, these investments being the highest in value (91% of the total funds absorbed through KAI 1.1), and the effects of these investments are multiple. On the same criterion of the financed values, the following target groups that have benefited the most from the interventions are those targeted by the rehabilitated or modernized social infrastructures: the elderly, children, people with disabilities, young people, disadvantaged people. Regarding the target groups of the interventions aiming at supporting the business environment (economic operators and their employees or people looking for a job),

69

there are two ways in which this target group has benefited from the interventions under KAI 1.1: direct benefits, as a result of the projects implemented in the field of business environment infrastructure, respectively indirect benefits, as a result of interventions in the field of urban infrastructure mainly and as a result of interventions in the field of social infrastructure in secondary. The direct benefits resulting from the implementation of projects in the field of business infrastructure are very limited at national level, considering that only 1.4% of the total value financed by KAI 1.1 belongs to this field. The 69 companies attracted to the newly created business structures, respectively the 803 newly created jobs and 938 jobs maintained in these companies, represent, at national level, a small result. In this regard, it can be stated that at least directly benefited from the intervention under KAI 1.1 the target group made up of economic operators and employees. However, for this target group, there are numerous indirect benefits resulting from the interventions under KAI 1.1 in the fields of urban infrastructure and social infrastructure. For example, the rehabilitation of the traffic arteries has led to the fluidization of the traffic which obviously benefits the economic agents with transport activities, the rehabilitation of the transport systems and in general the improvement of the connectivity between the urban areas has led to the increase of the mobility of the work force according to the reports of sustainability and the information collected through the case studies, the rehabilitation of some central areas of the cities led to the revitalization of the respective areas from a social and economic point of view, including through the advance of HORECA economic activities (hotels, restaurants, cafes), aspect indicated by the sustainability reports and confirmed by on-site visits in case studies. The rehabilitation of some social centres, such as homes for the elderly, has contributed, according to the sustainability reports, to the increase of the employability for their owners, etc. Apart from these target groups, the evaluation did not identify other groups that are the beneficiaries of the interventions or other propagation effects of these interventions, which does not mean that such other beneficiaries or other effects cannot exist. Obviously, any intervention always has a series of indirect effects on a large category of groups, but these are difficult to capture or quantify, for example the economic development to which KAI 1.1 interventions have had a contribution can bring benefits to the population from neighbouring localities to any city, not only to those in metropolitan areas, firms supplying local firms, located anywhere in the country or in other countries can benefit from urban development by increasing the volume of sales, social centres can bring indirect benefits beyond the families of the people who benefit from social services etc. Basically, there are no limits on the type of indirect benefits and the groups that can feel these benefits, but the analysis of such indirect and obvious benefits goes beyond the scope of the evaluation of KAI 1.1. Synthetic presentation of the findings identified under evaluation question 2 The interventions that gave the best results are those related to urban infrastructure in general, where the results obtained and analysed by this evaluation indicate a certain contribution to the increase of the quality of life and the attractiveness of the cities, as well as to the creation of jobs. A particular example refers to the rehabilitated heritage objectives, better results being obtained in the situations in which the beneficiaries have succeeded in developing the socio-cultural activity of these institutions by increasing the number of events organized here, thus enhancing the role of these institutions and contributing more to reinvigorate the cultural life of the city. The

70

interventions in the social infrastructure also had results that contributed to the increase of the quality of life, but being much smaller in number if the investments in video surveillance systems are associated with the urban infrastructure and not the social one. The investments in the infrastructure of the business environment had better results in the situations where they were based on an integrated thinking, of the economic cluster type, as well as in the situations in which the combination of the public and the private investments was successful.

71

5. CONCLUSIONS, RECOMMENDATIONS AND LESSONS LEARNED

Conclusions KAI 1.1 has reached its goal, opened new roads, consistently promoted the concept of integrated urban development and created the framework for the beneficiary cities to develop their capacity to develop and implement integrated urban development plans, thus contributing to achieving the objectives of KAI 1.1: increasing the quality of life and creating jobs. The integrated development plans developed by the beneficiary cities and metropolitan areas pursued an integrated approach to urban development, partially succeeding in achieving this goal. Thus, based on all the evidences already described, it can be concluded that within the field of urban infrastructure the integrated approach was largely present and had a high contribution to the increase of the quality of life in the beneficiary cities. In contrast, at the city level, the integrated approach was little present, due to the imbalance between the 3 areas financed at the level of the portfolio of implemented projects. As a result, the analysis of the approach of integrated urban development planning indicates an incipient level of local public administration regarding the concept of integrated urban development: what it means to develop a city, how this can be done. At this moment, most local public administrations associate urban development with the development of urban infrastructure, an area absolutely necessary but insufficient to achieve a sustainable development, according to the theories in the field. Through the intervention of DMI 1.1 we have obtained a series of results confirmed by the quantitative and qualitative information collected: the substantial improvement of the aspect of the beneficiary localities (ex: by rehabilitation of streets, sidewalks, historic centres), the stimulation of some economic activities (ex commercial activities and services) and social in the rehabilitated areas, increasing the comfort level of the citizens (e.g.: by building parking lots, rehabilitating lighting systems, rehabilitating or building parks and green areas, replacing street furniture, improving access for people with disabilities), reducing traffic congestion and increasing traffic safety (e.g. by making road passages, pedestrian crossings, widening of arteries, installing traffic management systems) increasing the number of visitors or revitalizing the cultural life to the rehabilitated heritage objectives, increasing the quality of social services provided by the institute the beneficiaries of the funded projects, the construction of new locations and the development of new social services (e.g. day centres, youth centres, specialized centres) for various target groups (young people, children, elderly, disadvantaged people, people with disabilities, etc.), increasing the number of beneficiaries (e.g. by rehabilitating previously unusable spaces and thus increasing the capacity), creating or maintaining jobs in the operating phase of some of the investments made (4,491 newly created jobs, 4,133 jobs maintained, nationally). In the case of the few projects that targeted the infrastructure of the business environment (18 projects out of 505 projects completed nationally within the framework of DMI 1.1), the main effects were the attraction of companies in the newly created structures, the creation or maintenance of jobs within the companies attracted. In general, the funded projects aimed to create clusters of companies by attracting in the financed infrastructures of companies from certain fields or by

72

attracting new companies that need support in the start-up phase, such desires being partially achieved. The interventions that gave the best results are those related to urban infrastructure in general, where the results obtained and analysed by this evaluation indicate a certain contribution to the increase of the quality of life and the attractiveness of the cities, as well as to the creation of jobs. The interventions in the social infrastructure also had results that contributed to the increase of the quality of life, but being much smaller in number if the investments in video surveillance systems are associated with the urban infrastructure and not the social one. The investments in the infrastructure of the business environment had better results in the situations where they were based on an integrated thinking, of the economic cluster type, as well as in the situations in which the combination of the public and the private investments was successful. The exercise of the implementation of KAI 1.1 2007-2013 had other very important indirect effects, more precisely the local capacity (of the public administration and of the local partners) of management of the integrated development plans has improved, due to the experience gained and due to the development of the collaboration between the local authorities and local stakeholders (local partners). The mechanism for coordinating inter-sectoral sectoral policies at local level is being developed, which is necessary for an integrated approach to urban development, this mechanism being stimulated by the collaboration of the administration - local partners found especially in the process of elaborating integrated plans. Similarly, a mechanism of collaboration between the local authorities and the authorities at county level (county council) is being developed, this mechanism working much better in the metropolitan areas. However, it is necessary to continue the development of the local capacity regarding the management of integrated interventions aimed at urban development, in two key directions: developing the degree of understanding by the local public administrations of the concept of urban development; developing the governance mode of urban development in the partnership system, by involving local partners in all stages of the process: planning, implementation, monitoring, evaluation. In particular, it is necessary to develop the system for monitoring and evaluating integrated plans, at the level of local administrations or metropolitan areas. At the level of the macroeconomic indicators used by this evaluation to analyse the overall effects of KAI 1.1 intervention, it was found that the financed urban centres had a better evolution in the population growth rate, respectively in the economic field at the employment level of population, than other urban centres that did not receive funding, this aspect indicating the effectiveness of KAI 1.1 intervention and also representing its net effect. At the level of growth poles and urban development poles, the intervention on KAI 1.1 had as a net effect the increase of the quality of life and the creation of jobs, without being possible to quantify the effect that can be attributed to the intervention on KAI 1.1. The main factors that contributed to achieving the effects were the determination of the beneficiary cities throughout the exercise of elaboration and implementation of the integrated plans, the fact that the local partners including the civil society were involved in the process, consistent promotion the ROP of the concept of integrated approach of urban development. One factor that limited the effects of the interventions is the fact that through DMI 1.1 only infrastructure investments were financed, since there is no possibility of financing and other types of activities, for example exchange of experience, studies, capacity development of the beneficiaries.

73

The main aspects that characterized the 3 types of financed cities (growth poles, urban development poles, urban centres) are the following: The growth poles had the most developed integrated plans, involved more local partners in the management of the implementation of integrated plans and they invested more in the transport infrastructure and the business environment. Urban development poles and urban centres have had similar approaches to integrated plans, differences appearing only in terms of the values of the project portfolios per city. On average, urban development poles have invested more, but there are also urban centres that have exceeded urban development poles as the value of the funded project portfolios. Keeping the proportions, the effects obtained are similar in the case of the urban and social infrastructure between the 3 categories of cities, the major difference being the infrastructure of the business environment, where due to the low level of investments from the urban development poles and the urban centres, the obtained results are much lower than in the case of growth poles. Growth poles have managed to configure metropolitan areas and thus support the polycentric approach to urban development. The results obtained through the interventions of DMI 1.1 were concentrated at the level of the localities where they were implemented, benefiting mainly the respective communities, but there were also projects implemented of metropolitan interest. For the next programming period, it is necessary to continue to support the development of metropolitan areas, in order to ensure the continuity of the process started in ROP 2007-2013 through the concept of Growth Poles, taking into account the essential role of metropolitan areas in the socio-economic development of the regions they belong to. The complex system of indicators under KAI 1.1 allowed a high level of heterogeneity of the indicators selected by each beneficiary and of the units of measure related to each selected indicator, so that difficulties were created in aggregating these indicators, both by ROP IB and MA , as well as by the evaluation team. As a result, only a relatively small proportion of the indicators reported by the beneficiaries were aggregated by MA ROP. For the next programming period, it is necessary to simplify the system of indicators and oblige the beneficiaries to use the same units of measurement and at the same time develop an electronic system for collecting and recording the values of these indicators, allowing their aggregation at national level and thus of the analyses necessary for ROP monitoring and evaluation. As a whole, cities are at the beginning of the road in terms of an integrated approach to urban development. The programming period 2007-2013 and the exercise of elaborating integrated plans represented a very good start and an important opportunity for cities to start thinking and planning integrated urban development, respectively to develop their management capacity for urban development. In the long term, the only feasible way that leads to the development of a city or metropolitan area is economic development. The development of the infrastructure is a basic element of the economic development, but without a critical mass of investments that directly support the economic development, the development of the city will always be dependent on the existence of sources of non-reimbursable financing and therefore it can be sustained only within the limit of existence of such sources. The evaluation team's opinion is that the long-term viable urban development scenario is based on the implementation of a city economic development plan, both

74

through specific policies that create a favourable environment, as well as through public investments or in public-private partnership, through concessions, etc., financed also from sources attracted (for example, the capital market or bank loans) in the medium term, this plan could lead to an increase in the city's revenues and thus to an increasingly solid financial base for continue to support the necessary investments in infrastructure. The investments made can directly affect the economic development or they can represent investments that have a domino effect - the stimulation of other private investments. This aspect is also underlined in the European Commission's 2019 Country Report on Romania, which considers as priority for integrated urban investments to be directed towards stimulating growth, innovation and productivity.3. Recommendations Considering the recommended scenario as a starting point, the recommendations addressed to the MA ROP for the priority dedicated to urban development under ROP 2021-2027 are: 1. Reconsidering economic development in the context of prioritization of the

investments, increasing its role and importance, as the only long-term feasible way for the development of a city or metropolitan area. MA ROR must stimulate cities to invest in this area, through the financing offer, in parallel with supporting the development of competences at local level regarding the integration of economic development into the vision of urban development and the implementation plans. The main elements of this approach include the following:

Cities have an adequate conceptual understanding of urban development;

Cities have a development strategy, supported by a balanced structure of objectives, priorities, projects that cover in a balanced manner all the planned objectives and development directions;

The development strategy includes objectives and projects aimed at economic development;

The specific needs of each city and the strategic choices made determine the formulation of the priorities and investments for each city or metropolitan area, with no model to follow, but only general principles of good practice..

2. In order to support the elaboration of quality plans and projects on integrated urban

development, MA ROP should support the development of cities' capacity to manage the urban development process as a whole: planning, implementation, monitoring, evaluation. The main actions required in this regard are: developing the degree of understanding by the local public administrations of the concept of urban development; developing the governance mode of urban development in the partner system, by involving local partners in all stages of the process: planning, implementation, monitoring, evaluation; developing the capacity of local administrations to monitor and evaluate integrated urban development plans. These

3 Working document of the Commission services. The 2019 Country Report on Romania {COM (2019) 150

final} Annex D: Investment Guidelines on Financing the Cohesion Policy for 2021-2027 for Romania.

75

objectives can be supported by financing activities such as training, exchanges of experience, technical assistance, carrying out studies, including by developing, within the framework of ADR, support activity for all cities in the region that are preparing urban development plans and who want to use this form of support. Promoting and explaining the concept of urban development can be put into practice by financing actions such as the ones below (there is also the possibility of collaborating with other institutions that undertake similar actions, for example, Ministry of Regional Development and Public Administration or the World Bank).): RDA should organise some regional seminars for UAT or ADI on topics on urban

development (concepts, examples from practice, EU directions in the field, etc.). These meetings can have a two-way role, both providing information to cities, as well as collecting information from them about the situation on the ground. For this purpose, the expertise of consultants in the field or of the experienced practitioners from Romania or other Member States can be used, and a number of existing materials can be used, for example: (i) materials elaborated at EU level (ex: Cities of Tomorrow, the EU Urban Agenda and the action plans related to the 12 thematic partnerships, etc.) (ii) materials elaborated at national level (e.g. World Bank studies, National Strategy for territorial development, etc.); (iii) materials developed internationally (e.g. McKinsey Urban Development Study).

Exchanges of experience between cities in Romania and between cities in Romania and cities in the European Union, on the topic of urban development.

3. ROP should finance development plans for cities (or metropolitan areas) that meet a

minimum level of quality (assessed on the basis of a technical selection grid). Thus, development plans must go through a selection process regarding both the eligibility and the technical quality for which they have to meet a number of criteria, for example: the extent to which the strategy formulated for the development of the city is adequate

to meet the urban development goals that the cohesion policy 2021-2027 aims to support4 (stimulating growth, innovation and productivity, access to new jobs and public services; local base etc.);

the quality of the structure of objectives and projects, which should cover in a balanced way the intervention areas chosen by the city: economic and social (minimum), cultural, environmental, etc.;

the modalities foreseen regarding the involvement of local partners both in the planning stage and in the implementation, monitoring and evaluation phase of the urban development plans. This aspect is in line with the partnership principle promoted in the proposal for EU Regulation 375 on structural and investment funds (e.g. Art. 6 of the document);

the quality of the structure of result indicators within the integrated plans (e.g. clarity, relevance, simplicity, existence of monitoring mechanism etc.).

4. As part of the development plans, ROP should finance a wide range of types of

investments, which should not be exhaustively specified in the ROP program document or subsequent documents (e.g. Applicant's Guides). The ROP should specify only the general funding framework (the funded areas, possibly a few examples within each domain, selection criteria, etc.). Thus, each beneficiary is given freedom to choose the investments considered most relevant, remaining within the general framework prescribed by the operational program. For situations where, however, there are project ideas that go beyond the priorities set by the

4 Working document of the Commission services. The 2019 Country Report on Romania {COM (2019) 150

final} Annex D: Investment Guidelines on Financing the Cohesion Policy for 2021-2027 for Romania

76

program, it is recommended to introduce a call dedicated to such atypical projects, which comply with certain selection criteria specified in the Applicant's Guide.

5. ROP should also fund activities to support and stimulate the emergence of the

targeted effects, not just investments in infrastructure, for example: Urban mobility: studies for the re-planning of transport systems, exchanges of

experience in the field between cities, technical assistance for the implementation of traffic management systems, etc.;

Patrimony objectives: elaboration of marketing studies to identify the possibilities of increasing the activity and the registered income, technical assistance or training for the elaboration and implementation of the resulting marketing plans;

Social services: studies needed target groups targeted, personal training, exchanges of experience between institutions providing social services, other activities to increase the institutional management capacity;

Business environment: studies, exchange of experience, technical assistance, training, various activities of institutional capacity development, addressed to local and county authorities for the elaboration of plans and their implementation on various topics on economic development where local authorities should intervene5 (ex. : facilitating a favourable business ecosystem, promoting the modernization of the local economy, training the workforce, promoting apprenticeship programs, supporting entrepreneurship, providing quality public services, controlling urban development and land use, reducing time and procedures for obtaining permits building, finding ways to stimulate the creation of local jobs, promoting financial instruments etc).

6. MA ROP should consider the possibility of financing integrated urban development

plans through combinations between the form of financing through non-reimbursable funds and the form of financing through financial instruments (loans, guarantee loans) in accordance with the guidelines in this regard in the proposal of EU Regulation 375 on structural and investment funds (ex: Title 5 - Financial support).

7. It is recommended to make the requirements regarding the format of urban

development plans more flexible, to accept any format provided the technical and eligibility criteria are met. There is no need for ROP to provide a model development plan. Cities can submit plans made according to any model, provided the technical and eligibility criteria are met, which must be clearly specified in the Applicant's Guide.

8. As regards the system of governance of urban development, the ROP MA should

support the development of the capacity to adopt and implement appropriate models and mechanisms. The measures can be targeted:

a. A model of local governance of urban development as close as possible to the community, including local partners both in the elaboration stage and in the implementation, monitoring and evaluation stages of urban development plans;

b. The use of the Community Led Local Development (CLLD) / DLRC (Local Development based on Community Responsibility) tool, especially for small cities, aimed at economic development, for example by developing the infrastructure needed for economic development;

c. Use of the ITI (Integrated Territorial Investment) instrument with multi-fund financing;

5 Examples extracted from the EU Urban Agenda: THE WORKPLACE PARTNERSHIP AND SKILLS IN LOCAL

ECONOMY, ACTION PLAN, July 24, 2018

77

d. Investments that stimulate, by domino effect, other investments (e.g. infrastructure investments that subsequently generate private investments and the development of the respective area);

e. Development of the system of monitoring and evaluation of urban development plans.

9. Calls with submission deadlines are organized allowing an adequate period of

preparation of development plans (8-12 months) or calls with continuous submission. The experience 2007-2013 (the deadline submission procedure was applied in the case of urban centres, respectively the system of direct allocation of funds in the case of growth poles and urban development poles) indicates that the typical disadvantage of the deposit system continues the first income, the first served that is, the fact that the budget is exhausted because of the risk of being unable to finance valuable projects that were submitted late, has not materialized. On the contrary, the budget allocated to KAI 1.1 was not fully absorbed, as a continuous deposit system could contribute to a higher degree of absorption. At the same time, the possibility of redistributing plans or projects in a short time is given, instead of waiting until a future call is launched.

10. Metropolitan areas are developing and need further support. In this respect, we

recommend MA ROP the following: Metropolitan areas should be explicitly identified as entities targeted by the urban

development priority within the ROP 2021-2027; The integrated development plans of the metropolitan areas will be subject to an

approval process, similar to the approval processes within the ROP 2007-2013 and ROP 2014-2020;

Projects from the entire metropolitan area are eligible within the urban development priority;

-2013, the eligible beneficiaries should be represented by: territorial-administrative units (local public administration authorities) at county or urban level; Inter-Community Development Associations; Partnerships between administrative-territorial units (local public administration authorities);

It is recommended to use the DLRC tool (Local Development under Community Responsibility) within territories within metropolitan areas, as a financing mechanism through Local Action Groups (LAGs) of needs specific to those territories within metropolitan areas, respectively some transposition actions in the respective territories of strategic directions within the integrated plans of the metropolitan areas. The benefits of using the DLRC tool can be multiple: empowering local actors to formulate specific territory-specific objectives within the territory strategy, the possibility to implement relatively small projects as value but effective for solving the needs of the territory or for transposing the strategic directions of the metropolitan area, the selection process of the projects being delegated to the LAG structure and thus avoiding overloading the ROP implementation structure. The disadvantage is that the contracting and payments process should also be managed by the ROP MA, similar to other ROP projects, which could represent a significant additional workload.

The integrated plans of the territories represented by the LAG should be subject to an approval process by the ROP implementation structures, similar to the approval process of the integrated plan of the metropolitan area.

Initially, within the metropolitan areas, the ROP should finance in a balanced way: projects that address needs of the entire metropolitan area, projects that address needs of the territories represented by the LAG, projects of local interest. In order to achieve such a balance, it is necessary that the process of approving the integrated plans of the metropolitan areas, respectively of the LAG strategies, should include a criterion regarding the balance in this respect from the objectives, measures, projects, etc. (more specifically, the interventions that address needs of the metropolitan area should be

78

represented in the integrated plans at least equal to the interventions that address needs of the LAG territories or of some localities).

For the implementation of the DLRC instrument, the existing LAGs (financed by the LEADER approach in the National Rural Development Program) will also be used, whose territories are partially overlapping with the metropolitan areas. Also, the new LAGs can be set up, having as main source of funding the ROP 2021-2027 program.

The types of projects that can be implemented by the Intercommunitary Development Association should be the same as the types of projects that can be implemented by any of its members, either individually or in partnership. In order to facilitate the implementation of projects by the Intercommunitary Development Association, harmonization of the existing legal framework is needed, in terms of creating the possibility of financing these structures for the elaboration and implementation of investment projects, but also of providing services of public interest. It should also be more clearly recognized in the law, respectively the law of local public finances, of the possibility of the Territorial Administrative Units not only to associate, but also to jointly finance certain projects, including through different co-financing rates and self-financing commitments. The interest in carrying out such projects is even greater in certain areas such as transport and connectivity infrastructure or that dedicated to the development of public services. The recognition of the strategic and integrative role of ADI, for example in the field of urban planning and spatial planning, the elaboration and monitoring of various programmatic documents, but also in terms of harmonizing / correlating common interventions and approaches at the level of the functional metropolitan territory, should be underlined in the legislation..

11. The program indicator system used in the future should be based on the following

elements: Defining and delimiting priorities (funded areas) regarding urban development; Formulate, for each field, a single specific objective and 1-2 outcome indicators related

to this specific objective, with a clear unit of measure; Defining a limited number of indicators for immediate achievement and units of

measurement for each indicator, by choosing the most relevant indicators for the type of investments financed;

The obligation of the beneficiaries to assume the indicators and targets corresponding to the types of investments implemented by each beneficiary;

Reporting by the beneficiaries of the realized values and their aggregation at the level of IB and MA, in an electronic database type system, allowing analysis and studies

12. Regarding the impact indicators at the relevant at the city level for measuring the

effects of future integrated interventions on urban development, it is recommended to use the following indicators, in addition to those used by the present evaluation: Dimension related to population: (i) Percentage of persons over 65 years of age in total

population, (ii) Demographic aging index, (iii) Resident population Economic dimension: (i) Number of registered companies, by activity categories, (ii)

Number of active companies, (iii), Employed population, (iv) Value of local government budget

Educational dimension: (i) School drop-out rate, (ii) Population structure by levels of education completed

In order to support future counterfactual impact assessments, the collection of data needed to analyse the evolution of these indicators (impact indicators and indicators used for matching) must be done as soon as possible and with a predetermined periodicity. In this regard, we recommend MA ROR to establish a collaboration protocol with the National Institute of Statistics to ensure a mechanism for collecting and making

79

available the necessary data at the locality level. The recommended frequency for collection is annual. The collection mechanism will have to cover the collection of all the indicators used in this evaluation and also proposed in this evaluation, at the locality level. It is recommended to build a mechanism that allows the calculation of a type indicator, the human development index, or the urban development index, at the city level, with an annual frequency, for all the Romanian cities. In this regard, the mechanism needs to cover the following elements:

Collecting the necessary individual indicators. This activity can be carried out by the National Institute of Statistics on the basis of a protocol. The list of indicators should include the following indicators, although not exhaustive: life expectancy, education stock (average number of school years, expected average number of school years), gross domestic product per capita (or average salary, as a proxy indicator), as well as measures of living standards. There is the possibility of increasing some research conducted by the NIS for collecting the necessary information followed by establishing methodologies for their calculation at city level.

Calculating the aggregate indicator value for all Romanian cities, with an annual frequency. The calculation of the indicator can be done by a professional organization (example: Romanian Statistical Society) or by a non-governmental organization (example: Econometrics and Applied Statistics Group). Also, any other organizations that can be identified and have the necessary expertise and availability can be considered as alternative solutions. It is preferable that the organization that calculates the indicator values annually is an independent organization.

Disseminating the values of the indicator. The values of the indicator, for all the cities in Romania will be able to be transmitted to the organizations that request them, based on a collaboration protocol and they can also be publicly disseminated through digital platforms, depending on the existing resources.

Lessons learned The main lessons learned from the experience of implementing KAI 1.1 under ROP 2007-2013 are:

The integrated urban approach is feasible and has good results; Cities can develop and implement integrated urban development plans, this capacity

developing over time; Cities need to increase the degree of conceptual understanding of urban development; MA ROP and RDAs must contribute to increasing the understanding of the concept of

urban development and to increasing the capacity of cities to develop and implement integrated urban development plans, through specific support measures and consequently in promoting this mode of action.

Date post:	03-Jun-2021
Category:	Documents
Upload:	others
View:	1 times
Download:	0 times

Lot 2 Impact evaluation of ROP 2007-2013 interventions · 2020. 4. 28. · Strategy&Consulting S.A,...

Documents