How can we know if EU cohesion policy is successful?Integrating micro and macro approaches to

the evaluation of Structural Funds

John Bradley

EMDA (Economic Modelling and Development Systems)

john.bradley@iol.ie

Copy of paper available on

www.gefra-muenster.de

National Development Plans and Structural Funds

• Large-scale investment aid for physical infrastructure, human resources, production incentives

• EC and local (public & private) co-finance• Targeted at lagging EU member states• Massively expanded after 1989• Implemented through multi-year National

Development Plans

RegionalGDP 2001

Part I Situation

and trends

GDP per head(PPS), 2001

< 5050 - 7575 - 9090 - 100100 - 125>= 125No data

Index EU 25= 100

Source: Eurostat

Two aspects of cohesion policy

• Policy design and implementation: usually a pragmatic process driven by local political wishes with local and EC oversight

• Implementation success depends on institutional capabilities and fiscal constraints

• Policy evaluation: (ex-ante, mid-term and ex-post)

• Logically the two aspects should be inter-linked. In practice they tend not to be (but ESRI “Investment Priorities” an exception)

Structural Fund impact evaluation:

micro versus macro techniques

Micro (bottom-up) Macro (top-down)

Level of disaggregation

High (individual projects) Low (sectoral aggregates, whole economy)

Use of theory Weak (judgemental, CBA)

Strong (macroeconomics)

Model calibration Judgemental/informal Scientific(?)/econometric

Policy impacts Informal/implicit/ranking/some quantification

Formal/explicit/quantified

Treatment of externalities

Limited or ignored Included/explicitly modelled

How can we know if EU cohesion policy is successful?

Integrating micro and macro approaches to the evaluation of Structural Funds

(Bradley, Mitze, Morgenroth & Untiedt, March 2006)

Paper available on: www.gefra-muenster.de

Implementing a micro-based approach to evaluation

• Welfare economics and the underlying rationale for public expenditure

1. Public goods 2. Corrective pricing (due to presence of

externalities)3. Targeted interventions (information

asymmetries)4. Redistribution (agriculture, social housing: but

mainly through tax and social welfare system)

Public goods: evaluation criteria

Corrective pricing: evaluation criteria

Targeted interventions: evaluation criteria

Redistribution: evaluation criteria

Two strands to the macro debate on cohesion policy effectiveness

• A political-economic literature that stands back from technical analysis, but argues in terms of theoretical paradigms

• An empirical literature that examines the issues empirically, using a variety of different analytical models.

Impact of recent research in economics

• New Trade Theory (Helpman & Krugman, 1985)

• New Growth Theory (Grossman and Helpman, 1991)

• New Economic Geography (Fujitsa, Krugman, Venables, 1999)

Empirical studies of cohesion:Two methodological approaches

• [1] “Testing” methodologies:

Testing a null hypothesis (e.g., cohesion policy has no effect)

• [2] “Impact evaluation” methodologies: Tracing out complex causal chains of policy consequences, and quantifying impacts

Macro evaluation:[1] Hypothesis testing

• Ederveen et al, 2002:“Funds and games: the economics of European cohesion

policy”“Fertile soil for Structural Funds? A panel data analysis

of conditional effectiveness of European cohesion policy”

• Midelfart-Knarvik & Overman, 2002 “Delocation and European integration: is structural

spending justified?”

Ederveen et al, 2002

• Used Barro-type regresions over the period 1960-65 to 1990-95 for 13 EU countries.

• Found no statistically significant “cohesion policy” effect (except for Ireland!)

• Critique: Looks only for growth impacts (i.e., ignores “level” impacts); inappropriate data sample; crude panel-regression model)

Mitelfart-Knarvik & Overman, 2002

• Focused on role of cohesion policy on industrial location, as it affects the interplay between agglomeration and dispersion forces

• Finds that cohesion policy influences endowments, but endowments do not appear to feed through to changes in production structure

• Ireland also an outlier, due to pre-cohesion policy investment in human capital.

Macro evaluation:[2] Modelling causality and impacts

• Be aware of the “built-in” limitations of the type of model selected: I-O, CGE, growth, macro-sectoral

• Implement an appropriate level of sectoral disaggregation on the production side

• Nest cohesion policy mechanisms within wider domestic and global drivers of growth

• Address the difficult issue of model “calibration”

The uses of the macro models

• Constructing internally consistent medium-term

baseline scenarios or forecasts

• Analysis of conventional policy shocks (external

environment, domestic policy, etc.)

• Analysis of complex policy shocks like a EU

Structural Funds)

Key issues arising from the macromodel-based research

• Need to have a more explicit treatment of FDI

• Need to evaluate carefully fiscal and monetary crowding out mechanisms

• Need to incorporate migration mechanisms, and treat labour inputs in more detail.

Contexts for Structural Fund impact analysis

• The model as a global framework for economic analysis

• The model as an explanatory framework for the study of growth and cohesion

• The model as an action framework for SF impact analysis

Construction phase vs Use phasein cohesion policy

• During construction phase, there will be large demand-side impacts. These vanish after completion (i.e., after 2013/15 for next NSRF)

• Increased stocks of infrastructure and human capital can generate long-tailed supply-side impacts

• The size of the supply-side impacts depend on the appropriateness and effectiveness of the NSRF

Physical infrastructure: PI• Demand-side impacts (implementation):

PI IG I (total investment) (Keynesian multiplier) impact on GDP

• Supply-side impacts (mainly post-implementation):

PI increased stock of infrastructure (KPI) boost to output/productivity

Human resources: HC• Demand-side impacts (implementation):

HC Income & Public expenditure Keynesian multiplier GDP

• Supply-side impacts (mainly post-implementation):

HC stock of human capital (KHC) boost to output/productivity

A serious methodological challenge

• Ex-ante impact analysis of “yet-to-be-implemented” NDPs

• Is the NDP appropriate? How effective will be the implementation?

• Strict monitoring and evaluation can help, but do not guarantee success

Infrastructure and human capital interaction effects

• The links between infrastructure and human capital are difficult to measure.

• A parallel improvement in both is probably necessary

• But we cannot say much about the optimum balance between them within an NDP

What macromodel?QUEST versus HERMIN

• QUEST: quarterly; one-sector; model-consistent expectations; no CEE models of new EU member states

• HERMIN: annual; four-sector (+); auto-regressive expectations; applied to “old” EU and new EU member states

HERMIN versus QUEST

• The issue of “crowding out” CEE economies operating below capacity

Public goods and externalities

Modest domestic co-finance requirement

Quasi-euro zone, so no monetary impacts

Presenting model-based cohesion policy impacts

• Difficult to define an appropriate counterfactual baseline scenario.

• Difficult to assign values to the spill-over (or externality) elasticities to different countries in he absence of empirical research.

• Macro impacts are complex, and GDP is an imperfect indicator

Long –run impact of cohesion policy

• Policy impacts build up gradually over time, so use accumulated change in GDP relative to the no-policy baseline

• Big SF injection implies big shock, so normalise SF expenditure as a percentage of GDP

• Define the SF cumulative multiplier as the accumulated percentage change in GDP compared to the no-poicy baseline caused by a one-percent of GDP SFshock

Ordinary policy multiplier

Change in GDP----------------------------------Change in public investment

Cumulative policy multiplier

Cumulative percentage change in GDP-------------------------------------------------------Cumulative percentage share of SFs in GDP

 

Evolution of accumulated SF injection (as % of GDP) and theaccumulated percentage increase in the level of GDP:

Czech Republic: NDP 2007-2013

0

10

20

30

40

50

60

70

80

90

100

Cum SF as % GDP Cum % incr GDP

 

Evolution of the Czech cumulative SF multiplier

Cumulative Multiplier

0

0.5

1

1.5

2

2.5

3

Classifying performance:NDP 2007-2013

• Star performers: Czech Republic, Slovenia, Estonia, Poland, Portugal

• Average performers: Latvia, Romania, Spain, Hungary

• Under performers: East Germany, the Italian Mezzogiorno, Greece

What explains differences in outcomes?

• A common set of “implementation” and “effectiveness” parameters

• Nimble Small Open Economies? Estonia, Slovenia, Czech Republic

• Structures oriented towards growth (Polish “eagle”, Portugal)

• Need for a “bottom-up” analysis (measure => operational programme => CSF)

• Mix of measures vital; also institutional & organizational abilities

Some conclusions 1. Structural Funds, on their own, will never produce

cohesion (for example, of the dramatic Irish variety)

2. However, returns to well-designed and effectively implemented NDPs are probably high

3. Micro-evaluation studies have not been systematic4. The macro “testing” literature conclusions are

probably overly negative and pessimistic5. The HERMIN/QUEST macro-modelling studies

and mechanisms may understate the potential for accelerated convergence

Towards a more constructive debate

• The Commission’s Cohesion Reports need to draw on available analytical research (micro and macro), even when critical

• Empirical approaches (“testing” and “impact evaluation”) can always be improved, but only examine limited aspects of cohesion

• Analysis needs to be broadened to include insights from industrial strategy and other policy frameworks (Vernon, Porter, Best),