How can we know if EU cohesion policy is successful?Integrating micro and macro approaches to
the evaluation of Structural Funds
John Bradley
EMDA (Economic Modelling and Development Systems)
Copy of paper available on
www.gefra-muenster.de
National Development Plans and Structural Funds
• Large-scale investment aid for physical infrastructure, human resources, production incentives
• EC and local (public & private) co-finance• Targeted at lagging EU member states• Massively expanded after 1989• Implemented through multi-year National
Development Plans
RegionalGDP 2001
Part I Situation
and trends
GDP per head(PPS), 2001
< 5050 - 7575 - 9090 - 100100 - 125>= 125No data
Index EU 25= 100
Source: Eurostat
Two aspects of cohesion policy
• Policy design and implementation: usually a pragmatic process driven by local political wishes with local and EC oversight
• Implementation success depends on institutional capabilities and fiscal constraints
• Policy evaluation: (ex-ante, mid-term and ex-post)
• Logically the two aspects should be inter-linked. In practice they tend not to be (but ESRI “Investment Priorities” an exception)
Structural Fund impact evaluation:
micro versus macro techniques
Micro (bottom-up) Macro (top-down)
Level of disaggregation
High (individual projects) Low (sectoral aggregates, whole economy)
Use of theory Weak (judgemental, CBA)
Strong (macroeconomics)
Model calibration Judgemental/informal Scientific(?)/econometric
Policy impacts Informal/implicit/ranking/some quantification
Formal/explicit/quantified
Treatment of externalities
Limited or ignored Included/explicitly modelled
How can we know if EU cohesion policy is successful?
Integrating micro and macro approaches to the evaluation of Structural Funds
(Bradley, Mitze, Morgenroth & Untiedt, March 2006)
Paper available on: www.gefra-muenster.de
Implementing a micro-based approach to evaluation
• Welfare economics and the underlying rationale for public expenditure
1. Public goods 2. Corrective pricing (due to presence of
externalities)3. Targeted interventions (information
asymmetries)4. Redistribution (agriculture, social housing: but
mainly through tax and social welfare system)
Public goods: evaluation criteria
Corrective pricing: evaluation criteria
Targeted interventions: evaluation criteria
Redistribution: evaluation criteria
Two strands to the macro debate on cohesion policy effectiveness
• A political-economic literature that stands back from technical analysis, but argues in terms of theoretical paradigms
• An empirical literature that examines the issues empirically, using a variety of different analytical models.
Impact of recent research in economics
• New Trade Theory (Helpman & Krugman, 1985)
• New Growth Theory (Grossman and Helpman, 1991)
• New Economic Geography (Fujitsa, Krugman, Venables, 1999)
Empirical studies of cohesion:Two methodological approaches
• [1] “Testing” methodologies:
Testing a null hypothesis (e.g., cohesion policy has no effect)
• [2] “Impact evaluation” methodologies: Tracing out complex causal chains of policy consequences, and quantifying impacts
Macro evaluation:[1] Hypothesis testing
• Ederveen et al, 2002:“Funds and games: the economics of European cohesion
policy”“Fertile soil for Structural Funds? A panel data analysis
of conditional effectiveness of European cohesion policy”
• Midelfart-Knarvik & Overman, 2002 “Delocation and European integration: is structural
spending justified?”
Ederveen et al, 2002
• Used Barro-type regresions over the period 1960-65 to 1990-95 for 13 EU countries.
• Found no statistically significant “cohesion policy” effect (except for Ireland!)
• Critique: Looks only for growth impacts (i.e., ignores “level” impacts); inappropriate data sample; crude panel-regression model)
Mitelfart-Knarvik & Overman, 2002
• Focused on role of cohesion policy on industrial location, as it affects the interplay between agglomeration and dispersion forces
• Finds that cohesion policy influences endowments, but endowments do not appear to feed through to changes in production structure
• Ireland also an outlier, due to pre-cohesion policy investment in human capital.
Macro evaluation:[2] Modelling causality and impacts
• Be aware of the “built-in” limitations of the type of model selected: I-O, CGE, growth, macro-sectoral
• Implement an appropriate level of sectoral disaggregation on the production side
• Nest cohesion policy mechanisms within wider domestic and global drivers of growth
• Address the difficult issue of model “calibration”
The uses of the macro models
• Constructing internally consistent medium-term
baseline scenarios or forecasts
• Analysis of conventional policy shocks (external
environment, domestic policy, etc.)
• Analysis of complex policy shocks like a EU
Structural Funds)
Key issues arising from the macromodel-based research
• Need to have a more explicit treatment of FDI
• Need to evaluate carefully fiscal and monetary crowding out mechanisms
• Need to incorporate migration mechanisms, and treat labour inputs in more detail.
Contexts for Structural Fund impact analysis
• The model as a global framework for economic analysis
• The model as an explanatory framework for the study of growth and cohesion
• The model as an action framework for SF impact analysis
Construction phase vs Use phasein cohesion policy
• During construction phase, there will be large demand-side impacts. These vanish after completion (i.e., after 2013/15 for next NSRF)
• Increased stocks of infrastructure and human capital can generate long-tailed supply-side impacts
• The size of the supply-side impacts depend on the appropriateness and effectiveness of the NSRF
Physical infrastructure: PI• Demand-side impacts (implementation):
PI IG I (total investment) (Keynesian multiplier) impact on GDP
• Supply-side impacts (mainly post-implementation):
PI increased stock of infrastructure (KPI) boost to output/productivity
Human resources: HC• Demand-side impacts (implementation):
HC Income & Public expenditure Keynesian multiplier GDP
• Supply-side impacts (mainly post-implementation):
HC stock of human capital (KHC) boost to output/productivity
A serious methodological challenge
• Ex-ante impact analysis of “yet-to-be-implemented” NDPs
• Is the NDP appropriate? How effective will be the implementation?
• Strict monitoring and evaluation can help, but do not guarantee success
Infrastructure and human capital interaction effects
• The links between infrastructure and human capital are difficult to measure.
• A parallel improvement in both is probably necessary
• But we cannot say much about the optimum balance between them within an NDP
What macromodel?QUEST versus HERMIN
• QUEST: quarterly; one-sector; model-consistent expectations; no CEE models of new EU member states
• HERMIN: annual; four-sector (+); auto-regressive expectations; applied to “old” EU and new EU member states
HERMIN versus QUEST
• The issue of “crowding out” CEE economies operating below capacity
Public goods and externalities
Modest domestic co-finance requirement
Quasi-euro zone, so no monetary impacts
Presenting model-based cohesion policy impacts
• Difficult to define an appropriate counterfactual baseline scenario.
• Difficult to assign values to the spill-over (or externality) elasticities to different countries in he absence of empirical research.
• Macro impacts are complex, and GDP is an imperfect indicator
Long –run impact of cohesion policy
• Policy impacts build up gradually over time, so use accumulated change in GDP relative to the no-policy baseline
• Big SF injection implies big shock, so normalise SF expenditure as a percentage of GDP
• Define the SF cumulative multiplier as the accumulated percentage change in GDP compared to the no-poicy baseline caused by a one-percent of GDP SFshock
Ordinary policy multiplier
Change in GDP----------------------------------Change in public investment
Cumulative policy multiplier
Cumulative percentage change in GDP-------------------------------------------------------Cumulative percentage share of SFs in GDP
Evolution of accumulated SF injection (as % of GDP) and theaccumulated percentage increase in the level of GDP:
Czech Republic: NDP 2007-2013
0
10
20
30
40
50
60
70
80
90
100
Cum SF as % GDP Cum % incr GDP
Evolution of the Czech cumulative SF multiplier
Cumulative Multiplier
0
0.5
1
1.5
2
2.5
3
Classifying performance:NDP 2007-2013
• Star performers: Czech Republic, Slovenia, Estonia, Poland, Portugal
• Average performers: Latvia, Romania, Spain, Hungary
• Under performers: East Germany, the Italian Mezzogiorno, Greece
What explains differences in outcomes?
• A common set of “implementation” and “effectiveness” parameters
• Nimble Small Open Economies? Estonia, Slovenia, Czech Republic
• Structures oriented towards growth (Polish “eagle”, Portugal)
• Need for a “bottom-up” analysis (measure => operational programme => CSF)
• Mix of measures vital; also institutional & organizational abilities
Some conclusions 1. Structural Funds, on their own, will never produce
cohesion (for example, of the dramatic Irish variety)
2. However, returns to well-designed and effectively implemented NDPs are probably high
3. Micro-evaluation studies have not been systematic4. The macro “testing” literature conclusions are
probably overly negative and pessimistic5. The HERMIN/QUEST macro-modelling studies
and mechanisms may understate the potential for accelerated convergence
Towards a more constructive debate
• The Commission’s Cohesion Reports need to draw on available analytical research (micro and macro), even when critical
• Empirical approaches (“testing” and “impact evaluation”) can always be improved, but only examine limited aspects of cohesion
• Analysis needs to be broadened to include insights from industrial strategy and other policy frameworks (Vernon, Porter, Best),
