+ All Categories
Home > Documents > Evidence Review 7 Transport · 2019. 1. 16. · Evidence Review: Transport - July 2015 7 To...

Evidence Review 7 Transport · 2019. 1. 16. · Evidence Review: Transport - July 2015 7 To...

Date post: 18-Aug-2020
Category:
Upload: others
View: 0 times
Download: 0 times
Share this document with a friend
42
Evidence Review 7 Transport July 2015
Transcript
Page 1: Evidence Review 7 Transport · 2019. 1. 16. · Evidence Review: Transport - July 2015 7 To determine policy priorities The Centre’s reviews consider a specific type of evidence

Evidence Review 7

Transport

July 2015

Page 2: Evidence Review 7 Transport · 2019. 1. 16. · Evidence Review: Transport - July 2015 7 To determine policy priorities The Centre’s reviews consider a specific type of evidence

Contents

Preface 3

ExecutiveSummary 4

Introduction 8

Impactevaluation 11

Methodology 14

Definition 17

Findings 20

Summaryoffindings 31

References 38

AppendixA:Findingsbyoutcome 39

AppendixB:EvidenceReviewed 40

00

Page 3: Evidence Review 7 Transport · 2019. 1. 16. · Evidence Review: Transport - July 2015 7 To determine policy priorities The Centre’s reviews consider a specific type of evidence

Evidence Review: Transport - July 2015 3

01

Preface

Thisreportpresentsfindingsfromasystematicreviewofevaluationsofthelocaleconomicimpactoftransport.Itcoversevidenceonroads,rail(includinglightrailandsubways),trams,buses,cyclingandwalking–areasofexpenditurewhichaccountforthemajorityoftransportschemesconsideredbylocaldecisionmakers.Evidenceonportsandairportswillbecoveredinafurtherreport.

ThisreportistheseventhreviewproducedbytheWhatWorksCentreforLocalEconomicGrowth.Ourreviewsconsideraspecifictypeofevidence–impact evaluation–thatseekstounderstandthecausaleffectofpolicyinterventionsandtoestablishtheircost-effectiveness.Toputitanotherwaytheyask‘didthepolicywork’and‘diditrepresentgoodvalueformoney’?Withthisreviewweareparticularlyinterestedindemonstratingthatthelocaleconomicimpactsoftransportcanberigorouslyevaluatedandindrawingoutthewiderlessonsforpolicy–includingquestionsofschemeappraisalandprioritisation.

Evidenceonimpactandeffectivenessisacrucialinputtogoodpolicymaking.Inthecaseoftransportthemainaimisnotnecessarilytoimprovethelocaleconomy.However,policymakersoftenclaimeconomicbenefitsfortheseinterventions,andsoitisimportanttoundertakeeconomicimpactevaluationtounderstandiftheseclaimsarejustified.Otherwaysofconsideringtheimpactoftransport(e.g.casestudies)provideavaluablecomplementtoimpactevaluation,butwedonotfocusontheseinthisreport.

We see these impact-focused reviews as an essential part of more effective policy making.Weoftensimplydonotknowtheanswerstomanyofthequestionsthatmightreasonablybeaskedwhenimplementinganewpolicy–notleast,doesitwork?Figuringoutwhatwedoknowallowsustomakebetterdecisionsandtostartfillingthegapsinourknowledge.This also helps us to have more informed discussions and to improve policy making.

Thesereviewsthereforerepresentafirststepinimprovingourunderstandingofwhatworksforlocaleconomicgrowth.Inthemonthsahead,wewillbeworkingwithlocaldecisionmakersandpractitioners,usingthesefindingstohelpthemgeneratebetterpolicy.

HenryOverman;Director,WhatWorksCentreforLocalEconomicGrowth

Page 4: Evidence Review 7 Transport · 2019. 1. 16. · Evidence Review: Transport - July 2015 7 To determine policy priorities The Centre’s reviews consider a specific type of evidence

Evidence Review: Transport - July 2015 4

Executive Summary

Thisreportpresentsfindingsfromasystematicreviewofevaluationsofthelocaleconomicimpactoftransportprojects.Itcoversevidenceonroads,rail(includinglightrailandsubways),trams,buses,cyclingandwalking.Evidenceonportsandairportswillbeconsideredinafurtherreport.ThisreviewistheseventhproducedbytheWhatWorksCentreforLocalEconomicGrowth.

Thereviewconsideredmorethan2,300policyevaluationsandevidencereviewsfromtheUKandotherOECDcountries.Itfound29impactevaluationsthatmettheCentre’sminimumstandards.

ApproachTheCentreseekstoestablishcausalimpact–anestimateofthedifferencethatcanbeexpectedbetweentheoutcomeforareasthatbenefitfromtransportinvestmentandtheaverageoutcometheywouldhaveexperiencedwithoutinvestment(seeFigure1).OurmethodologyforproducingourreviewsisoutlinedinFigure2.

02

Figure 1: Evaluating impactEvaluating impact

VS

Change inoutcome for those

with improved transport

Change inoutcome for those without improved

transport

Page 5: Evidence Review 7 Transport · 2019. 1. 16. · Evidence Review: Transport - July 2015 7 To determine policy priorities The Centre’s reviews consider a specific type of evidence

Evidence Review: Transport - July 2015 5

Findings

Thissectionsummarisesthedetailedfindings.Weemphasisethatmanyofthesefindingsdependonasmallnumberofstudies.Theyare,however,consistentwithotherresearchontheeconomicimpactoftransportimprovements.

What the evidence shows

• Roadprojectscanpositivelyimpactlocalemployment.Buteffectsarenotalwayspositiveandamajorityofevaluationsshowno(ormixed)effectsonemployment.

• Roadprojectsmayincreasefirmentry(eitherthroughnewfirmsstartingup,orexistingfirmsrelocating).However,thisdoesnotnecessarilyincreasetheoverallnumberofbusinesses(sincenewarrivalsmaydisplaceexistingfirms).

• Roadprojectstendtohaveapositiveeffectonpropertyprices,althougheffectsdependondistancetotheproject(andtheeffectscanalsovaryovertime).

• Theimpactofroadsprojectsonthesizeofthelocalpopulationmayvarydependingonwhethertheprojectisurban,suburbanorrural.

• Thereissomeevidencethatroadprojectshavepositiveeffectsonwagesorincomes.

• Thereissomeevidencethatroadprojectshaveapositiveeffectonproductivity.

• Railprojectstendtohaveapositiveeffectonpropertyprices,althougheffectsdependondistancetotheproject(andtheeffectscanalsovaryovertime).

Where there is a lack of evidence

• Wefoundnohighqualityevaluationsthatprovideevidenceontheimpactofrailinfrastructureonemployment,andonlyalimitednumberofevaluationsshowingthatroadprojectshaveapositiveeffect.

Figure 2: Methodology

government

34

5

1 & 2

user panelacademic panel

1scope

2search

3sift

4score

5

synthesis

To identify what works, each policy review finds and evaluates the evidence which is robust and demonstrates clear outcomes in a 5 stage process

Evaluation evidence is collected using a wide range of sources

Each study is scored based on the quality of

method and quality of implementation

The full set of evidence is refined based on its relevance and the robustness of the research method

Conclusions drawn are based on a combination of these findings and existing literature

academiathinktanks

call forevidence

Existing literature and evidence is reviewed on the basis of an agreed review question,

specific search terms, and a set of inclusion criteria

Page 6: Evidence Review 7 Transport · 2019. 1. 16. · Evidence Review: Transport - July 2015 7 To determine policy priorities The Centre’s reviews consider a specific type of evidence

Evidence Review: Transport - July 2015 6

• Wefoundnohighqualityevaluationsthatprovideevidenceontheimpactsoftrams,buses,cyclingandwalkingschemesonanyeconomicoutcomes.

• Evenwhenstudiesareabletoidentifyapositiveimpactonemployment,theextenttowhichthisisasaresultofdisplacementfromothernearbylocationsisstillunresolved.Moregenerally,thespatialscaleofanyemploymenteffectsvariesandwedonothaveenoughevidencetobeabletogeneraliseaboutthespatialdistributionofeffectsiftheyoccur.Thesameistrueforotheroutcomes.ThescaleatwhichthestudiesevaluateimpactvariesfromadjacentneighbourhoodstomuchlargerUScounties.

• Surprisingly,veryfewevaluationsconsidertheimpactoftransportinvestmentonproductivity(wefoundjustthreestudies,twoforroadsandoneforrail).Althoughtheuseofsuchproductivityeffectstocalculate‘widereconomicbenefits’intransportappraisalisunderpinnedbyalargerevidencebase,itisstillworryingthatsofewevaluationscandemonstratethattheseeffectsoccurinpractice.

• Wehavelittleevidencethatwouldallowustodrawconclusionsonwhetherlarge-scaleprojects(e.g.highspeedrailormotorwayconstruction)havelargereconomicgrowthimpactsthanspendingsimilaramountsonacollectionofsmall-scaleprojects(e.g.lightrailorjunctionimprovements).

• Moregenerally,wedonotknowhowdifferencesinthenatureofimprovements(e.g.journeytimesavedornumberofadditionaljourneys)affectlocaleconomicoutcomes.

• Thereissomeevidencethatcontextmatters.Forexample,propertypriceeffectsmaydependonthetypeofproperty,whilewageeffectsmaydifferbetweenlowskilledandhighskilledworkers.But,onceagainwedonothaveenoughevidencetobeabletogeneralise.

How to use these reviewsTheevidencereviewhighlightsanumberoffactorsforpolicymakerstobeawareofwhenconsideringtransportpolicy:

• Muchmoreempiricalworkremainstobedoneonunderstandingtheimpactofinfrastructureimprovementsonlocaleconomicgrowth.Theeconomicbenefitsoftransportinfrastructurespending–particularlyasamechanismforgeneratinglocaleconomicgrowth–arenotasclear-cutastheymightseemonfacevalue.

• WhileitisunderstandablethatpoliticaldebatefocusesonexpenditurefiguresacrossdifferentpartsoftheUK,theydonothelpanswerthequestionofwhatwouldhappenifexpenditurewasdistributeddifferently.Argumentsforspendingmoreinareasthatarelesseconomicallysuccessfulhingeonthehopethatnewtransportisacost-effectivewaytostimulateneweconomicactivity.Asthisreviewshows,wedonotyethaveclearanddefinitiveevidencetosupportthatclaim.

• Thesefindingsraisefundamentalquestionsaboutschemeappraisalandprioritisation,andabouttheroleofimpactevaluationinimprovingdecision-makingaroundtransportinvestment.SomepreliminaryrecommendationsbaseduponourworkwithDfTandLEPsareoutlinedinsection8ofthefullreport.

Page 7: Evidence Review 7 Transport · 2019. 1. 16. · Evidence Review: Transport - July 2015 7 To determine policy priorities The Centre’s reviews consider a specific type of evidence

Evidence Review: Transport - July 2015 7

To determine policy priorities

TheCentre’sreviewsconsideraspecifictypeofevidence–impactevaluation–thatseekstounderstandthecausaleffectofpolicyinterventionsandtoestablishtheircost-effectiveness.Inthelongerterm,theCentrewillproducearangeofevidencereviewsthatwillhelplocaldecisionmakersdecidethebroadpolicyareasonwhichtospendlimitedresources.Figure3illustrateshowthereviewsrelatetotheotherworkstreamsoftheCentre.

HelpingtofilltheevidencegapsAsshouldbeclearfromthisreview,therearemanythingsthatwedonotknowaboutthelocaleconomicimpactofinfrastructure.Tohelpfilltheseevidencegaps,thefinalpartofthereviewprovidesanumberofrecommendationsaimedatimprovingtheevaluationandappraisaloftransportschemes.

TheCentre’slongertermobjectivesaretoensurethatrobustevidenceisembeddedinthedevelopmentofpolicy,thatthesepolicesareeffectivelyevaluatedandthatfeedbackisusedtoimprovethem.Toachievetheseobjectiveswewantto:

• workwithlocaldecisionmakerstoimproveevaluationstandardssothatwecanlearnmoreaboutwhatpolicieswork,where.

• setupaseriesof‘demonstrationprojects’toshowhoweffectiveevaluationcanworkinpractice.

Interestedpolicymakerspleasegetintouch.

Evidence reviews

Demonstrationprojects

You are here

Capacitybuilding

Understanding what works

More effective policy

Capacitybuilding

Capacitybuilding

Figure 3: What Works Centre work programme

Page 8: Evidence Review 7 Transport · 2019. 1. 16. · Evidence Review: Transport - July 2015 7 To determine policy priorities The Centre’s reviews consider a specific type of evidence

Evidence Review: Transport - July 2015 8

Introduction

Thisreviewlooksatthelocaleconomicimpactsoftransportinvestment.Itcoversevidenceonroads,rail(includinglightrailandsubways),trams,buses,cyclingandwalking–areasofexpenditurewhichaccountforthemajorityoftransportspendingthatwillbeconsideredbylocaldecisionmakers.

Transportinfrastructureandservicesarefundamentalpublicgoodsthataffectthewaysocietiesandeconomiesfunction.Localdecisionmakerswillwanttotakemanyfactorsintoaccountwhendecidinglocaltransportpolicy,butourfocusisonthenarrowerissueofunderstandingtheeconomicimpact.

Therearetwomaineconomicaimsoftransportspending.First,toreducetransportcoststobusinessesandcommuters(forexamplebyreducingcongestion–andthussavingtime-orbyreducingfares).Second,andrelated,tostimulatetheUKandlocaleconomies,forexample,byraisingtheproductivityofexistingfirmsandworkersorbyattractingnewfirmsandprivatesectorinvestment.Tomeetthesepolicyaimsrequiresanunderstandingofwhetherwearespendingenoughandontherightthings.

Tohelpanswerthisquestion,thisreviewsummarisessomeofthekeytheoriesandevidenceregardingtheimpactoftransportontheeconomy–withaparticularfocusonthelessonsthatwecandrawfromthelimitednumberofavailableimpactevaluations.

Thebasicmessagethatemergesfromthisreviewisthattheeconomicbenefitsoftransportinfrastructurespending–particularlyasamechanismforgeneratinglocaleconomicgrowth-arenotasclear-cutastheymightseemonfacevalue.Inturn,thisraisesfundamentalquestionsaboutschemeappraisalandprioritisationandabouttheroleofimpactevaluationinimprovingdecisionmakingaroundtransportinvestment.Thelatterpartofthisreviewaddressessomeoftheserelatedquestions.

The economic aims of transport spendingForacountryliketheUKwithawell-developedtransportnetwork,wecanidentifytwokeypolicyaims(Gibbons,2015).Thefirstistorespondtogrowingdemandsothatincreasedcongestion,longertraveltimesandhighercoststoproducersandconsumers,donotconstraingrowth.Onthebasisofthiskindof“ameliorative”argument,weshouldinvestmoreinplaceswheretheeconomyandtransportdemandisgrowing.

03

Page 9: Evidence Review 7 Transport · 2019. 1. 16. · Evidence Review: Transport - July 2015 7 To determine policy priorities The Centre’s reviews consider a specific type of evidence

Evidence Review: Transport - July 2015 9

Oneconcernwiththisapproachisthatmakingtraveleasierinthiswaysimplyencouragesmoretravel.Ifthishappens,itmaydivertresourcesfromotherplacesandsectors,withlittleeconomicgainandbigenvironmentalcosts.Anotherconcernisthatthiskindofpolicymayexacerbatespatialinequalitiesbytargetingresourcesatplaceswhicharealreadyprosperousandgrowing.

Asecondaimoftransportspendingistostimulatelocaleconomies.Thatis,todrivegrowthinthelocaleconomy,ratherthanjustrespondtoit.Argumentsforgreaterinvestmenttomeetthisobjectivearebasedontheideathatlowertransportcostsallowforthemoreefficientallocationofexistingresources.Forexample,aconsiderablebodyofevidencesuggeststhatconnectingpeople,firmsandplacesmorecloselygenerates“agglomerationeconomies”,whichincreaseproductivity.Loweringtransportcostsalsoincreasesprivatesectorreturnsandthismaystimulateinvestment.

Buildingontheseideas,anumberofrecentreportshavearguedforgreaterinvestmenttostimulatenationalgrowth,andalsototacklespatialdisparitieswithintheUK(e.g.CityGrowthCommission2014).Tomeetthelatterobjective,suchreportsarguethatweshouldtargetmoreresourcestoplaceswhereeconomicperformanceislagging,inordertostimulategrowth.

ThehighprofileEddingtonReviewoftheUK’stransportnetworkfocusedmoreonthefirstoftheseissues.Ithighlightedtheproblemsofcongestionandthepotentialeconomicbenefitsofanimprovedsystemestimatingthata5%reductionintraveltimesnationallywouldbewortharound0.2%ofGDPannually(Eddington2006).ThereportarguedthattheUKwasalreadywellinterconnected,andrecommendedthatimprovementsshouldfocusonincreasingtheperformanceoftheexistingnetworkthroughmanagementandpricing.

ThekeypolicyprioritiestheEddingtonReviewidentifiedweregrowingandcongestedareas,urbanareas,andmajorcongestedinter-citylinks.Accordingtothisanalysis,transportinfrastructureinvestmentshouldaimtorelaxtheconstraintsthatacongestedsystemimposesontravelandbusinesscosts.Investmentshouldbetargetedtoplaceswherethereisgrowingdemandfortransport,implyingthatinvestmentshouldflowtothefastestgrowingcitiesandregions.

TheLSEGrowthCommission(Aghionetal2013)echoedmanyoftheseconclusions,andproposedasetofnewindependentinstitutionstounblockmajortransportinfrastructureplanningdecisions–includingaStrategyBoardtodeterminelong-terminfrastructureplans(thenratifiedbyParliament),aCommissiontodeliverthisplan(includinggenerouscompensationforloserstodeflectNimbyism)andanInfrastructureBanktohelpwithbothfinanceandprivateexpertise.

AstheLSEGrowthCommissionreportdemonstrates,littlehaschangedaboutourunderstandingoftheinteractionsbetweentransportandtheeconomysincetheEddingtonreportwaswritten.However,since2007theGreatRecessionhasledtoarenewedfocusondisparitiesbetweenmajorcities(Londoninparticular)andtherestofthecountry.Inturn,thishasraisedquestionsabouttheextenttowhichtransportinvestmentcouldhelpnarrowthesedisparities.Forexample,arecentreportbyIPPR(CoxandDavies2013)onregionalinfrastructureissueshighlightedstarkdifferencesinplannedspendingperpersonindifferentregions,andarguedforgreaterspendinginlaggingareasintheNorthofEngland.1Recentreportssuchasthesehaveonceagainraisedthequestionofwhetherwecanstimulateeconomicactivity–locally,regionallyornationally–throughinfrastructure

1 Theextentofdisparitiesdependscruciallyonwhethertheexpenditurefiguresusedforcomparisonincludeonlypublicinvestmentorarebasedontotalinvestmentwherethereissomeelementofpublicsupport.Disparitiesinthelatterlookmuchlargerthandisparitiesinpublicinvestmentalone.Itisalsoimportanttonotethathistoricaldisparities–whichunderpintoday’sdifferencesineconomicperformance-aremuchsmallerthandisparitiesinplannedspending.Finally,differentwaysofpresentingthesefigureseliminateorevenreversethesedisparities.SeeGibbons(2015)forfurtherdiscussion.

Page 10: Evidence Review 7 Transport · 2019. 1. 16. · Evidence Review: Transport - July 2015 7 To determine policy priorities The Centre’s reviews consider a specific type of evidence

Evidence Review: Transport - July 2015 10

investment,ratherthansimplytargetingittomeetunderlyingdemand.Thisquestioniscentraltoourunderstandingoftheroleoftransportinvestmentinimprovinglocaleconomicgrowthandisthemainfocusoftheremainderofthisreview.

The effect of transport investment on local economic growthTherearetwowaysofstructuringourthinkingaboutthelikelyeconomicimpactofinfrastructureinvestments.Thefirstviewspublicsectorinfrastructureinvestmentasprovidingacapitalstockthatiscomplementarytoprivatesectorphysicalcapital(i.e.machinesandbuildings)andtohumancapital(i.e.skills).Thesecondthinksofinfrastructureasprovidinganetworkthatconnectsdifferentplacessothatpublicsectorinvestmentreducesthetransportcostsbetweenplaces.

Thefirstwayofthinkingsuggeststhatprovidingmoreinfrastructurewillalwaysimprovearealevelproductivity(Jones,2013).Ofcourse,infrastructurecanbeveryexpensivesotheseproductivitybenefitsmightbeoutweighedbythecostsofprovision.Thisdisparitybetweenproductivitybenefitsandcostsmaybeparticularlyacutewheninfrastructureisusedtotrytoturnaroundstrugglinglocaleconomies.Becauseinfrastructureisdurable,placesthathaveseenslowgrowthwilltendtohaverelativelylargeamountsofinfrastructureperperson.Theconcretemanifestationofthisarerelativelylowcongestionlevelsinpoorlyperformingcities.Economictheory–supportedbyempiricalevidence-suggeststhataddingfurthertransportinvestmentinthoseplacesmaynotdomuchtoimproveproductivity.2Incontrast,investingincongestedplaceswilltendtodeliverhigherreturnsbecausethecongestionreflectsthefactthattheseplaceshavelowinfrastructureperperson.Ofcourse,thesearegeneraltendencieswhichdon’truleoutthepossibilitythatspecificprojectsmayhavelargerimpactsinpoorlyperformingcities(andvice-versa).

Thesecondwayofthinkingaboutinfrastructure–asanetworkthatconnectsdifferentplaces–providesmoremixedmessages;particularlywhenitcomestobetterconnectingrichandpoorregionsregions(Baldwin,etal;2005).Onewaytothinkaboutthesetypesoftransportinvestmentistoviewenhancedintegrationasawayofincreasingtheeffectivesizeofthelocaleconomies.Asalargerlocaleconomymeanshigheragglomerationeconomiesthisshouldhelpfirmsbemoreproductive.

Therearetwoimportantcaveatsconcerningthislineofreasoning.First,theavailableempiricalevidencesuggeststhatagglomerationeconomiesmayattenuatequitequicklywithdistance.Itisnotclear,therefore,whetherconnectingdifferentcitieswillalwaysgeneratesignificantagglomerationbenefits.

Second,loweringtransportcostsmayencouragefirmstomoveintotherichermarketandservetheircustomersfromthere.This‘twowayroadsproblem’ispoorlyunderstood,leadingsomepolicymakerstofocussolelyonthebenefitstothepoorermarket–ratherthanthinkingthroughthe‘threats’fromgreatercompetition.

Aswillbecomeclearfromtheevidencereviewedbelow,muchmoreempiricalworkremainstobedoneonunderstandingtheimpactofinfrastructureimprovementsonlocaleconomicgrowth.Theoreticalanalysiscertainlyurgescautioninassumingthatinfrastructureinvestmentcanstimulategrowthinpoorlyperformingareas.Inshort,whileinfrastructureinvestmentmaybevitallyimportantforgrowingcities,itsroleinstimulatinggrowthisnotasclear-cutasassumedbymanydecisionmakers.

2 Thisisbecauseinvestmentsinphysicalcapitalarelikelytobesubjectto‘diminishingmarginalreturns’.Thismeansthat,whenaplacehaslotsofcapitalperpersonaddingextracapitalwillnotdomuchtoincreaseproductivity.See,forexample,Solow(1956)andthelargeeconomicgrowthliteraturethatbuildsonthiswork.

Page 11: Evidence Review 7 Transport · 2019. 1. 16. · Evidence Review: Transport - July 2015 7 To determine policy priorities The Centre’s reviews consider a specific type of evidence

Evidence Review: Transport - July 2015 11

Impact evaluation

Governmentsaroundtheworldincreasinglyhavestrongsystemstomonitorpolicyinputs(suchasspendingoninfrastructureprovision)andoutputs(suchasthenumberandspeedofjourneysmadeonanewroad).However,theyarelessgoodatidentifyingpolicyoutcomes(suchasthewidereffectoftransportonlocalemployment).Inparticular,manygovernment-sponsoredevaluationsthatlookatoutcomesdonotusecrediblestrategiestoassessthecausalimpactofinfrastructureinvestment(henceforth,werefertotheseas‘projects’).

Bycausalimpact,theevaluationliteraturemeansanestimateofthedifferencethatcanbeexpectedbetweentheoutcomeforareasundertakingaproject(inthiscase,improvingtransportprovision)andtheaverageoutcometheywouldhaveexperiencedwithouttheproject.Pinningdowncausalityisacruciallyimportantpartofimpactevaluation.Estimates of the benefits of a project are of limited use to policy makers unless those benefits can be attributed, with a reasonable degree of certainty, to that project.

Thecredibilitywithwhichevaluationsestablishcausalityisthecriteriononwhichthisreviewassessestheliterature.

Using CounterfactualsEstablishing causality requires the construction of a valid counterfactual–i.e.whatwouldhavehappenedtoanarea(orpartofanarea)iftheprojecthadn’thappened.Thatoutcomeisfundamentallyunobservable,soresearchersspendagreatdealoftimetryingtorebuildit.Thewayinwhichthiscounterfactualis(re)constructedisthekeyelementofimpactevaluationdesign.

A standard approach is to create a counterfactual group of similar places not undertaking the kind of project being evaluated. Changesinoutcomescanthenbecomparedbetweenthe‘treatmentgroup’(locationsaffectedbyimprovedtransportation)andthe‘controlgroup’(locationsnotaffected).Aswediscussbelow,inthecaseoftransportprovision,suchtreatmentandcontrolgroupsarenotalwayseasytoidentify.

04

Page 12: Evidence Review 7 Transport · 2019. 1. 16. · Evidence Review: Transport - July 2015 7 To determine policy priorities The Centre’s reviews consider a specific type of evidence

Evidence Review: Transport - July 2015 12

A key issue in creating the counterfactual group is dealing with the ‘selection into treatment’ problem.Selectionintotreatmentoccurswhenlocationsthatundergotransportimprovementsdifferfromthosewhodonotdoso.

Anexampleofthisproblemfortransportprojectswouldbewhenagovernmentfocusestransportinvestmentonitsbestperformingcities.Ifthishappens,estimatesofpolicyimpactmaybebiasedupwardsbecauseweincorrectlyattributebettereconomicoutcomestotheproject,ratherthantothefactthatthecityisalreadyperformingbetterthanaverage.

Selectionproblemsmayalsoleadtodownwardbias.Forexample,ifalocalauthorityprojectexplicitlytargetsslowgrowingareasfortransportimprovementsthenwemaymistakenlyattributepooreconomicperformancetotheprojectratherthantounderlyingconditionsinthearea.

Thesefactorsareoftenunobservabletoresearchers.So the challenge for good programme evaluation is to deal with these issues, and to demonstrate that the control group is plausible.Iftheconstructionofplausiblecounterfactualsiscentraltogoodpolicyevaluation,thenthecrucialquestionbecomes:how do we design counterfactuals? Box1providessomeexamples.

Box 1: Impact evaluation techniques

Onewaytoidentifycausalimpactsofaprojectistorandomlyassignparticipantstotreatmentandcontrolgroups.Forresearchers,suchRandomised Control Trials(RCTs)areoftenconsideredthe‘goldstandard’ofevaluation.Properlyimplemented,randomisationensuresthattreatmentandcontrolgroupsarecomparablebothintermsofobservedandunobservedattributes,thusidentifyingthecausalimpactoftheproject.However, implementation of these ‘real world’ experiments is challenging and can be problematic. RCTsmaynotalwaysbefeasibleforlocaleconomicgrowthpolicies–forexample,policymakersmayunderstandablybeunwillingtorandomisethelocationofprojects.3

Whererandomisedcontroltrialsarenotanoption,‘quasi-experimental’approachesofrandomisationcanhelp.Thesestrategiescandealwithselectiononunobservables,by(say)exploitinginstitutionalrulesandprocessesthatresultinsomelocationsquasi-randomlyundertakingprojects.

Evenusingthesestrategies,though,thetreatmentandcontrolgroupsmaynotbefullycomparableintermsofobservables.StatisticaltechniquessuchasOrdinary Least Squares(OLS)andmatching canbeusedtoaddressthisproblem.

Notethathigherqualityimpactevaluationfirstusesidentificationstrategiestoconstructacontrolgroupanddealwithselectiononunobservables.Thenittriestocontrolforremainingdifferencesinobservablecharacteristics.Itisthecombinationthatisparticularlypowerful:OLSormatchingaloneraiseconcernsabouttheextenttowhichunobservablecharacteristicsdeterminebothtreatmentandoutcomesandthusbiastheevaluation.

3 Gibbons,NathanandOverman(2014).

Page 13: Evidence Review 7 Transport · 2019. 1. 16. · Evidence Review: Transport - July 2015 7 To determine policy priorities The Centre’s reviews consider a specific type of evidence

Evidence Review: Transport - July 2015 13

Evidence included in the review

Weincludeanyevaluationthatcomparesoutcomesforareasimprovingtransportprovision(thetreatedgroup)aftertheprojectwithoutcomesinthetreatedgroupbeforetheproject;relativetoacomparisongroupusedtoprovideacounterfactualofwhatwouldhavehappenedtotheseoutcomesintheabsenceoftheproject.

Thismeanswelookatevaluationsthatdoareasonablejobofestimatingtheimpactoftheprojectusingeitherrandomisedcontroltrials,quasi-randomvariationorstatisticaltechniques(suchasOLSandmatching)thathelpmaketreatmentandcontrolgroupscomparable.Weviewtheseevaluationsasprovidingcredibleimpactevaluationinthesensethattheyidentifyeffectsthatcanbeattributed,withareasonabledegreeofcertainty,totheprojectinquestion.AfulllistofshortlistedstudiesisgiveninAppendixB.

Evidence excluded from the review

Weexcludeevaluationsthatprovideasimplebeforeandaftercomparisononlyforthoseplacesundertakingtransportprojectsbecausewecannotbereasonablysurethatchangesforthetreatedgroupcanbeattributedtotheeffectoftheproject.

Wealsoexcludecasestudiesorevaluationsthatfocusonprocess(howtheprojectisimplemented)ratherthanimpact(whatwastheeffectoftheproject).Suchstudieshavearoletoplayinhelpingformulatebetterpolicybuttheyarenotthefocusofourevidencereviews.

Page 14: Evidence Review 7 Transport · 2019. 1. 16. · Evidence Review: Transport - July 2015 7 To determine policy priorities The Centre’s reviews consider a specific type of evidence

Evidence Review: Transport - July 2015 14

Methodology

ToidentifyrobustevaluationevidenceonthecausalimpactoftransportweconductedasystematicreviewoftheevidencefromtheUKandacrosstheworld.Ourreviewfollowedafive-stageprocess:scope,search,sift,scoreandsynthesise.

Stage 1: Scope of Review WorkingwithourUserPanelandamemberofourAcademicPanel,weagreedthereviewquestion,keytermsandinclusioncriteria.Wealsousedexistingliteraturereviewsandmeta-analysestoinformourthinking.

05

Figure 1: Methodology

government

34

5

1 & 2

user panelacademic panel

1scope

2search

3sift

4score

5

synthesis

To identify what works, each policy review finds and evaluates the evidence which is robust and demonstrates clear outcomes in a 5 stage process

Evaluation evidence is collected using a wide range of sources

Each study is scored based on the quality of

method and quality of implementation

The full set of evidence is refined based on its relevance and the robustness of the research method

Conclusions drawn are based on a combination of these findings and existing literature

academiathinktanks

call forevidence

Existing literature and evidence is reviewed on the basis of an agreed review question,

specific search terms, and a set of inclusion criteria

Page 15: Evidence Review 7 Transport · 2019. 1. 16. · Evidence Review: Transport - July 2015 7 To determine policy priorities The Centre’s reviews consider a specific type of evidence

Evidence Review: Transport - July 2015 15

Stage 2: Searching for EvaluationsWesearchedforevaluationevidenceacrossawiderangeofsources,frompeer-reviewedacademicresearchtogovernmentevaluationsandthinktankreports.Specifically,welookedatacademicdatabases(suchasEconLit,WebofScienceandGoogleScholar),specialistresearchinstitutes(suchasCEPRandIZA),UKcentralandlocalgovernmentdepartments,andworkdonebythinktanks(suchastheOECD,ILO,IPPRandPolicyExchange.)Wealsoissuedacallforevidenceviaourmailinglistandsocialmedia.Thissearchfoundjustover2,300books,articlesandreports(thefulllistofsearchtermscanbefoundonlinehere:whatworksgrowth.org/policies/transport/search-terms).

Stage 3: Sifting EvaluationsWescreenedourlong-listonrelevance,geography,languageandmethods,keepingimpactevaluationsfromtheUKandotherOECDcountries,withnotimerestrictionsonwhentheevaluationwasdone.WefocusedonEnglish-languagestudies,butwouldconsiderkeyevidenceifitwasinotherlanguages.Wethenscreenedtheremainingevaluationsontherobustnessoftheirresearchmethods,keepingonlythemorerobustimpactevaluations.WeusedanadjustedversionoftheMarylandScientificMethodsScale(SMS)todothis.4TheSMSisafive-pointscalerangingfrom1,forevaluationsbasedonsimplecrosssectionalcorrelations,to5forrandomisedcontroltrials(seeBox2).WeshortlistedallthoseimpactevaluationsthatcouldpotentiallyscorethreeoraboveontheSMS5.Inthiscasewefoundnoevaluationsscoringfive:forexamplesofimpactevaluationsthatscorethreeorfourontheSMSscaleseethecasestudiesandourscoringguideavailableat:www.whatworksgrowth.org/resources/scoring-guide.

Stage 4: Scoring EvaluationsWeconductedafullappraisalofeachevaluationontheshortlist,collectingkeyresultsandusingtheSMStogiveafinalscoreforevaluationsthatreflectedboththequalityofmethodschosenandqualityofimplementation(whichcanbelowerthanclaimedbysomeauthors).Scoringandshortlistingdecisionswerecross-checkedwiththeacademicpanelmembersandthecoreteamatLSE.Thefinallistofincludedstudiesandtheirreferencenumbers(usedintherestofthisreport)canbefoundinAppendixB.

Stage 5: Synthesising EvaluationsWedrewtogetherourfindings,combiningmaterialfromourevaluationsandtheexistingliterature.

4 Sherman,Gottfredson,MacKenzie,Eck,Reuter,andBushway(1998).5 Shermanetal.(1998)alsosuggestthatlevel3istheminimumlevelrequiredforareasonableaccuracyofresults.

Page 16: Evidence Review 7 Transport · 2019. 1. 16. · Evidence Review: Transport - July 2015 7 To determine policy priorities The Centre’s reviews consider a specific type of evidence

Evidence Review: Transport - July 2015 16

Box 2: Our robustness scores (based on adjusted Maryland Scientific Methods Scale)

Level 1:Either (a) across-sectional comparison of treated groups with untreated groups, or (b) a before-and-after comparison of treated group, without an untreated comparison group.Nouseofcontrolvariablesinstatisticalanalysistoadjustfordifferencesbetweentreatedanduntreatedgroupsorperiods.

Level 2:Use of adequate control variables and either (a) across-sectional comparison of treated groups with untreated groups, or (b) a before-and-after comparison of treated group, without an untreated comparison group. In(a), controlvariablesormatchingtechniquesusedtoaccountforcross-sectionaldifferencesbetweentreatedandcontrolsgroups.In(b),controlvariablesareusedtoaccountforbefore-and-afterchangesinmacrolevelfactors.

Level 3:Comparison of outcomes in treated group after an intervention, with outcomes in the treated group before the intervention, and a comparison group used to provide a counterfactual (e.g. difference in difference). Justificationgiventochoiceofcomparatorgroupthatisarguedtobesimilartothetreatmentgroup.Evidencepresentedoncomparabilityoftreatmentandcontrolgroups.Techniquessuchasregressionand(propensityscore)matchingmaybeusedtoadjustfordifferencebetweentreatedanduntreatedgroups,buttherearelikelytobeimportantunobserveddifferencesremaining.

Level 4:Quasi-randomness in treatment is exploited, so that it can be credibly held that treatment and control groups differ only in their exposure to the random allocation of treatment.Thisoftenentailstheuseofaninstrumentordiscontinuityintreatment,thesuitabilityofwhichshouldbeadequatelydemonstratedanddefended.

Level 5: Reserved for research designs that involve explicit randomisation into treatment and control groups, with Randomised Control Trials (RCTs) providing the definitive example. Extensiveevidenceprovidedoncomparabilityoftreatmentandcontrolgroups,showingnosignificantdifferencesintermsoflevelsortrends.Controlvariablesmaybeusedtoadjustfortreatmentandcontrolgroupdifferences,butthisadjustmentshouldnothavealargeimpactonthemainresults.Attentionpaidtoproblemsofselectiveattritionfromrandomlyassignedgroups,whichisshowntobeofnegligibleimportance.Thereshouldbelimitedor,ideally,nooccurrenceof‘contamination’ofthecontrolgroupwiththetreatment.

Note:TheselevelsarebasedonbutnotidenticaltotheoriginalMarylandSMS.Thelevelsherearegenerallyalittlestricterthantheoriginalscaletohelptoclearlyseparatelevels3,4and5whichformthebasisforourevidencereviews.

Page 17: Evidence Review 7 Transport · 2019. 1. 16. · Evidence Review: Transport - July 2015 7 To determine policy priorities The Centre’s reviews consider a specific type of evidence

Evidence Review: Transport - July 2015 17

Definition

Transportimprovementprojectsarebroadinscope,notonlyintermsofthetransportmode(seebelow),butalsointermsofthetypeofinterventions.Threebroadtypesofinterventionswereconsideredaspartofthisreview:

• Physicalintervention–i.e.theexpansionandimprovementoftransportinfrastructure.Thiscouldeitherincludethebuildingofnewroutesandfacilities,orthroughmakingcapitalimprovementstoexistingones(e.g.increasinghighwaycapacitythroughjunctionupgradesorextralanes).

• Serviceenhancement–i.e.wherethephysicallayoutofthetransportinfrastructureremainsunchangedbutwhereitsqualityisincreased(e.g.improvementstoreliability,increasingservicefrequency).

• Revenueprojects–i.e.changestothewayexistingtransportinfrastructureissuppliedandconsumed.Thiscanbesplitintotwofurthergroups:

• Pricinginterventions/subsidies–e.g.faresubsidies,car-poollanes,congestionchargesetc.

• Sectoralservicechange–changingtheownershiporoperationoftransportservices,e.g.privatisationornationalisation.

Whilstevaluationsfromallthreegroupswereincludedduringthesearchphaseofthereview,ultimatelythemajorityofthearticlesmeetingtheCentre’sstandardsfocusonphysicalinterventionstoexpand/improveinfrastructure.

Tohelporderthelargeamountofliterature(around2,300policyevaluationsandevidencereviews),studiesweresplitbymodeasfollows:

• Road.

• Rail–coveringarangeoftypes,includinghighspeed,regional,urban,andlight(e.g.subway)railinfrastructure.

• Non-railpublictransport–e.g.tramsandbuses.

• Walkingandcycling.

06

Page 18: Evidence Review 7 Transport · 2019. 1. 16. · Evidence Review: Transport - July 2015 7 To determine policy priorities The Centre’s reviews consider a specific type of evidence

Evidence Review: Transport - July 2015 18

• Ports.

• Airports.

• Multi-modal.

Thisreportcoversevidenceonroads,rail(includinglightrailandsubways),trams,buses,cyclingandwalking–areasofexpenditurewhichaccountforthemajorityoftransportspendingthatwillbeconsideredbylocaldecisionmakers.Evidenceonportsandairportswillbeconsideredinafurtherreview.

Impact evaluation for infrastructure projectsAsdiscussedabove,evaluatingtheeconomiceffectsoftransportprojectsischallenging:transportwillaffectmultipleeconomicoutcomesinwaysthatarehardforresearcherstodisentangle.

Therearealsospecificchallengesinundertakinghighqualityimpactevaluation.Itisfairlyeasytounderstandhowwemightconstructcontrolgroupsandundertakeevaluationforpoliciestargetedatindividuals,householdsorfirms.Itishardertothinkabouthowwemightdothisforpolicies–suchasrailandroad–thattargetareas.Inadditiontooursubstantiveinterestintheimpactsofpolicy,oneofourmotivationsinconsideringtransportistohelpconvincedecisionmakersthatbetterevaluationispossible.Thissectionprovidesabriefexplanationofhowthereportsweconsideredhavetriedtodothis.Furtherdetailsonspecificexamplescanbefoundinourscoringguideavailablefromwww.whatworksgrowth.org/resources/scoring-guide.

Evaluationofthelocaleconomicgrowtheffectsoftransportisparticularlychallenging.Theuseofcost-benefitanalysismeansthatmuchinfrastructurespendingoccursinareaswherethereisexpectedtobestrongandgrowingdemand.Oftentheselocationswillalreadybeexperiencingeconomicgrowthandincreasesinjobsandwages–underlyingfactorsthataredrivingthegrowthindemand.Theeffectsoftheseunderlyingfactors(‘selectioneffects’)mustbeaccountedforifwewanttounderstandtheextenttowhichtransportspendingactuallyincreasesgrowth.

Selectionislikelytobeamuchbiggerproblemfortransportprojectsthanforsomeofourpreviousreviewsthatconsideredsimilarareabasedpolicies.Forexample,whenreviewingtheeffectsofsportsandculturalfacilitiesorofestaterenewal,economicfactorsmayoftenbeoneconsiderationamongmanywhenmakingdecisionsonprojects.However,fortransportprojects,economicfactorsarelikelytobeacoreconsideration.Forthisreason,treatedareasarealmostalwayslikelytobedifferenttountreatedareas.Someofthesedifferenceswillbehardtoobserveinavailabledata,makingitverydifficulttoconstructanappropriatecontrolgroup.Furthermore,itisunlikelythattheseunderlyingdifferenceswillbeconstantovertime.

Inmanycircumstancesevaluationscould,inprinciple,userandomisedcontroltrialstoaddresstheseconcernsoverselection.Forcapitalexpenditure,whereinvestmentsaredurable,itishardtoimaginesituationsinwhichtruerandomisationofprojectplacementwouldbeeitherfeasibleordesirable.Thismeansthatweneedtorelyonalternativeevaluationapproachestotrytoaddresstheproblemofselectionandthusidentifythecausalimpactoftransportinvestment.

Manystudiesinthisreviewattempttoaddressthese‘selectionproblems’usingvariationsondifference-in-differenceorpanelfixedeffectsmethods.Inthesemethods,thechangeinoutcomeinthe‘treatment’areas(thosethatundertakeprojects)iscomparedwiththechangeinoutcomeinagroupofsimilarcontrolareas(whichdonot).Thecontrolgroupisconstructedtobesimilartothetreatmentgroupeitherbymatchingonobservedcharacteristicsorbyusingcontrolvariables.By

Page 19: Evidence Review 7 Transport · 2019. 1. 16. · Evidence Review: Transport - July 2015 7 To determine policy priorities The Centre’s reviews consider a specific type of evidence

Evidence Review: Transport - July 2015 19

takingabefore-and-afterdifference,thismethodeliminatesallfixedunobservabledifferencesbetweenthetreatmentandcontrolgroups.However,asalreadydiscussed,therearealsolikelytobetime-varyingunobservabledifferencesthatleadtoinvestmentintransportinfrastructure.Thesemethodscannotaccountfortheseunderlyingfactors.

Inordertoallowfortheseunobservablefactors,andthusmorereliablyassesstheimpactoftransportprojectsitisimportanttoexploitsomesourceofrandomnessinthewaytransportinfrastructureisdelivered.Althoughtheoverallnumberofevaluationswehaveavailableissmall,aroundonethirdofthemhaveattemptedtousemethodsthatexploitsomesourceofrandomness.Thisisalargersharethanformanyofourotherreviews(itisaboutthesameasforbroadband),reflectingtheimportanceofsuchmethodsforevaluationinthispolicyarea.

Forexample,study1067looksattheeffectofhighwaysonemploymentandwagesofskilledlabourusinganinstrumentbasedonthefactthattheUShighwaysystemwasplannedalongagridpattern.6Thismeansthathighwaysaremorelikelytorunthroughruralcountriesthataredirectlynorth,south,eastorwestofthenearestmajorcity.Thisarbitraryfeatureofthesystemprovidesquasi-randomvariationinthedeliveryofroadstotheruralcountiesthatcanbeexploitedtoestimateacausaleffect.Onaverage,countieslyingdirectlynorthofamajorcityarenotexpectedtobedifferentfromcountieslyingsaynorth-eastofamajorcityapartfromthefacttheyaremorelikelytoreceiveahighway.Therefore,anydifferenceintheemploymentandwagesofskilledlabourmaymoreconfidentlybeattributedtotheeffectsofthehighwayinfrastructure.

Inasecondexample,paper1017examinestheeffectofroadsonfirmperformance,exploitingthefactthatthefirstfewsectionsofaninter-citymotorwayprovideimprovedaccessbetweenlocationswithinalocalarea.Sincetheintercityconnectionisprovidedtoincreaseaccessbetween,ratherthanwithinlocalareas,thislocalimprovementisconsideredquasi-random.Thereforeanyimprovementsinfirmperformancefortheimprovedareascomparedwithsimilarunimprovedareasinthesamelocalareacanbeattributedtotheeffectoftheroad.

Thesemethodsarepotentiallytheonlywaytoachievereliableestimatesoftheimpactoftransportinvestmentonlocaleconomicgrowthoutcomes.Futuretransportevaluationsshouldpaycloseattentiontotechniquesusedinstudiessuchasthese,anissuetowhichwereturnbelow.

6 AllstudynumbersrefertospecificevaluationsaslistedinAppendixB:EvidenceReviewed.

Page 20: Evidence Review 7 Transport · 2019. 1. 16. · Evidence Review: Transport - July 2015 7 To determine policy priorities The Centre’s reviews consider a specific type of evidence

Evidence Review: Transport - July 2015 20

Findings

Thissectionsetsoutthereview’sfindings.Webeginwithadiscussionoftheevidencebase,andthenexploretheoverallpatternofresults.Afterthisweconsiderspecificoutcomesinmoredetail.

Thereviewinitiallyconsidered2,300policyevaluationsandevidencereviewsfromtheUKandotherOECDcountries,identifiedduringtheinitialkeywordsearch.Thisisasignificantlylargerstartingevidencebasethanourearlierreviews.

Followingafurtherhighlevelreview,over1,800weresiftedoutasnotrelevant(e.g.becausetheyweretheoreticalratherthandata-based;reviewednon-OECDcountries;orbecauseofsubjectrelevance).Fromtheremainingevaluations,wediscardedover250furtherevaluationsastheywerefoundnottobeeconometricallyrobust.Finally,232studieswereshortlistedfordetailedreview.Theresultsofthatdetailedreviewareoutlinedinthefollowingsections,whichsplittheevaluationsbymode.

ThescaleatwhichthestudiesevaluateimpactvariesfromadjacentneighbourhoodstomuchlargerUScounties.

Roads

Quantity and quality of the evidence base

Ofthe232shortlistedstudiesreviewedindetail,80consideredtheimpactofroadsprojects.

Ofthese80studies,anadditional62studieswerediscounted:Eightongroundsofrelevance,and54ongroundsofnotmeetingtheCentre’sminimumstandardofevidence(i.e.scored2orbelowontheSMSscale).Theremaining17studieshavebeenincludedinthisreview.

Thisisasmallerevidencebasethanmostofourreviewstodate(onemploymenttraining,businessadvice,sportsandcultureprojects,accesstofinanceandestaterenewal)butroughlyonparwithourreviewofbroadband.Asdiscussedabove,thispartlyreflectsthedifficultiesinevaluatingtransportprojectsbutisalsoindicativeofafailuretocarefullyevaluateexistingpolicyinterventions.Table1showsthedistributionofthestudiesrankedbySMSscore.

07

Page 21: Evidence Review 7 Transport · 2019. 1. 16. · Evidence Review: Transport - July 2015 7 To determine policy priorities The Centre’s reviews consider a specific type of evidence

Evidence Review: Transport - July 2015 21

Table 1: Implementation Quality Scores

SMS Score No. of studiesEvaluation reference

numbers

3

12 1009,1011,1015,1016,1027,1031,1035,1050,1052,1055,1061,1062

4 5 1005,1017,1037,1063,1067

Total 17

Wefoundnostudiesthatusedrandomisedcontroltrials,butfivestudiesthatusedcrediblerandomsourcesofvariation.Asdiscussedintheprevioussection,thisisnotthatsurprisinggiventhenatureoftheseprojects.Theremaining12studiesusedvariationsofdifference-in-differenceandpanelmethods(scoring3ontheSMS).Thetechniquesappliedinthesestudiesmeanthatwecanbereasonablyconfidentthattheyhavedoneagoodjobofcontrollingforobservablecharacteristicsofareas,individualhouseholdsandfirmsaffectedbytheprojects.However,itislikelythatunobservablecharacteristicsmaystillbeaffectingtheresults.

Type and Focus of Support

Inmostofourpreviousevidencereviewswehavefocusedonspecificpolicyinterventionsaimedatdeliveringparticularobjectives(e.g.governmentfundedemploymenttraininginourfirstreview).Incontrast,thevastmajorityofstudiesinthisreviewfocusonevaluatingtheimpactofspecificinvestmentprojectsoroverallspendingratherthanevaluatingaspecificpolicywithexplicitobjectivesandrationales.Thisisunsurprisinggiventhenatureofmosttransportinvestment.

Themajorityofprogrammeswerepubliclyfunded(eitheratanational,localorEUlevel).Oftheevaluations:

• Twostudiesevaluatenamedpolicieswhichprovidedfundsforroadbuildingandimprovementprojects:

• EuropeanRegionalDevelopmentFund–Trans-EuropeanNetworks(TENs),EU.7

• TheuseofEuropeanStructuralFundstoupgraderoadsinSpain.8

• Tenevaluationsexaminedconstructionandimprovementworksrelatedtoroadnetworksgenerally:

• TheexpansionofPortugal’smotorwaynetworkintoeconomicallylaggingregions.9

• TheexpansionoftheUnitedStatesInterstateHighwaySystem.10

• InvestmentintheSpanishroadsnetworkgenerally.11

• RoadconstructionandimprovementprojectsintheUnitedKingdomgenerally.12

• DevelopmentofroadsintheUnitedStatesgenerally.13

7 Study1055.8 Study1016.9 Study1061.10 Study1067.11 Study1062.12 Study1017.13 Study1037.

Page 22: Evidence Review 7 Transport · 2019. 1. 16. · Evidence Review: Transport - July 2015 7 To determine policy priorities The Centre’s reviews consider a specific type of evidence

Evidence Review: Transport - July 2015 22

• ThegrowthofroadnetworksinthestateofMinnesota,USA.14

• RoadwideningprojectsinthestateofNorthCarolina,USA.15

• OnestudyreferstotheUnitedStates1947highwayplanasaproxyfordatasetconstruction.16

• Anotherstudyfocussesonfederalroad-buildingintheUnitedStatesfollowingtheHighwayAct1944andtheInterstateHighwayAct1956.17

• Onestudyfocussesontollroads,constructedinOrangeCountyinCalifornia,USAandoperatedbyTransportationCorridorAgencies.18

• Fivestudiesfocusontheconstructionofspecificroadsorroadnetworks:

• TheopeningofInterstate105inCalifornia,USA.19

• TheconstructionandopeningoftheInterstate210extensioninCalifornia,USA.20

• TheconstructionandopeningofStateRoutes87,85and237inLosAngeles,USA.21

• TheextensionofPresidentGeorgeBushTurnpikeandDallasNorthTollwaytollroadsinDallas,USA.22

• TheconstructionoftheM6andM60highwaysinHungary.23

Oftheseventeenstudiesontheroadsfinalshortlist,onlyonefocussesontheeffectsofroadconstructionandimprovementsintheUK.Themajorityofstudies(eleven),examineprogrammesintheUSA.TheremainingstudiesevaluateprogrammesinSpain(two),PortugalandHungarywithonestudyexaminingprogrammesthroughouttheEU.

Findings by outcomeAbreakdownofthestudiesbyoutcomeandoverallfindingisprovidedinthetablesinAppendixA.

Employment

Road projects can positively impact local employment. But effects are not always positive and a majority of evaluations show no (or mixed) effects on employment.

Table 2: Road investment evaluations by outcome on employment

Outcome No. of studiesEvaluation reference

numbersPositive 2 1011,1017

Zero 3 1027,1031,1067

Mixed 1 1015

14 Study1027.15 Study1031.16 Study1063.17 Study1005.18 Study1009.19 Study1011.20 Study1035.21 Study1015.22 Study1050.23 Study1052.

Page 23: Evidence Review 7 Transport · 2019. 1. 16. · Evidence Review: Transport - July 2015 7 To determine policy priorities The Centre’s reviews consider a specific type of evidence

Evidence Review: Transport - July 2015 23

Ifweareinterestedintherolethattransportmayplayindrivingeconomicgrowth,thenacentralquestionistheextenttowhichprojectscausechangesinemployment.Sixevaluationsconsidertheimpactofroadprojectsonlocalemployment.Ofthese,twoevaluationsfindpositiveimpacts,threefindnoimpactandoneevaluationshowsmixedresults.

Ofthetwoevaluationsreportingpositiveeffects,onefindsimpactsthatarerelativelylarge:employmentinthetreatmentgroupincreasedby200%against10%inthecontrolgroupoverthestudyperiod,between1980and1997.24Theauthorssuggestthatthestrengthoftheseeffectsmayreflectnegativespillovers–i.e.positiveeffectsonareasalongthehighwaycorridor(the‘treatment’group),accompaniedbylossesforareasatagreaterdistance(the‘control’group).Movingjobsaroundisnotthesameascreatingjobs,therefore,thisissueofdisplacementshouldbeamajorconcernforlocaldecisionmakersinterestedindistinguishingbetweentotalandadditionaleconomicgrowth.

Forthe‘noimpact’evaluations,onestudyfoundtherelationshipbetweenroadnetworkexpansionandemploymentgrowthdisappearedaftercontrollingforlocationalfactorssuchashumancapitallevelsandtaxrates.25Similarlyonestudyfoundthatevenaftercapacityenhancements,thehighwaynetworkquicklybecamesaturatedwithtraffic.26Employmentbetween1985and1997remainedunchangedin‘treated’countieswithanincreaseindensityofhighwaylane-milesduringthatperiod.

Themixedresultlookedatseveralcasestudieswhichshowedincreasing,staticanddecreasingtotalemploymentintheareasaroundhighwayexpansionorimprovementprogrammesinthreeneighbouringCaliforniacounties.27

Firm Entry and Number of Businesses

Road projects may increase firm entry, although not necessarily the overall number of businesses (as new entrants may displace existing firms).

Table 3: Road investment evaluations by outcome on firm entry

Outcome No. of studiesEvaluation reference

numbersPositive 2 1017,1061

Zero 1 1016

Whenemploymenteffectsarepositive,thismaybedrivenbybothexpansionofexistingfirmsandentryofnewfirms.Evenintheabsenceofemploymentgrowth,effectsonfirmentry,exitandtheoverallnumberofbusinessmaybeofinteresttolocaldecisionmakers.

Threeevaluationsconsidertheseeffects,withtwofindingpositiveeffectsandonefindingzeroeffect.Thetwostudiesreportingpositiveeffectsbothlookatfirmentry.28Inoneofthesestudies,plantbirthwasaffectedpositivelyacrossmostsectorswithin10kmofnewmotorways,althoughtheeffectwaslargestforsectorsrequiringproximitytomarketsandclientssuchasprimaryindustries.29Theshareofindustrialsectorplantbirthsinmunicipalitieswithin10kmofmotorwaysincreasedatamuchhigherratethantheshareofservicesectorfirmbirthsinthesamemunicipalitiesbetween1986and1997.The

24 Study1011.25 Study1027.26 Study1031.27 Study1015.28 Studies1017and1061.29 Study1061.

Page 24: Evidence Review 7 Transport · 2019. 1. 16. · Evidence Review: Transport - July 2015 7 To determine policy priorities The Centre’s reviews consider a specific type of evidence

Evidence Review: Transport - July 2015 24

authorsattributethisdifferenceineffectacrosssectorstoapossiblegreaterrelianceontransportbymanufacturingfirms.Thepositiveeffectstendedtodecreasebeyond10km,withzeroeffectsbeyond50km.Again,thisraisesthepossibilitythatthereissomedisplacementtoareasnexttotheproject,fromareasclosetobutnotnexttotheproject.Thesecondpositivestudy30consideredbusinessaccessibilitytoroadimprovementsbyUKelectoralwardsandfoundthatwardsincloseproximitytoroadimprovementsrecordedanincreaseinnewplantsoverthestudyperiod.Inthisevaluationitwasfoundthata10%improvementinaccessibilityleadstoa3%increaseinthenumberofbusinessesandemploymentupto30kmfromthesiteoftheimprovement.Incontrasttothesetwostudies,thethirdstudy,foundthatnationalroadcapacityinSpainhadnoeffectonthetotalnumberoffirms.31

Property Prices

Road projects tend to have a positive effect on property prices, although the effect in prices may depend on distance to the project (and the effects can vary over time).

Table 4: Road investment evaluations by outcome on property prices

Outcome No. of studiesEvaluation reference

numbersPositive 3 1009,1035,1052

Mixed 1 1050

Fourevaluationsconsiderpropertyprices.Threefindpositiveimpactswithoneshowingmixedresults.

Twoofthestudieslookataveragehousepricesin‘treated’areasandfindpositiveimpactsonhousepricesrelativeto‘untreated’areas.

Theothertwostudiessuggestthatpriceeffectsdependondistancetotheroadproject(consistentwiththehedonicpricingliteraturethatlooksatthelinkfrompropertycharacteristicstoprices).Housesclosetotheprojectdonotexperiencethesamepositivepricerisesasthoseclose,butnotimmediatelyadjacentto,theproject.Theymayevendepreciate.Thesizeofthe‘buffer’zoneinwhichthesenon-positiveeffectsoccurvaries:inonestudynegativeeffectsarepresentupto0.2miles,butpositivefrom0.25milesaway32;whileasecondstudyshowsoverallpositiveeffectsonpropertypricesinalltreatmentareasalbeitwithslightlysmallerincreasesupto0.4milesfromtheintervention.33

Oneofthetwoevaluations(study1035)thatconsideredthespatialpatternofdistanceeffectsalsolookedatwhetherthesechangedovertime.Itfoundlittleevidenceofannouncementeffects(2yearspriortoconstruction)butpriceswerealreadyincreasingclosetotheprojectinthefirstthreeyearsofthefiveyearconstructionperiod.34Afterprojectcompletion,thegreatestpriceappreciationwas0.4-0.8milesawayduringthefirst3yearsfollowingcompletion,0.8-1.2milesaway4yearsfollowingcompletionwitheffectsdisappearinginthefifthyear.

30 Study1017.31 Study1016.32 Study1050showsnegativeeffectsofaround10%upto0.2milesfromthetoll-roadcorridor,whilepricesincrease13%

at0.25-1miledistantand19%at1-2milesdistant.33 Study1035showsthathouses0.4milesfromthehighwayarebetween$22,000and$33,000moreexpensivethan

thoseadjacent,withthesepositiveeffectsdiminishingaspropertiesarelocatedfurtherfromthehighway.34 Theseeffectsdisappearedforyears4and5ofconstruction,whichtheauthorsspeculatethiscanbeattributedto

increasednoiseexternalityduringthatperiod.

Page 25: Evidence Review 7 Transport · 2019. 1. 16. · Evidence Review: Transport - July 2015 7 To determine policy priorities The Centre’s reviews consider a specific type of evidence

Evidence Review: Transport - July 2015 25

Thegeneralfindingthatpriceeffectsvarybydistancefromtheprojectandtimefromconstructionmayapplytoothercontexts(andasalreadynotedthedistanceeffectisconsistentwiththewiderhedonicliterature).35Thiscouldhaveimportantimplicationsforpredictingtheimpactofschemesandforincorporatinglandpriceupliftinappraisalandevaluation.Wereturntothisissueintheconclusions.

Population

The impact of roads projects on local population may vary depending on whether the project is urban, suburban or rural.

Threeevaluationsconsideredtheimpactofroadsonlocalpopulation.Allthreeevaluationslookedattheeffectofoverallroadinvestmentsratherthanaspecificproject.

Study1005foundthatanewhighwaypassingthroughacitycentreleadstoan18%fallinpopulation,whileeach‘ray’(ahighwaysegmentconnectingtheCentralBusinessDistrictwiththesuburbs)causesa9%drop.Thisimpliesthattheconstructionofaroadmayleadtosuburbanisation.

Consistentwiththis,study1015foundpositiveeffectsonhousingdevelopment(andhencepopulation)fornon-urbanareaswithin0-0.75milesofnewroads.36

Study1027comestoasimilarconclusionthatthelinkagebetweenhighwayinfrastructureandgrowthpatternsvariesdependingonthetypeofimprovementandcharacteristicsofthelocation.Inurbansettings,highwaysmayleadtopopulationdecline37whileinasuburbanorruralcontext,populationincreasesclosetothehighway38.

Aswiththepropertypriceeffects,itishardtoknowwhethertheseresultsgeneralise.However,aswiththeemploymentresults,theyemphasisethefactthatlocaleffectsofroadprojectsneednotnecessarilybepositive.

Income/Wages

There is some evidence that road projects have positive effects on wages/income.

Onlytwoevaluationsconsideredtheimpactofroadconstructiononincomeand/orwageswithonestudyfindingpositiveeffects,theotherreportingmixedfindings.39

Thepositiveeffectsinstudy1017varywiththeextentofchangesinaccessibility.Within20kmofnewroadconstructionprojects(includingnewjunctions,dualling,widening,upgradesandroadconstruction)afirmexperiencingthemeanincreaseinaccessibilitysawa0.2%averageincreaseinwages(calculatedastotalwagebillperworker).

Study1067reportsmoremixedfindingsforruralcountiesinthevicinityoftheUnitedStatesInterstateHighwaySystem.Countieswithahighendowmentofskilledworkerssawincreasesinwages,whilethosewithalowproportionofskilledworkerssawdecreases.

35 Howeveritisunlikelythattheexactpatternofeffectsreportedinthesetwoevaluationswillgeneralise:indeed,study1050showthattheyvarydependingonlocalcontextandtheparticularstretchofroadevaluated.

36 Theseeffectscanbequitelarge.Inonearea(inMercedCounty),therewas65,501moresquarefeetofhousingconstructedperkilometersquaredwithin0-0.75milesfromthehighway.Note,however,thatthiswaspartlyoffsetbynegativeeffects2.7-3milesfromthehighway.

37 Study1005.38 Study1015.39 Studies1017and1067.

Page 26: Evidence Review 7 Transport · 2019. 1. 16. · Evidence Review: Transport - July 2015 7 To determine policy priorities The Centre’s reviews consider a specific type of evidence

Evidence Review: Transport - July 2015 26

Itishardtogeneralisefromtheseresults–althoughstudy1017istheonlystudythatfocusesonroadprojectsintheUK.Theydoprovidesome,albeitlimited,evidencethatproductivityeffects(whichunderpinUKWebTAGcalculationsofwiderbenefits)occurinpractice.Theseproductivityeffectsarealsoconsidereddirectlyintwofurtherstudieswhichweconsidernext.

Productivity

There is some evidence that road projects have a positive effect on productivity.

Twoevaluationsconsidertheimpactofroadsonproductivity.Onestudyfindsthat,ingeneral,theconstructionoforimprovementtomajorroadsleadstoa0.4%upliftinGVAperworker(thougha0.2%increaseinworkers’wagessuggeststhatpartofthisproductivitygainispaidoutinincreasedsalaries–thesefindingsarecomplementary).40

Thesecondstudyalsofindspositiveeffectsonprovincialproductivitywithstrongereffectsforareasintensiveinsectorsthataremoredependentonroads(e.g.manufacturingandlogistics).41Roadusebyprovincialindustriesisproxiedbytheseindustries’vehicleintensitywiththestudyfindingaparticularincreaseinuseofroadsbyindustrialsectors.

Alongwiththeresultsonwages,thisprovidesmoredirectevidencethattheproductivityeffectsthatunderpinWebTAGappraisalguidanceoccurinpractice.Although,asshouldbeclearfromthediscussionsofar,thenumberofevaluationsthatcandemonstrateacausallinkfromroadprojectstoproductivityisextremelylimited.

Other Outcomes

Twoevaluationsconsideredimpactonbusinessandtradevolume42.Bothfoundpositiveimpacts.Thefirstoftheseobserveda1.4%upliftinthevalueoftradeanda1.9%increaseinthevolumeoftradeforevery1%reductionintraveldistancebetweentradingpartners.43Thesamestudyalsofounda10%increaseinthestockofurbanhighwaysincreasedexportweightby5%(butdidnotinduceanupliftinvalue).Thesecondstudyfoundthattruckingactivityincreasedby7-10percentagepointspercapitainruralcountiescrossedbyhighways.44

Twoevaluationsconsideredtheimpactoninnovation(oneintermsofoutputs,theotherintermsofinputs).ThefirstoftheseconsideredtheimpactofInterstatehighwaysintheUSAonpatentingactivity45.Thefindingssuggestthata10%increaseinaregion’shighwaystockcauseda1.7%increaseinregionalinnovationgrowthoverafiveyearperiod.ThesecondstudyconsideredtheeffectofroadinfrastructureonGDPinNUTS1andNUTS2EUregions.46Theevaluationfoundthattheeconomicperformanceofregionswithagoodendowmentofmotorwayinfrastructureisenhancedwhentheyhave–andaresurroundedbyregionswith–highlevelsofR&Dinvestment.

40 Study1017.41 Study1062.42 Studies1037and1067.43 Study1037.44 Study1067.45 Study1063.46 Study1055.

Page 27: Evidence Review 7 Transport · 2019. 1. 16. · Evidence Review: Transport - July 2015 7 To determine policy priorities The Centre’s reviews consider a specific type of evidence

Evidence Review: Transport - July 2015 27

Rail

Quantity and quality of the evidence base

Ofthe232shortlistedstudiesreviewedindetail,95consideredtheimpactofrailprojects.

Ofthese95studies,anadditional83studieswerediscounted:Eightongroundsofrelevance(e.g.theylookedatanoutcomenotrelatedtolocaleconomicgrowth,suchasridershiplevels),47and65ongroundsofnotmeetingtheCentre’sminimumstandardofevidence(i.e.scored2orbelowontheSMSscale).Theremaining12studieshavebeenincludedinthisreview.

Thisisasmallerevidencebasethanourreviewstodate(onemploymenttraining,businessadvice,sportsandcultureprojects,accesstofinanceandestaterenewalandbroadband)aswellasbeingsmallerthanthatavailableforroads.Aswithroads,thispartlyreflectsthedifficultiesinevaluatingtransportprojectsbutisalsoindicativeofafailuretocarefullyevaluateexistingpolicyinterventions.Table5showsthedistributionofthestudiesrankedbySMSscore.

Table 5: Ranking Studies by Quality of Implementation

SMS ScoreNumber of

studies Evaluation reference

numbers

3 10

1070,1071,1074,1083,1107,1108,1111,1112,

1114,1116

4 2 1075,1109

Total 12

Wefoundnostudiesthatusedrandomisedcontroltrials,buttwostudiesthatusedcrediblerandomsourcesofvariation.Asdiscussedforroads,thisisnotthatsurprisinggiventhenatureoftheseprojects.Theremainingtenstudiesusedvariationsofdifference-in-differenceandpaneltechniques(scoring3ontheSMS).Thetechniquesappliedinthesestudiesmeanthatwecanbereasonablyconfidentthattheyhavedoneagoodjobofcontrollingforobservablecharacteristicsofareasandindividualhouseholdsandfirmsaffectedbytheprojects.However,itislikelythatunobservablecharacteristicsmaystillbeaffectingtheresults.

Types and Focus of Support

Aswithroads,thestudiesincludedinthefinalshortlistdidnotgenerallyevaluatespecificpolicies(e.g.nationalisation/privatisation,co-ordinatedrollingstockimprovementschemesetc.).Instead,theyeither:

• Focussedonindividualnewrailprojectsinspecificlocations.

• Evaluatedtheimpactofaccesstorailmoregenerally.

Theevaluatedprojectsvariedbybothscaleandtype:

• Twoevaluationslookedathighspeedrail.ThefirstlookedattheimpactofnewrailservicesbetweenCologneandFrankfurt,ontwosmalltownsthatgotnewstationsontheline.48The

47 Thisincludedonehighqualitystudy(SMSlevel4)whichevaluatedtheimpactofrailexpansioninnineteenthcenturyAmericaon‘farmimprovement’showingthatcountiesthatgainedaccesstorailwaysbetween1850and1860experiencedsignificantlygreaterincreasesinthepercentageoffarmsthatwereimproved.

48 Study1075.

Page 28: Evidence Review 7 Transport · 2019. 1. 16. · Evidence Review: Transport - July 2015 7 To determine policy priorities The Centre’s reviews consider a specific type of evidence

Evidence Review: Transport - July 2015 28

secondevaluatedtheopeningofahighspeed(Shinkansen)raillineinJapan.49

• Threeevaluationslookedatregionalrail(i.e.connectingdifferentcitiesorwiderregions).50Allthreeofthesestudieslookedatrailaccessgenerally,ratherthanspecificprojects.

• Sevenevaluationslookedatlightrail:

• Study1108and1114bothfocusedonanewlightraillineinCharlotte,NorthCarolina.

• Study1071evaluatedtheHudson-BergenLightRailsysteminNewJersey.

• Study1070evaluatedtheRiverLineraillineinNewJersey.

• Study1083focusedontheMetrorailinMiami.

• Study1111evaluatedlightrailtransitinMinneapolis.

• Study1112studiedtheimpactofextensionstotheDocklandsLightRailwayandJubileeLineinLondon.

Fundinganddeliveryfortheprogrammesarenotstatedinmanyoftheevaluations,butwherethisismentionedtheyareoverwhelminglypubliclyfunded.Similarly,theobjectiveoforrationalefortheinterventionisoftennotreported,thoughinsomecasesitisimpliedthatatleastpartoftherationalewastoboosteconomicgrowth.

Findings by outcomeAbreakdownofthestudiesbyoutcomeandoverallfindingisprovidedinthetablesinAppendixA.

Property Values

Rail projects tend to have a positive effect on property prices, although the size of the effect varies considerably.

Table 6: Rail investment evaluations by outcome on property values

SMS ScoreNumber of

studies Evaluation reference

numbersResidentialPositive 5 1070,1071,1107,1108,1112

Zero 2 1074,1083

CommercialZero 1 1108

Sevenevaluationsconsiderpropertyprices.51Allsevenstudiesconsidertheeffectofproximitytonewrailstationsonresidentialpropertyprices,withstudy1108alsolookingatcommercialpropertyprices.Fiveoutofthesevenstudiesthatconsideredresidentialpropertyfoundpositiveeffectsofproximitytostations,whiletwostudiesfoundnoeffectofproximity.Theonestudythatconsideredcommercialpropertypricesfoundnoeffectsofproximitytostations.

49 Study1117.50 Studies1107,1109,1074.51 Studies1071,1107,1074,1070,1083,1108and1112.Mostevaluationsmeasuredchangesinpropertyvaluesusing

hedonicmodelsappliedtorepeatedsalesdatasets.

Page 29: Evidence Review 7 Transport · 2019. 1. 16. · Evidence Review: Transport - July 2015 7 To determine policy priorities The Centre’s reviews consider a specific type of evidence

Evidence Review: Transport - July 2015 29

Forevaluationsshowingpositiveeffects,thedegreeofpriceappreciationrangedfromextremelysmalltoquitesubstantial.Forexample,Study1108(whichlookedattheimpactoflightrailinCharlotte,NorthCarolina)foundeffectsthatrangedfromnearzerouptoaround13%,dependingon:thetypeofproperty(forexample,condominiumsseeagreaterincreasethansingle-familyproperties);andproximityfromthestation(forexample,single-familyhomeswithinhalfamileofthestationseenoimpact,whilstcondominiumswithinhalfamilearesubjecttoagreaterincreasethanthosefurtheraway).Study1071foundeffectsashighas18.4%.Thishighvariationinpriceeffectsacrossstudiesimpliesasimilarlyhighvariationintheimpliedvalueofimprovementinrailaccessintermsofwillingnesstopayforresidentialhousing.

Incontrast,Study1074foundthataccesstointercityrailconnectionsinpost-unificationBerlinhadnoimpactonpropertyprices.Interestingly,theevaluationalsofoundthatthenewmainlinenetworkhadonaverageanadverseimpactonmainlineaccessibilityatthecitylevel(asaresultoftheallocationoftransportcapacityfavouringsomelinesoverothers,includingthecompletedisconnectionofastationinanareawhichhadservedastheCBDofWestBerlinfordecades),whichmayexplainthelackofeconomicbenefit.Study1083,whichlookedattheimpactoftheMiamiMetrorailonthevalueofhousesnearstationlocations,alsofoundnostatisticallysignificanteffectsonresidentialpropertyprices.

Aswithroads,anumberofevaluationssuggestthatthepriceeffectsdependondistancefromtheproject(consistentwiththehedonicpricingliteraturethatlooksatthelinkfrompropertycharacteristicstoprices).Study1071findspositiveeffectsuptoaquartermilefromthestation,witheffectsdecreasingwithdistance.Study1107foundsimilarevidencethateffectsdecayedwithdistance,buteffectswerestillpositiveupto2.2milesfromthestation.Resultsweresimilarforstudy1070,althoughhereeffectswerepositiveupto4miles.Finally,study1108suggeststhattheeffectofdistancemaydifferbypropertytypewiththelargesteffectsforcondominiumsathalfamile,butthelargesteffectsforone-familyhomesatamile.Incontrasttothefindingsforroads,noneoftheevaluationsreportsmallereffectsforpropertiesveryclosetostations.52

Anumberofevaluationsalsoconsideredthetimingofpricechanges.Forexample,Study1107(intheNetherlands)and1083(inMiami)foundweakevidenceofanannouncementeffect–i.e.theappreciationofpropertypricespost-announcement,butpre-completionoftheproject.Thelengthoftimeoverwhichpriceeffectsareobservedalsovariesbystudy.Somestudiesonlylookateffectsoneortwoyearsaftercompletionofastation(forexample,Studies1071,1047);othersconsideredchangesoveramuchlongertimeperiod(forexample,Studies1070,1112).

Other outcomes

Asidefromtheeffectonpropertyprices,theevaluationevidenceonotherlocaleconomicimpactsisextremelylimited.Wefoundnoevaluationsthatconsideredtheimpactonemployment–whichisstartlinggiventhedegreeofinterestintheexistenceandsizeoftheseeffects.

Oneevaluation(Study1116)lookedattheimpactofanewJapanesehighspeed(Shinkansen)passengerraillineonbusinessperformanceandbusinessproductivity,findingpositiveeffectsinbothareas.Businessperformanceismeasuredbysalesrevenueandbusinessproductivityismeasuredbysalesrevenueperemployee.Theauthorssuggestthatthesebenefitsoccurredasaresultofincreasedaccessandlowersearchcoststoothermarkets,resultinginfirmsbeingabletofindbettersuppliers.Thesebenefitsoccurreddespitethefactthattheinterventiononlyloweredthecost/timeof

52 AlthoughanumberoftheSMS2levelstudiesdidreportsmaller,ornegative,priceeffectsforpropertiesveryclosetoimprovements.

Page 30: Evidence Review 7 Transport · 2019. 1. 16. · Evidence Review: Transport - July 2015 7 To determine policy priorities The Centre’s reviews consider a specific type of evidence

Evidence Review: Transport - July 2015 30

passengertravelanddidnotaffectfreighttransportationcosts.Overall,theresultsindicatedthatforinput-intensivefirmssalesperemployeeincreasedrelativetotheindustryby42%.

Asecondhighspeedrailstudy53lookedattheimpactonGDPfortwosmalltownsthatlieonthenewhighspeedlineconnectingCologneandFrankfurt.Thestudyreportsquitelargeeffectsamountingtoa2.7%increaseintotalGDP.Unfortunately,thecontextthatmakesthisstudyattractivefromananalyticalpointofview(thisstudyscoreslevel4ontheSMSscale)alsomakesthefindingshardtogeneralise.ThesmalltownsofMontabaurandLimburg(populationsof12,571and33,84354)happenedtogetstationsfollowingcomplexnegotiations,despitetheirsmallsizeandperipherality.Thishelpsaddressconcernsabout‘selectionintotreatment’,55butmakesitimpossibletoknowwhethertheseGDPeffectswouldextrapolatetothelargercities(e.g.Birmingham,LondonandManchester)thatwouldtypicallybehometonewhighspeedrailstations.

Theresultsfromtheonlyevaluationthatconsiderspopulation(study1109)aresimilarlyhardtoextrapolate.ThisstudylookedattheimpactofrailwayconstructioninSwedeninthenineteenthcentury,evaluatingitsimpactonthegrowthofcitiessincethattime.Thestudyfoundthatcitieswithearlyaccesstothenetworkcontinuedtogrowfasteroverthefirsthalfofthetwentiethcentury,andthatthetreatmentgroupcitiesareonaverage51%largertodaycomparedtocitiesthatdidnotgainaccesstotherailroadnetworkinthefirstwaveofexpansion.Allofthissuggeststhateffectsmayplayoutovertheverylongterm,butitishardtoknowwhetherthesegeneralisetoadditionstoanalreadyexistingwell-developedrailnetwork.

Perhapsofmoreinteresttolocaldecisionmakersisstudy1111whichexaminedtheimpactofametrolineinMinneapolisonlandusechanges,findingonlysmallandverylocalisedimpact.Specifically,single-familyandindustrialpropertieswithinhalfamileofoperationalstationsexperiencedasmallincreaseinthelikelihoodoflandusechangeawayfromtheseuses.However,onalargerscaletheintroductionofthenewlinedidnotincreasethelikelihoodofchangesinlanduseabovenormallevels,nordidithaveanyeffectonthelikelihoodofchangesinlanduseawayfromvacantland,commercialpropertiesandmulti-purposefamilies.

Afinalstudy56lookedattheimpactofrailonlevelsofcrime.Study1114,lookingattheimpactofnewlightrailtransitinCharlotte,NorthCarolina,foundthattheannouncementoftherailtransitledtoadecreaseinpropertycrimes,whichwasmaintainedevenafterthestationswereopened.Thiswasattributedtopublicandprivatedecisionstoinvestalongtransportcorridors,whichgentrifiedsurroundingneighbourhoodsanddecreasedcriminalactivity.Oncethestationsopened,thedecreaseincrimewasmaintainedanddidnotreturntopre-announcementlevels.Aswithanumberofpreviousreviews,thisfindingservestohighlightthefactthatinfrastructureinvestmentcandeliveramenitybenefitsthatareimportant,butseparateto,theeffectonlocaleconomicgrowth.

Other modes: Trams, Buses, Cycling and Walking Ofthe232shortlistedstudiesreviewedindetail,10consideredtheimpactofbuses,1theimpactoftrams,1cycling;1walking.Unfortunately,fromthistotalof13studiescoveringthesefourareas,wefoundnohighqualityevaluationsthatprovideevidenceontheimpactsoftrams,buses,cyclingandwalkingschemesonanyeconomicoutcomes.

53 Study1075.54 PopulationstatisticsfromStatistischesBundesamt(2014).55 SeetheImpactEvaluationforTransportInfrastructuresectionforfurtherdiscussionofthisselectionproblem.56 Study1114.

Page 31: Evidence Review 7 Transport · 2019. 1. 16. · Evidence Review: Transport - July 2015 7 To determine policy priorities The Centre’s reviews consider a specific type of evidence

Evidence Review: Transport - July 2015 31

Summary of findings

Thissectionsummarisesthedetailedfindings.Weemphasisethatmanyofthesefindingsdependonasmallnumberofstudies.Theyare,however,consistentwithotherresearchonthebroaderimpactoftransportimprovements.

What the evidence shows• Roadprojectscanpositivelyimpactlocalemployment.Buteffectsarenotalwayspositive

andamajorityofevaluationsshowno(ormixed)effectsonemployment.

• Roadprojectsmayincreasefirmentry(eitherthroughnewfirmsstartingup,orexistingfirmsrelocating).However,thisdoesnotnecessarilyincreasetheoverallnumberofbusinesses(sincenewarrivalsmaydisplaceexistingfirms).

• Roadprojectstendtohaveapositiveeffectonpropertyprices,althougheffectsdependondistancetotheproject(andtheeffectscanvaryovertime).

• Theimpactofroadsprojectsonthesizeofthelocalpopulationmayvarydependingonwhethertheprojectisurban,suburbanorrural.

• Thereissomeevidencethatroadprojectshavepositiveeffectsonwagesorincomes.

• Thereissomeevidencethatroadprojectshaveapositiveeffectonproductivity.

• Railprojectstendtohaveapositiveeffectonpropertyprices,athougheffectsdependondistancetotheproject(andthesecanalsovaryovertime).

Where there is a lack of evidence • Wefoundnohighqualityevaluationsthatprovideevidenceontheimpactofrailinfrastructure

onemployment,andonlyalimitednumberofevaluationsshowingthatroadprojectshaveapositiveeffect.

• Wefoundnohighqualityevaluationsthatprovideevidenceontheimpactsoftrams,buses,cyclingandwalkingschemesonanyeconomicoutcomes.

• Evenwhenstudiesareabletoidentifyapositiveimpactonemployment,theextenttowhichthisisaresultofdisplacementfromothernearbylocationsisstillunresolved.Moregenerally,

08

Page 32: Evidence Review 7 Transport · 2019. 1. 16. · Evidence Review: Transport - July 2015 7 To determine policy priorities The Centre’s reviews consider a specific type of evidence

Evidence Review: Transport - July 2015 32

thespatialscaleofanyemploymenteffectsvariesandwedonothaveenoughevidencetobeabletogeneraliseaboutthespatialdistributionofeffectsiftheyoccur.Thesameistrueforotheroutcomes.ThescaleatwhichthestudiesevaluateimpactvariesfromadjacentneighbourhoodstomuchlargerUScounties.

• Surprisingly,veryfewevaluationsconsidertheimpactoftransportinvestmentonproductivity(wefoundjustthreestudies,twoforroadsandoneforrail).Althoughtheuseofsuchproductivityeffectstocalculate‘widereconomicbenefits’intransportappraisalisunderpinnedbyalargerevidencebase,itisstillworryingthatsofewevaluationscandemonstratethattheseeffectsoccurinpractice.

• Wehavelittleevidencethatwouldallowustodrawconclusionsonwhetherlarge-scaleprojects(e.g.highspeedrailormotorwayconstruction)havelargereconomicgrowthimpactsthanspendingsimilaramountsonacollectionofsmall-scaleprojects(e.g.lightrailorjunctionimprovements).

• Moregenerally,wedonotknowhowdifferencesinthenatureofimprovements(e.g.journeytimesavedornumberofadditionaljourneys)affectanylocaleconomicoutcomes.

• Thereissomeevidencethatcontextmatters.Forexamplepropertypriceeffectsmaydependonthetypeofproperty,whilewageeffectsmaydifferbetweenlowskilledandhighskilledworkers.But,onceagainwedonothaveenoughevidencetobeabletogeneralise.

How to use these reviewsTheevidencereviewhighlightsanumberoffactorsforpolicymakerstobeawareofwhenconsideringtransportpolicy:

• Muchmoreempiricalworkremainstobedoneonunderstandingtheimpactofinfrastructureimprovementsonlocaleconomicgrowth.Theeconomicbenefitsoftransportinfrastructurespending–particularlyasamechanismforgeneratinglocaleconomicgrowth–arenotasclear-cutastheymightseemonfacevalue.

• WhileitisunderstandablethatpoliticaldebatefocusesonexpenditurefiguresacrossdifferentpartsoftheUK,theydonothelpanswerthequestionofwhatwouldhappenifexpenditurewasdistributeddifferently.Argumentsforspendingmoreinareasthatarelesseconomicallysuccessfulhingeonthehopethatnewtransportisacost-effectivewaytostimulateneweconomicactivity.Asthisreviewshows,wedonotyethaveclearanddefinitiveevidencetosupportthatclaim.

• Thisraisesfundamentalquestionsaboutschemeappraisalandprioritisation,andabouttheroleofimpactevaluationinimprovingdecision-makingaroundtransportinvestment.

Helping to fill the evidence gaps: improving evaluation and appraisalInmanyinstanceslocaleconomicimpactisanimportantpartofthecasefortransportinvestment.Suchinvestmentalsoformsacentralcomponentofmanygovernmentalpolicyinitiativesaimedatincreasinglocaleconomicdevelopment(e.g.theUKgovernment’sLocalGrowthDealprocess).Itisthereforevitalthatprogressismadeinfillingtheevidencegapsandinimprovingourunderstandingoftheeffectoftransportimprovementonlocalgrowth.

Inthisfinalsection,wemakesomepreliminaryrecommendationsbuildingonrecentworkwiththeDepartmentforTransport,aswellasthediscussionsofaLEPworkinggroupconvenedbytheWhatWorksCentreforLocalEconomicGrowth.Ourrecommendationsfocusontheneedformore,andbetter,ex-postimpactevaluationandtheneedtoembedsuchevaluationintotheappraisalprocess.

Page 33: Evidence Review 7 Transport · 2019. 1. 16. · Evidence Review: Transport - July 2015 7 To determine policy priorities The Centre’s reviews consider a specific type of evidence

Evidence Review: Transport - July 2015 33

Considerableresourcesarealreadydevotedtotheex-anteappraisaloftransportschemesaspartofthedecisionmakingprocess.Cost-benefitanalysisplaysacentralroleinsuchappraisals.Theincreasedinterestintheeffectoftransportinvestmentonthelocaleconomyhasalsobeenaccompaniedbygrowingcriticismoftheappraisalapproachesusedtohelpfacilitateschemeprioritisation.Inparticular,thereisgrowingcriticismthatthecurrentapproachtocost-benefitanalysisdoesnotcaptureallofthebenefitsthatmaybeassociatedwithtransportinvestment.

Ifthemainaimofnewtransportinfrastructureisfasterjourneys,thenbenefitstotheeconomymaterialisebecausetimesavedcanbeusedonproductiveorotherwisevaluableactivities(eitherinbusinessorleisure).Thisiswhythemostfundamentalinputintotransportinfrastructurecost–benefitsanalysishastraditionallybeenthesocalled‘valueoftraveltimesavings’.Thisistraveltimesaved,convertedintomonetaryunits.Thesemonetisedtimesavingsareacrucialmeasureoftheeconomicbenefitfromtransportinvestment(andcanbesupplementedbymonetisedestimatesofthebenefitsofreductionsinothercostslikeaccidentsandunreliability).

Butoverthepast15yearstherehasbeengreaterinterestinthepotentialfortransporttogenerate‘widereconomicbenefits’thatgobeyondthesetraveltimessavings(inadditiontoarangeofotherwidersocialandenvironmentalbenefits).InthecontextofUKappraisalaparticularfocus,intermsofwidereconomicbenefits,hasbeenonthosethatcomefromeffectivelybringingpeopleandbusinessesclosertogethertoformagglomerationsofeconomicactivity.Thelogicfollowsfromtheobservationthatcitiesaremoreproductivethanruralplacesandbigcitiesaremoreproductivethansmallercities.Solinkingplacestogethermayhelpgenerateproductivityimprovements.

Despitetheseimprovementstoappraisalpractice,therecontinuetobeconcernsthatappraisalmissesimportantbenefitsoftransportinvestment–particularlyintermsoftheimpactonlocaleconomicgrowth.SomeoftheseissueswererecentlyconsideredinanindependentreportfortheDepartmentforTransport(Laird,OvermanandVenables,2015).Thereportconcludesthat,insomecases,traditionalcost-benefitanalysismayindeedmissimportantbenefitsthatshouldbeincludedintheanalysis(althoughtheDepartment’sWebTAGguidanceincludesalmostallofthem).

Thereportalsoarguesthattheremaybeinstancesinwhichlocaldecisionmakersareinterestedinthelocaleconomiceffectsoftransport–e.g.onemploymentandinvestment–evenwhentheseshouldnotbeincludedinacost-benefitanalysiswhichseekstoevaluatetheoverall(i.e.national)gainsfromanewproject.Aconcreteexamplewouldoccurwhenemploymentgrowthneartonewtransportinvestmentispurelydrivenbydisplacementfromelsewhereintheeconomy.Atraditionalcost-benefitanalysis–whichtriestoassesstheoverallgainstosociety–wouldignoresuchdisplacement.Buttheseeffectsmaybeoflegitimateinteresttolocaldecisionmakers.

DfTisplanningtorefreshWebTAG(thesetofprocedureswhichoutlinehowappraisalsshouldbeconducted)torespondtotheseobservations.Whilesuchanexercisewillbewelcomedbymany,thefindingsinthisreviewalsohighlighttheimportanceofcomplementinganyfurtherworkontheex-anteappraisalframework(i.e.analysistopredictwhatmighthappen)withadditionalworktostrengthentheex-postevaluationoftransportinvestment(i.e.analysisofwhatactuallyhappened).Notleastbecause,asthisreportmakesclear,thereislimitedevidencethattheemployment(andothereffects)thatwouldunderpinanychangestotheguidanceactuallyoccurinpractice.

Whatformshouldsuchex-postevaluationtake?Asdiscussedabove,forcapitalexpenditure,whereinvestmentsaredurable,itishardtoimaginesituationsinwhichtruerandomisationofprojectplacementwouldbeeitherfeasibleordesirable.Thismeansthatweneedtorelyonalternativeevaluationapproaches

Page 34: Evidence Review 7 Transport · 2019. 1. 16. · Evidence Review: Transport - July 2015 7 To determine policy priorities The Centre’s reviews consider a specific type of evidence

Evidence Review: Transport - July 2015 34

totrytoidentifythecausalimpactoftransportinvestment.Onthebasisofourreviewwork,plusourwiderworkontheissueoftransportevaluationwethinkthatworktodevelopanewapproachisurgentlyneeded.Anysuchapproachneedstobebothfeasibleandproportional.Italsoneedstoproduceevidencethatishelpfulinimprovingfuturedecisionmaking.Unfortunately,manyexistingstudiesappeartohavecostmuchbutwitharguablylittlebenefitinimprovingdecisionmaking.Thereareanumberofpossibleavenuesthatcanbeexploredandissuesthatwillneedtobeconsidered.Forexample:

1. AtpresenttheHighwaysAgencyundertakesPostOpeningProjectEvaluation(POPE)ofalargenumberofschemes.Thistakestwoforms–one,lightertouch,forsmallerprojects(LocalNetworkManagementSchemes);andamoreextensiveevaluationforlargerprojects(MajorSchemes).POPEaimstodeterminehowschemeshaveperformedintheiropeningyearand,formajorprojects,fiveyearsafteropening.Findingsarecomparedtoex-anteappraisaltoassessaccuracyandanannualmeta-studypullstogetherfindingsfromallPOPEundertakenthatyear.POPEusesbeforeandafteranalysisofschemedata–anapproachwhichwouldscorelevel2ontheSMSscale.Morerecently,theDfThasissuedaMonitoringandEvaluationFrameworkforLocalAuthorityMajorSchemes(LAMS)thatprovidesguidancethatoutlinesaPOPEstyleapproachforthoseschemes.Thisguidanceoutlinesthreeapproaches:standardandenhancedmonitoringwhichparallelthesmallerprojectsapproachinPOPE;andfullerevaluationwhichisclosertothePOPEguidanceforMajorSchemes.AswithPOPE,thereisastrongemphasisonbeforeandaftercomparisons.57

2. Itwouldbehelpfultoconsiderhowtheuseofappropriatecontrolgroupscouldrefinetheseprocesses.Interestingly,themostrecentPOPEguidanceformajorschemeshasrecognisedtheimportanceofcontrollingforthebackgroundreductioninthenumberofcollisionswhenassessingbenefitsfromaccidentreduction.LAMSalsoplacesmoreemphasisontheuseofcontrolgroups–atleastforthefullerevaluations(see,forexample,thediscussionofcontrolgroupsintheassessmentofchangesintravelbehavior).Itwouldbeusefultoundertakefurtherworktoidentifyappropriatecontrolgroupsandtoencouragetheiruseforbenefitswhichmightbemostaffectedbyother‘background’changes.Controlgroupscouldbeconstructedinanumberofways:Forexamplea)forsimilarpartsofthenetworkthathavenotbeensubjecttoimprovement58b)fromschemesthatarelikelytobefundedinthefuturebuthavenotyetbeenfunded;c)fromschemesthathavesimilarbenefit-costratiosbutweredeclinedfunding;d)forareasclosetofundedschemesthatarenotdirectlyaffectedbythescheme.Moresimply,appropriateareawideaverages(whichwouldideallyexcludenewschemes)couldbeusedtoprovideaverybasiccontrolgroup.Similarapproachescouldbeusedtoidentifysuitablecontrolgroupswhenusingindividualleveldata(e.g.ontravelbehavior).Therewill,ofcourse,beprosandconstoalloftheseapproachesandfurtherworkwouldbeneededtoconsiderthealternatives(andwhetherthebenefitsintermsofimprovedPOPEandLAMSanalysis,outweightheadditionalcosts).

3. AlthoughPOPEconsidersperformanceofeachschemeagainsttheGovernment’sfourWebTAGobjectives(economy,environment,societyandpublicaccounts)theanalysisofeconomicimpactstendstofocusondirectbenefits–particularlyintheformofreducedaccidentsandimprovedjourneytimes.ThisisunsurprisinggiventhecurrentfocusofPOPEonperformance.Interestingly,whilethelightertouchapproachesinLAMSgivelessconsideration

57 AthirdsetofguidelinesconsiderthemonitoringandevaluationframeworkfortheLocalSustainableTransportFund.ManyofthepointswemakeherecouldalsoapplytodevelopmentoftheLSTFframeworkbutthetextfocusesonPOPEandLAMSwhichhaveclearerparallelsandastrongerimpactevaluationfocusthanLSTF.

58 Thisistheapproachissimilartothatusedforcollisions–atleastforkeylinks–whereadjustmentisbasedonnationaltrendsforthattypeofroad(althoughsomeproportionofthenetworkwillalsohavebenefitedfromimprovements).

Page 35: Evidence Review 7 Transport · 2019. 1. 16. · Evidence Review: Transport - July 2015 7 To determine policy priorities The Centre’s reviews consider a specific type of evidence

Evidence Review: Transport - July 2015 35

tosomeWebTAGobjectives59theyplacemoreemphasisoneconomicbenefits–particularlyintermsofeffectsonemploymentandrentalvalues.POPEformajorschemesdoesallowfortheassessmentofwidereconomicbenefitswiththeconsiderationgiventothesebenefitsvaryingaccordingtothelevelofPOPEthatisundertaken.Atitsmostextensive,thiswillinvolveasurveyoflocalbusinessatoneyearto‘identifyemergingconcernsorpositiveoutcomesassociatedwiththescheme’andatfiveyearsa‘focusedsurveyofbusinessestoidentifywidereconomicimpacts’.Giventheincreasedinterestinwidereconomiceffects,wethinkthatthisprocesscouldbeimprovedtobetteralignthePOPEandLAMSprocessestoensurethatbothcarefullyassesstheseeconomiceffects.60AsismadeclearinLAMS,notallschemeswouldwarrantsuchananalysis,butthisshouldbeconsideredwhenemployment,orotherlocaleconomyeffects,areanimportantcomponentofthestrategicoreconomiccaseformajorschemes.Resultsfromthisanalysisareunlikelytobeusefulinisolation.BothLAMSandPOPEhighlighttheimportanceofcomparingoutcomestokeyappraisalassumptions.61Butwewouldalsohighlighttheimportanceofbroadercomparisonstoboththestrategicandeconomiccasesthatformpartoftheappraisalprocess.

4. Onceagain,itwillbeimportanttoconsiderhowtheuseofsuitablecontrolgroupscouldplayapartintheanalysisoftheseeconomiceffects.AsLAMSrecognises,therearestrongargumentsinfavourofdevelopingsuchanapproach–atleastforlargerschemes.Someoftheoptionsforconstructingsuchcontrolgroupswerediscussedunderpoint(2).Itwouldalsobeusefultoconsiderwhetheralighttouchapproachcouldbedevelopedforsmallerschemes.

5. Thereneedstobeamuchcloserlinkbetweentheex-anteappraisalandex-postevaluationofschemes.Ourreviewoftheliteraturediscoveredalargenumberofex-postevaluationsthatappeartoliveinavacuum,62withnoattemptmadetolinkthefindingsfromthesereportsbacktoschemeappraisals.Higherqualityimpactevaluations–i.e.thosethatseektoidentifythecausalimpactofinvestmentsusingchangesinoutcomescomparedtoacontrolgroup(i.e.arescoredSMS3andaboveandincludedinourreview)–arestillhelpfulevenintheabsenceofsuchcomparisons.Thesearethestudiesthatwehaveusedinthisreview.Thisis,unfortunately,notsotrueforlessrobustevaluations(e.g.thoseinvolvingsimplebeforeandaftercomparisons).Embeddingevaluationintotheschemeprioritisationprocessisanimportantstepinensuringthatmoneyspentonex-postevaluationsiscost-effectiveinimprovingprioritisationforfuturespending.OneoftheadvantagesofincorporatingtheevaluationofwidereconomicimpactswithinanimprovedPOPEmethodologyisthatthesecomparisonsarealreadypartofthePOPE‘meta-analysis’process.Itwillbeimportanttodevelopasimilar‘meta-analysis’forLAMSthatparallelsthePOPEprocess.Thesecomparisonsacrossevaluationsshouldallowfindingsonschemeeffectsandthecomparisontoappraisalassumptionstobeusedtoimproveschemeprioritisation(forexample,throughtheuseofoptimismbiastoadjustpredictedemploymenteffects).GiventheinterestintheeconomicimpactsofinvestmentinothertransportmodesweshouldconsiderhowandwhenasimilarapproachcouldbeextendedtosuchschemesnotcoveredbyPOPEorLAMS.

6. InlinewiththerecommendationsoftheDfTTIEPreport,thereshouldbefargreaterattentionpaidtothecriticalanalysisofboththeeconomicandstrategiccasesforsupport.This

59 Forexample,inLAMSenvironmentisconsideredinenhancedmonitoring,butnotinstandardmonitoring.60 InPOPE,inparticular,thiswouldinvolvebringingtheapproachtowidereconomiceffectsinlinewiththoseusedtomore

carefullymeasurechangesinjourneytimes,etc.61 Forexample,POPEsystematicallycomparesthemonetisedvalueofchangesinaccidentsandjourneytimestotheex-

anteappraisalpredictions.62 Indeed,anumberofevaluationsarenot(easily)accessibleevenwhentheyhavebeenpublicallyfunded.Whileconcerns

overcommercialconfidentialitymaybeproblematicforsomeaspectsoftheex-postevaluation(asforappraisal)routinepublicationofpublicallyfundedevaluationsshould,arguably,bethenorm.

Page 36: Evidence Review 7 Transport · 2019. 1. 16. · Evidence Review: Transport - July 2015 7 To determine policy priorities The Centre’s reviews consider a specific type of evidence

Evidence Review: Transport - July 2015 36

criticalanalysisshouldoccurbothex-ante(onthebasisofavailableevidence–includingthatcoveredinthisreview)andex-post(onthebasisofappropriateevaluation–includinganalysisdevelopedaccordingtotherecommendationsabove).

7. Ex-postevaluationneedstoincludedueconsiderationoftheextenttowhichanyemploymenteffectsarelikelytoresultfromdisplacement(theshiftingofjobsfromoneplacetoanother).Addressingconcernsoverdisplacementwillneedtobeakeyquestioninunderstandingthenetimpactofinvestments.Therearesimilarconcernsoverspillovereffectswherebyemploymentgrowthresultingfromtransportimprovementsdrivesgrowthinareasnotdirectlyaffectedbythescheme.However,forboththesequestionscarefullyidentifyinganydirectemploymentimpactisafirststepinunderstandingtheseotherfactors.Concernsoverdisplacementandspilloversshouldnotpreventprogressinaskingthesimplerquestionastowhetheranychangesinemploymentoccurdirectlyasaresultofthescheme.Workisneededtotackleallthreequestions–whatistheemploymenteffect;areanylocalemploymentchangesadditional;dothesespillovertowiderareas?Asimilarpointholdswithregardtovariationsineffectsacrossschemes.Worktoidentifytheaverageeffectshouldbeafirststepinunderstandinghowvariationineffectsdependsoncontext.Again,concernsoverheterogeneityofeffectsshouldnotpreventprogressonthesimplerquestionofidentifyingaverageeffects.

8. ThecurrentLAMSguidance,includingacomparisontoappraisalassumptions,wouldappeartoprovideanappropriateframeworkforundertakingandimprovingevaluationandschemeprioritisationforindividualLAs/LEPs.However,thereisaroleforDfTinhelpingdeveloptheguidelinesforhowthisanalysiscouldbeconductedandimprovedalongthelinesofpoints(2)to(7).Thiswillensure,aswithPOPE,thatresultsforspecificLAs/LEPsaretransferableacrossareas.Thedevolutionagendaraisesquestionsabouttheextenttowhichsuchanapproachcould(orshould)bemandatory.Regardlessoftheoutcomeofthatdebate,manyLAs/LEPswouldstillwelcomeguidanceonhowbesttoproceed–especiallygivenlocalconstraintsonanalyticalcapacity.

9. ConsiderationneedstobegivenastohowtoensureLAs/LEPshavetheincentive(andtheresources)tocollectdataincontrol/comparisonareas.Itispossiblethatcentralgovernmentdepartmentscouldprovideappropriateareadata(andtheuseofsuchsecondarydatawouldsubstantiallyreducethecostimplicationsofundertakingevaluations).Wheredataiscollectedatthelocallevelitwillbeimportanttoensurethatsuchdataareavailabletoresearchersforuseinaggregated/multi-interventionanalysis.

10. Giventhecomplexityofmanyoftheissuesraisedabove,andtheneedforcomparisonacrossareas,itislikelythatDfTwillneedtoplayacoordinatingroleinaddressingmanyoftheseevaluationchallenges.ThereisalsoaroleforDfTinundertakingmulti-interventionex-postanalysisusingthekindofapproachesusedbythehigherqualitystudiesconsideredaspartofthisreport.

Furtherworkwouldbeneededtodeveloptheissuesdiscussedhereandtoconsiderappropriatesolutions.Itiscrucialthatfurtherworkrecognisestheimportanceofembeddingevaluationintotheschemeprioritisationprocess(DfTiscurrentlyundertakingworkonthisissue).Thismeansbetteraligningappraisalandevaluation,particularlyiftheobjectiveistoimproveschemeprioritisation.Withoutcloserintegration,thereisadangerthatweundertakerefinementstotheappraisalprocess–e.g.toincludeemploymentandinvestmenteffects–withoutknowingthelikelymagnitudeofeffects,whethertheyareadditional,etc.Similarly,whileex-postevaluationcanservesomeroleintermsofmonitoringandaccountability,itsmainaimshouldbeinimprovingfuturedecisionmaking.Thismeansthinkingaboutwaysinwhichevaluationcanfeedbackintotheschemeprioritisationprocess–bothintermsofdevelopingex-anteappraisal,butalsoinprovidingameansofscrutinisingstrategiccasesforfutureinvestment(again,anareainwhichDfTiscurrentlyundertakingwork).

Page 37: Evidence Review 7 Transport · 2019. 1. 16. · Evidence Review: Transport - July 2015 7 To determine policy priorities The Centre’s reviews consider a specific type of evidence

Evidence Review: Transport - July 2015 37

References

Aghion,P.,Besley,T.,Browne,J.,CaselliF.,Lambert,R.,LomaxR.,Pissarides,C.,Stern,N.,VanReenen,J.(2013)InvestingforProsperity,Skills,InfrastructureandInnovation,ReportoftheLSEGrowthCommissionLondon:LSEGrowthCommissionhttp://www.lse.ac.uk/researchAndExpertise/units/growthCommission/documents/pdf/LSEGC-Report.pdf

Baldwin,R.,Forslid,R.,Martin,P.,Ottaviano,G.,andRobert-Nicoud,F.(2005)EconomicGeographyandPublicPolicy,PrincetonUniversityPress

CityGrowthCommission(2014)UnleashingMetroGrowth,London:RSAhttp://www.citygrowthcommission.com/publication/final-report-unleashing-metro-growth

Cox,E.andDaviesB.(2013)StillontheWrongTrack:AnUpdatedAnalysisofTransportInfrastructureSpending,IPPRNorth

http://www.ippr.org/publications/still-on-the-wrong-track-an-updated-analysis-of-transport-infrastructure-spending

Eddington,R.(2006)TheEddingtonTransportStudy,Thecaseforaction:SirRodEddington’sadvicetoGovernment,London:HMSOhttp://www.thepep.org/ClearingHouse/docfiles/Eddington.Transport.Study%20-%20Rod.pdf

Gibbons,S.(2015)Planes,TrainsandAutomobiles:TheEconomicImpactofTransportInfrastructure.SERCPolicyPaper#13

Gibbons,S.,Nathan,M.,andOverman,H.G.(2014)Evaluatingspatialpolicies.TownPlanningReview,85(4).pp.427-432.

Jones,C.andVollrathD.(2013)IntroductiontoEconomicGrowthW.W.Norton&Co.;3editionSherman,L.W.,Gottfredson,D.C.,MacKenzie,D.L.,Eck,J.,Reuter,P.,&Bushway,S.D.(1998).PreventingCrime:WhatWorks,WhatDoesn’t,What’sPromising.WashingtonDC:USDepartmentofJustice.https://www.ncjrs.gov/pdffiles/171676.PDF

Solow,R.M.(1956)AContributiontotheTheoryofEconomicGrowth,TheQuarterlyJournalofEconomics,Vol.70,No.1.(Feb.,1956),pp.65-94

StatistischesBundesamt(2014)AllepolitischselbständigenGemeindenmitausgewähltenMerkmalenam31.12.2013.https://www.destatis.de/DE/ZahlenFakten/LaenderRegionen/Regionales/Gemeindeverzeichnis/Administrativ/Archiv/GVAuszugJ/31122013_Auszug_GV.html

Page 38: Evidence Review 7 Transport · 2019. 1. 16. · Evidence Review: Transport - July 2015 7 To determine policy priorities The Centre’s reviews consider a specific type of evidence

Evidence Review: Transport - July 2015 38

09

Appendix A: Findings by outcome

Table A1: Roads

Outcome +ve Zero -ve Mixed No. +ve

Employment 1011,1017, 1027,1031,1067

- 1015 2/6

PropertyValues/Rents 1052,1009,1035

- - 1050 3/4

BusinessVolume/Sales 1037,1067 - - - 2/2

Firmentryandnumberofbusinesses

1017,1061 1016 - - 2/3

Productivity 1017,1062 - - 2/2

Innovation 1063,1055 - - - 2/2

Income/Wages 1017 - - 1067 1/2

Population - - 1005 1015,1027 0/3

Table A2: Rail

Outcome Total evaluated

+ve Zero -ve Mixed No. +ve

EconomicPropertyValues

61070,1071,1107,1108,

11121074 - - 5/6

GDP 1 1075 - - - 1/1

BusinessProductivity 1 1116 - - - 1/1

Non-economicPopulation 1 1109 - - - 1/1

Crime-reduction 1 1114 - - - 1/1

Note:Inadditiontooutcomesreportedhere,Study1111showedevidenceofchangesinlanduseasdiscussedinthetext.

Page 39: Evidence Review 7 Transport · 2019. 1. 16. · Evidence Review: Transport - July 2015 7 To determine policy priorities The Centre’s reviews consider a specific type of evidence

Evidence Review: Transport - July 2015 39

Appendix B: Evidence Reviewed

Roads

Ref No. Reference1005 Baum-Snow,N.(2007).Didhighwayscausesuburbanization?TheQuarterlyJournalof

Economics,775-805.

1009 Boarnet,M.G.&Chalermpong,S.(2001)NewHighways,HousePricesandUrbanDevelopment:ACaseStudyofTollRoadsinOrangeCounty,CA.HousingPolicyDebate,12(3),575-605

1011 Chalermpong,S.(2002).EconomicSpilloversofHighwayInvestment:ACaseStudyoftheEmploymentImpactsofInterstate105inLosAngelesCounty.UniversityofCaliforniaTransportationCenter.

1015 Funderburg,R.G.,Nixon,H.,&Boarnet,M.G.(2010)NewHighwaysandLandUseChange:ResultsfromaQuasi-experimentalResearchDesign.TransportationResearch:PartA:PolicyandPractice,44(2),76-98

1016 Garcia-Mila,T.&Montalvo,J.G.(2013)ANewApproachtoMeasuretheImpactofHighwaysonBusinessLocationwithanApplicationtoSpain,”UniversitatPompeuFabra,BarcelonaGSEandIVIE.

1017 Gibbons,S.,Lyytikainen,T.,Overman,H.G.,&Sanchis-Guarner,R.(2012).Newroadinfrastructure:theeffectsonfirms.

1027 Iacono,M.,&Levinson,D.(2013).CausalityintheLinkBetweenRoadNetworkGrowthandRegionalDevelopment(No.000112)

1031 Jiwattanakulpaisarn,P.,Noland,R.B.,Graham,D.J.,&Polak,J.W.(2009).Highwayinfrastructureinvestmentandcountyemploymentgrowth:Adynamicpanelregressionanalysis*.JournalofRegionalScience,49(2),263-286.

1035 Chernobai,E.,Reibel,M.,&Carney,M.(2011).Nonlinearspatialandtemporaleffectsofhighwayconstructiononhouseprices.TheJournalofRealEstateFinanceandEconomics,42(3),348-370.

1037 Duranton,G.,Morrow,P.M.,&Turner,M.A.(2014).RoadsandTrade:EvidencefromtheUS.TheReviewofEconomicStudies,81(2),681-724.

1050 Vadali,S.(2008).Tollroadsandeconomicdevelopment:exploringeffectsonpropertyvalues.TheAnnalsofRegionalScience,42(3),591-620.

1052 Márk,L.(2013).TheEffectofHighwaysonNearbyResidentialPropertyPricesinHungary(Doctoraldissertation,CentralEuropeanUniversity).

1055 Crescenzi,R.,&Rodríguez-Pose,A.(2012).InfrastructureandregionalgrowthintheEuropeanUnion*.Papersinregionalscience,91(3),487-513.

1061 Holl,A.(2004).TransportInfrastructure,AgglomerationEconomies,andFirmBirth:EmpiricalEvidencefromPortugal*.JournalofRegionalScience,44(4),693-712.

1062 Montolio,D.,&Solé-Ollé,A.(2009).Roadinvestmentandregionalproductivitygrowth:theeffectsofvehicleintensityandcongestion*.PapersinRegionalScience,88(1),99-118.

1063 Agrawal,A.,Galasso,A.,&Oettl,A.(2014).RoadsandInnovation.RotmanSchoolofManagementWorkingPaper,(2478752).

1067 Michaels,G.(2008).Theeffectoftradeonthedemandforskill:Evidencefromtheinterstatehighwaysystem.TheReviewofEconomicsandStatistics,90(4),683-701.

Page 40: Evidence Review 7 Transport · 2019. 1. 16. · Evidence Review: Transport - July 2015 7 To determine policy priorities The Centre’s reviews consider a specific type of evidence

Evidence Review: Transport - July 2015 40

Rail

Ref No. Reference1070 Chatman,D.G.,Tulach,N.K.,&Kim,K.(2012).Evaluatingtheeconomicimpactsoflightrail

bymeasuringhomeappreciationafirstlookatNewJersey’sRiverLine.Urbanstudies,49(3),467-487.

1071 Kim,K.,&Lahr,M.L.(2014).TheimpactofHudson-BergenLightRailonresidentialpropertyappreciation.PapersinRegionalScience,93(S1),S79-S97.

1074 Ahlfeldt,G.M.(2011).Thetrainhasleftthestation:domarketsvalueintracityaccesstointercityrailconnections?.Germaneconomicreview,12(3),312-335.

1075 Ahlfeldt,G.M.,&Feddersen,A.(2010).Fromperipherytocore:economicadjustmentstohighspeedrail.DocumentsdetreballIEB,(38),1.

1083 Gatzlaff,D.H.,&Smith,M.T.(1993).TheimpactoftheMiamiMetrorailonthevalueofresidencesnearstationlocations.LandEconomics,54-66.

1107 Koster,H.R.A.,Ommeren,J.N.&Rietveld,P.(2010).Estimatingthebenefitsofimprovedrailaccess;geographicalrangeandanticipationeffects.TinbergenInstituteDiscussionPaper(TI2010-094/3).

1108 Billings,S.B.(2011).Estimatingthevalueofanewtransitoption.RegionalScienceandUrbanEconomics,41(6),525-536.

1109 Enflo,K.,Berger,T.(2013)LocomotivesofLocalGrowth:TheShortandLong-TermimpactofRailroadsinSweden.EHESWorkingPapersinEconomicHistory.No.42.

1111 Hurst,N.B.,&West,S.E.(2014).Publictransitandurbanredevelopment:TheeffectoflightrailtransitonlanduseinMinneapolis,Minnesota.RegionalScienceandUrbanEconomics,46,57-72.

1112 Ahlfeldt,G.M.(2011).Ifwebuild,willtheypay?:predictingpropertypriceeffectsoftransportinnovations.

1114 Billings,S.B.,Leland,S.,&Swindell,D.(2011).Theeffectsoftheannouncementandopeningoflightrailtransitstationsonneighborhoodcrime.JournalofUrbanAffairs,33(5),549-566.

1116 Bernard,A.B.,Moxnes,A.,&Saito,Y.U.(2014).ProductionNetworks,GeographyandFirmPerformance.TuckSchoolofBusinessWorkingPaper.

Findthefulllistofsearchtermsweusedtosearchforevaluationsonourwebsitehere:whatworksgrowth.org/policies/transport/search-terms.

Page 41: Evidence Review 7 Transport · 2019. 1. 16. · Evidence Review: Transport - July 2015 7 To determine policy priorities The Centre’s reviews consider a specific type of evidence

The What Works Centre for Local Economic Growth is a collaboration between the London School of Economics and Political Science (LSE), Centre for Cities and Arup.

www.whatworksgrowth.org

Page 42: Evidence Review 7 Transport · 2019. 1. 16. · Evidence Review: Transport - July 2015 7 To determine policy priorities The Centre’s reviews consider a specific type of evidence

ThisworkispublishedbytheWhatWorksCentreforLocalEconomicGrowth,whichisfundedbyagrantfromtheEconomicandSocialResearchCouncil,theDepartmentforBusiness,InnovationandSkillsandtheDepartmentofCommunitiesandLocalGovernment.ThesupportoftheFundersisacknowledged.TheviewsexpressedarethoseoftheCentreanddonotrepresenttheviewsoftheFunders.

Everyefforthasbeenmadetoensuretheaccuracyofthereport,butnolegalresponsibilityisacceptedforanyerrorsomissionsormisleadingstatements.

Thereportincludesreferencetoresearchandpublicationsofthirdparties;thewhatworkscentreisnotresponsiblefor,andcannotguaranteetheaccuracyof,thosethirdpartymaterialsoranyrelatedmaterial.

Photocredit:“AdelaideTracks”byLesHaines

July2015

WhatWorksCentreforLocalEconomicGrowth

[email protected]@whatworksgrowth

www.whatworksgrowth.org

©WhatWorksCentreforLocalEconomicGrowth2015


Recommended