1

Evaluation of the Coastal and

Sustainable Tourism Projects

Executive Summary

1. Introduction

1.1 This is the final evaluation for two of the Environment for Growth (E4G)

projects – Sustainable Tourism and Coastal Tourism (including the Green

Sea programme) – which were developed and managed by Visit Wales. E4G

projects received funds from the European Regional Development Fund

(ERDF) programmes for Wales 2007-2013. The goals of the projects were to

improve the attractiveness of natural and manmade facilities, develop

“Centres of Excellence” and spin-out activities; develop marinas, cycle and

walking trails; improve access to the coast and countryside; and develop

potential for sustainable recreation and economic activity.

1.2 The E4G programme as a whole consisted of six project packages: Valleys

Regional Park; Heritage Tourism; Communities and Nature; Wales Coastal

Path; Sustainable Tourism; and Coastal Tourism. Short summaries of the

Coastal Tourism and Sustainable Tourism projects are provided below:

The Coastal Tourism project received a total of £18.9 million of which £8.7

million was from the ERDF. The programme consisted of three initial Centres

of Excellence: Watersports Swansea Bay, Aberdaron National Trust, and

Pembrokeshire. A fourth, Saundersfoot Marine Harbour, was added later. The

project also included the investment of the Green Sea Programme in both

North and South Wales.

The Sustainable Tourism project also received a total of £18.9 million of

which £8.2 million was from the ERDF. The programme consisted of four

Centres of Excellence: South Wales Cycling (Cognation), One Historic

Garden, North Wales Cycling Centre of Excellence, and the Eryri Centre of

Excellence (One Big Adventure).

1.3 Across these eight Centres of Excellence and the Green Programme, there

were up to 50 different investment activities spread across multiple sites

throughout the two ERDF programme areas. The ERDF funding was spread

SOCIAL RESEARCH NUMBER: 1/2018

PUBLICATION DATE: 09/01/2017

2

widely with relatively modest sums reaching some particular places and

activities.

1.4 The investments in Sustainable and Coastal Tourism projects reflect the

strategic priorities that were set in The Tourism Strategy for Wales “Achieving

Our Potential” which set out aims to meet customer needs, work innovatively,

manage destinations responsibly, and maximise business profitability,

alongside broad policy goals to support social justice, equalities, and

sustainability. This set a wide ranging policy context about what these tourism

related investments should achieve.

1.5 The projects were developed during what has been a challenging time for the

tourism economy in Wales. For the period covered by this report, overnight

visits from within Great Britain increased, while international visits decreased.

The economic environment also remained challenging with constraints for

tourism enterprises in securing private sector or public sector investment.

2. Purpose And Approach to the Evaluation

2.1 The scope for evaluation of the Sustainable and Coastal Tourism projects is

ambitious. Visit Wales commissioned Regeneris Consulting and The Tourism

Company to answer six essential questions and meet eight specific

objectives:

Questions on how project activity reflected commitments set out in the

business plans; perceived outcomes of the project for beneficiaries; how

project outcomes compared to if investment had not been made; potential

long term impacts and implications of withdrawal of funding; project delivery

and good practice; and lessons learned.

Objectives to provide an independent understanding of how projects

performed; review delivery and partner project management; assess

achievement against cross cutting aims; review progress against social

impacts; review match-funding and support; review monitoring and

evaluation; consider the legacy of the projects; and consider marketing.

2.2 In order to answer these questions, we drew on a range of evaluation strands

including:

Reviewing project business plans, monitoring returns and annual reports for

the project as a whole and the Centres of Excellence

Consultations with project managers, the Centre for Excellence managers,

and responsible officers and relevant partners and stakeholders for many of

the 50+ investment activities that were funded

Examination of the baseline, monitoring and evaluation evidence held for the

investment activities

Specific commissioned research for the two projects such as that undertaken

by Cardiff Business School estimating visitor and construction expenditure

related economic impacts.

3

2.3 Within the scope of such an ambitious evaluation, the extent to which all

these questions and objectives can be addressed is dependent on the

availability and quality of data, the trail of documentary evidence, and

perspectives and views of officials and beneficiaries. A number of challenges

arose during the course of the evaluation which constrained the evaluators

ability to gather the evidence that was required:

Although Visit Wales set targets and then gathered and reported monitoring

data on a sensible basis for the Centres of Excellence in order to address

their contractual requirements to WEFO, the evaluators were unable to easily

access comprehensive disaggregated analysis of these outputs and results

across the investment activities – this is important in terms of judging the

effectiveness of particular activities in generating outputs, results and value

for money.

Whilst guidance was provided by WEFO and Visit Wales on the

measurement of output and results indicators, this has not always been

applied consistently by the Centres of Excellence or the delivery bodies for

the visitor related result indicator in particular. There has been occasional

confusion about whether this is measuring visits or visitors and whether it is

gross or additional visits.

Given the time limited project funding for the Centres of Excellence and the

timing of the evaluation towards the end of the project’s lifespan, a number of

the Centre of Excellence managers and staff responsible for the

implementing the investments were no longer in post at the time of the

evaluation. This has made it difficult to gather the project perspective in some

instances.

The economic impact research commissioned by Visit Wales (and other E4G

partners) from Cardiff University has provided a helpful source of evidence of

the visitor and construction related impacts for the Centres of Excellence,

underpinned by extensive visitor survey work. However, the researchers

struggled to obtain robust baseline data (and in some instances post

investment visitor data) and hence an inherent weakness of the research is

that it does not distinguish between gross and net additional impacts. This is

highlighted further in the lessons section below for the new ERDF

programme.

3. Key Findings

3.1 The overall ERDF expenditure target was met at a little under £17 million,

although the composition changed slightly (£8.2 million for Sustainable

Tourism and £8.7 million for Coastal Tourism compared to the original

allocation of £8.5 million for both). Total expenditure was £37.7 million, with

substantial contributions from local authorities, other public sector agencies

and £4.1m funding from the private sector and charitable trusts (such as the

National Trust).

4

3.2 This funding has enabled the delivery of a wide range of visitor infrastructure

and facilities tailored to a range of target markets across Wales. At the level of

the Centres of Excellence, a reasonably clear message has emerged that the

original business plans were implemented largely as intended. Given the

extent of capital activities being implemented as part of the Centres of

Excellence, it is inevitable that some further planning and development

activities occurred following the approval of the project funding. This did lead

to some delays with the implementation of some initiatives, especially where

original plans needed to be amended to reflect development, planning or

costs constraints. This is in part due to the absence of a mobilisation and

planning period at the start of the funding period for Centres of Excellence

and their initiatives, although this has been addressed in the design of the

new ERDF programmes.

3.3 However, the Centres of Excellence were flexible around changes in funding

and partners – when changes were made or initiatives substituted, there were

understandable and justifiable reasons for this. Those that appear well-

conceived at the start were less likely to need to make changes in

composition or funding than those projects which were comparatively diffuse

and difficult to coordinate.

3.4 A number of the Centres of Excellence are judged by the evaluators to be

particularly well conceived, developed and delivered, including Cognation,

Aberdaron and Saundersfoot Marine Harbour. These were all underpinned by

strong strategies, carefully targeted investments to enhance the existing or to

provide new visiting infrastructure and offer, and often more straight forward

delivery arrangements.

Outputs

3.5 Overall, most of the projected outputs targeted were achieved and in some

instances exceeded:

The headline output metric on “initiatives developing the natural and/or

historic environment” was exceeded (35 compared to a target of 20), although

the increase largely reflects a lack of clarity in the definition of the indicator

about what counts as an “initiative” rather than a major change in the

composition of the investment programmes.

The final outturn for “managed access to the countryside” (measured in

kilometres) was slightly less that the original target (460 km compared 400

km achieved), mainly associated with cycling trails.

The final outturn for “enterprises assisted” was similar to the initial projection

in 2011 (44 for the Sustainable Tourism project compared to an initial target

of 49 and 18 compared to 15 for the Coastal Tourism project). Whilst our

consultations have indicated that there are differences in how this indicator

has been interpreted by the Centres of Excellence and delivery staff (and we

have not always had access to the enterprises being claimed), our research

5

provides reasonable evidence that the projects have provided opportunities

for new and existing enterprises to commence trading locally.

Table 1. Summary of Outputs by Centre of Excellence

Total (£m) Initiatives km Enterprises

Assisted

Jobs

Overall 37.7 44 430.9 >70 130

Sustainable

Tourism

18.9

Eryri 4.6 4 27 6 29

North Wales

Cycling

1.8 1 270 11 2

South Wales

Cycling -

Cognation

4.4 1 72 25 14

One Historic

Garden

4.6 7 34 11 14

Remainder

(e.g. central

marketing)

3.5

Coastal

Tourism

18.8

Swansea Bay

Watersports

4.2 4 10 6 33

National Trust

– Aberdaron

3.4 1 1 3 7.5

Pembrokeshire

Coastal

4.0 8 1 n/a 24

Saundersfoot

Trust

0.9 n/a 0.4 n/a 5

Of which Green

Sea:

0

The Green Sea

Programme

South

1.9 14 13 9 1

The Green Sea

Programme

North

0.9 6 2.5 0

Remainder

(e.g. central

marketing)

3.5

6

Outcomes, Impacts and Added Value

3.6 The evidence on outcomes is also generally positive but less clear cut in

terms of the robustness of the underpinning evidence:

The target for the number of “gross jobs created” was significantly

underachieved (i.e. these are direct jobs associated with the enhanced

facilities rather than associated with the indirect visitor expenditure). The

target for the Sustainable Tourism project was met (a target of 59), whilst the

revised target for the Coastal Tourism project (reduced from 240 to 53) was

exceeded (71).

The number of ‘visits’ reported suggest that both projects exceeded the

targets set in 2011. The Sustainable Tourism reported over 1.6 million

visitors compared to the target of 0.76 million and the Coastal Tourism

reported 1.15 million compared to a target of 250,000.

3.7 As noted earlier, our analysis identified a number of concerns about the

recorded ‘visits’. There is some inconsistency across the investment activities

about whether the measured outcome is ‘total visits’ or ‘additional visits’,

some confusion over whether the measured outcome is ‘visits’ or ‘visitors’ and

inconsistencies in the baseline information recorded by initiatives. This

confusion sits alongside significant limitations on how visits/visitors have been

counted or estimated. Whilst the reported visitor numbers need to be treated

with a degree of caution, it is reasonable to assume that the much lower

original targets have been exceeded.

3.8 Similarly, there is little evidence on the experience of visitors and whether

investment in improved facilities resulted in higher satisfaction or more

positive perceptions. Information about visitor perceptions was not recorded in

a systematic way.

3.9 It has not been possible to estimate the economic impacts associated with the

visitor activity as part of this evaluation. However, the Cardiff University study

concluded that:

The economic impacts take place largely away from the visitor sites

themselves, often with limited on-site spending opportunities and hence less

potential for impact.

Nevertheless, at the time of the study (2015), the impact of visitor spending

associated with the Sustainable Tourism project generated around £4.9

million of gross GVA per year and around 230 gross full time equivalent jobs

(approximately £21,300 per job). For the Coastal Tourism project the impact

of visitor spending was £8.8 million of gross GVA per year and around 399

full-time equivalents (approximately £20,000 per job). It is important to note

that these are gross rather than net additional impacts due to the absence of

clear pre-investment baseline data and some of this economic impact will

already be counted in the gross jobs reported in the previous section.

7

The projects also involved estimated construction expenditure of £28 million

across the two projects, supporting one off economic impact of around £20

million of GVA and 480 years of employment.

3.10 The consultations provide clear evidence that the investments have had value

in enhancing the reputation of particular attractions and areas, especially

where the investments were carefully targeted at existing assets, investing in

new facilities or spatially concentrated. It is where there were more

concentrated investments (such as Cognation and the new investment in Bike

Park Wales, Aberdaron or Saundersfoot Marine) that a clearer story about

additional outcomes can be demonstrated. In other instances, the

investments will have helped in providing opportunities to compete in the

tourism market place, although it is difficult to demonstrate this where the

investment was modest, fragmented and incremental.

3.11 The evaluation also explored the issue of deadweight or, in other words, what

would have happened in the absence of ERDF funding. The consultations

confirmed that in the absence of the ERDF funding from the two projects the

majority of the investments would not have occurred or would have been

delayed while alternative funding was sourced. The funding has therefore

been critical in enabling much of these activities to proceed.

Project Development and Delivery

3.12 The evaluation has pointed to different experiences with development and

implementation of the plans and investments across the Centres of

Excellence. The overarching strategic concepts of the Centres of Excellence

appear to have offered an initial vision that helped shape early ideas and

choices. But the large number of centres and investment activities that were

subsequently supported with relatively modest sums indicates that resources

and efforts were in some instances spread widely and therefore thinly.

3.13 As a result, much of the early strategic focus diminished in some instances.

The different centres developed largely independently with little binding them

together. Meanwhile, match-funding was almost wholly dependent on public

sector financing from local authorities or government agencies. The

leveraging of private sector funding was limited and confined to only a few

investment activities.

3.14 A strong message from the evaluation is that in the case of many of the

investments, there was insufficient pre-planning. This could have helped to

avoid some of the issues which arose during delivery, for example around

practical issues such as planning permissions, site constraints and costs.

Changes made to delivery plans were often time consuming and added

pressure to the delivery of projects. Whilst in practice there may be limitations

on the ability for delivery partners to undertake more detailed planning and

development of projects prior to submitting business plans and agreeing

funding agreements, an initial post funding agreement development or

mobilisation period could have helped to avoid these challenges and delays

(this has been adopted in the new programmes).

8

3.15 There are positive perceptions of overall project management for both the

Coastal and Sustainable Tourism projects, including relationships with Visit

Wales and other partners which worked well. The commitment of local

authorities to the delivery of investments, often working with local partners

and communities, was seen as a key aspect of successful project delivery.

This also included cross sector working with the third and private sectors both

at an overall project level and specific Centres of Excellence.

3.16 The administrative requirements of European funding were a potential barrier

although the engagement of Visit Wales and local authorities helped to

address this. However, some of the associated activity, including marketing

support and the environmental training, would have benefited from much

more advanced planning and better integration.

3.17 The delivery of the programme though European funding mechanisms may

also have been a barrier to integration of the Centres of Excellence with other

activities being undertaken by local authorities or by Visit Wales. There is a

sense that the project did not link in with other parts of local authorities or

wider tourism plans, as operations were often channelled through officials

responsible for European funds rather than those responsible for wider

economic development or tourism.

Marketing

3.18 Visit Wales undertook a successful national marketing campaign themed

around adventure and outdoor activity, closely linked to the sustainable and

coastal tourism themes pursued by the Centres of Excellence. This has been

the focus of a separate evaluation commissioned by Visit Wales.

3.19 In terms of the marketing of the local initiatives, the main concern of most

managers of each of Centres of Excellence was to deliver a set of projects, on

time and within budget. The reviews of the Centres of Excellence suggest

insufficient attention was paid in some instances to complementary local

marketing support and activities required when bringing a new product to the

market place. This was especially true where those responsible for delivery

were based within a European Funding team.

3.20 There was considerable variation in the extent to which marketing formed a

significant or prominent part of the original bid and subsequent allocation of

funding; for example marketing was a significant share of funds for One

Historic Garden but negligible for Aberdaron, and some were designed to

form a cluster with a common branding, such as Cognation and One Historic

Garden. Only One Historic Garden appears to have directed significant

resources to employing an external marketing team to deliver a campaign.

3.21 There was also variation in the extent to which different Centres of Excellence

were suitable for standalone marketing campaigns and co-ordination in joint

marketing was lacking. The diversity of projects, for example in Snowdonia’s

One Big Adventure, with their different target audiences presents a marketing

challenge. In some cases, dedicated standalone websites were established

9

but it is not clear how traffic was driven to these websites. Commercial or

semi-commercial partners (e.g. Saundersfoot Harbour and National Trust)

have an ongoing commitment to marketing their facilities. This is different to

the situation in which local authorities have found themselves, especially at a

time of reducing budgets.

Cross cutting Themes and Social Impacts

3.22 The ERDF funded activities over 2007-2013 were expected to incorporate the

cross-cutting themes of environmental sustainability and equal opportunities

as themes for a more balanced, sustainable and innovative economy. Each of

the business plans for centres under the project umbrella of ‘Sustainable’,

‘Coastal’ and ‘Green Sea’ projects had ambitions around environmental goals;

and each set ambitions around equality – in particular with physical access for

people with disabilities and leisure and educational opportunities for people in

low income groups as a theme across the projects. Visit Wales was proactive

in providing support to the Centres of Excellence to plan for and achieve

these cross crossing themes, including various themed workshops run by the

Carbon Trust and WRAP. The consultations largely brought out positive

qualitative perceptions that the projects had set out to achieve these goals

and made progress. There are examples of good practice such as Aberglasni

Gardens (see the reviews of the Centres of Excellence).

3.23 However, there are understandable challenges in monitoring progress and

value in these kind of environmental and social goals – these are not financial

or physical outcomes. There was limited guidance from WEFO at a

programme level and, whilst bearing this in mind, there is little indication of a

systematic and consistent approach to doing so being applied within or across

all the different centres. No clear set of indicators has been used and these

have not been reported alongside the core ERDF performance indicators.

Although the need for a social impact assessment framework is noted in the

Mid Term Evaluation report, it does not appear to have been adopted as a

formal method of monitoring and reporting impacts.

4. Legacy Impacts

4.1 The investments made through the Centres of Excellence were intended to be

financially self-sustaining in the longer term wherever possible. The extent to

which the different centres and investment activities will continue beyond the

end of ERDF funding is mixed. The range of capital investments in harbours

and marinas, paths and bike trails, buildings and facilities provides a clear

physical infrastructural legacy and enhances the offer to visitors and

residents. What is less clear is how the on-going costs of maintaining or

running the facilities will be met, either through mainstreaming into public

sector budgets or self-supporting through income generation.

4.2 Many of the centres anticipated the longer term funding required and made

investments with goals of becoming self-sustaining in raising their own

revenues, for example through ticketing (such as Bike Park Wales) and car

10

parking fees, or through shops and café facilities. This picture naturally differs

across centres with those that are privately-led (or Trust-led) more

commercially focused compared to those that are public sector led and

providing local public amenities. Some projects (such as parts of Swansea

Watersports) began as public sector initiatives and have moved towards a

more commercial footing. Some other delivery partners were more optimistic

about activities becoming self-sustaining and generating revenues from

visitors without further public assistance.

4.3 However, the evaluation identified significant concerns amongst some public

sector led projects about the availability of future resources from within the

public sector to maintain, market and reinvest in these facilities. These

concerns have clearly been heightened by the cuts in public sector budgets.

4.4 In short, the legacy impacts and the sustained effect on targeted sectors,

products and businesses is not yet clear and this will need to be assessed in

the years ahead.

5. Recommendations and Lessons

5.1 The evaluation points to a number of lessons which will help to guide the

development of future tourism centres of excellence or specific tourism

infrastructure and facilities investments:

At the level of the Centres of Excellence. There are clear benefits in tourism

development and economic impact in terms of achieving a greater

concentration of resources rather than spreading resources more thinly

across large numbers of tourism priorities, spatially and thematically. This

lesson is relevant to Visit Wales and delivery bodies involved in designing

centres of excellence.

The importance of delivery bodies undertaking more thorough early

development for the proposed investment activities within the Centres of

Excellence, which will help to avoid or at least better manage the delivery

risks. As there may be a limit on how much development delivery bodies are

willing or able to undertake prior to grant approval, this can be overcome by

building in a mobilisation and development phase into delivery schedules.

This lesson is relevant to WEFO in particular, but should also be borne in

mind by Visit Wales and delivery bodies involved in designing centres of

excellence.

The provision of support and guidance from Visit Wales to the Centres of

Excellence and delivery partners to develop appropriate marketing activities

and to ensure integration with the overarching national marketing campaigns

as well as local efforts. Where appropriate, this can be achieved by Visit

Wales providing guidance and support to the Centres of Excellence and the

delivery partners, to enable them to develop their own strategies and to

integrate this with the national marketing campaigns.

The development by the Centres of Excellence and their delivery partners of

a clear monitoring and evaluation framework built upon a clear theory of

11

change to enable data collection before, during and at the final stages of an

investment; the inclusion of CCT targets at the investment levels; providing

clarity about the baseline against which change is measured, ensuring

monitoring requirements understood by all management and delivery

partners. Whilst all of these will ultimately be the responsibility of the Centres

of Excellence and their delivery partners, Visit Wales needs to provide the

guidance to enable them to do this effectively and in a coordinated manner.

It is particularly important that this framework establishes the approach to and

underpinning data requirements for assessing overall counterfactual impact of

the investments (i.e. what could be expected to be incurred in the absence of

the ERDF backed investments). Whilst this will always be challenging for

smaller and spatially dispersed investments, it is more achievable for larger,

spatially focused and more impactful investments. Visit Wales should take a

lead in investigating and developing the feasibility of these approaches and

providing guidance to delivery partners as appropriate.

In the case of projects like the Centres of Excellence, which have multiple

layers of management and delivery, providing clear guidance on the

responsibilities of partners at all levels for accurate monitoring, reporting and

record keeping. The approach has clearly worked well for ensuring that data

can be aggregated for the purposes of overall reporting to WEFO, however it

also needs to provide the scope for disaggregation to individual investments

for evaluation purposes (and make clear to Centre of Excellence managers

that the data needs to be made available to evaluators on this basis).

Earlier engagement with delivery partners on how to promote the cross-

cutting principles of sustainability, equality and social impact into their

projects. This also applies to planning for the long-term sustainability and

legacy of the investments themselves, providing clarity on how new visitor

infrastructure and facilities will be maintained, reinvested in and marketed.

Ensure that all delivery partners are clear about the basis on which they are

monitoring visits and visitors, how these relate to their original targets and

whether they are on a gross or net additional basis. It is clear that there has

been confusion amongst some partners and changes in the measures used

over time and in different reports.

1

Report Authors: Regeneris Consulting Ltd and The Tourism Company Ltd

Full Research Report: Regeneris Consulting and the Tourism Company 2017. Evaluation of the Coastal

and Sustainable Tourism Projects. Cardiff: Welsh Government, GSR report number 1/2018.

Available at: http://gov.wales/statistics-and-research/evaluation-coastal-sustainable-tourism-

projects/?lang=en

Views expressed in this report are those of the researchers and not necessarily those of the Welsh

Government

For further information please contact:

Joanne Coates

Social Research and Information Division

Knowledge and Analytical Services

Welsh Government, Cathays Park

Cardiff, CF10 3NQ

Email: Joanne.coates@gov.wales

Mae’r ddogfen yma hefyd ar gael yn Gymraeg.

This document is also available in Welsh.

© Crown Copyright Digital ISBN 978-1-78903-322-9