Defra Evaluation of the Catchment Based...

Defra

Evaluation of the Catchment Based Approach

Economic Assessment of the Catchment Based Approach September 2015

In Association with

YJRees

Consulting

Client: Defra Title: Evaluation of the Catchment Based Approach: Economic

Assessment of the Catchment Based Approach Project No: CC772 Date of Issue: September 2015 Status: Final Version Version No: 1.0 Produced By Authorised for Release By ……………………………….. …………………………………… Allan Provins Dr Kieran Conlan eftec Chief Executive CONTACT DETAILS

CASCADE CONSULTING

Enterprise House Manchester Science Park Lloyd St North Manchester M15 6SE Tel: 0161 227 9777 Fax: 0161 227 1777

Defra Evaluation of the Catchment-based Approach: Economic Assessment of the CaBA

Cascade Consulting

CONTENTS

1 INTRODUCTION .............................................................................. 1 1.1 Overview .................................................................................................. 1 1.2 Catchment Based Approach ..................................................................... 1 1.3 Economic Assessment ............................................................................. 1 1.4 Objective and Scope ................................................................................. 3 1.5 Report Structure ...................................................................................... 4 2 NATIONAL CONTEXT ...................................................................... 6 2.1 Background .............................................................................................. 6 2.2 Draft Programmes and Measures for RBMP2 ......................................... 6 2.3 Potential Contribution of Catchment Partnerships ................................. 7 2.4 Prevent Deterioration and Achieve Protected Area Objectives .............. 11

2.4.1 Eutrophication (As Part of Measures to Prevent Deterioration and Achieve Protected Area Objectives) 13

2.5 Improving Status ................................................................................... 14 2.6 RBMP3 and Beyond ............................................................................... 14 3 ASSESSMENT FRAMEWORK ......................................................... 16 3.1 Approach ................................................................................................ 16 3.2 Process Versus Impact Evaluation ........................................................ 18 3.3 Establishing the Additionality of Partners ............................................20 3.4 Availability of Data and Evidence .......................................................... 22 3.5 Assessment Structure ............................................................................ 24 4 Economic assessment .................................................................... 28 4.1 Inputs ..................................................................................................... 28

4.1.1 Total Inputs 29

4.2 Type of Inputs ........................................................................................ 29 4.2.1 Funding 29

4.2.2 In-Kind Inputs (Time Given Freely) 30

4.3 Source of Funds ..................................................................................... 31 4.3.1 Defra Funding 31

4.3.2 Other Public Sector Funding 32

4.4 Non-Public Sector Funding ................................................................... 33 4.5 Leveraging of Additional Resource Inputs ............................................ 34 4.6 Summary of Findings ............................................................................. 36 4.7 Activities ................................................................................................ 38 4.8 Expenditure ........................................................................................... 38 4.9 Cost Savings ........................................................................................... 42 4.10 Outputs .................................................................................................. 43 4.11 Number of Projects ................................................................................ 43 4.12 Type of Project ....................................................................................... 44 4.13 Waterbody Element Addressed ............................................................. 44 4.14 Expected Waterbody Improvement ....................................................... 46 4.15 Outcomes and Impacts .......................................................................... 46 5 CASE STUDIES ............................................................................... 47 5.1 Selected Projects .................................................................................... 47 5.2 Cost-Benefit Analysis ............................................................................. 48 5.3 Summary Results ................................................................................... 50 6 SUMMARY, FINDINGS AND CONCLUSIONS ................................. 52 6.1 Summary ................................................................................................ 52


Cascade Consulting

6.2 Findings ................................................................................................. 52 6.3 Conclusions ............................................................................................ 56 APPENDICES ......................................................................................... 58


Cascade Consulting 1

1 INTRODUCTION

1.1 OVERVIEW

The purpose of this report is to outline a framework for an economic assessment of

catchment partnerships within the Catchment Based Approach (CaBA). An initial

assessment is set out within this framework using available data and evidence. This

preliminary analysis highlights a number of gaps in the evidence base, which future

work could address in order to provide a more complete analysis of the benefits of the

CaBA within the structure of the framework.

This current report augments the main report ‘Evaluation of the Catchment Based

Approach – Phase 2’ and the summary of the evaluation survey (“S2”) and in-depth

interviews undertaken in April – May 2015. It should be read in conjunction with

these documents to gain a fuller understanding of the implementation of the CaBA.

The Phase 2 survey follows from the initial evaluation survey (“S1”) conducted in July

– August 2014.

1.2 CATCHMENT BASED APPROACH

The aim of the CaBA policy framework is set out in Defra (2013) ‘Catchment Based

Approach: Improving the quality of our water environment’. Overall, the aim is to

deliver improved water environment quality in accordance with targets set under the

European Water Framework Directive (WFD).

The CaBA establishes local partnerships that are expected to work with key

stakeholders in a catchment, particularly those that have an interest in water and the

wider environment in a catchment. Within their role, local partnerships may agree

and deliver the strategic priorities for the catchment and support the Environment

Agency in developing an appropriate River Basin Management Plan, as part of the

implementation of the WFD. In 2013, following a 2-year pilot phase, the CaBA was

widely adopted across England with over 100 catchment partnerships formed in 93

catchments1.

1.3 ECONOMIC ASSESSMENT

Economic assessment is an integral component of the ROAMEF2 policy cycle

described in HM Treasury Guidance for public sector bodies for appraising and

evaluation the funding of a policy, programme or project (Figure 1.1). There are three

main stages in the cycle where economic assessment can be applied:

1 A catchment is a geographic area defined naturally by surface water hydrology. There are 93 management catchments in England and a total of 107 catchment partnerships. 2 Rationale, Objectives, Appraisal, Monitoring, Evaluation and Feedback (ROAMEF).



Appraisal: to help identify the best way of delivering a policy prior to implementation,

by identifying alternative options which meet the stated objectives, and assessing the

costs and benefits that they are expected to lead to;

Monitoring: as part of the process of checking progress against planned targets and

compiling evidence that funding and outputs are being are being successfully

delivered; or

Evaluation: providing an assessment of the policy effectiveness and efficiency during

and after implementation. It seeks to measure outcomes and impacts in order to

assess whether the anticipated benefits have been delivered.

Figure 1.1 ROAMEF Cycle

Source: HM Treasury (2003) The Green Book Appraisal and Evaluation in Central Government.

Within the wider evaluation of the CaBA conducted by this study (see Main Report),

the economic assessment component largely falls between the monitoring and

(during implementation) evaluation phases of the ROAMEF policy cycle. This is

because the overall policy objective - improved water environment quality –

represents outcomes that will be delivered in the medium to longer term across a

number of iterations of the river basin planning process (e.g. RBMP2, RBMP3).

As evidenced in the main report, catchment partnerships are at varying levels of

maturity and many have only been set up since 2013. Necessarily the initial activities



of partnerships are focused on coordination and developing partnership working.

Across the partnerships, there remains a considerable number that are yet to identify

issues and appraise options for improvement within the catchment. Therefore

strategic plans and formal prioritisation of actions remain to be developed in a

number of cases. These are core building blocks that typically need to be put in place

before a coordinated process of establishing funding and delivering practical projects

and environmental improvements can be realised.

Consequently it is not possible to provide a formal (ex-post) economic evaluation at

this point of the CaBA in terms of the outcomes it is expected to deliver, and a

systematic assessment of the ‘value for money’ of catchment partnerships is

challenging.

It is possible, however, to set out a framework for an assessment that is consistent

with the key principles of an economic evaluation. The purposes of this would be to

monitor the progress of catchment partnerships towards delivering improved

environmental outcomes. Moreover the economic assessment framework can be

updated and refined as the CaBA and catchment partnerships mature and more

evidence on the environmental improvements that are being delivered becomes

available.

The economic assessment framework can therefore provide the basis for continued

and improved monitoring of the CaBA, particularly in relation to ensuring better

value for money from Defra funding.

1.4 OBJECTIVE AND SCOPE

The objective of the economic assessment is to review the evidence that is currently

available to help answer the policy question “Has Defra’s funding of catchment

partnerships been worthwhile?” Termed differently, this can also be phrased as “Is it

value for money for Government to fund partnership working in catchments?”

Notwithstanding the current limitations of this exercise that are highlighted in

Section 1.3, this evidence is required by Defra to assess the case for continued

funding of the CaBA beyond 2015.

The main way in which to conduct the economic assessment is via cost-benefit

analysis (CBA), which quantifies in monetary terms as many of the costs and benefits

of a policy, programme or project in order to examine the net public value generated

(to the UK/England as a whole). CBA represents one way in which ‘value for money’

(VfM) of a policy can be assessed. Section 3 sets out the economic assessment

framework along with definitions for key terms and VfM criteria.

The scope of the assessment is the national level implementation of the CaBA. This



can be interpreted to include the ‘hosting role’ of partnerships which is principally

resourced via Defra’s core funding (approx. £1.7million in total across all catchment

partnerships in 2014/15; £1.1million in 2013/14) and the role of partnerships in

securing funding for projects in catchments which aim to deliver water environment

improvements, and/or wider environmental, social and economic benefits.

Practically the scope of the assessment is constrained by the available data, allowing

only for a partial and, for the most part, indicative analysis at this stage. For example,

the Phase 2 evaluation survey provides economic data and information from 56 of the

107 catchment partnerships. This evidence underpins a summary analysis of the

levels of funding being drawn in by partnerships. It is indicative of the national

‘picture’ but is not fully representative and results should be interpreted accordingly.

A series of case studies are also presented, which are intended to be illustrative of the

projects and outcomes that partnerships are leading the delivery of. These are

selected examples of local level working in catchments and, again, cannot be

interpreted as representative of the national situation.

1.5 REPORT STRUCTURE

Following this introduction the remainder of this reported is structured as follows:

Section 2 – sets the CaBA against the implementation of the WFD across the

2nd and 3rd river basin planning cycles. The objectives and targets set out in

the WFD provide the underpinning policy rationale for the CaBA, and hence

the ultimate ‘value for money’ assessment is to understand how catchment

partnerships can impact on the expected costs and benefits of implementation

of the WFD (the ‘national context’).

Section 3 – outlines a framework for the economic assessment of the CaBA.

This developed around a ‘logic model’ approach that is typical policy

evaluation tool. The framework identifies quantitative evidence that can be

compiled to develop a formal economic analysis of the policy intervention. A

critical aspect of the analysis is determining the ‘additionality’ of catchment

partnerships.

Section 4 – provides an initial and partial economic assessment of the CaBA,

making use of data and evidence that is currently available. It includes

indicative assessments of levels of funding and leveraging of additional funds

by partnerships, the cost of hosting (coordination activities), and the types of

environmental outcomes to be delivered by partnerships through part of the

current funding that is available.

Section 5 – presents a summary of a set of case study examples that examine



the costs and benefits of individual that have been brought about by

catchment partnerships. These focus on assessing the additional benefits that

can attributed to the involvement of the partnership.

Section 6 – provides a summary of the main findings and conclusions, along

with recommendations for enabling a fuller and more systematic economic

assessment of the CaBA as the approach matures.

A supporting Appendix A is also included, which provides a summary of the case

study examples.



2 NATIONAL CONTEXT

2.1 BACKGROUND

The key objectives of the Water Framework Directive (WFD) are to prevent

deterioration and achieve good status in all water bodies across Europe3. The second

6-year cycle of management actions (RBMP2) are currently being prepared. These

will run from 2016 and update on progress towards WFD objectives in 2021.

The draft programme of measures for RBMP2 has been set out by the Environment

Agency in its consultation documents published in 2014. The draft programme

includes projects where catchment partnerships, or their information on costs or

benefits, have contributed to the Environment Agency’s planning and decision-

making.

2.2 DRAFT PROGRAMMES AND MEASURES FOR RBMP2

In its consultation documents, the Environment Agency sets out a series of future

management scenarios. This includes a scenario with the stated aim to ‘prevent

deterioration, achieve protected area objectives and improvements in status where

benefits exceed cost’ (subject to no affordability constraint) 4. It can be interpreted as

a reasonable representation of the scale of WFD implementation that is expected to

be required over RBMP2 and RBMP35. It is referred to as ‘Scenario 4’ by the

Environment Agency and is expected to result in 75% of water bodies achieving good

status overall by 2027 (compared to 30% currently). It sets out measures to address

three distinct WFD objectives:

New measures to help prevent deterioration in status. A changing environmental

baseline resulting from pressures of population growth, climate change and the

impact of invasive non-native species is set out to result in deterioration in

environmental quality without additional measures in waterbodies.

The programme of measures is based on nationally held information about

measures, costs and benefits broadly allocated to the sectors whose activities

cause the problem (e.g. polluter pays).

3 The European Water Framework Directive (WFD) is a legal framework for managing the water environment across Europe. It requires measures to be taken to ensure the sustainable use of water and to protect and improve inland surface waters, groundwaters and coastal waters with the aim of achieving ‘good status’. Implementation of the Directive is dependent on holistic and integrated approach to managing the water environment, which requires many stakeholders working together to design and implement improvements in waterbodies. 4 Environment Agency (2014) Economic Analysis Extended Report for River Basin Management Planning. 5 In the sense that the alternative scenarios examined by the Environment Agency – ‘do nothing (Scenario 1), ‘only do things that prevent deterioration or are appropriate to protected areas’ (Scenario 2) and ‘do everything technically feasible’(Scenario 3) – do not represent cases that reflect the overall objectives of the WFD and application of certain exemptions related to technical feasibility and disproportionate cost. Scenario 5 provides an illustration of potential progress towards Scenario 4 by 2021.



New measures to achieve protected area objectives. Additional measures needed

to achieve protected area objectives for:

Drinking water protected areas: surface water and groundwater

Economically significant species (shellfish waters)

Recreational waters (bathing waters)

Nutrient sensitive areas (urban waste water treatment directive)

Natura 2000: water dependent special areas of conservation and special

protection areas for wild birds.

New measures to improve status. Additional measures – from the technically

feasible set - to achieve good status by 2027 in failing elements or water bodies

where there is no driver based on preventing deterioration or achieving protected

areas objectives. These measures are promoted by the Environment Agency

following economic appraisals undertaken at a catchment level. These appraisals

assess the costs and benefits of ‘bundles’ of measures needed to improve and

restore most of the catchments in England. Promoted measures are those

considered by the Environment Agency as technically feasible and economic

appraisal indicates that benefits justify costs. No measures have been ruled out on

the basis of affordability constraints or available funding.

2.3 POTENTIAL CONTRIBUTION OF CATCHMENT PARTNERSHIPS

As noted in Section 1.2, the CaBA promotes the engagement of local stakeholders to

establish solutions and develop partnerships to implement actions at the local level.

Balancing environmental, economic and social demands and aligning funding and

actions within catchments is intended to bring about long term environmental

improvements that contribute to WFD objectives and wider.

The Environment Agency’s estimate of the cost of ‘Scenario 4’ (‘prevent deterioration,

achieve protected area objectives and improvements in status where benefits exceed

cost’, subject to no affordability constraint) reveals both the scale and burden that is

expected to be shared across sectors in implementing the WFD:

The estimated cost for the draft programme of new measures set out in RBMP2 is

substantial, totalling £12.1billion in present value terms6.

The split of cost by sector is: Government 14%; rural land management 40%;

industry, services and other sectors 10%; and water industry 36%.

The associated benefits are estimated to be approximately £20.6billion in present

6 All present value cost and benefits cited are discounted over a 37 year time horizon from 2015 at the start of Cycle 2 and ending 2052. A discount rate of 3.5% is applied for the first 30 years and then 3% thereafter, in accordance with HM Treasury (2003) Green Book guidance.



value terms, with some 2,256 waterbodies improved to good status.

The potential for catchment partnerships to contribute to: (a) lessening the burden of

WFD implementation and/or (b) enhancing the benefits; has not been (fully)

assessed. It is though potentially substantial and an indicative assessment is

presented in Table 2.1. This is based on the interpretation of the Environment

Agency’s consultation documents (and other RBMP2 planning documentation),

which illustrates that with even relatively modest assumptions, substantial benefits

from the working of catchment partnerships can be suggested.

The indicative assessment set out in Table 2.1 is predicated on three general ‘added

value’ objectives:

A. Catchment partnerships can contribute to achieving WFD objectives at lower cost

than currently estimated (i.e. reduce the cost of WFD implementation); and/or

B. Catchment partnerships can contribute WFD objectives being achieved more

quickly than currently expected (i.e. deliver the benefits of WFD implementation

earlier); and/or

C. Catchment partnerships can contribute by delivering more environmental

outcomes for the same cost (i.e. increase the benefits of WFD implementation

and/or other policy objectives).

In effect, the Environment Agency’s programme of measures and associated costs

and benefits provide ‘targets’ for out-performance. For example a 5% cost-efficiency

from local level ‘ground truthing’ of proposed measures to prevent deterioration and

achieve protected area objectives implies a national level benefit and added value in

the region of £200million (undiscounted), through avoided costs. Similarly, a 5%

enhancement of environmental outcomes through local level input these measures

imply a national level benefit and added value in the region of £500million. The

following sub-sections support the assessment provided in Table 2.1 by outlining in

more detail the underpinning assumptions.

The realisation of these benefits – and the proportion that can be realistically

delivered at the national level - is though critically dependent on a number of factors

beyond the scope of this report that require further examination, including the ability

of partnerships to influence plans in relation to the specific water body management

issues highlighted, access to funding to address these, and scope for leading delivery.

Indeed there are many critical dependencies in such judgements and these require

further examination and review as part of the continued policy implementation.



Table 2.1 Measures of Success and Mapping to Logic Model

Broad WFD objective

Estimated cost/benefit*

Potential contribution of catchment partnership

Indicative assessment – potential scope

Sources of funding

Prevent deterioration and achieve protected area objectives

Costs: £4.6billion (PV)

Govt: £0.4bn ISI: >£0.1bn RLM: £4.3bn WI: £2.2bn (undiscounted)

1. Out-perform (reduce) the costs set out in draft plans by providing more locally applicable values and local option appraisal of cost-effectiveness.

5 - 10% efficiency (for example in relation to RLM measures):

approx. £215 - £430m (undiscounted)

Funding of local-scale decision making (central Government funding of catchment partnerships)

Match funding (funds - Grants e.g. HLF or People’s Postcode Lottery Dream Fund, Local Growth Fund, Princes Countryside Fund) Contribution in Kind (staff time, staff time, provision of services, volunteer time for surveying, overheads of project deliverer)

Funding to address:

Physical modification (flood protection, urban areas, land management associated sediment, impoundments, barriers to fish, land drainage)

Invasive non-native species (bio-security and access to volunteer activity for eradication works)

2. Increase the partnership funding, volunteer activity and benefit in kind from the low assumption used in planning purposes (reduce overall costs/change distribution).

Benefits: £10.9billion (PV) Prevent deterioration: £6.8bn (PV)

Achieve protected area objectives: £4.1bn (PV)

3. Out-perform (increase) the benefits identified in planning purposes:

Increase to Good status elements at risk of deterioration

Increase the status of other failing elements to Good

Where measures are set out (at planning) to achieve protected area objectives, also increase the status of failing elements to Good.

Bring forward the date by which measures would be delivered, thereby increasing the number of years over which the benefit value applicable.

5 - 10% additionality of benefits: approx. £500 - £1,000m

Eutrophication Costs/benefits of addressing individual pressures is not available

4. Enhance measures to prevent deterioration of other failing elements can help to reduce eutrophication pressures

Potential benefits are a subset of prevent deterioration and achieve protected area objectives reference in (3) above.

Government: grant in aid funding at 2014 level (including WFD catchment restoration fund); New Environmental Land Management Scheme and rural development grants. Rural land management: Rural Development Programme



Broad WFD objective

Estimated cost/benefit*

Potential contribution of catchment partnership

Indicative assessment – potential scope

Sources of funding

Funding to address:

Phosphorus, plant and algal communities, groundwater Nitrate Vulnerable Zones (NVZs) and Urban Wastewater Treatment Directive Nutrient Sensitive Areas (NSAs).

B. Improving status

Costs: £7.5billion (PV)

Govt: £1.9bn ISI: >£1.2bn RLM: £2.2bn WI: £3.7bn (undiscounted)

Potentially limited - local evidence has informed the assessment of measures to be promoted to improve status.

Limited -

Benefits: £9.7billion (PV)

C. RBMP3 and beyond

Not available Setting of less stringent objectives in relation to exemptions for natural conditions and technical feasibility (no known technical solution) and where costs are not justified by benefits (disproportionate cost).

5. Determining measures to address:

Eutrophication

Flow pressures

Chemicals

Barriers to fish movement

Programme of measures is not well defined at present so there is opportunity for partnerships to influence it and shape it going forward; i.e. RBMP3 revise £12.1bn (PV) cost

Source of funding across all sectors – with particular emphasis on Water Industry and Rural Land Management funding streams for eutrophication; Water industry and Industry, services and other sectors for flow pressures; Water Industry and Government (SUDS) for chemicals; and Government for barriers.

Notes: Govt = Government; ISI = Industry , services and other sectors; RLM = Rural land management; WI = Water industry. *Cost and benefit estimates sourced from Environment Agency (2014) Economic Analysis Extended Report for River Basin Management Planning.



2.4 PREVENT DETERIORATION AND ACHIEVE PROTECTED AREA

OBJECTIVES

Addressing WFD objectives to prevent deterioration and achieve protected

area objectives represents a sizeable proportion of the ‘Scenario 4’ costs

outlined by the Environment Agency:

The estimated cost for the draft programme of new measures set out in

RBMP2 to prevent deterioration and achieve protected area objectives

is reported to be £4.6billion in present value terms.

The split of cost by sector is: Government 6%; rural land management

62%; industry, services and other sectors 1%; and water industry 31%.

The associated benefits are estimated to be approximately £10.9billion

in present value terms, with some 60 waterbodies achieving good

status.

Review of evidence from the Environment Agency indicates that catchment

partnerships have not significantly influenced the projects promoted to

prevent deterioration or achieve protected area objectives in RBMP2. The

development of the programme has been based mostly on nationally

available information. It has not been subject to CBA7; however, the overall

benefit-cost ratio (approx. 2.3:1) suggests a reasonable ‘value for money’

case for the measures (noting that affordability has not yet been tested).

The associated benefits are interpreted to be composed of £6.8bn in

present value terms for preventing deterioration together and £4.1bn in

present value terms for improvements in protected areas8. Whilst

catchment partnerships have not substantially influenced the planning

process, there would appear to be significant scope for partnerships to

influence delivery of RBMP2 for the range of measures set out to prevent

deterioration and achieve protected area objectives.

The significant water management issues pressures with the biggest risk of

negative impact on water body status by 2027 are physical modification and

invasive non-native species. The highest pressure on physical modification

on surface waters is associated with flood protection, urban areas and land

7 CBA of these measures was not a requirement as noted in Environment Agency (2014) Economic Analysis Extended Report for River Basin Management Planning. 8 This is based on interpreting the Scenario (dis-benefits) of approximately -£6.8billion (present value terms) as representing the cost of deterioration in status. However this should interpreted as indicative as it is no information is available on the assumed timeline for deterioration in status, nor how assumptions underpinning Scenario 1 correspond to Scenario 2 (i.e. whether there some deterioration over time because measures cannot be brought forward in time).



management associated sediment. Impoundments, barriers to fish and

land drainage are next highest pressure on physical modification.

The collaborative working approach from catchment partnerships offer

significant input into the national strategy for the Invasive non-native

Species Framework Strategy for Great Britain – through enhanced

education on bio-security and access to volunteer activity for eradication

works.

Some initial evidence compiled for this report, indicates that catchment

partnerships are leading projects that are targeted at these water

management issues; particularly urban areas, land management, and

particularly impoundments/barriers to fish (see Section 4.3). The caveat to

support the assessment provided in Table 2.1 is that these measures need to

be applied in a ‘prevent deterioration’ setting.

Roughly 20% of the projects applying for current funding by partnerships

are targeted at “prevent deterioration” or “contribute to the achievement of

protected area objectives”9. There is also evidence from the Environment

Agency that partnership working can work for flood protection10.

The key assumption in this assessment is that the draft programme of

measures selected from planning purposes would benefit from extensive

ground-truthing with local stakeholders. Such action could be expected to

substantially focus the range of measures selected to:

1. Out-perform (reduce) the costs from those used in planning purposes

by providing more locally applicable values and local option appraisal of

cost-effectiveness.

2. Increase the incidence of partnership funding, volunteer activity and

benefit in kind from the low assumption used in planning purposes

(and thereby reducing overall costs and/or changing the distributions

by sector).

3. Out-perform (increase) the benefits identified in planning purposes:

9 See Table 4.1 and Figure 4.1 (Section 4.3). 10 Defra (2015) Central Government Funding for Flood and Coastal Erosion Risk Management in England, February 2015. Expenditure by Central Government, including expenditure by the EA funded by local levy and EA income raised through the Partnership Funding scheme, Internal Drainage Board Precepts; General Drainage Charges; and Sales of Assets, averaged out at £660m per year from 2010/11-2013/14. About £606m of this was funded by government (ie 92%), of which 45% was capital expenditure, and £28m from the local levy (4.2%). In addition to the funding set out above, there is further funding which does not come through Central Government at all. This includes the proportion of Partnership Funding which does not come through EA and Local Authorities who may choose to spend their settlement funding on FCERM activity (separate to the element given to Lead Local Flood Authorities). This funding is a matter for Local Authorities, who are free to set their own priorities and are accountable to local communities for the effectiveness of their spending decisions.



Where measures are set out (at planning) to only reverse the risk

of status deterioration, catchment partnership working offers

opportunity to increase the status to Good for the elements at risk

of deterioration; for example in a river water body where current

status of fish is Moderate status and the risk to which measures

are planned is to prevent deterioration to Poor status.

Where measures are set out (at planning) to only reverse the risk

of status deterioration, catchment partnership working offers

opportunity to increase the status of other failing elements to

Good; for example in a river water body where current status of

fish is Moderate status and the risk to which measures are

planned is to prevent deterioration to Poor status, enhanced

targeting of measures may also improve the status of

macroinvertebrates from Moderate to Good status.

Where measures are set out (at planning) to achieve protected

area objectives, there appears significant potential to also

increase the status of failing elements to Good.

Bring forward the date by which measures would be delivered,

thereby increasing the number of years over which the benefit

value applicable.

A reduction in costs of delivery would also likely enhance the affordability

of the programme of measures.

2.4.1 Eutrophication (As Part of Measures to Prevent Deterioration

and Achieve Protected Area Objectives)

The programme to prevent deterioration and achieve protected area

objectives has no focus on eutrophication, unless this is a target of Natura

2000 sites that are not at favourable condition for nutrient pressures.

Consequently no measures are targeted at river and lake water body status

for phosphorus, plant and algal communities (each of which are WFD

status elements), groundwater Nitrate Vulnerable Zones (NVZs) and Urban

Wastewater Treatment Directive Nutrient Sensitive Areas (NSAs).

Where measures aimed at preventing deterioration of other failing

elements can also reduce eutrophication pressures, these could be explored

through ground-truthing with local parties as they represent the greatest

opportunity for increasing benefit for this part of the promoted programme,

for reducing costs in the “improving status” part of the programme, and

overall increasing the number of water bodies that can achieve Good status

for nutrients, plants and algae.



2.5 IMPROVING STATUS

The draft programme of new measures set out to improve status in

RBMP211 is also substantial:

This is estimated to be approximately £7.5bn in present value terms12.

The split of cost by sector is: Government 21%; rural land management

24%; industry, services and other sectors 1%; and water industry 24%.

The associated benefits are estimated to be £9.7bn in present value

terms, with some 2,196 waterbodies achieving good status.

Review of evidence from the Environment Agency indicates that catchment

partnerships have had opportunity to and, to some extent have influenced

the projects promoted to improve status. As local evidence has already

been used in the assessment of measures to be promoted to improve status,

at face value there are expected to be fewer opportunities for catchment

partnership, in particular ground-truthing with local parties, to enhance the

delivery of these measures in RBMP2.

2.6 RBMP3 AND BEYOND

The Scenario 4 programme of measures set out by the Environment Agency

for RBMP2 will not achieve Good status in all waterbodies. Less stringent

objectives would be set for water bodies where exemptions for natural

conditions and technical feasibility (no known technical solution) apply.

Less stringent objectives would also be set where costs are not justified by

benefits. Although the balance of costs and benefits should be taken into

account in setting water body objectives, this scenario does not fully take

account of all disproportionate cost considerations, for instance

distributional impacts (affordability).

At present the Environment Agency has not set out information on the

costs and benefits for a programme of measures to address these less

stringent objectives. It is not clear to how many waterbodies this may apply

or what the range of elements that would remain less than Good status are.

From available documentation, however, the main significant water

management issues pressures that would require this approach to less

stringent objectives appear to be:

11 Calculated as the difference between reported costs for Scenario 4 (approx. £12.1bn, PV) and Scenario 2 (approx. £4.6bn, PV) by the Environment Agency. 12 PV using 37 year assessment period (from 2015 at the start of cycle 2, ending 2052) and a discount rate of 3.5% for the first 30 years and then 3% thereafter, as per Treasury Green Book guidance.



Eutrophication13: research is underway in RBMP2 to review measures

for river nutrient pressures affecting reactive phosphorus status which

are not currently technically feasible and this will influence the decision

making in RBMP3.

Flow pressures: In RBMP2 consultation documentation, the

Environment Agency state that flow pressures on river ecology

(principally linked to the status of fish and macroinvertebrates) will not

be fully addressed by cost-beneficial measures; although the decision

making process for RBMP3 is currently unclear.

Chemicals: In RBMP2 consultation documentation, the Environment

Agency consider that many of the measures to deliver Good Chemical

Status of water bodies lie outwith the WFD and have excluded these

from RBMP2.

Barriers to fish movement: Without new legislation the ability to

address barriers to the movement of fish are limited; and the decision

making process for RBMP3 is currently unclear.

13 The AMP6 National Environment Programme phosphorus removal technologies trial.



3 ASSESSMENT FRAMEWORK

Addressing the policy question “Has Defra’s funding of catchment

partnerships been worthwhile - i.e. is it value for money for Government to

fund partnership working in catchments?” requires careful framing. In

effect, there are two perspectives that can be adopting in addressing this

question:

The value for money of partnership hosting and whether catchment

partnerships are effective as a mechanism for local coordination and

input catchment planning; or

The value for money of the Catchment Based Approach, and its

potential to contribute to implementation of the WFD as highlighted in

Section 2 (the ‘added value’ in terms of the scale of potential cost

savings and additional benefits).

A narrow focus on core funding for (primarily) partnership hosting

(approx. £1.7m in 2014/15) presents a challenge since, conventionally,

economic assessments are concerned with the outcomes and impacts of

policy interventions. Coordination and input to planning are ‘processes’

and not outcomes or impacts, so the relevant question is largely whether

catchment partnerships represent a more cost-effective way of ensuring

local level input to planning.

This perspective, will not though, inform on the ‘added value’ and whether

the CaBA is worthwhile, since the ultimate policy objective is to deliver an

outcome of improved water environment quality in accordance with targets

set under the European Water Framework Directive (WFD). This implied a

broader perspective is required, which tracks the ‘additional’ impact that

catchment partnerships can deliver in terms of the cost of WFD

implementation or the associated benefits. This latter perspective is

adopted for the framework set out in this section; note also though that the

framework also incorporates the narrower cost of hosing perspective.

3.1 APPROACH

A ‘logic model’ is a commonly applied tool for framing assessments of

policy interventions. As recognised in the HM Treasury (2011) Magenta

Book Guidance for Evaluation a logic model represents a formal way of

describing how a policy intervention is intended to achieve its aims (Figure

3.1). It represents a ‘theory of change’ via series of steps describing the

relationship between an intervention’s inputs, activities, outputs, outcomes,



and impacts:

Inputs: resources required to achieve the policy objectives (e.g. funding,

capital and operating expenditure);

Activities: the actions and work that are undertaken by the

organisations and agencies tasked with implementing the policy (e.g.

hosting, coordinating stakeholders, producing plans);

Outputs: what is delivered by the organisations and agencies through

their activities (e.g. km of waterbody improved, number of fish passes

created);

Outcomes: the consequences of the policy, in terms of the policy

objectives and/or any unintended outcomes (e.g. improvement in water

body status as measured by WFD criteria).

Impacts: the ultimate impact on social wellbeing from economic, social

and environmental outcomes (e.g. the benefits associated with

improved WFD status of a water body).

Adapting the logic model concept and developing a framework for the

assessment of catchment partnerships within the CaBA requires

consideration of a number of factors:

The evaluation ‘perspective’ (process vs. impacts);

The appropriate counterfactual for assessing the ‘additionality’ of

catchment partnerships; and

The available monitoring data and evidence.

Figure 3.1 Outline of logic model



3.2 PROCESS VERSUS IMPACT EVALUATION

A key determination to be made is whether an evaluation should focus on:

(a) how a policy was/is being delivered (the process); or (b) what difference

the policy has made (the impact)14. Turning this to the CaBA, the relevant

perspectives to understand are the processes that catchment partnerships

are engaged in (e.g. activities – see Figure 3.1) versus the impact that

catchment partnerships are delivering, in terms of improved water

environment outcomes along with wider environmental, social and

economic outcomes.

The scope and objectives for an economic assessment are more readily

interpreted in terms of an impact evaluation, since this extends the

evaluation question from whether the intended outcomes were generated

by a policy, to whether those outcomes were justified, in terms of the

balance of costs (inputs) and benefits (outcomes/impacts).

The stated ‘measures of success’ for CaBA, are however, as outlined in the

Defra policy framework, indicative of a process evaluation15. Primarily they

focus on the activities and processes that catchment partnerships are

expected to undertake, rather than the outcomes that the policy is intended

to deliver. Table 3.1 summarises the Defra measures of success in

conjunction with the applicable logic model step.

Table 3.1 Measures of success and mapping to logic model

Measure of success1 Logic model mapping

Stakeholders are closely involved in

identifying priorities and taking action

to address them.

Activity – relates to engagement and

coordination role of catchment

partnerships (process evaluation focus)

Stakeholders understand their role in

the CaBA and their influence on river

basin planning.




Additional joint action and external

investment is leveraged, with Defra

delivery body officers providing key

input into the securing of resources

Input – relates to cost of policy

implementation (leveraging - impact

evaluation focus) Activity – relates to identifying funding

sources and coordination role of

catchment partnerships (joint action -

process evaluation focus)

14 The HM Treasury (2011) Magenta Book describes ‘process evaluation’ as concerned with the activities and pathways involved in its implementation and delivery of a policy. This is contrasted to an ‘impact evaluation’ which focuses on the outcomes of the policy, defined as the measurable achievements through either the objectives of the policy (outcomes in the logic model concept) or the benefits generated (impacts in the logic model concept). 15 The proposed measures of success are set out in Defra (2013) ‘Catchment Based Approach: Improving the quality of our water environment’ [p14].



Measure of success1 Logic model mapping

Measures are planned in a more

coordinated, strategic and better

targeted way that takes account of

ecosystem function




Different planning systems and plans

are brought together in a synergistic

way, with common strategic and spatial

points of reference




Notes: 1 See Defra (2013) Catchment Based Approach: Improving the quality of our water environment, p14-15.

As Table 3.1 highlights, only one aspect of the measures of success that are

set out for catchment partnerships can be readily interpreted as a

component of an impact based evaluation and economic assessment. This

is in relation to leveraging of external funding, which relates to the ‘inputs’

stage of the logic model, and consequently the ‘cost’ element of an

economic assessment.

The process evaluation focus of the Defra measures of success is also

consistent with the catchment partnership contracts, which establish

expectations in relation to the host role as follows:

Partnerships are to work collaboratively and in a way that enables all

catchment stakeholders to input in to the planning and delivery of

environmental outcomes;

Partnerships are to set out priorities in a way that is “meaningful for

them” (possibly by production of a formal catchment plan, although

this is not a requirement);

Partnerships to work towards becoming financially self-sustaining; and.

Partnerships are to respond to the Environment Agency’s updated

River Basin Management plan consultation.

The significance of noting the process evaluation focus of the Defra

measures of success and the catchment partnership contracts is to highlight

that the expectations that have been set to date for monitoring and

evaluating the CaBA do not explicitly recognise economic assessment as a

core consideration. This has potential implications for the data and

evidence that are available to support an economic assessment since

indicators and metrics that are integral to economic assessment are not



central to the policy implementation and evaluation perspective16.. Defra

considered mandating various requirements (such as a Catchment

Management Plan) during the development of the programme, however it

was decided that the final contract with the partnerships would not include

such specific requirements.

3.3 ESTABLISHING THE ADDITIONALITY OF PARTNERS

Regardless of data and evidence constraints (see below), a critical aspect of

impact-based evaluation and economic assessment is determining the

‘additionality’ of the policy intervention. In effect, this is the outcome(s)

delivered by the policy over and above the outcome(s) that would be

expected to be delivered (by Defra) in the absence of the policy (the so-

called ‘reference case’/’counterfactual’). This can be regarded as the ‘added

value’ that is generated by the CaBA and catchment partnerships.

For CaBA and local partnerships the frame of reference is implementation

of the WFD. Section 2 recognised the broad value for money opportunities

for CaBA in terms of: (i) achieving WFD implementation more cheaply; (ii)

achieving WFD implementation more quickly; and/or (iii) delivering more

benefits for the same cost. The implied reference case is a strategy for

implementing the WFD with alternative mechanism for locally focussed

input and delivery. One potential alternative would be for the Environment

Agency to resume the functions of the catchment partnerships.

At the partnership level, the reference case needs to be aligned to the logic

model, since the counterfactual needs to be determined at each point along

the inputs – activities – outputs – outcomes - impacts chain to assess how

the additionality may be assessed in economic terms:

Inputs: a local partnership may draw in additional funding to support

its activities and deliver projects that lead to environmental and social

outcomes. If funding is sourced from the private sector to match-fund

public sector investment, this ‘leveraging’ can be interpreted as an

‘additionality’ if the reference case is for public sector funding-only for

the activities and projects delivered by partnerships.

Leveraging, however, does require careful interpretation. From an

economic analysis perspective, both public and private sector

investment represent the cost of delivering a policy intervention. Hence

16 For example, the HM Treasury (2011) Magenta Book states: “Economic approaches value inputs and outcomes in quite particular ways, and it is crucial that the needs of any economic evaluation are considered at the [policy] design stage. Otherwise, it is very likely that the evaluations will generate information which, although maybe highly interesting and valid in itself, is not compatible with a cost-benefit framework, making it very difficult to undertake an economic evaluation” [p20].



measuring leveraging does not provide an indication of whether the use

of resources (public and private funding) is/was worthwhile. This

requires consideration of the outputs and outcomes that are delivered

relative to the cost of the policy implementation, in order to make value

for money assessments (see below).

Activities: if it is taken as given that implementation of the WFD

requires locally focussed decision-making to influence river basin

planning (which is the policy ambition stated in Defra’s framework for

the CaBA), then a key aspect of the scope for additionality for local

partnerships is the cost of coordination and activities that influence

river basin planning. A reasonable assumption for the reference case

would be that these activities related to local level input would be

carried out by the Environment Agency (the competent authority

responsible for preparing river basin plans) in the absence of catchment

partnerships.

Cost savings against the reference case therefore represent a measure of

additionality in relation to activities. This, however, requires an

assumption that the equivalent activities and results are delivered as

would be under the counterfactual. Cost saving therefore should be

balanced against measures of effectiveness, which in the context of

activities can include qualitative or (semi-)quantitative assessments of

coordination/plan influencing activities. Assessing cost-effectiveness in

these terms, is though, more relevant to the process evaluation

perspective, rather than impact evaluation.

Outputs and outcomes: for local partnerships the reference case with

respect to outputs and outcomes are the measures delivered via river

basin management plans. Hence there are two main aspects for

additionality:

o Water body improvements: projects delivered via partnerships

result in: (i) a greater level of output/outcome than would be the

case if measures were delivered in the absence of the

partnership – including the case of the measures not being

taken forward; and/or (ii) the output/outcome is achieved

earlier due to the partnership, such as within RBMP2 as

opposed to RBMP3.

o Wider economic, social and environmental outputs/outcomes:

projects delivered via partnerships give rise to wider positive

impacts; for example improvements to terrestrial habitats,



hazard/risk reductions (e.g. flood risk), employment, etc.

Relating (physical) measures of outputs and outcomes to costs of

delivery (e.g. project capital and operating costs) provides the basis for

cost-effectiveness assessments; i.e. whether partnerships help to deliver

WFD outputs/outcomes more cheaply relative to the reference case.

Impacts: measuring the benefits associated with WFD and wider

economic, social and environmental outcomes in monetary terms

provides the basis for a value for money assessment within a CBA

setting. This allows the additionality of partnerships (cost savings

and/or additional outputs and outcomes relative to the reference case)

to be assessed in net terms; i.e. whether CaBA is justified in terms of the

balance of costs and benefits, based on the additional outcomes that are

generated.

Following the logic model, an economic assessment of the CaBA can be

built up with data and evidence and appropriate interpretation against the

reference case to determine the additionality delivered by catchment

partnerships. This would then allow a ‘complete’ examination of whether

the partnership working represented value for money for public funds.

However this aspiration needs to be set against the context of the policy,

which is to deliver improved water environment quality through medium to

long term planning processes. Consequently – as highlighted in Section 1.3

- only a partial assessment is possible in the short term, reflecting progress

made in the initial 2 years of the widespread adoption of the CaBA.

Crucially, though, this assessment is dependent on the data and evidence

currently available.

3.4 AVAILABILITY OF DATA AND EVIDENCE

Data and evidence that are presently available to underpin an economic

assessment of the CaBA and catchment partnerships includes:

The Phase 2 evaluation survey data (“S2”) (see Main Report), which

includes data for 56 of the 107 catchment partnerships. This includes

levels of funding by source reported by catchment partnerships for the

period 2014/15.

In-depth interviews with a sample of catchment partnerships. A total of

6 partnerships (Combined Essex, Wye, Wear, Test and Itchen, Middle

Dee, and Soar) participated in in-depth interviews and follow-up with

the project team in May and June 2015. However, the information from



only 5 partnerships was used as the Middle Dee evidence was not used

to assist with this exercise. Information and data sourced from

partnerships has informed the case study example presented in Section

5.

RBMP2 appraisal summaries. Environment Agency coordinators have

provided appraisal summary evidence for the catchments that

participated in the in-depth interviews to support the case study

examples. This includes details of RBMP2 waterbody measures for

these catchments and the associated monetary estimates of costs and

benefits.

Catchment Partnership Action Fund (CAPF) (2015/16). A summary of

catchment partnership projects to be funded by CAPF during 2015/16.

This is a Defra fund administered by the Environment Agency with a

total funding level of approximately £4.78 million. Data provided by the

Environment Agency (May 2015) reports the project aims, waterbody

pressures to be addressed, and expected outcomes, along with details of

match funding and in-kind contributions17.

Overall, the data and evidence available to underpin an economic

assessment at this stage is limited, and has to be pieced together from

various sources. By and large this reflects the observation that the

expectations that have been set for monitoring of the CaBA focus on

process evaluation, rather than impact evaluation and economic

assessment.

In particular there is no coherent reporting structure for CaBA and

partnerships that is consistent with an economic assessment logic model

and requirements for determining the value for money of public funding.

The practical implications of this are described subsequently in Section 3.2.

Whilst the aspiration is to provide a national level assessment of the CaBA,

this has not been possible within the scope of this study. However, case

study evidence has been compiled to assess the value for money of

individual projects that have been led by catchment partnerships. The

examples that are subsequently provided (Section 5) cannot be interpreted

as representative of the national situation, but they do provide examples

where projects are being implemented through the work of partnerships.

The analysis focuses on the additional outcomes that are to be delivered,

consistent with the assessment framework that is outlined.

17 Note that data provided to the project team in May 2015 is incomplete and subject to further review by the Environment Agency.



There is potential for these ‘bottom-up’ analyses to be expanded to provide

a more comprehensive evidence base to underpin future assessments. This

is dependent though on improved data and evidence, which would make

project level analysis more readily replicable.

3.5 ASSESSMENT STRUCTURE

Following the logic model approach described in Section 3.1, Table 3.2 sets

out a structure for an economic assessment of the CaBA and local

partnerships. It is intended that the structure could be updated and refined

as the CaBA and catchment partnerships mature and more evidence on the

environmental improvements that are being delivered becomes available.

This would be facilitated by future data gathering in line with the ROAMEF

policy cycle and the economic assessment of impacts.

The assessment structure is framed in terms of an impact evaluation and

the examination of the ‘additionality’ of catchment partnerships. A process

focused evaluation concentrating on activities undertaken by catchment

partnerships in relation to Defra’s measure of success is provided in the

Main Report, based around the Phase 2 evaluation survey responses and in-

depth interviews with 5 catchment partnerships and a range of beneficiary

organisations.

Table 3.2 sets out the structure for the economic assessment in terms of

the:

Role of the partnership: the purpose of catchment partnerships within

each step of the logic model;

Reference case: the assumed counterfactual for assessing the

additionality of catchment partnerships;

Additionality: the basis for assessing the added value (net benefit)

delivered by catchment partnerships in comparison to the reference

case;

Indicators/metrics: the quantitative and monetary metrics for evidence

to assess the contribution of catchment partnership (both overall and

the component that is additional to the reference case);

Available evidence: the sources of data and information currently

available to populate the economic assessment framework; and

Scope of assessment: the extent of the analysis that can be currently

undertaken at present.



Table 3.2 Framework for economic assessment of catchment partnerships

Logic model step

Inputs Activities Outputs Outcomes Impacts

Partnership role

Leveraging external investment

(private sector, in-kind) in

catchments to deliver

environmental improvements

Hosting and coordination,

influencing planning, delivery

of projects/measures

Delivery of waterbody

measures/improvements and

other economic, social and

environmental outputs

Improvements in water body

status (overall and component

elements) and other economic,

social and environmental

outcomes

Improve social wellbeing

through improved water

environment quality

Reference case Public sector funding only Environment Agency led-local

planning activities

Outputs delivered via RBMP in

absence of partnerships





Additionality Investment over and above

public sector funding

Delivery of activities at lower

cost

Outputs over and above RBMP

measures and outputs

Outcomes over and above

RBMP outcomes

Benefits over and above RBMP

outcomes

Indicators/

metrics

Total funding by source (£) Cost of activities (£) Range of physical metrics,

including: no. projects; no.

measures (e.g. fish passes);

waterbody length (km); etc.

Water body status (overall

ecological/chemical status)

Water body element status (e.g.

fish, invertebrate, phosphate,

etc.)

Monetary value of benefits (£)

Available evidence

(source)

Catchment partnerships

(survey responses)

CPAF applications

(Environment Agency)

Catchment partnerships

(survey responses)

Pilot Evaluation (Environment

Agency response)

Catchment partnerships (in-

depth interviews )

CPAF applications


RBMP2 appraisals



depth interviews )

CPAF applications


RBMP2 appraisals



depth interviews )

NWEBS (Environment Agency)

Current scope for

economic

assessment

Assessment of leveraging in

2014/15 (indicative national):

Core funding: all

other sources

Core funding: other

Estimation of annual cost

saving (indicative national)

Summary of CAPF projects

2014/15 (national)

Case study examples – CBA

(individual projects)

Case study examples only –

CBA (individual projects)

Case study examples only –

CBA (individual projects)



Logic model step

Inputs Activities Outputs Outcomes Impacts public sector

Core funding: private

sector



Table 3.3 evaluates the current scope for the assessment against a set of

value for money assessment criteria that are typically considered in

economic evaluations. As noted the framework can be updated in future to

reflect improved data and evidence, if this becomes available, in order to

provide a ‘complete’ assessment against these criteria.

Table 3.3 VfM Assessment Criteria

VfM criteria Measurement Evidence needs Possible in

current

assessment?

Economy

Cost of resource

inputs

Inputs (funding/

investment) and cost

of activities

Yes – indicative

national analysis

provided

Efficiency

Relationship between

inputs (resources)

and outputs

Inputs and

quantitative metric

for outputs (e.g. km

water body

improved)

No – insufficient data

to assess

Effectiveness

Achievement of

policy objective

(actual vs. intended)

Measurements of

outputs and

outcomes

Limited – policy

target outputs and

outcomes are to be

delivered over

medium to long-term

planning cycles

Cost-effectiveness

(cost-effectiveness

ratio)


inputs (resources )

and outcomes

Inputs (investment)

and quantitative

(physical) metric for

outcome (e.g.

waterbody status)


to assess

Value for money

(benefit-cost ratio)


inputs (resources )

and impacts

Inputs (investment)

and monetary value

of impacts (benefits)


to assess policy

implementation

Only individual case

study examples

possible (specific

projects)



4 ECONOMIC ASSESSMENT

This section sets out the data and evidence that are currently available to

inform an economic assessment of the CaBA. This includes a summary of:

Inputs: levels of funding and leveraging of additional funds by

partnerships (Section 4.1);

Activities: the cost associated with partnership activities (Section 4.2);

Outputs: the types of environmental outputs that are expected to be

delivered by partnerships through part of the current funding that is

available (Section 4.3).

No data and evidence is available to provide an indicative national level

assessment of outcomes and impacts (Section 4.4).

4.1 INPUTS

Inputs are the resources that are needed to fund and deliver a policy,

programme or project. For CaBA, the resource inputs are funds (from the

public and non-public sectors) and in-kind support (time given freely). The

focus of this aspect of the assessment is on quantifying and monetising the

additional inputs that have flowed through the catchments as a result of the

catchment hosting role that is funded by Defra. In economic terms these

resource inputs are considered as costs of the policy implementation.

For the purposes of this economic assessment it is relevant to value in

monetary terms the following resource inputs:

The total value of resource inputs that are flowing through a

catchment;

The different types of resource inputs (i) monetary funds (ii) in-kind

support (time given freely);

The different sources (i.e. public/non-public) of resource inputs;

The leveraging of additional resource inputs into the catchment as a

result of the partnership (i.e. over and above the initial Defra

investment).

The latter is a key measure of success of the catchment partnerships as



detailed in Table 3.2. Information on total resource inputs by source and

type is available for the 56 catchments that responded to the Phase 2

evaluation survey (“S2”). This represents approximately half of the total

number of catchment partnerships. Results should therefore be interpreted

as indicative of the overall national picture.

4.1.1 Total Inputs

The total value of resource inputs (costs) associated with the 56 catchment

partnerships that responded to this study’s survey was approximately

£4.7million/year in 2014/15.

4.2 TYPE OF INPUTS

4.2.1 Funding

The total value of additional non-core Defra funds flowing through all 56

catchments was approximately £3.2million/year in 2014/15 with the

average (mean) value of additional funds flowing through each catchment

being approximately £57,500/year.

Figure 4.1 shows the variation in the scale of additional funds attracted into

catchments. It shows a significant variation in the amount of leveraged

funds. About a quarter of catchments (14 out of 56) do not receive any

additional funding yet about one in ten catchments receives hundreds of

thousands of pounds and a small minority (1 out of 56) receive additional

funds of (just) over a million pounds. Most (63%) catchments receive tens

of thousands of pounds, with a quarter (13 out of 56) receiving single

thousands of pounds in additional funds.

Figure 4.1. The scale of total additional funds attracted



Source: Phase 2 evaluation survey (“S2”) (sample size = 56 catchment partnerships).

4.2.2 In-Kind Inputs (Time Given Freely)

An estimated 18% of the total resource inputs to the 56 catchments was in

the form of in-kind support. This is calculated to be worth approximately

£0.7million in 2014/1518. The average additional time given per

catchment was approximately £12,600/year in 2014/15. Figure 4.2 shows

the variation in the scale of additional time given freely that is attracted

into catchments. It shows that over forty percent of catchments gain time

given freely that is equivalent to a labour cost of tens of thousands of

pounds and a third of catchments receive additional time given that is

equivalent to thousands of pounds in labour costs. The range of time given

freely in a catchment as a result of the partnership is between

approximately £1,000/year and £66,000/year.

Figure 4.2. The scale of in-kind resources attracted

18 In kind support is measured based on the assumption that it is professional people engaging in

catchment partnership working, at an average (mean) FTE rate of £158/day (8 hours a day) (ONS,

2014). This gives an estimate of £707,357/year.

0

0.05

0.1

0.15

0.2

0.25

0.3

0.35

0.4

0.45

None £1k's £10k's £100k's >£1m

Total additional funds into catchment

Pro

po

rtio

n o

f ca

tch

men

ts




4.3 SOURCE OF FUNDS

4.3.1 Defra Funding

The value of Defra funding providing a mechanism to encourage

involvement of interested parties via the catchment partnerships, since the

national roll-out has been:

£1.1m in 2013/14; and

£1.7m in 2014/15.

For the 56 partnerships that responded to the Phase 2 evaluation survey

(“S2”), the value of Defra funding was approximately £820,700/year in

2014/1519. This represents 21% of the total funds flowing through the

catchments.

There were two catchments that received no Defra funding and for those

that received funding, the range was between £4,600/year and £32,000,

with an average of around £14,650/year per catchment in 2014/2015.

Figure 4.3 illustrates the variation in the scale of Defra funding of each

catchment. It shows that just over 70% of catchments receive Defra funds

in the order of tens of thousands of pounds, about 20% receive single

19 This represents roughly half of the £1.7m of Defra funding, consistent with the 56 responding partnerships representing approximately half of the total number of catchment partnerships.

0

0.05

0.1

0.15

0.2

0.25

0.3

0.35

0.4

0.45

None £1k's £10k's £100k's

Additional in-kind resource inputs

Pro

po

rtio

n o

f ca

tch

men

ts



thousands of pounds and none receive hundreds of thousands of pounds.

Figure 4.3. Scale of Defra funding


4.3.2 Other Public Sector Funding

The total value of additional public sector was approximately

£1.9million/year in 2014/15, or 48% of the total funds flowing through

the catchment. It is relevant to consider the resource inputs from other

government bodies as these resources are a cost to the exchequer and in

many cases will come out of Defra budgets from arms-length bodies such as

the Environment Agency and Natural England. Figure 4.4 illustrates the

variation in the scale of additional other public sector funds that are

attracted into catchments. It shows that sixty eight percent of responding

catchments receive no additional public sector funding, that fourteen

percent receive single thousands.

Given the scale of total funding being nearly two million, this suggests that

the funds from other public sector organisations are predominantly large

payments to a small number of catchments (i.e. the remaining eighteen

percent).

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8


Defra funds

Pro

po

rtio

n o

f ca

tch

men

ts



Figure 4.4. The scale of public sector additional funds attracted


4.4 NON-PUBLIC SECTOR FUNDING

The total value of additional non-public sector was approximately

£1.3million/year in 2014/15 or 32% of the total funds flowing through

the catchment. The average (mean) value of additional non-public sector

funds flowing through each catchment is £22,894/year in 2014/2015.

Figure 4.5 illustrates the variation in the scale of additional non-public

sector funds that are attracted into catchments.

There is a skew towards smaller sums being provided by the non-public

sector, spread out across a greater number of catchments. Approximately

one third of catchments receive four figure sums, another third receive five

figure sums and less than five percent receive sums over one hundred

thousand pounds.

Figure 4.5. The scale of non-public sector additional funds attracted

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8


Additional other public sector funds

Pro

po

rtio

n o

f ca

tch

men

ts




4.5 LEVERAGING OF ADDITIONAL RESOURCE INPUTS

The leveraging of additional resource inputs into the catchment as a result

of the partnership (i.e. over and above the initial Defra investment) is key

measure of success of the catchment partnerships. Defra’s investment of

£0.8million in the 56 surveyed catchments was matched by approximately

£3.2million of other funds. This represents a funding leveraging ratio of

3.9:1. That is for every £1 of core-Defra funding, there is another £3.90 of

public and non-public sector funding into the catchment.

Figure 4.6, however, illustrates the variation in leveraging ratios across the

catchments. It shows that a quarter of catchments achieved no additional

funds (i.e. no leverage) and the majority (36%) achieved leverage ratios of

less than one, meaning the leveraged funds were less than the Defra funds.

0

0.05

0.1

0.15

0.2

0.25

0.3

0.35

0.4


Additional non-public sector funds

Pro

po

rtio

n o

f ca

tch

men

ts



Figure 4.6 Variation in leverage ratios across the catchments for funds

only


Some of the partnerships have high levels of additional funding; e.g. two

catchments had leveraging ratios of over 20, meaning for every pound

spent an additional twenty pounds was sourced from elsewhere. These

partnerships have been operating as partnerships for some time but have

only recently come under the CaBA umbrella. In these cases it is likely that

Defra investment has not resulted in a ‘leveraging’ insofar as attracting new

funds into the catchment. These two cases substantially skew the overall

leveraging ratio, which drops to 2.1:1 without their inclusion.

Including time given freely in the leveraging ratio alongside additional the

funding results in a ratio of 4.7:1. Figure 4.7 shows the variation in

leveraging ratios across the catchments when time given freely is included

in the ratios. There remains an eighth of catchments with no leveraging at

all and just over a quarter with a leveraging ratio less than 1. Half of the

catchments have a leveraging ratio of between 1 and 20 when funds and

time given freely are included. Taking out the catchments with funding

leveraging of over 20:1 results in a reduction in the leveraging ratio (funds

and time given freely) to 2.9:1.

0

0.05

0.1

0.15

0.2

0.25

0.3

0.35

0.4

0 <1 1> <1.5 1.5> <2 2> <5 5> <20 >20 n/a

Leveraging ratio

Pro

po

rtio

n o

f ca

tch

men

ts



Figure 4.7. Variation in leverage ratios across the catchments including

funds and in-kind resource


If funds from other government bodies are included alongside Defra’s

funding, this results in a leverage ratio of 0.5:1. That is for every £1 of

public sector funding, there is another £0.50 of non-public sector funding

into the catchment. If time given freely is included in this ratio, then it rises

to 0.7:1. Taking out the catchments with funding leveraging of over 20:1

results in a drop in the leveraging ratio (funds and time given freely) to

0.1:1.

If funds from other government bodies are excluded from the calculation

entirely, this results in a leverage ratio of 1.6:1. That is for every £1 of core-

Defra funding, there is another £1.60 of non-public sector funding into the

catchment. If time given freely is included in this ratio, then it rises to

2.4:1. Taking out the catchments with funding leveraging of over 20:1

results in a drop in the leveraging ratio (funds and time given freely) to

0.4:1.

4.6 SUMMARY OF FINDINGS

The Phase 2 evaluation survey (“S2”) responses provide an indicative

picture of the success of catchment partnerships in sourcing funding. Based

on approximately half of the total number of partnerships (56 in total), the

0

0.05

0.1

0.15

0.2

0.25

0.3

0 <1 1> <1.5 1.5> <2 2> <5 5> <20 >20 n/a

Leveraging ratio

Pro

po

rtio

n o

f ca

tch

men

ts



findings of the survey are likely to provide a reasonable account of the

national picture. However, the analysis of inputs does not inform on why

there are differences in the potential for each catchment to leverage

resource inputs or whether the partnerships can be viable without funding

from Defra/EA post April 2015. Largely these are process evaluation type

questions, and as such are considered in the Main Report using qualitative

results Phase 2 evaluation survey (“S2”).

The main findings are:

The amount of additional funding leveraged varies significantly across

the catchments, with about a quarter of catchments (14 out of 56) not

receiving any additional funding and about one in ten catchments

receiving additional funding in the order of hundreds of thousands of

pounds. This suggests an equivalent variation in the scale of activities

that are being funded (and the benefits arising) across catchments.

There is an aggregate leveraging ratio including funds and in-kind

inputs of 4.7:1, which means that £4.70 of additional funds are sourced

for every £1 of Defra spend. Removing ‘outlier’ partnerships for which

core-Defra funds are unlikely to have attracted new funds results in a

fall of this ratio to 2.9:1. It is difficult to determine whether this is ‘good

result’ or otherwise, as Defra’s measure of success for partnerships does

not quantify expectations as to leveraging.

A quarter of catchments receive less in additional resource inputs than

the Defra funding provided (i.e. leveraging ratio of <1 :1) when funds

and time given freely are included in the leveraging calculation. Half of

the 56 catchments that responded to this study’s survey have a

leveraging ratio of between 1 and 20. This suggests that the long term

viability of a significant number of partnerships is not clear-cut; but it is

difficult to make such a determination based on leveraging data alone.

At this stage, qualitative responses to the Phase 2 evaluation survey

(“S2”) are likely to provide the best indicator of the continued viability

of partnerships.

Including funds from other public sector organisations alongside Defra

funds gives a leveraging ratio of 0.7:1, meaning that £0.70 in non-public

sector resources is leveraged for every £1 of public sector spend. This

falls to a ratio of 0.1:1 if ‘outlier’ catchments are removed. This suggests

a low level of leveraging of private sector funds against total public



funding.

Overall, assessing inputs in isolation provides only a limited account of

catchment partnership. Whilst for many partnerships their additionality

appears to be relatively modest in terms of attracting extra sources of

funding, it provides no information on how funding is used, either in terms

of activities undertaken, or the extent of outputs/outcomes that funding

helps to deliver. This should be a focus of any future funding arrangements.

4.7 ACTIVITIES

Activities are undertaken once the resource inputs are accessed/secured.

It is the outputs from these activities that lead to environmental, social

and economic outcomes that can be valued within an impact evaluation.

The following summarises the types of activities undertaken by catchment

partnerships as a result of the resource inputs identified in Section 4.1:

Partnership development and administration;

Local community engagement and conflict resolution;

Co-ordinating activities across organisations, geographic areas and

delivery areas;

Data collation and interpretation, and planning including influencing

others plans and policies;

Delivering practical environmental work, advice or awareness raising in

the catchment; and

Other, including primarily grant/funding applications, with bespoke

projects (website development, ES mapping), website maintenance,

scoping and feasibility studies, room hire costs, mileage, and training.

4.8 EXPENDITURE

A breakdown of expenditure across the key activities in 2014/15 shows that

overall, most of the funding of the 56 surveyed catchment partnerships in

2014/15 has been spent on delivering practical environmental work, advice

and awareness raising (66%). Partnership administration and development

(16%) and other activities (6% or lower) account for much less, as shown in

Figure 4.8.

Whilst two-thirds of expenditure by catchment partnerships is reported to

be for ‘practical environmental work, advice and awareness raising’, this

represents a broad category of activity. It has not been possible to break this



down further between the advisory activities and any projects undertaken

to directly improve environmental quality. The case study examples

presented in Section 5, however, provide illustrative examples of the actual

projects led by partnerships that are targeting water environment and

wider outcomes (benefits).



Figure 4.8 Partnership spend by activity 2014/1520


Figure 4.8 shows the allocation of the £4.7million funding across different

activities according to the funding source. The main observations are:

Of the approximately £0.8million/year of core-Defra funding that is

received by the catchments:

o nearly half of this (approx. £0.39million/year) is spent on

partnership development;

o partnership development along with co-ordinating activities and

local community engagement represent the ‘partnership hosting’

role funded by Defra at a cost of approximately £0.58million/year

(70% of core-Defra funds);

Approximately £0.40 million (66%) of in-kind resource is spent on

partnership development or delivering practical environmental work;

96% (approx. £1.9million/year) of other public sector funds and 70%

(approx.£0.9million/year) of non-government funds are spent on

delivering practical environmental work, advice or awareness in the

catchment.

20 “Other” activities partnerships reported were: Developing proposals, grant applications and fund raising to support the partnership; Scoping and feasibility studies; Bespoke Projects (website development, and maintenance, ES mapping); room hire costs/mileage etc.; Training, conferences and Project development.



Figure 4.8 Source and value of resource inputs by activity




This suggests that the core-Defra funding is crucial for the operation of the catchment

partnerships since these funds are relied on for developing and nurturing the

partnerships, creating links and co-ordinating across organisations and delivery areas

and engaging with local communities.

In addition, the leveraging (Section 4.1) is dependent on the “co-ordinating” activities

that are reliant on core-Defra funding. Figure 4.8 highlights that leveraged funds

(from other government bodies but also non-public sector) are the primary resource

inputs for the delivery of practical environmental work, advice and awareness raising.

However, since this activity definition mixes activities that directly lead to

environmental outputs via projects, and those that do not or indirectly (e.g.

awareness raising and advice) it is difficult to link outcomes (and therefore

additionality) to these funds. Where practical environmental work is being funded via

projects this should lead to improvements in environmental outcomes.

The case study examples in Section 5 illustrate a selection of outcomes and benefits

delivered by partnership projects.

4.9 COST SAVINGS

The assessment structure (Section 3.2) recognises ‘cost savings’ with respect to

undertaking activities as a specific aspect of the additionality that can be delivered by

catchment partnerships. In particular if partnerships can ‘outperform’ a realistic

reference case in delivering the necessary activities and processes for ensuring local

support, consensus, effective coordination, delivery etc. within the river basin

planning process.

During the CaBA pilot stage (2011-2013) ten catchment partnerships were hosted by

the Environment Agency. This is interpreted as a reasonable reference case to use as

a counterfactual (reference case) against which to assess the cost savings of Defra’s

funding of the catchment partnership hosting role.

During 2011-2013, the Environment Agency collected information on the time spent

by Environment Agency hosts in fulfilling the hosting role. The estimated cost of

funding one coordinator per catchment is approximately £75,000/year. This includes

labour costs and overheads (includes hire of meeting venues, etc). Based on this

estimate, the total aggregate cost for the 93 management catchments nationally

would be around £7m/year21.

The current hosting role that is funded by Defra includes substantial support from

Environment Agency catchment coordinators and not all catchment hosts are full-

21 Estimated provided by the Environment Agency, May (2015).



time. To adjust for this and make a reasonable comparison of the current situation

with the Environment Agency counterfactual, a 50% reduction in the role is assumed,

equating to £3.5m/year for the Environment Agency to fulfil the catchment hosting

role.

The total cost of Defra funding of £1.7m investment in partnership hosts across 107

catchments in 2014/15 is notably lower. Furthermore, as highlighted above and

shown in Figure 4.8, not all of Defra’s funds are spent on the hosting role. It is

estimated that 70% of core-Defra funds is spent on hosting. On this basis the cost to

Defra of hosting is calculated to be approximately £1.2million/year.

Following the above assumptions the estimated cost savings associated with Defra’s

funding of the catchment hosting role as opposed to Environment Agency performing

the role is estimated to be between £1.8million/year and £2.3million/year.

4.10 OUTPUTS

At present, there is limited data availability for assessing the outputs (and the

additionality) that have been delivered, or will be delivered, through currently

available funding sources by catchment partnerships. The Environment Agency has

provided summary data for the Catchment Partnership Action Fund (CAPF), which is

administered by the Environment Agency on behalf of Defra22. Under CPAF, each

partnership has the opportunity to submit an application to deliver projects in line

with CPAF guidelines. A proportion of these funds can also be used to support

hosting if required23.

The Environment Agency data relates to funding for the 2015/16 financial year. It

captures all the applications for CPAF projects received from the partnerships for

funding 2015/16 and provides for a number of the proposed projects a summary of

waterbody improvements that will be delivered (i.e. outputs/outcomes). All projects

are 100% partnership led and they are match funded.

4.11 NUMBER OF PROJECTS

The CAPF data shows 106 project applications from 105 partnerships. All but one

partnership have a single application (one has two applications). Project applications

can include activities associated with different water bodies within the same

catchment. The 105 applications cover 270 water bodies (out of approximately 7,000

in total).

22 Note that this data is subject to further review by the Environment Agency and subject to further change. 23 Environment Agency (2015) Catchment Partnership Action Fund (CPAF) for England CPAF 3 - Project Application Guidance February 2015.



4.12 TYPE OF PROJECT

Table 4.1 provides a breakdown of the 106 projects according to the “project

category”. Approximately half (51%) of the projects are primarily focused on reducing

the impact of diffuse pollution from either urban or rural land uses. Around a third

(32%) are focused on reducing the impacts of man-made structures on wildlife in

watercourses and just under a fifth (17%) are concerned with preventing

deterioration or contributing to achieving protected area objectives (see Section 2.2).

Table 4.1 Number of CPAF applications by project category

Primary project category No. of applications

Prevent deterioration or contribute to the achievement of protected area objectives

18 (17%)

Reduce the impact of diffuse pollution that arises from rural land use 29 (27%)

Reduce the impact of diffuse pollution that arises from urban land use 25 (24%)

Reduce the impact of man-made structures on wildlife in watercourses 34 (32%)

Total 106 Source: CAPF applications (2015/16) (Environment Agency, May 2015).

Figure 4.1 plots the distribution of projects nationally, by project categories reported

in Table 4.1. This shows the coverage of projects by management catchment.

4.13 WATERBODY ELEMENT ADDRESSED

Table 4.2 shows the specific water body elements that are intended to be addressed

by the CPAF project applications. This gives an indication of the type of

environmental outcomes that these projects will contribute to (noting that some

projects will address multiple issues in water bodies). Across the 105 applications

which cover 270 water bodies, the most commonly addressed WFD elements are

stated to be fish, phosphate, and invertebrates. There is though a relatively wide

spread of elements that are expected to benefit from the improvements.



Figure 4.1. CPAF applications by project category

Prevent deterioration or contribute to the achievement of

protected area objectives

Reduce the impact of diffuse pollution that arises from rural

land use

Reduce the impact of diffuse pollution that arises from urban

land use

Reduce the impact of man-made structures on wildlife in

watercourses



Table 4.2 Waterbody elements to be addressed within CPAF applications

Waterbody element description No. of references Fish, 100 Phosphate 77 Invertebrates 49 Macrophytes 37 Phytoplankton 33 Biological oxygen demand 23 Chemical 24 Hydromorphology 18 Mitigation measures 15 Ammonia 9 Protected Areas 9 Sediment 8 pH 6 Copper 6 Draining and sewerage discharge 5 Heavily Modified Waterbody 3 Other 36 Undisclosed 11 Overall status 58 Total 527

Source: CAPF applications (2015/16) (Environment Agency, May 2015).

4.14 EXPECTED WATERBODY IMPROVEMENT

Quantitative data is not available for the length of water body (km) that will be

improved by the projects. However, overall 36 of the 270 water bodies that are

covered in the applications have CPAF projects that are expected to result in an

improvement in overall water body status. This includes projects that are stated as

addressing multiple elements within a waterbody. In addition, of the 270 water

bodies that are referred to in the CAPF applications, nearly 40% (100) are to include

measures that will improve water bodies for fish and nearly 30% (77) are address

phosphates/phosphorous. There are overlaps across these categorisations and so the

degree of additionality associated with each element is unclear.

4.15 OUTCOMES AND IMPACTS

No data and evidence is available to provide an indicative national level assessment

of outcomes and impacts.



5 CASE STUDIES

This section outlines the economic analysis for a selected set of case study examples.

The purpose of the case studies is to highlight the type of work that is being

undertaken by catchment partnerships, but is not well evidenced in Section 4 due to

limited reporting and data at the national level.

The focus of the case studies is the ‘additional’ environmental, social and economic

impacts delivered by catchment partnerships. They demonstrate part of the potential

scope of what can be delivered by partnerships going forward, linking to the

assessment outlined in Section 2 and the contribution towards overall WFD

implementation. Significantly, the case studies provide some initial evidence that

partnerships in some situations are a mechanism for delivery enhanced

environmental outcomes against the WFD.

5.1 SELECTED PROJECTS

Table 5.1 summarises the basic details of the case studies, listing the catchment

partnerships and selected projects:

All appraised projects within the case study catchments are concerned with

tackling the impacts of diffuse pollution;

Four out of 5 catchments have water body (WFD) improvements for which it has

been possible to monetise the expected benefits;

Additional non-WFD benefits have been monetised for housing development,

reductions in flood risk, avoided costs of topsoil replacement, de-silting and

maintenance and habitat restoration.

The reference case assumes that water body improvements would be delivered via

river basin management plans (either RBMP2 or RBMP3).



Table 5.1 Catchment partnership case study summary

Catchment

Project(s) Objective Additional

outcomes that have

been monetised

Combined

Essex

Chelmer Valley River

Restoration Project

Urban diffuse

pollution impacts

Restoration of grazing marsh and woodland

Reduction in flood risk

Reduced de-silting costs

Reduced maintenance costs

Test and

Itchen

Sediment Pathways

Project

Dunn

Rural diffuse

pollution

Man-made

structures

Improve 2 elements

of water body status

by 1 RBMP cycle

Wye Save our Soils Rural diffuse

pollution

Improve 2 elements


by 1 RBMP cycle

Development of 12k

houses forward by

2years

Avoided costs of

replacing topsoil

Wear Castle Eden Man-made

structures

Rural diffuse

pollution

Improve water body status

Restoration of saltmarsh

Soar Improving Water

Management in the

Willow Brook

Rural diffuse

pollution

Improve 3 elements


by 1 RBMP cycle

5.2 COST-BENEFIT ANALYSIS

Figure 5.1 illustrates the link between the HM Treasury (2011) Magenta Book logic

model for policy assessment and the cost-benefit analysis framework outlined in the

HM Treasury (2011) Green Book. The key additional element for CBA is to establish a

counterfactual which sets out what would have happened in the absence of the policy

intervention (i.e. without the core-Defra funding through CPAF). This enables the

‘additionality’ of the CaBA partnerships to be established.



Figure 5.1. Comparison of logic model and CBA steps

Within the CBA ‘inputs’ are ‘costs’ which represent the use of resources. The CBA

framework defines ‘impacts’ as ‘benefits’. Where possible these costs and benefits

should be valued in monetary terms and compared to estimate the value for money

(VfM) of a particular course of action. A VfM assessment of CaBA would therefore

examine the ratio of benefits delivered by the partnerships to the costs in monetary

terms, in the following way:

Value for money = £ return / £ spent

An initial VfM category can be identified based upon the benefit-cost ratio (BCR) of

the scheme, using monetised impacts. The Department for Transport has categories

for BCR’s in line with its WebTAG guidance24. (HM Government, 2014):

Poor VfM if the BCR is less than 1.0

Low VfM if the BCR is between 1.0 and 1.5

Medium VfM if the BCR is between 1.5 and 2.0

High VfM if the BCR is between 2.0 and 4.0

Very high VfM if the BCR is greater than 4.0

24 See: https://www.gov.uk/transport-analysis-guidance-webtag

https://www.gov.uk/transport-analysis-guidance-webtag



The WebTAG guidance notes that not all costs and benefits might be monetised and

an assessment should be reported of whether consideration of these non-monetised

impacts is likely to alter the VfM category of the proposal.

5.3 SUMMARY RESULTS

Table 5.2 outlines the results from the economic appraisal for each of the case studies

in Table 5.1. It shows:

The BCRs of the appraised projects are between 2:1 and 16:1;

Funding is derived from a variety of sources and ranges from between £40,000

and £95,000 in present value terms;

There are some significant estimates of water body (WFD) improvements in

monetary terms, in the order of hundreds of thousands of pounds as a result of

the partnerships;

Non-WFD improvements are also significant in some cases;

The calculated net present value (NPV) for each project is reported as:

The overall project (NPV); i.e. the net benefit calculated from total benefits and

total costs. This is the overall social wellbeing gain, which can only be attributed

to the actions of the catchment partnership if the project is 100% additional; i.e. it

would not have been delivered through RBMP2 or RBMP3.

The best estimate of the contribution of the catchment partnership to the overall

project NPV (‘Additional NPV’). This takes account of the fact that some

proportion of benefits of the project are likely to have been delivered anyway,

and therefore provides an estimate of the additionality of the project.

The additional contribution of the catchment partnerships is estimated to be between

14% and 100% for the example projects. In each case this is in terms of delivering a

greater level of benefits (either WFD or non-WFD).

Whilst all projects have a BCR greater than 1, indicating that they do represent ‘value

for money’, it is not possible to assess the incremental BCR for the projects in terms

of additionality (i.e. compare the difference in costs to a reference case cost estimate

with the additional benefit generated). This is because reference case cost estimates

are not available. However, there is no strong case to expect that reference case costs

would be significantly different from those reported in Table 5.2, implying that the

calculated additional NPV for each example is a reasonable indicator of the

contribution of the catchment partnership.



Table 5.1 Catchment partnership case study summary

Catchment

Funding (funds and in-kind) Estimated benefits (£, PVB) Total Project

NPV (100%

attribution)

Additional

NPV

BCR

(total

project) Source(s) Amount (£,

PVC)

WFD non-WFD

Combined Essex Defra, Environment Agency, other funding

£43.8k £0 £155.7k £111.9k £111.9k

(100%)

4 : 1

Test and Itchen Defra , Natural

England, In-kind,

GIA, Other funding

£95.9k £263.0k £0 £681.5k £167.2k

(25%)

3 : 1

Wye Defra, Business,

Other funding

£80.6k £450.2k £875.8k £5,297.1k £1,245.4k

(24%)

16 : 1

Wear Environment

Agency, Natural

England, Other gov.,

Water company,

eNGO

£63.6k £110.8k £11.4k £58.5k £58.5k

(100%)

2 : 1

Soar Defra, Environment

Agency, Trent Rivers

Trust, In-kind

£53.8k £233.0k £0 £1,295.4k £179.1k

(14%)

4 : 1



6 SUMMARY, FINDINGS AND CONCLUSIONS

6.1 SUMMARY

Economic assessment is a fundamental component of the policy

implementation cycle. When applied in policy monitoring and evaluation

settings, it enables judgements to be made about the effectiveness of

measures that have been implemented and whether these have been

worthwhile. This includes ‘value for money’ considerations, which in broad

terms concern the balance of costs and benefits; i.e. whether the resource

inputs that have gone into a policy are ‘justified’ by the resulting outcomes

and impacts.

Judging the value for money of the CaBA and catchment partnership is a

challenging task. The national level implementation of the policy is just two

years in the running and catchment partnerships are at different stages of

development. The initial actions for partnerships have been to engage and

coordinate local stakeholders and develop partnership working with a

catchment. Activities that lead to more integrated planning and

prioritisation of measures follow from these initial stages, which in turn

should result in the delivery of environmental improvements in

catchments. Delivering improved water environment quality in accordance

with targets set under the Water Framework Directive is a medium to long

objective and as a minimum subject to two river basin planning cycles (12

years from 2015).

As a result the present scope for undertaking an economic assessment is

limited. This is due to two factors: (i) the current stage of the policy

implementation, where catchment partnerships are only now (as a

collective) stating to shift towards activities and projects that directly

impact the quality of the water environment; and (ii) a lack of a systematic

and coherent evidence base for establishing the ‘additionality’ of catchment

partnerships. Determining additionality is a critical component of an

economic assessment, as this establishes the benefits that are delivered by

partnerships over and above those that would have occurred anyway in

their absence (the so-called counterfactual); i.e. the added value in terms of

enhanced benefits or reduced costs.

6.2 FINDINGS

This reports sets out a structure for economic assessment that is framed



around the logic model concept that is commonly applied in policy

evaluations. It involves a sequence of steps (inputs – activities – outputs –

outcomes – impacts) that describe how a policy should achieve it aims. The

added value of the policy implementation is measured against a reference

case that represents the counterfactual.

The key findings include:

National context: overall there are potentially significant

opportunities for catchment partnerships to deliver added value

through implementation of the WFD. Set against the current

programme of measures presented in the Environment Agency’s

RMBP2 consultation documents, there is potential for partnerships to

contribute in terms of: (i) achieving WFD targets more cheaply; (ii)

achieving WFD targets more quickly; and/or (iii) delivering more

benefits for the same cost.

An indicative assessment suggests that the key opportunities are likely

to be in relation to WFD objectives to prevent deterioration and

achieve protected area objectives. This includes measures to address

physical modification (e.g. flood protection, urban areas, land

management associated sediment, impoundments, barriers to fish,

land drainage) and invasive non-native species (bio-security and

access to volunteer activity for eradication works).

The scale of the estimated costs and benefits of the measures required

to achieve these WFD objectives implies that even with only modest

assumptions as to the contribution of partnerships, the potential

benefits are substantial. For example a 5% cost-efficiency from local

level ‘ground truthing’ of proposed measures implies a national level

benefit in the region of £200million (present value terms), through

avoided costs. Similarly a 5% enhancement of environmental

outcomes through local level input to measures implies a national level

benefit in the region of £500million (present value terms). The

realisation of these benefits is though critically dependent on a

number of factors that require further examination, including the

ability to influence plans, access to funding, and scope for leading

delivery.

Inputs: refers to the resources required to achieve a policy objective

(e.g. funding). The analysis presented in this report focuses on



leveraging, and the extent to which catchment partnerships have been

able to draw-in additional funding. This is based on survey responses

from 56 of the 107 partnerships and can only be interpreted as

indicative of the national situation.

The main finding is that the amount of additional funding leveraged

varies significantly across partnerships. A significant proportion of

partnerships report no additional funding (approx. 25%), whilst a

handful report substantial amounts. Accounting for outliers, the

average leveraging ratio including other sources of public funding,

private funding and in-kind inputs is 2.9:1, which means that £2.90 of

additional funds are sourced for every £1 of Defra core-funding.

Assessing leveraging in terms of all public sector funding against

private sector funding gives a ratio of 0.1:1, meaning that £0.10 of

private sector funding is sourced for every £1 of public sector funding.

This suggests that partnerships, at present, represent a mechanism for

coordinating and distribution of public sector funding at the local

level, rather than a means for generating substantial (additional)

private sector investment in catchments.

Activities: are the actions and work that are undertaken by

partnerships in implementing the CaBA (e.g. hosting, coordinating

stakeholders, producing plans, etc.). The analysis presented in this

report considers expenditure on different types of activity by

partnerships and also the potential for cost savings in the hosting role.

Data on expenditure is sourced from survey responses from 56 of the

107 partnerships and can only be interpreted as indicative of the

national situation. Overall, it is found that the majority of expenditure

by partnerships (approx. 66%) is on ‘practical environmental work,

advice and awareness raising’. This suggests the expected focus on

understanding waterbody issues within a catchment and undertaking

the groundwork need to inform and influence strategic planning.

Approximately 70% of core funding received from Defra is attributed

to ‘partnership development’. This suggests that Defra’s funding is

critical for developing and ensuring continued local level partnership-

working, including co-ordinating across organisations and delivery

areas and engaging with local communities.

In relation to potential cost savings, data provided by the Environment

Agency indicates that Defra’s funding of the partnership hosting role



via partnerships represents approximately £2million saving per year,

against a reference case of the Environment Agency performing the

hosting role.

Outputs: at present there is limited data availability for assessing the

outputs (and the added value) delivered by catchment partnerships, in

terms of physical environmental improvements in waterbodies (e.g.

km of waterbody improved, number of fish passes created). Available

national data has been provided by the Environment Agency, based on

the Catchment Partnership Action Fund (CAPF). This data, however,

is interim and subject to further review by the Environment Agency.

The CAPF data provides a partial indication of the types of

environmental improvement projects that partnerships are facilitating

and leading in catchments. Current project funding applications cover

270 water bodies (out of approximately 7,000 in total). These are

expected to improve overall status in 36 water bodies, and

improvements in WFD elements in many others. Quantitative data is

not available for the length of water body (km) that will be improved.

The projects show a relatively even spread across WFD objectives,

covering:

o Prevent deterioration or contribute to the achievement of

protected area objectives (17%)

o Reduce the impact of diffuse pollution that arises from rural

land use (27%)

o Reduce the impact of diffuse pollution that arises from urban

land use (24%)

o Reduce the impact of man-made structures on wildlife in

watercourses (32%)

Whilst the available data is high level, it does illustrate that

partnerships are attempting to address the types of issue highlighted

in the ‘national context’ findings where there is potential for

significant ‘added value’ contribution to overall WFD implementation.

Outcomes/impacts: no data and evidence is available to provide an

indicative national level assessment of outcomes and impacts, in terms

of monetised costs and benefits and overall value for money.

Case studies: the analysis presented in this report is augmented by a

series of case study examples. These provide value for money



assessments at the individual project level, focussing on the added

value that has been delivered by catchment partnerships. Whilst they

are small sample of selected examples and cannot be interpreted as

representative, they do provide initial evidence that partnerships (in

some situations) are a mechanism for delivery enhanced

environmental outcomes.

Each case study is found to deliver added value against a reference

case of WFD implementation without the involvement of catchment

partnerships. Benefit-cost ratios for the individual projects range

between 2:1 and 16:1, indicating that each project delivers a net

benefit, whilst the added value attributable to the catchment

partnership ranges between 15 – 100%. This suggests that as

catchment partnerships continue to develop and work to integrate

with local planning processes, they may be in a position to realise the

potential added value highlighted in the ‘national context’ findings.

6.3 CONCLUSIONS

The purpose of this report is to outline a framework for an economic

assessment of catchment partnerships within the Catchment Based

Approach (CaBA). An initial assessment has been set out within this

framework using available data and evidence. This preliminary analysis

highlights a significant number of gaps in the evidence base, which future

work could address in order to provide a more complete analysis of the

CaBA. Within this there is scope to revise and update the structure

economic assessment framework.

Well-evidenced conclusions that can be drawn from this report are limited.

The majority of findings need to be interpreted as ‘indicative’ or

‘suggestive’. This reflects both the stage of the policy implementation and

the data gaps. However, it is envisaged that there is potential for significant

‘added value’ to be delivered by the CaBA and partnerships within the

overall WFD implementation process. This though is reliant on interpreting

the overall objective of the policy as being concerned with delivering

improved environmental outcomes in the long term. If this is the case then

the economic assessment framework should be consistent with this

perspective and focus on ‘ultimate’ outcomes in terms of water body

improvements and wider environmental, social and economic benefits. This

is the perspective adopted in this report.



Alternatively a narrower focus may be relevant, concerned with only the

hosting role of catchment partnerships. Here the question is essentially one

of effectiveness; i.e. do partnerships play an effective role in providing local

level input to planning? This focus, however, does not permit much

consideration of the added value of partnerships, unless it is broadened to

take account of the environment improvements that partnerships affect.

Either way, a conclusion of this assessment is that a fundamental review is

required to determine how the performance of partnerships will be

evaluated moving forward. This is dependent on multiple factors, beyond

the scope of this report, including the policy direction which Defra wishes to

take with partnerships including the continuation of core funding.

The findings from this report should also be interpreted in light of the Main

Report and wider findings from the evaluation of catchment partnerships.

However, focusing only on the economic assessment component, it is not

possible at this stage to conclude whether the CaBA and catchment

partnerships overall represent value for money. There are indications that

individual partnerships are engaged in activities that will deliver added

value. Moreover these appear to be directed in ways that can potentially led

to the realisation of significant benefits at the national level. There are

though many critical dependencies in such judgments and these require

further examination and review as part of the continued policy

implementation.


Cascade Consulting

APPENDICES


Cascade Consulting

APPENDIX A. CASE STUDIES


Cascade Consulting

COMBINED ESSEX CATCHMENT PARTNERSHIP

Summary

This note provides a summary of the economic assessment of the projects that have

emerged as a result of the existence of a catchment partnership in the Combined

Essex catchment. The project(s) considered were those for which sufficient

evidence could be gathered on the marginal impact of the catchment partnerships

on environmental outcomes. For Combined Essex this was the Chelmer Valley

River Restoration Project. The objective of this project is to reduce the impact of

diffuse pollution that arises from urban land use. Table 1 sets out the headline

results of the analysis.

Table 1: Headline results for economic evaluation of selected projects in the Combined Essex catchment

Low Central High Present value costs - £43,800 - Present value benefits - £155,700 - Net present value - £111,900 -

BCR (£ benefit per £ cost) - 4 : 1 -

% attribution to partnership n/a

Switching value (% attribution required BCR = 1)

n/a

Note: Estimates are rounded to nearest £100. Present values are calculated over 25 years and discounted at 3.5% in accordance with HM Treasury Green Book guidance.

The impact of catchment partnerships has been examined within a cost-benefit

analysis (CBA) framework. The general approach to CBA consists of five steps as

described below, which are consistent with HM Treasury guidance for economic

appraisals and evaluations as described in the Green Book (HMT, 2003) and

Magenta Book (HMT, 2011).

Baseline

This project has resulted entirely from the existence of the catchment partnership.

Therefore all of the outcomes from this project are deemed to be additional to what

would have happened in the absence of the partnership.

Physical Output of the Catchment Partnership and Outcomes

The project is expected to lead to the physical outputs and outcomes (benefits) set

out in Table 2.


Cascade Consulting

Table 2: Outcomes and outputs as a result of the Chelmer Valley River Restoration Project Output Outcome

Restoration of a 2km watercourse

Maintain/improvement water body status or reduction in risk of failing to meet GES – not valued - see Annex 1 A reduction in costs of de-silting to the Environment Agency Lower costs associated with river and land management maintenance to the Local Authority

Restoration of 3.6ha floodplain grazing marsh

An assumed improvement in condition and associated ecosystem service provision equivalent to that achieved by improving the condition of floodplain grazing marsh under SSSI designation to favourable status

Restoration of 0.6ha woodland

An assumed improvement in condition and associated ecosystem service provision equivalent to that achieved by improving the condition of what’s assumed to be broadleaved woodland under SSSI designation to favourable status

A reduction in flooding risk from 1 in 100 years to 1 in 200 years

Avoided damages to 95 (assumed to be residential) properties from flooding

MONETARY VALUE OF IMPACTS

The total cost of funding for the project is £31,000 from Defra, £10,000 from the

Environment Agency for flood risk modelling, and £3,512 from other sources in the

form of in-kind inputs.

Table 3 sets out the unit value benefit estimates that are applied to the outcomes

reported in Table 2. These have been adjusted for inflation and are reported in

2014 prices (HMT, 2015). The total marginal value is the unit value multiplied by

the relevant change in units as a result of the project.


Cascade Consulting

Table 3: Unit value estimates - outcomes from the Chelmer Valley River Restoration Project Outcome valued Unit value Source Annual

benefit Reduction in costs of de-silting to the Environment Agency

£2,000/yr Mark Iley pers comm, Essex Wildlife Trust (2015)

£2,000/yr

Lower costs associated with river and land management maintenance to the Local Authority

£2,000/yr Mark Iley pers comm, Essex Wildlife Trust (2015)

£2,000/yr

Estimated value of ecosystem services delivered by floodplain grazing marsh SSSI conservation activities by SSSI habitats under an ‘Increase funding’ scenario - floodplain grazing marsh

£479/ha/yr Christie and Rayment (2012)

£1,726/yr

Estimated value of ecosystem services delivered by broadleaved SSSI conservation activities by SSSI habitats under an ‘Increase funding’ scenario - woodland

£565/ha/yr Christie and Rayment (2012)

£339/yr

(Avoided) weighted annual average damages (AAD) from flooding for residential properties assuming an increase in Standard of Protection from 1:100 to 1:200 (assumed flood warning lead time of <8hrs)

£42/yr FHRC (2005)

£3,971/yr

Note: All unit values are reported in 2014 price terms.

ANALYSIS OF COSTS AND BENEFITS OVER TIME

Costs and benefits are appraised over a 25 year time horizon. Costs and benefits

occurring in the future (post-2015) are discounted at a rate of 3.5% per annum

based on HM Treasury Green Book guidance.

It is assumed that costs are split evenly over the first 2 years of the project from

2015 to 2016, in line with the project lifetime of 01/04/2015 to 31/03/2016 giving

a present value of costs over 25 year appraisal period of £43,759 (PVC, 25years).

Benefits are assumed to increase over the first 5 years of the project to a level

where they remain for the rest of the 25year appraisal period. This gives a present

value of benefits over 25 year appraisal period of £155,703 (PVB, 25 years).

The calculated net present value is £111,943 (NPV, 25 years), with a corresponding

benefit-cost ratio of 4 : 1, meaning that for every pound spent on the project an


Cascade Consulting

estimate benefit of £4 is achieved. These figures are subject to the assumptions and

datasets that have been available for the analysis. The benefit estimate does not

include any account of the potential contribution of the project to improved

waterbody status under implementation of the Water Framework Directive (see

Annex 1).

Sensitivity Testing

Given that this analysis is an illustrative project level case study, sensitivity testing

has not been undertaken. This, though, is a fundamental aspect of CBA, which is

critical to establishing the validity of results in light of data and information

uncertainties that all practical analyses face; for example determining the

circumstances and assumptions under which benefits outweigh costs. Sensitivity

testing can include, for example, the: (i) use of ranges of values for key variables

(e.g. high, medium, low); (ii) use of ‘expected’ values so that uncertain outcomes

are assigned a probability of occurrence; and/or (iii) Estimation of benefits

thresholds and switching values for key variables.

Key assumptions used in the analysis are:

Funding is a one-off cost occurring between 2015 and 2016 in line with lifetime

of the project;

The 'enhancement' in floodplain grazing marsh and woodland are of the same

marginal improvement as that associated with improving SSSI's to favourable

status

Woodland enhanced is broadleaved woodland;

Current risk of flooding is 100yr SoP (1 in 100 annual probability of river

flooding) and this is reduced to 200yr SoP (1 in 200 annual probability of river

flooding) and flood warning lead time is <8hours;

Flooding risk is reduced for residential properties only;

Benefits occur from 2017 every year over 25 year appraisal period (to 2040) reflecting an increasing profile of benefits over the first 5 years of the project.


Cascade Consulting

ANNEX TO COMBINED ESSEX CATCHMENT

PARTNERSHIP CASE STUDY

The Chelmer Valley River Restoration Project aims to offset the impact of

modifications along a 2.5 km section of the River Chelmer river (which is 34km in

length). These modifications have resulted in a relatively uniform habitat with little

flow or habitat diversity and a waterbody which is not at good ecological status for

a number of elements. The project will implement measures identified in the draft

2nd River Basin Management (RBMP2), as part of a cost-beneficial bundle that

will contribute towards the WFD objectives of achieving Good Ecological status.

The project aims to:

Directly address the lack of in-channel morphological diversity;

Improve habitat and provide refuge for fish from predators and high flows,

reversing the deterioration in fish status;

Reduce water temperatures, thus increasing dissolved oxygen concentrations

and ensuring no deterioration; and

Provide shade and marginal aquatic habitat to improve macrophyte status,

whilst offsetting the biological impacts of the elevated phosphate

concentrations.

Improvement in Water Body Status

Based on the information provided by the catchment partnership (Mark Iley, Essex

Wildlife Trust), no change is expected in the overall waterbody status, nor any

components (elements) of the waterbody (Table A1).

Table A1: Expected change in WFD status of different water quality elements Element/ Area or Site type

Current Status (WFD status classes)

Post-project Outcome (WFD status classes)

Biological status Moderate Moderate Diatoms No data No data Macrophytes Moderate Moderate Macro-invertebrates Good Good Fish Moderate Moderate Dissolved Oxygen Good Good pH High High Phosphate Poor Poor Ammonia High High Hydro-morphology status Not high Not high Hydrology Good Good Morphology Moderate Moderate


Cascade Consulting

If a change in these outcomes was predicted then NWEBS values (Metcalfe, 2012)

could be used to value the incremental change associated with any of the following

six equally weighted ecosystem components (Tamar Consulting, 2012): (i) fish; (ii)

other animals such as invertebrates; (iii) plant communities; (iv) The clarity of

water; (v) the condition of the river channel and flow of water; and (vi) the safety of

the water for recreational contact.

For example the (central) value:

Per component that is improved from moderate to good ecological status in the

Combined Essex catchment is £5,304/km/year.

For improvement of the water body status in the Combined Essex catchment is

£32,392/km/year.

Reduction in the Risk of not Meeting GES under WFD

Whilst the expected outcome of the project is not to improve the status of the water

body, it might be expected that the measures that are implemented do reduce the

risk of failing to meet GES under WFD. Currently, however, the information

provided by the catchment partnership (Mark Iley, Essex Wildlife Trust) does not

include any assessment of the potential contribution to ensuring that the

combination of all measures in the catchment leads to GES being attained.

Appraisal undertaken by the Environment Agency do though include an

assessment of the ‘risk of failure’ of proposed measures, categorised by Low (0-

30%), Med (35%-65%) High (70%-100%). If this risk of failure data can be

obtained for the River Chelmer then it may be possible to provide an assessment of

how the project contributes to the overall GES objectives. The project is targeting

approximately 7.5% of the waterbody, in terms of total length. A proxy indicator

can be derived based on mitigating risk along this length of the river (i.e. that the

measures brought about as a result of the project along this stretch of river will

lead to the risk from that section failing to meet GES either being reduced or

avoided). This then can valued relative to overall benefit of attaining good status in

the catchment.


Cascade Consulting

SOAR CATCHMENT PARTNERSHIP

SUMMARY

This note provides a summary of the economic evaluation of the projects that have

emerged as a result of the existence of a catchment partnership in the Soar

catchment. The costs and benefits of these project(s) should be viewed within the

wider context of the costs and benefits of measures required for the catchments

water bodies to reach good ecological status as appraised under RBMP2, as shown

in Table 1 (EA, 2014).

Table 1: RBMP2 economic evaluation of measures to reach good ecological status under WFD (risk adjusted, central estimates)

Management catchment PVB PVC BCR

Soar 46,000,000 57,700,000 0.8

The project(s) considered were those for which sufficient evidence could be

gathered on the marginal impact of the catchment partnerships on environmental

outcomes. For the Soar this was the Improving Water Management in the Willow

Brook project. The objective of this project is to reduce the impact of diffuse

pollution that arises from rural land use. Table 2 sets out the headline results of the

analysis.

Table 2: Headline results for economic evaluation of selected projects in the Soar catchment

Low Central

High (100%

attribution) Present value costs - £53,824 £53,824 Present value benefits - £232,960 £1,349,187 Net present value - £179,136 £1,295,362

BCR (£ benefit per £ cost) - 4 : 1 25 : 1

% attribution to partnership - 20% -


- 5% -


Comparing Tables 1 and 2 it can be seen that the present value of costs (PVC)

attributable to the catchment partnerships is £0.053million compared to

£57.7million worth of projects throughout the entire Soar catchment that are

required to reach GES. The estimated benefit cost ratio of the assessed projects

under this study for the catchment partnerships of 4 : 1 is much higher than the


Cascade Consulting

appraisal of measures required to reach GES across the entire catchment of 0.8 : 1

(where the costs outweigh benefit and the net present value is negative).






Baseline

This project(s) has resulted entirely from the existence of the catchment

partnership. Therefore all of the outcomes from this project(s) are deemed to be

additional to what would have happened in the absence of the partnership. It is

assumed that the partnership brings forward actions to deliver WFD objectives by

a RBMP cycle (i.e. 6 years) and the attribution of outcomes to the partnership

(versus other non-partnership based projects) is based on expert opinion.


The project has led to the physical outputs and outcomes (benefits) set out in Table

2.

Table 2: Outcomes and outputs as a result of the Improving Water Management in the Willow Brook project Output Outcome

Reducing:

Phosphate/NH3/BOD;

Tackling diffuse

pollution.

20% contribution towards an improvement in water quality brought forward by 6years (1 RBMP cycle) specifically in relation to the:

Clarity of water (related to phosphate/NH3/BOD)

from “poor” to “moderate” along 30km;

Plant communities (biology) from “moderate” to

“good” along 15km;

Invertebrates from “bad” to “poor” to “moderate”

along 15km.

Monetary Value of Impacts

The total cost of the project is £31,000 from Defra, £12,500 from the EA, £7,500

from Trent Rivers Trust and £3,750 from in-kind resource.


Cascade Consulting


reported in Table 2. These have been adjusted for inflation so they are all in 2014

prices (HMT, 2015). The total marginal value is the unit value multiplied by the

relevant change in units as a result of the project.

Table 3: Unit value estimates - outcomes from selected projects in the Soar catchment Outcome valued Unit

value (£/km/yr)

Source Annual benefit (100% attribution, £/yr)

Annual benefit (20% attribution, £yr)

Value of improving ecosystem components (Tamar Consulting, 2012) in the Soar catchment from 'Poor' to 'Moderate'

£4,049 Metcalfe (2012) NWEBS

£121,480 £24,296

Value of improving ecosystem components (Tamar Consulting, 2012) in the Soar catchment from 'Bad' to 'Poor'


£52,384 £10,476

Value of improving ecosystem components (Tamar Consulting, 2012) in the Soar catchment from 'Bad' to 'Poor'


£52,384 £10,476


Analysis of Costs and Benefits Over Time

Costs and benefits are appraised over a 7 year time horizon as the benefits are the

bringing forward water quality improvements by 6years (1 RBMP cycle) this which

are expected to occur from year 2. Costs and benefits occurring in the future (post-

2015) are discounted at a rate of 3.5% per annum based on HM Treasury Green

Book guidance.

It is assumed that costs are split over the first 2 years of the project from 2015,

giving a present value of costs over the appraisal period of £53,824 (PVC, 7years).

This is in line with the project timeline stated by catchment respondent.

Benefits are assumed to occur in years 2 to 7. This gives a central estimate of the

present value of benefits over 7year appraisal period of £232,960 (PVB, 7 years).




Cascade Consulting


datasets that have been available for the analysis.

Sensitivity Testing

Given that this analysis is an illustrative project level case study, extensive

sensitivity testing has not been undertaken. However switching values are

produced for the percentage attribution of total water quality benefits that is

required in order for the benefits from the catchment partnership to be equal to the

costs of funding project(s) under the partnership.

The percentage attribution that has been estimated from consultation with the

partnership is 20%, as shown in Table 1. This means that 20% of the benefit of

improving water quality from one WFD status to the next is estimated to be

attributable to the partnership. The estimated switching value is 5%. This suggests

that the partnership is likely to be producing a net benefit (because it only needs to

deliver 5% of the change in status and is estimated to be producing 20%) even

given the uncertainty of attribution of impacts to the partnership.


Central NWEBS values are used to value the water quality benefits ; and

Benefits to water quality elements 'Clarity of Water' and 'Fish' are brought

forward by 6 years or one RBMP cycle occurring in years 2017 to 2022

The contribution of the catchment partnership is a % of the total contribution to

improving the quality of the relevant water bodies, this has been established by

catchment representatives based on expert knowledge.


Cascade Consulting

TEST AND ITCHEN CATCHMENT PARTNERSHIP

Summary


emerged as a result of the existence of a catchment partnership in the Test and

Itchen catchment. The costs and benefits of these project(s) should be viewed

within the wider context of the costs and benefits of measures required for the

catchments water bodies to reach good ecological status (GES) as appraised under

RBMP2, as shown in Table 1 (EA, 2014).


Management catchment PVB PVC BCR

Itchen £20,276,630 £1,622,314 12.5 : 1

Lower Test and Southampton £11,438,856 £8,859,677 1.3 : 1

Upper and Middle Test £3,301,655 £1,305,107 2.5 : 1

Total £35,017,141 £11,787,097 3 : 1



outcomes. For the Test and Itchen this was the Sediment Pathways Project and

Dunn projects. The objective of these projects is to reduce the impact of diffuse

pollution that arises from rural land use. Table 2 sets out the headline results of the

analysis.

Table 2: Headline results for economic evaluation of selected projects in the Test and Itchen catchment

Low Central

High (100%

attribution) Present value costs - £95,851 £95,851 Present value benefits - £263,049 £777,326 Net present value - £167,198 £681,475 BCR (£ benefit per £ cost) - 3 : 1 8 : 1 % attribution to partnership (average by length)

- 6% to 50% (34%)

-


- 12% -



Cascade Consulting



£11.8million worth of projects throughout the entire Test and Itchen catchment

that are required to reach GES. The estimated benefit cost ratio of the assessed

projects under this study for the catchment partnerships of 3:1 is consistent with

appraisal of measures required to reach GES across the entire catchment.






Baseline









2.

Table 2: Outcomes and outputs as a result of the Sediment Pathways Project and Dunn projects Output Outcome

Reducing:

Phosphate/NH3/BOD;

Tackling diffuse

pollution.

Between a 6% and 50% contribution (across different water bodies) towards an improvement in water quality brought forward by 6years (1 RBMP cycle) specifically in relation to the:

Clarity of water (related to phosphate) from

“moderate” to “good” along 15.4km (50%

attribution) and 9km (6% attribution);

Fish from “moderate” to “good” along 4.2km (33%

attribution);


Cascade Consulting


The total cost of the project is £30,000 from Defra, £5,000 from Natural England,

£35,000 from GIA, £25,000 in other funding from a non-disclosed source and

£2,500 from in-kind resource.





Table 3: Unit value estimates - outcomes from selected projects in the Soar catchment Outcome valued Unit

value (£/km/yr)


Annual benefit (stated attributions, £yr)

Value of improving ecosystem components (Tamar Consulting, 2012) in the Test and Itchen catchment from 'Moderate' to 'Good'


£71,818 £35,909



£40,520 £2,431



£38,648 £12,754




bringing forward water quality improvements by 6years (1 RBMP cycle) this which

are expected to occur from year 2. Costs and benefits occurring in the future (post-

2015) are discounted at a rate of 3.5% per annum based on HM Treasury Green

Book guidance.


Cascade Consulting




Benefits are assumed to occur in years 2 to 7. This gives a central estimate of the

present value of benefits over 7year appraisal period of £263,049 (PVB, 7 years).


benefit-cost ratio of 3:1, meaning that for every pound spent on the project an



Sensitivity Testing







partnership is between 6% and 50% for different water bodies, as shown in Table 1.

This means that for one water body (9km in length) 6% of the benefit of improving

water quality from one WFD status to the next is estimated to be attributable to the

partnership, for another water body (15.4km in length) 50% of the benefits are

attributable and there is another water body (4.2km in length) where 33% of

benefits are deemed to be attributable. On average, taking into account the length

of the water bodies, this translates into an average attribution across the water

bodies of 34%. The estimated switching value is 12%. This suggests that the

partnership is likely to be producing a net benefit (because it only needs to deliver

12% of the change in status and is estimated to be producing 34%) even given the

uncertainty of attribution of impacts to the partnership.



forward by 6 years or one RBMP cycle occurring in years 2017 to 2022 ;

The contribution of the catchment partnership is a % of the total

contribution to improving the quality of the relevant water bodies, this has

been established by catchment representatives based on expert knowledge;


Cascade Consulting

Additional benefits to biodiversity are unquantified and not monetised;

Average % contribution across the three water bodies is according to their

% attribution and length.


Cascade Consulting

Wear Catchment Partnership

Summary


emerged as a result of the existence of a catchment partnership in the Wear



water bodies to reach good ecological status as appraised under RBMP2, as shown

in Table 1 (EA, 2014).


Management catchment PVB PVC BCR Wear Lower and Estuary 81,706,941 61,803,268 1.3 : 1

Wear Middle 11,724,543 11,475,042 1.0 : 1

Wear Upper 15,436,140 8,160,392 1.9 : 1

Total 108,867,624 81,438,703 1.3 : 1



outcomes. For the Wear this was the Castle Eden project. The objective of this is to

reduce the impact of diffuse pollution that arises from rural land use. Table 2 sets

out the headline results of the analysis.

Table 2: Headline results for economic evaluation of selected projects in the Wear catchment

Low Central

High (100%

attribution) Present value costs - - £63,640 Present value benefits - - £122,158 Net present value - - £58,518

BCR (£ benefit per £ cost) - - 2 : 1

% attribution to partnership - - 100%


- - 52%




£61.8million worth of projects throughout the entire Wear catchment that are

required to reach GES. The estimated benefit cost ratio of the assessed projects


Cascade Consulting

under this study for the catchment partnerships of 2 : 1 is slightly higher than the

appraisal of measures required to reach GES across the entire catchment of 1.3 : 1.






Baseline





a RBMP cycle (i.e. 6 years) from 2021 and the attribution of outcomes to the

partnership (versus other non-partnership based projects) is based on expert

opinion.



2.

Table 2: Outcomes and outputs as a result of the Castle Eden project Output Outcome

Reducing:

Erosion;

Risk of deterioration to

failing for chemicals/diffuse

pollution by evidencing the

impacts

100% contribution towards an improvement in water quality status for the entire water body brought forward by 6years (1 RBMP cycle)

Restoring 1ha of UK BAP intertidal saltmarsh

An assumed improvement in ecosystem service provision equivalent to that achieved by improving the condition of intertidal saltmarsh under SSSI designation to favourable status


The total cost of the project is £67,000 this includes £32,000 from Defra, £10,000

from the EA, £10,000 from other government bodies, £5,000 from water


Cascade Consulting

company, £10,000 from an environmental non-government organisation. This

includes a mix of funding and in-kind resource.





Table 3: Unit value estimates - outcomes from selected projects in the Soar catchment

Outcome valued Unit value (£/km/yr)


Value of improving ecosystem components (Tamar Consulting, 2012) in the Wear catchment from 'Moderate' to 'Good'


£24,690

Aggregate consumer surplus values of ecosystem services delivered by saltmarsh SSSI conservation activities by SSSI habitats under an ‘Increase funding’ scenario

£734 Christie and Rayment (2012)

£734



Costs and benefits are appraised over a 25 year time horizon. Costs and benefits

occurring in the future (post-2015) are discounted at a rate of 3.5% per annum

based on HM Treasury Green Book guidance.




Habitat benefits are assumed to increase over the first 5 years of the project to a

level where they remain for the rest of the 25year appraisal period. Water quality

benefits are assumed to occur for 6years from 2021, in line with expert judgement

from catchment respondents. This gives a present value of benefits over 25 year

appraisal period of £122,158 (PVB, 25 years).




Cascade Consulting



Sensitivity Testing







partnership is 100%, as shown in Table 1. This means that 100% of the benefit of

improving water quality from one WFD status to the next is estimated to be

attributable to the partnership. The estimated switching value is 52%. This

suggests that the partnership is likely to be producing a net benefit (because it only

needs to deliver 52% of the change in status and is estimated to be producing

100%) even given the uncertainty of attribution of impacts to the partnership.


Funding is a one-off cost occurring between 2015 and 2017

The improvements in water quality and habitat creation are entirely

attributable to the existence of the catchment partnerships (i.e. there's 100%

additionality);

The 'enhancement' in intertidal saltmarsh is of the same marginal

improvement as that associated with improving SSSI's to favourable status ;

Saltmarsh habitat benefits occur from 2017 every year over 25 year appraisal

period (to 2040) reflecting an increasing profile of benefits over the first 5

years of the project;

Benefits to water quality status for the entire water body are brought forward

by 6 years or one RBMP cycle occurring in years 2021 to 2026.


Cascade Consulting

Wye Catchment Partnership

Summary This note provides a summary of the economic evaluation of the projects that have

emerged as a result of the existence of a catchment partnership in the Wye



water bodies to reach good ecological status as appraised under RBMP2. For the

Wye catchment, an economic appraisal under RBMP2 was not performed because

it is failing for SAC standards.



outcomes. For Wye this was the Save our Soils project. The objective of this is to

reduce the impact of diffuse pollution that arises from rural land use. Table 1 sets

out the headline results of the analysis.

Table 1: Headline results for economic evaluation of selected projects in the Wye catchment

Low Central

High (100%

attribution) Present value costs - £80,600 £80,600 Present value benefits - £1,325,967 £5,377,805 Net present value - £1,245,354 £5,297,191

BCR (£ benefit per £ cost) - 16 : 1 67 : 1

% attribution to partnership - 10% to 100% -


- n/a -







Baseline





Cascade Consulting






2.

Table 2: Outcomes and outputs as a result of the Save our Soils project Output Outcome

Reducing:

Phosphorous loss

from poultry

operations;

Soil loss from

farmland;

Nutrient runoff

5% contribution towards an improvement in water quality brought forward by 6years (1 RBMP cycle) specifically in relation to the:

Clarity of water (related to phosphate as leads to

eutrophication) from “poor” to “moderate” along

100km;

Plant communities (macrophytes) from “moderate” to

“good” along 100km;

Fish from “poor” to “moderate” along 100km;

Bringing forward the development of 12,000 houses

Bringing forward the land value enhancement (value to society, DCLG, 2015) associated with development of 364ha of land for residential use by 2 years based on 33 dwellings per hectare using DCLG’s Land Use Change Statistics on density (DCLG, 2013)

Enabling industry development

Realisation of 2years of industry profits (not valued here)

Conserved top soil in the catchment

Avoided costs of replacing an estimated 5,140tonnes of topsoil with the assumption of the responsible rainfall event (and therefore the avoided costs being incurred) occurring once every 5years


The total cost of the project is £82,000, with £46,000 from Defra, £21,500 from

business and £14,500 from other funding. This includes costs in the form of funds

and in-kind inputs.

Table 3 sets out the unit value benefit estimates, that are applied to the outcomes





Cascade Consulting

Table 3: Unit value estimates - outcomes from selected projects in the Wye catchment Outcome valued Unit value

(£/km/yr) Source Annual

benefit (100% attribution, £/yr)

Annual benefit (5% attribution, £yr)

Value of improving ecosystem components (Tamar Consulting, 2012) in the Wye catchment from 'Poor' to 'Moderate'

£2,354.2

Metcalfe (2012)

NWEBS

£243,087

£24,309

Value of improving ecosystem components (Tamar Consulting, 2012) in the Wye catchment from 'Moderate' to 'Good'

£5,304.4

Metcalfe (2012)

NWEBS

£547,716

£54,772

Value of improving ecosystem components (Tamar Consulting, 2012) in the Wye catchment from 'Poor' to 'Moderate'

£2,354.2

Metcalfe (2012)

NWEBS

£243,087

£24,309

Value of bringing forward land value enhancement associated with development of land for residential use by 2 years

£140,000/yr

DCLG (2015) ; Savills

(2014) ; DCLG

(2013) ; DTZ

(2008) ; DCLG (2011)

£137,672/yr n/a

Value of avoided cost of topsoil replacement

£50/tonne Tony Bostock

£257,000/5yrs n/a


Analysis of costs and benefits over time


avoidance of replacement costs for topsoil every 5 years, the bringing forward of

housing development by two years and water quality improvements by 6years (1

RBMP cycle). Costs and benefits occurring in the future (post-2015) are discounted

at a rate of 3.5% per annum based on HM Treasury Green Book guidance.


Cascade Consulting




Benefits are assumed to occur from year 2. This gives a central estimate of the

present value of benefits over 25 year appraisal period of £1,325,967 (PVB, 25

years).

The calculated net present value is £1,245,354 (NPV, 25 years), with a

corresponding benefit-cost ratio of 16 : 1, meaning that for every pound spent on

the project an estimate benefit of £16 is achieved. These figures are subject to the

assumptions and datasets that have been available for the analysis.

SENSITIVITY TESTING

Given that this analysis is an illustrative project level case study, sensitivity testing

has not been undertaken. This, though, is a fundamental aspect of CBA, which is

critical to establishing the validity of results in light of data and information

uncertainties that all practical analyses face; for example determining the

circumstances and assumptions under which benefits outweigh costs. Sensitivity

testing can include, for example, the: (i) use of ranges of values for key variables

(e.g. high, medium, low); (ii) use of ‘expected’ values so that uncertain outcomes

are assigned a probability of occurrence; and/or (iii) Estimation of benefits

thresholds and switching values for key variables.


Funding is a one-off cost occurring between 2015 and 2016 in line with the

project lifetime

Benefits to housing development are brought forward by 2 years occurring in

year 2017 as opposed to 2020


forward by 6 years or one RBMP cycle occurring in years 2017 to 2022 and for

'Macrophytes' from 2027 to 2032 based on expert opinion of catchment

respondent.

Phosphate benefits are equivalent to NWEBS "Clarity of water" and that

Macrophytes are equivalent to "Plant communities"


Cascade Consulting

Fail SAC is equivalent to Poor WFD status and Pass SAC is equivalent to

Moderate WFD status

The % attribution is based on catchment respondents estimates of less than

10% being attributable to the catchment

12,000 houses equates to 364ha of land used based on 33 dwelling per hectare

using DCLG’s Land Use Change Statistics on density (DCLG, 2013)

Land developed for housing is previously undeveloped agricultural land.

DCLG (2015) residential land value estimates (appraises land projects from a

social perspective), Herefordshire £1,545,000/ha.

Land value enhancement associated with development of land for residential

use = £1.504million/ha (average land values for Herefordshire £1,545,000/ha

minus agricultural land values £21,000/ha) (DTZ, 2008 ; DCLG, 2011).

The improvement in water quality brings the housing development forward by

2 years. An assumed discount rate is used of 3.5% didn't use commercial rates

as the land value is to society not market value. The marginal benefit of

bringing forward development by 2 years at this rate is equal to £1.5m minus

the discounted value of the properties after 2 years (£1,310,158) = £189,842

Costs to the agricultural industry associated with avoided loss of topsoil is

quantified and monetised in addition to water quality benefits. This is based on

a 1 in 5 year likelihood of a rainfall event leading to loss of topsoil in the Lugg

catchment applied to the relevant area of the catchment for targeted farm visits

based on modelling of high risk areas, based on a loss of 200k/tonnes across

the entire catchment.

Cost savings to agricultural industry begin in year 2017 and occur every 5 years

over the appraisal period.

Date post:	13-Apr-2020
Category:	Documents
Upload:	others
View:	4 times
Download:	0 times

Defra Evaluation of the Catchment Based...

Documents