MODULE 3 – DATA, MONITORING AND EVALUATION

DR DARREN PERRIN

The aim / learning outcome of this module is to understand how to gather and use data effectively to plan the development of recycling and composting schemes; identify improvements and then use data effectively to monitor and assess the performance of these schemes.

• Understand the different types of data and why it is important to collect data

• Understand the limitations of data

• Explain why monitoring and evaluation is important

• Understand how to translate data into action

MODULE OUTLINE

• Provides the basis of any sound decision making

• “If you can’t measure it ... You can’t manage it”

• Commercial / Public sector - Increased efficiency = saves money!

• Reduces risk, increases certainty, subject to:

• Understanding the limitations of no / poor data:• Inaccurate estimates• Incompatibility of infrastructure and markets• Poor planning and missed opportunities• No data better than poor data!!!

WHY IS DATA IMPORTANT TO COLLECT?

• Type of Data• Waste Generation & Flows • Waste Composition• Financial • Social Profiling • Capacity and Infrastructure• End Markets• Performance Assessments

• Fit for purpose?• Affordability and Priorities • Be aware of poor data / Data gaps

WHAT DATA DO I NEED?

• Current position on waste material flows

• Issues and Opportunities

• Inefficiencies

• Ability to track changes and impact of new policy, strategy objectives / targets

• Performance against Key Performance Indicators (KPI)

• Ability to plan and forecast

WHAT WILL WASTE DATA TELL ME?

• How to Prioritise

• Weight versus volume

• Data not static

• Composition • System performance• Population / household • Financial • ……?

• External influences over time

• Change in material revenue• Change

• Material Properties

• Bulk density • CV• Chemical properties

SOME CHALLENGES IN DATA COLLECTION

PLANNING EFFECTIVELY?

• Data not always available / affordable

• No data sometimes better than poor data

• Gaps in data may require assumptions to be made:• Waste composition• Number of households or business waste generation rate• Potential performance e.g. Material capture rates

• Where assumptions are critical to outcomes, sensitivity analyses can be used to:• Provide range of values on which to base decision• Highlight potential areas of risk

DATA LIMITATIONS

• Need to forecast the quantities of waste to help in planning

• Future quantities of waste dependent on:

• - Waste generated• - Households (rather than population)• - Business activity • - Economic Activity • - Specific elements of waste stream (e.g. recycled content)• - Waste prevention activities• Predict a range not single line growth• Use previous trends to inform assumptions• Dependant on future workload, business expansion, type of

activities/production

IMPORTANCE OF FORECASTING

FORECASTING A RANGE

• Recycling Rate

• Landfill Diversion Rate

• Dry Recycling Contamination Rate

• Participation Rate

• Capture Rate

• Recognition rate

• Collection Yield

KEY PERFORMANCE INDICATORS (KPI)

DATA STRATEGY

Document which clearly sets out data requirements and approach to obtaining it • What data is required and priorities ?

• Why is the data required (Mandatory, Required, Useful, “Nice to have”) ?

• When will the data be collected and at what repeat frequency ?

• Who will collect the data ?

• How will the data be collected ?

• Units of measurement

• How will the data be reported?

• How much will the data cost to collect? ROI?

DATA STRATEGY PROCESS

MONITORING AND EVALUATION PLANNING

What are you trying to find out? Define

Investigate

Assess

Learn

D

I

A

L

What tools / indicators are you going to use?

What are you going to do with the information?What have you learnt and what is going to change as a result of the new information

• What question are you trying to answer?

• What would the answer look like?

• Is it SMART?

• What are you going to measure?

• Do you need to compare data and is this data available?

Define D

• What indicators are you going to use?

• Will your indicators selected answer your question?

• How are you going to use them?

• Quantitative or qualitative data?

• Single or multiple sources of data?

• Plan to collect data• Costs• Audits• Field data

Investigate Investigate I

• How are you going to analyse the data?• How are you going report it?

Assess A

Exercise

• Set out Rate

• Participation Rate

• Capture Rate

• Recognition Rate

• Contamination Rate

Remember how to calculate them? What’s new?

RELATIONSHIP BETWEEN INDICATORS

• Low set out, low participation, high recognition

• High set out, low contamination, high participation, low recognition

• High participation, low set out, high recognition, High contamination

• Low capture, high set out

• Consider: • Describe scenarios and implications • How would you resolve each situation ?• What would be preferred ?

EXERCISE – RELATIONSHIP INDICATORS

• Appropriate format to data • Show trend• Compare with baseline

• Data should be clear and consistent

• Tailor reporting to audience

REPORTING

0.0%

20.0%

40.0%

60.0%

80.0%

100.0%

Tuesday 62.9% 61.2% 50.7% 81.6%

Wednesday 52.8% 44.3% 43.4% 69.5%

Thursday 51.8% 56.4% 49.0% 72.3%

Week 1 Set Out Week 2 Set Out Week 3 Set Out Participation

• What are you going to differently in response to the monitoring and evaluation data

• Is there further monitoring required?• Continuous improvement • Ongoing

Learn L