Scottish Government funded Projects:

Monitoring, Evaluation and Learning (MEL)

… and reporting

4 September 2013

Outline of presentation:• Log Frames as a project planning,

management and MEL tool• Outcomes of Comic Relief / Bond

research on MEL by international organisations

• Setting up MEL systems ... then:• The SG presentation about using

the log frame to report on project progress and outcomes/impact

Log frame – a tool:

If Activities are implemented, will outputs be produced?

If Outputs are produced, will outcomes result?

If Outcomes result, will the Impact be achieved?

What can we learn from this project and the impact achieved?

Log frame – use of it:

MEL research – DFID, Comic Relief/Bond/NIDOS:• Aimed to review how NGOs view MEL and what resources and processes they commit to MEL• Reviewing the ‘cost of MEL’ • Case studies of good practice• Some key learning on how MEL could be improved

MEL research – results:• MEL is compartmentalised and ‘per project’ largely• Log frames being seen mainly as a tool for project reporting to funders• Data collection largely done by staff and partners in the field• Data analysis largely done by grant holder in the ‘north’

MEL research – continued:• Not enough embedding of MEL in organisational review – results from projects not used in impact review• Lots of hidden MEL costs not budgeted for • MEL more successful when have: •leadership buy-in and support;•appropriately skilled and resourced staff;•investment in MEL at a strategic level

Log frame – pairs discussion:Who in your organisation/partner organisation is involved in developing & managing the Log Frame?Who owns the Log Frame – and who understands it and knows about it?How often does the Log Frame get looked at and used – and for what and by whom?

Monitoring...How and when is the monitoring system set up – who decides / delivers the following:• baseline setting and measurement • what data will be collected, from where and who will do this• creating forms/tools for data collection• training / induction of data collectors• setting up data storage system

Data quality and storage• Who oversees the data collected and who looks at it and how often? • How is consistency checked?• Do data collectors understand the data storage system or is there someone dedicated to ensuring data is stored accurately?

Evidence Principleshttp://www.bond.org.uk/effectiveness/principlesVoice and Inclusion: focused on/ includes the perspectives of people living in povertyAppropriateness: generated through methods that are justifiable given the purposeTriangulation: the evidence comes from a mix of methods, data sources, and perspectivesContribution: the evidence explores how change happens and the specific contributionTransparency: open about the data sources, methods used, the results achieved, limitations in the data or conclusions.

Review and Analysis• Who is involved in review and analysis of monitoring data during the project delivery? Are participants involved, project field staff/frontline staff, project managers, grant holder?• How does learning from monitoring get fed into the review of project activities and purpose to improve project delivery?

Where and how: data review and analysis? • Who is involved?• Who decides whether the project is achieving useful outcomes - if priorities change, who decides?• What does this mean for the project and for the funder?

Evaluation and Learning• Who is involved in shaping evaluation questions and who will be stakeholders in the evaluation?• Is it external and if so, how are the evaluators chosen?• Who looks at the evaluation results?

Reporting on the Log Frame• This is just the last stage in an on-going process of data collection, review, analysis and evaluation • The log frame should not be a dusty document!

Some useful tools:Impact Builder: tool for strengthening MEL systems – with NIDOS area ... http://my.bond.org.uk/impact-builder NIDOS Effectiveness toolkit: section on project management and MEL: www.effectiveness.nidos.org.uk/ Or Health Check: www.bond.org.uk/effectiveness/health-check

IATI: www.aidtransparency.net/