Purposes and uses of evaluation

transcript

A. For accountability

A1. Accountability upwards

Accounting “upwards” (to donors or sponsors, to government, to management) is one of the commonest reasons for monitoring during implementation and for showing successful results at the end.Staff and managers of projects and services need to show that the job has been properly done, e.g. funds have been spent and activities carried out as planned.

The first principle“We hold ourselves accountable both to

those we seek to assist and to those

from whom we accept resources.”

(IFRC 2011)

A1. Accountability upwardsThis involves keeping accounts, paying attention to costs, planning work, measuring indicators, collecting data, producing reports and much more. In state programs the annual budget sometimes depends on proof that the job is done properly and the outcomes were worth the expense.

In your own project (1):In your own project who are you accountable to, upwards? What are you doing to keep them satisfied that you are doing it properly?

A1. Accountability upwards

A2. Accountability to participants and stakeholders

An intervention should also be accountable to the participants and stakeholders - individuals, households, groups, communities.Often “participatory evaluation” is simply consultative: managers discuss aims and progress with participants and stakeholders, take their advice, report back to them periodically and discuss outcomes. This gives everyone a voice, shares essential information and experience and helps to ensure consumer satisfaction.

However, it is not fully participatory. In a fully participatory evaluation, the main actors have their own objectives; monitor their own actions, their progress and the situation around them – as you yourself did when you made your own dietary change. This can build ownership and commitment and makes changes more sustainable: essential in improving personal and family diet. It can be done in parallel with a formal evaluation.

In your own project (2):How do participants check progress/outcomes for themselves?Do they see it as their own business or as something imposed upon them?

B. For development – to see what works and what does not, and why

B1. To show progress and success (or failure)

One of the main motives for evaluation is to see and measure success.Some kinds of evaluation are internationally standardised. In IYCF, for example, the indicators and evaluation framework developed by experts are available for use worldwide (WHO 2008). This enables countries and organizations to say where progress is being made, to monitor national progress and to compare with other countries and regions.

More reasons for evaluating1. What gets measured gets done.2. If you don't measure results, you can't tell success

from failure.3. If you can't see success, you can't reward it.4. If you can't reward success, you're probably rewarding

failure.5. If you can't see success, you can't learn from it.6. If you can't recognize failure, you can't correct it.7. If you can demonstrate results, you can win public

support.(Popular saying, attributed to Osborne & Gaebler 1992)

Apart from wanting to see and measure success, there is a real need to show it too (to donors, participants, the public, the world). Visible success inspires participants and implementers and can also mean more funds and ammunition for advocacy.For nutrition education, which has been so little evaluated, it is a top priority to show that it can produce lasting changes in practices, attitudes and understanding.

In your own project (3):What progress have you and your participants been able to see?

B2. To improve the approach or select the most effective actions

Evaluation findings must be used - to improve the intervention as it goes along, to show up gaps in the design or the need to change the approach, to feed into policy decisions. This sounds obvious, but findings are often not used.

“Evaluations need to demonstrate success, but also need to learn about how to do better.

These ideas do not always go well together!”(adapted from O’Flynn 2010)

Some reasons why findings are not used are:• Not everyone has the time, resources or courage to learn

lessons and try new strategies. It requires a powerful (and shared) desire to fix things, a “can-do mentality”.

• Some organizations are open to change but have the opposite problem: they have so many recommendations that they can’t absorb them all (McHarg 2012).

• For learning, failure is as valuable as success, often more valuable. However for many people failure means blame and shame: they are tempted to suppress negative evidence and report only positive results (McHarg 2012, Pritchard 2012).

In your own project (4):Is there anything that you will do better next time? Give an example

C. Research - to build the evidence base for policy and

strategy

C. Research - to build the evidence base for policy and strategy

In all development work, governments, donors and development organizations are looking for evidence-based solutions.

The need for evidence“Initiatives to promote intake of fruit and vegetables

must be based on scientific evidence. Evaluating projects is essential to learn what works and what

does not.” (FAO/WHO 2005)

Evidence-based solutions come from purpose-built randomized controlled trials or from reviews of research which compare alternative solutions and cost-effectiveness. For example, here are some names you should know:• The SUN movement was inspired by the Lancet 2008

series of research reviews which identified ten cost-effective solutions for maternal and child undernutrition (Lancet Series 2008).

• Organizations such as J-PAL at MIT research large-scale strategies needed to solve priority problems (e.g. What are the secrets of success in community programs? What keeps children at school?), comparing solutions and assessing cost-effectiveness.

Research examples (contd.) • The Cochrane Collaboration produces systematic,

internationally recognized reviews of research in human health care and health policy. For example a review of the effect of 14 WASH (water, sanitation and hygiene) interventions on child growth concluded “some WASH interventions …. may slightly improve height growth in children under 5 years of age” (Haddad 2013). Since this is not a very strong finding, it suggests that we may need to look more closely at WASH interventions!

In your own project (5): Do you see any potential for research? If so, what?

In your own project (6): Finally, what do you conclude is the main purpose of evaluating your own project?

This will be mentioned in your final project report.

Summary of evaluation purposes and uses

• A1. For accountability to donors, employers, governments

• A2. For accountability to participants and stakeholders

• B1. For development, to show progress, success or failure

• B2. For development, to improve the approach/ select actions

• C. For research, to provide the evidence base for policy/planning

Purposes and uses of evaluation

Education