Post on 21-May-2015
description
transcript
Development Project Planning 5
Monitoring and Evaluation
Indicators
This Session
Recap – about M&E and Indicators
M&E Planning
Some Indicators o choosing
o setting up
Monitoring
Performance Measurement
o of the Process
o against the sections of the Logical Framework
o routine
Inputs
Outputs
Outcomes (Results Based Management)
What Indicators Show
Inputs Shows what goes in to providing a service - the resources used, e.g.
• Amount spent on travel per week
Activity or
Output
Shows what a service has done or provided, e.g.
• Number of condoms distributed
Utilisation Utilisation Shows if a service is being used, e.g.
• Number of people attending a nutrition course
Coverage Coverage Shows what proportion of people / groups in need receive a service, e.g.
• Proportion of schools with an AIDS awareness club
Performance Performance shows how well something was done, e.g.
• Number of reported cases of STIs
Monitoring (Performance Measurement)
Indicators show success at different levelso Strategic
• Goals, Objectiveso Sustainability
• after project completeo Effectiveness
• on the objectiveso Efficiency
• inputs against outputs
Quality, Quantity,
Time
Monitoring Criteria
o Relevance to goal/purpose and in-country needs
o Efficiency in providing inputs and converting to outputs
o Effectiveness – has production of outputs achieved objectives?
o Impact – is purpose making anticipated level of contribution to high-level goals
o Sustainability – the benefits will continue, without future negative impact
Exercise 5 mins
“Look back to our proposal for a watsan project in Ratanakiri.”
Come up with one indicator for ‘pump maintenance’ AT EACH LEVEL o Activity (efficiency)
o Objectives (effectiveness)
o Strategic - Goals (impact)
o Sustainability – (continues after project)
Data Collection Flow in Projects
What should you monitor and how?
You should monitor information that will help you to track what you are doing and to measure the success of your work.
So you should review your objectives and indicators and decide what information needs to be collected to allow the indicators to be measured.
You should only collect information that you will use.
Evaluation
“episodic” – happens at certain times, not routine
Looks at the Impact (Impact Assessment)
Evaluation: Impact Assessment
Approacho Impact from beneficiaries’ point of viewo What do they think is significant?o To whom is it important?o Ex-ante and Ex-post
Criteriao Efficiency – relate inputs to outputso Effectiveness- extent to which achieved objectiveso Consistency- methods/approaches with objectiveso Impact – change to lives/environment
Audit
Mainly look at financial operations and statementso compliance with legal and contractual
obligationso external, imposed
Can have “Performance Audits”o efficiency and good managemento Data Quality audits
What is an Indicator ?
a variable …
that measures one aspect of a project
that is directly related to the program’s objectives.
Break
Please – back here ON TIME !
A good indicator should:
produce the same results when used repeatedly to measure the same condition or event;
measure only the condition or event it is intended to measure;
show changes in the state or condition over time;
have reasonable measurement costs; and
be defined in clear and unambiguous terms.
Good Indicator - is…
Valid - measures what it is intended to
Reliable - accurate, repeatable
Precise - measures intended condition only
Independent - gives a line of possible results
Timely - available when needed
Comparable - with other similar situations Varies - shows changes in the metric over time; Costed - reasonable measurement costs; and Defined - in clear and unambiguous terms.
Other Factors InIndicator Choice
Data availability
Resources
Program needs
Donor requirements
Indicators: An ExampleObjective: Polluted water put into the Bassac is reduced
Select the indicator: Concentration of heavy metals (Pb, Cd, Hg)
Define the targets:
• the quantity: Concentration of heavy metal compounds (Pb, Cd, Hg) is reduced by 75% from year 2008 levels …
• the quality: ... to meet the limits for irrigation water ...
• the target group: ... used by the farmers of Phan village, ...
• the place : ... in Buphon District, ...
• the time: ... 2 years after the project has started .
Reporting Times
Level of Indicator
Frequency of Reporting
Examples of Data Collection Methods
Input/Process
Continuously ●Health services statistics●Health facility surveys●Program monitoring
Output Quarterly, semi-annually, or annually
●Health services statistics●Health facility surveys●Program monitoring
Outcome 1 to 3 years ●Population-based surveys●Health facility surveys●Special studies
Impact 2 to 5 years ●Surveillance●Population-based surveys●Special studies
Means of Verification
M&E plans should include multiple data sources. they should include indicators that use data from each sector, and include data from the program facilities, population measures, and special surveys.
The M&E plan should clearly state the sources for collecting data and how often the data should be collected.
Means of Verification
Data source must be:o Available
• time• cost
o Accurate• Showing what you want to show• Show what is really there
Exercise
Look at our watsan logframe
Fill in the MoV column
Data Systems
Tools:
o Progress reports
o Team meetings, team briefing reports
o HIS records
Evaluation Plan
A part of the project documents
Use to plan staff needs and budget
Contents of a Typical Evaluation Plan (1)
Brief project description
Objectives of the project
Objectives of the evaluation
Contents of a Typical Evaluation Plan (2)
Evaluation methodology – type of evaluation, indicators for each objective, data collection methods, sample size, methods of sampling / selecting participants
Resources needed, timetable, and budget – material, human, financial, transportation and logistics
Planned use of results – for example, improve the project mid-course, plan future projects, guide decision-making
Thankyou GrKuN
Produced by Tony Hobbs
Health Unlimited,
Ratanakiri, Cambodia
www.healthunlimited.org
With the support of Australian Volunteers Internationalwww.australianvolunteers.com
© 2009 HU. Use with Acknowledgement