Post on 05-Jul-2020
transcript
Annexe 1. TERMS OF REFERENCE for the External Evaluation of ACF’s Nutrition Programme
1.1. Key Evaluation Dates
Expected Start Date: 11th June 2012 End Date: 16th July 2012 Submission of Draft Report
7th July 2012 Submission of Final Report
16th July 2012
1.2. Language of the Evaluation
Language Requirements for the Evaluation:
English Language of the Report: English
1.3. Workplan & Timetable
Activities Workin
g
Days Briefing HQ (teleconference) 1 Travel to mission 1 Briefing mission, review of documents, preparation of fieldwork + evaluation framework.
Detailed methodology to be proposed to ACF prior to commencing fieldwork.
3
Field work 13 Collection of secondary information in capital 1 Data analysis and preparation of the draft report, common editing of the report with ACF
Pakistan
5
Debriefing and presentation of preliminary findings in-country on the basis of the draft
report
1
Travel back from mission 1 HQ debriefing with desk officer 1 Finalization of the report on the basis of Field, HQ and ACF-UK Feedback 3
Total 30
1.4. Budget Details/Conditions Payment will be done on the basis of the above mentioned time table, daily fees shall be
negotiated.
20% of the fees will be paid upon signature of contract, 40% upon reception of draft report and
60% will be paid after validation of the final report by ACF-UK. Travel, accommodation (guest house for in-country nights), food, will be provided at ACF
guesthouses at field level. The application of the visa is the responsibility of the consultant however these costs will be
reimbursed in full upon receipt of the expenses claim form and supporting receipts. Evaluation costs (in country transport, evaluation team, translator etc.) will be covered by ACF,
but the evaluator may be required to absorb the cost initially. Insurance costs will not be covered, and the evaluator shall manage their own insurance, and
provide the details of this cover to ACF-UK before departure.
2. DETAILS OF THE PROGRAMME
Name of the Programme: Emergency Nutrition intervention in Sindh Location:
Thatta and Badin districts, Sindh province,
Pakistan Nutrition interventions implemented under two different
programs: -Pakistan Emergency Food Security Alliance (PEFSA2):
Emergency food security and nutrition support to flood affected
populations in Pakistan (emergency response to 2010 floods) - Emergency nutrition, food security and livelihood support to
flood affected populations in Pakistan (emergency response to
2011 Monsoon)
ECHO/PAK/BUD/2011/91009
Start Date: 01/07/2011 End Date: 31/05/2012 ECHO/PAK/BUD/2011/91025
Start Date: 01/11/2011 End Date: 15/06/2011
2.1. Map of Programme Thatta and Badin districts:
-PEFSA 2: Thatta district, Union Councils (UCs): Bano, Bachal Gugo, Kinjhar, Goongani and
Ladium -Monsoon: Thatta district, UCs: Liakpur, Jokh Sharif, Mehar Shah and DK Suho; and Badin
district, UCs: Shaheed Fazal, Kario, Rahuki and Kadi Kazia
2.2. Programme Overview
ACF in Pakistan: A Brief History
ACF's first intervention in Pakistan was in 1979 in the context of the Afghan crisis and the
arrival of thousands of Afghan refugees. Since then, ACF has been active in responding
to the needs of Pakistani and Afghan people caught in disaster and conflict in Pakistan.
More recently, in October of 2005, ACF intervened in response to the massive
earthquake that struck Northern Pakistan, beginning operations only 24 hours after. Able
to draw on its expertise in the country and the region (with programs having been
implemented in Afghanistan, Nepal, and Tajikistan), ACF was able to provide effective
relief to communities through distributions of food aid, water, sanitation services, and food
security and nutrition programs in areas such as Jared, Kaghan Valley and Rashang,
Allai Valley, Balakot and Battagram. Following the floods caused by a cyclone in 2007,
ACF implemented relief operations in Dadu and Kamber Shahdadkot districts of
Sindh Province, assisting 36,000 flood victims with latrines, water trucking, drilling of wells,
hygiene promotion, and hygiene kits. Also in 2007, ACF conducted Food Security and
Nutrition surveys in these areas as well as Rawalpindi city, to document humanitarian needs
among the populations.
Since 2009, ACF has been implementing an integrated (FSL/WASH) project in Sindh
Province, Thatta District, and in 2010 an ECHO-funded integrated (WASH/FSL) operation
in KPK province so as to begin targeting the underlying causes of malnutrition. When
the 2010 flooding occurred, ACF responded in Thatta District with major WASH and
FSL responses, covering 400,000 people. ACF current strategy is to maximize the impact
of our presence by targeting those areas within our current operational areas. ACF will
respond to immediate needs but also recognizes the need to mitigate and minimize the
negative impact of recurrent floods. Hence ACF will also focus on activities that will
ensure protection of ongoing livelihood recovery. This experience has given ACF a detailed
institutional knowledge of the area.
The Present Nutrition Situation
In Sindh Province, child malnutrition rates have remained persistently high for a number of
years. National nutrition surveys reported the provincial prevalence of wasting to be equal
to 21.3% in 2001-02, and to be equal to 17.5% in 20111. Another nutrition survey2
conducted six months after the 2010 monsoon floods hit revealed critical levels of
malnutrition as well, showing i) Severe Acute Malnutrition (SAM) rates of 6.1% and 2.9%
in Northern and Southern Sindh respectively, and ii) a Global Acute Malnutrition (GAM)
rate of 23.1% in Northern Sindh and of 21.2% in Southern Sindh.
It has been more than a decade since malnutrition rates have been well above the World
Health Organization's (WHO) emergency threshold level (GAM>15%). At district level,
in 2010, Mirpur Bathoro Taluka in Thatta district was highly affected by the floods. In the
four most affected Union Councils of Mirpur Bathoro, ACF assessed the nutritional
status of 6-59 months children and confirmed the severity of the situation with above
emergency threshold level of GAM (GAM=19.5% and SAM=2.4%, based on weight-for-
height z-scores, and using WHO 2006 standards)3, and the
1National Nutrition Survey, Aga Khan University, Pakistan, Sep 2011
2Flood Affected Nutrition Survey (FANS) Dept of health Gov of Sindh Province, Unicef and ACF-Canada
3ACF Integrated Survey, Mirpur Bathoro, Thatta, Pakistan, December 2010
presence of aggravating factors such as high disease burden and poor immunization
coverage. One of the recommendations made was for ACF to start interventions in order
to treat malnourished children and to prevent those who were at risk of becoming
malnourished.
ACF Nutrition Programs
NUM CODE DONOR DEPARTMENT PROVINCE DISTRICT STARTING
DATE ENDING
DATE Status BUDGET COMMENTS
3 PKA1D ECHO-
PEFSA II FSL/NUT SINDH THATTA 1/7/2011 30/03/2012 2,000,000
EURO 30/5/2012NCE
4 PKA1F ECHO-
Monsoons FSL/NUT SINDH BADIN/THATTA 1/11/2011 1/4/2012 850,000EURO 15/6/2012NCE
The direct beneficiaries for the above ECHO-funded programs were flood-affected
households that were rendered acutely food insecure having lost access to food and income
and were still in need for external support. With a multi sectoral approach, the projects
aimed i) to assist the targeted households in regaining their capacity to generate income
and/or produce food while at the same time covering their immediate food needs; and ii) to
reduce mortality and morbidity by improving the general health and nutritional status of
children under five (U5) and Pregnant and Lactating Women (PLW) through a
Community-based Management of Acute Malnutrition (CMAM) approach.
Coordination
ACF coordinates with the Early Recovery Working Groups, individual UN Agencies
and other humanitarian organizations at district, provincial, and national levels in Pakistan
in order to avoidduplication and direct as much aid as possible to those in need.
2.3. General Objective
The humanitarian situation of vulnerable people affected by the food crisis in Southern Sindh
is improved.
2.4. Programme Activities
ACF Nutrition interventions included all components of the Community-based
Management of Acute Malnutrition (CMAM): identification (screening) of under 5
children and pregnant and lactating women (PLW), referral and treatment of those
suffering from acute malnutrition in the appropriate program component (supplementary
feeding programme, outpatient treatment programme or stabilisation centre).
Awareness raising sessions (Hygiene, Health, Breastfeeding and Nutrition best practices)
were conducted to raise awareness among beneficiaries and their caretakers. The programs
also aimed at training i) local health workers on the management of acute malnutrition,
and ii) community volunteers and outreach workers on screening and referral of
malnourished cases to the adequate program sites.ACF established decentralized Out-patient
Therapeutic Programs (OTPs, for the treatment of SAMcases) and Supplementary Feeding
Programs (SFPs, for the management of MAM cases) sites for
the delivery of CMAM services and conducted CMAM trainings to build the capacity of
the health workers and worked to collaborate with the Executive District Offices
(Health), District Coordination Office and other relevant government ministries and
departments to ensure that MoH will in the future handle cases of acute malnutrition in
non emergency situations at localhealth centers and basic health units using the national
CMAM guidelines.
3. AIM OF THE EVALUATION
3.1. Target User(s) of the Evaluation
ACF ELA Unit Implementing HQ Pakistan Nutrition Advisor, Desk Officer Field Level Nutrition team, Country Director, Deputy Country Director Other -
3.2. Overall Objective of the Evaluation
To evaluate the implementation of the ACF ECHO funded Nutrition programs in Thatta and
Badin to assess alternative/improved modes of operation and provide solid recommendations
for the future of the programme.
3.3. Specific Objectives of the Evaluation
1. Assess the capacity building of MoH/PPHI and provide recommendations for
future programming.
2. Assess the current approach to community mobilization using volunteers and
outreach workers such as Lady Health Workers (LHWs), and provide
recommendations for future programming.
3. Assess current community sensitization activities and provide recommendations for
future programming.
4. Assess the extent of the integration of Nutrition with Food Security& Livelihood
(FSL)
activities
5. Analyze implementation strategy and efficiency of intervention and identify best
practices
(activities within the context)
6. Assess relevance and methodology used for targeting and reaching of beneficiaries
on the field
3.4. Scope of the Evaluation
The evaluator should answer the following questions with the aim of making
specific recommendations on how ACF can improve the nutrition programme in the future.
Relevance/Appropriateness
How adequately did ACF assess the needs of the population?
Assess the appropriateness of the objectives and results pursued by the program in
relation
to the identified needs.
How appropriate is the use of community volunteers for case-finding?
Assess its effectiveness and how the community receive this approach.
Is ACF’s current strategy of integration of CMAM services in the MoH appropriate?
Assess the relevance of the chosen partnership with the MoH, integration of
CMAM vs. direct implementation and the relevance of an increased advocacy push.
Coherence
How coherent are ACF’s activities with national policy, ACF country strategy
and otherhumanitarian actors?
Did ACF adequately involve the community in each stage of the programme cycle?
(Needs assessment, programme design and implementation).
Sustainability
Were communities and authorities sufficiently informed of changes in programme
design? How have the current partnerships been developed with UNICEF, WFP and
how can it be improved for the coming period
Assess the level of local-ownership and financial sustainability of the programme.
Did the project decrease the vulnerabilities of the targeted population? Any
linking with recovery (LRRD)? Coverage
How appropriate was the coverage (selection of geographical area and target
group selection) of the CMAM programme?
In UCs where both Nutrition and FSL sectors operate, what was the extent
of the integration of activities and/or of the collaboration between ACF Nut
and FSL
departments?
Impact
How well were nutrition activities integrated with ACF’s WaSH and FSL activities?
To what extend has ACF taken into account principles of ‘Do-no-Harm’ and
mitigated
potential negative environmental impacts?
Effectiveness
Assess the effectiveness of ACF’s ability to deal with foreseen and unforeseen
challenges in the programmes
Sensitization: How effective were the sensitization and awareness sessions given
to the community?
Screening and Case-Finding: How effective was the training in screening methods
for each chosen group (LHWs, community volunteers etc.)?
Efficiency
How could the current monitoring system be strengthened in order to measure
impact more effectively in the future programming?
What level of emergency preparedness (scale-up capacity) is/should be considered
within the programme?
To what extend could program cost have been reduced/made more efficient
without sacrificing the quality of the result?
Cross cutting issues
What measures have ACF put in place in order to ensure gender equality and
sensitivity to other gender issues throughout the programme cycle?
What measures have ACF taken to ensure the security of its employees and other
stakeholders (including beneficiaries’ and non-beneficiaries) in the programme?
3.4. Evaluation Criteria ACF subscribes to the Development Assistance Committee (DAC) criteria for evaluation:
Impact, Sustainability, Coherence, Coverage, Relevance / Appropriateness, Effectiveness and
Efficiency. ACF also promotes systematic analysis of the monitoring system and cross cutting
issues (gender, HIV/AIDS etc). All external evaluations are expected to use DAC criteria in data
analysis and reporting. In particular, the evaluation must complete the following table and include
it as part of the final report. The evaluator will be expected to use the following table to rank the performance of the
overall intervention using the DAC criteria. The table should be included as an Annex to the
report.
Criteria Rating
(1 low, 5 high) Rationale
1 2 3 4 5 Impact Sustainability Coherence Coverage Relevance/Appropriateness Effectiveness Efficiency
3.5. Best Practices The evaluation is expected to provide one key example of Best Practice from the
project/programme. This example should relate to the technical area of intervention, either in
terms of processes or systems, and should be potentially applicable to other contexts where
ACFIN operates. This example of Best Practice should be presented in the Executive Summary
and/or the Main Body of the report.
3.6. Evaluation Outputs The result of this evaluation should be presented in a written report and through
several oral presentations:
One on the mission (to Head of Mission and relevant technical
staff) One at HQ (in person or via teleconference).
3.7. Methodology
3.7.1. Preparation
Review of project documents (proposal, logical framework, donor reports, activity and
monitoring reports, assessment reports, capitalisation documents, budget follow-up, etc.)
3.7.2. Briefing Prior to the evaluation taking place, the evaluator is expected to attend a briefing at HQ level,
and at field level with the Country Director and/or the relevant technical focal point. Briefings
by telephone must be agreed in advance. 3.7.3. Field activities
Consultants are expected to collect an appropriate range of data. This includes (but not
limited to):
Direct information: Primary data collection using standard and participatory
evaluation methods (e.g. semi structured interviews, FGD and observation)
Indirect information: Interviews with local representatives; interviews with project
staff expatriate and national staff; meeting with local and provincial authorities,
groups of beneficiaries, humanitarian agencies, donor representatives and other
stakeholders. For indirect data collection, standard and participatory evaluation methods
are expected to be used (HH interviews and FGDs with beneficiaries, non-beneficiaries,
key informants – health workers and leaders).
Secondary information analysis: including analysis of project monitoring data or of any
other relevant statistical data related to the nutrition implementation. 3.7.4. Report The report shall follow the following format.
Cover Page
Table of Contents
Executive Summary: must be a standalone summary, describing the programme, main
findings of the evaluation, and conclusions and recommendations. This will be no more
than 2 pages in length.
Main Body: The main body of the report shall elaborate the points listed in the
Executive Summary. It will include references to the methodology used for the evaluation
and the context of the action. In particular, for ACF key conclusion there
should be a corresponding recommendation. Recommendations should be as
realistic, operational and pragmatic as possible; that is, they should take careful
account of the circumstances currently prevailing in the context of the action, and of the
resources available to implement it both locally and in the commissioning HQ. Annexes:
Listed and correctly numbered. Format for the main body of the report is:
o Background Information o Methodology o Findings & Discussions o Conclusions Recommendations o Annex I (Best Practice) o Annex II (DAC-based Rating Table)
The report should be submitted in the language specified in the ToR. The report should not be
longer than 30 pages excluding annexes. The draft report should be submitted no later than 10
calendar days after departure from the field. The final report will be submitted no later than
the end date of the consultancy contract. Annexes to the report will be accepted in the working
language of the country and programme subject to the evaluation.
3.7.5. Debriefing & Learning Workshop The evaluator should facilitate a learning
workshop:
To present the draft report and the findings of the evaluation to the ACF Pakistan Team and other stakeholders.
To gather feedback on the findings and build consensus on recommendations.
To develop action-oriented workshop statements on lessons learned and proposed
improvements for the future. 3.7.6. Debriefing with ACF HQ
The evaluator should provide a debriefing with ACF HQ in New York on her/his draft report,
and on the main findings, conclusions and recommendations of the evaluation. Relevant
comments should be incorporated in the final report.
4. RESOURCES AND DOCUMENTATION
The following documentation will be made
available: Project proposals (ECHO; PEFSA,
Monsoon) Budgets
Quarterly reports
5. PROFILE OF THE EVALUATOR
Significant experience in the design and implementation of community-based nutrition programmes,
Experience in the evaluation of CMAM programmes,
Experience with nutrition programmes in Pakistan
desirable,
Significant field experience in the evaluation of humanitarian / development
projects, Relevant degree / equivalent experience related to the evaluation to be
undertake, Good communications skills and experience of workshop facilitation,
Ability to write clear and useful reports (will be required to produce examples of previous
work) Fluent in English,
A strong understanding of donor requirements,
Ability to manage the available time and resources and to work to tight
deadlines, Independence from the parties involved.
6. RIGHTS
The ownership of the draft and final documentation belong to the agency and the funding
donor exclusively. The document, or publication related to it, will not be shared with
anybody except ACF before the delivery by ACF of the final document to the donor. ACF is to be the main addressee of the evaluation and its results might impact on both
operational and technical strategies. This being said, ACF is likely to share the results
of the evaluation with the following groups:
Donor
Governmental partners
Various co-ordination bodies
Intellectual Property Rights
All documentation related to the Assignment (whether or not in the course of the
evaluator’s duties)
shall remain the sole and exclusive property of the Charity.
Annexe 2. Chronogram
Date Day Activities
20.6.12 Wednesday Preparation and Document Review + Briefing HQ
21.6.12 Thursday Flight Madrid-Islamabad
22.6.12 Friday Briefing in Islamabad. Evaluation framework
23.6.12 Saturday Review of documents
24.6.12 Sunday Preparation of fieldwork. Detailed methodology
25.6.12 Monday Discussions with nutrition coordinator, WASH and FSL
26.6.12 Tuesday Travel to the field. Briefings in Thatta base with programs coord.
27.6.12 Wednesday Travel to TMK to meet nutrition Teams. Worked local data bases
28.6.12 Thursday Interviews and FGD with beneficiaries in Badin.
29.6.12 Friday Interviews and FGD with beneficiaries in Badin
30.6.12 Saturday Interviews and FGD with beneficiaries in Thatta
1.7.12 Sunday Resting day
2.7.12 Monday Interviews and FGD with beneficiaries in Thatta district
3.7.12 Tuesday Interviews and FGD with beneficiaries in Thatta District
4.7.12 Wednesday Meetings with EDO, PPHI, Focal Nutrition and visit to SC
5.7.12 Thursday Interviews and FGD with beneficiaries in Thatta district, meetings
with LHW and MERLIN
6.7.12 Friday Failed to meet UNICEF and WFP in Karachi, Phone interviews
7.7.12 Saturday Flight to Islamabad
8.7.12 Sunday Data analysis and preparation of the draft report
9.7.12 Monday Collection of secondary information in ACF base
10.7.12 Tuesday Data analysis and preparation of the draft report
11.7.12 Wednesday Discussions with Nut Coordinator, Consultant for surveys & FSL
12.7.12 Thursday Meetings with UNICEF, OXFAM, MERLIN and SC in Islamabad
13.7.12 Friday Debriefing and presentation of preliminary findings
14.7.12 Saturday Report writing.
15.7.12 Sunday Sending draft report
16.7.12 Monday Flight Islamabad Madrid
Annex 3 Selected UC and villages
District Unit Council Village Comments
Badin Dubi Saeed Khan Chandio
Kario Ganhwar Not feasible due to time
constraints
Karyio Sohrab Khan
Nizamani Aprox 2000 inhabitants
Thatta Bachal Gogo Mohad Khan Samoo
Khamoon Malha
Jock Sharif Abdula Palijo Not feasible due to demonstration
on the road
Atal Sha
Mehar Shah Wali Sha
Qadir Dino
Kiinjhar Rahib Amro
Nodo Bara
Darro Darro
Leemo Malha
Annex 4 List of persons met for interviews
Place Name Organization Position
Madrid-NY
via skype
Charmaine Brett ACF Desk Officer
Cecile Basquin ACF Nutrition Advisor
Islamabad Shahid Fazal ACF Nutrition Coordinator
Paola Maria Valdettaro ACF FSL Coordinator
Eric ACF WASH Coord
Dewi Dwiyanti ACF Admin Coord
Onno Van Mannen ACF Country director
Sarma Mazish OXFAM PEFSA Coord
Alison Donnelly SC Nut Advisor
Silvia Koffman UNICEF Nut Chief
Dr Ijaz Habib MERLIN Nut Coord
Thatta John Batley ACF Field Coord
Jackeline ACF Nut PM
Milton Zhakata ACF FSL PM
Fayaz Ahemd CNV
Razia beneficiary
Salma beneficiary
Momal beneficiary
Asilam Khan beneficiary
Akhter Ali ACF FSL MEAL
Dr Arhum MD Dahro
Abdul Wahid CNV
Ghulam Haibar beneficiary FSL CFW
Gulzar beneficiary Grant
Ahmed beneficiary FSL CFW
Mohamed Haseem beneficiary FSL CFW
Suleiman beneficiary OTP
Ali Mohamed beneficiary FSL CFW
Karan Bux beneficiary OTP
Mohamed Mussa beneficiary SFP
Tallat beneficiary PLW
Mohamen Mussa beneficiary FSL CFW
Khalis beneficiary OTP
Ali Khan beneficiary Grant
Dr Nider Ahmed Mimon MD Nodo Baro
Dr Khalid EDO Nut focal point
Dr Ifthikar EDO Exec. District Off. Heal.
Manzo MERLIN
Dr Paras MD SC
Maqsood PPHI Distric Support Manag.
Jawaid Nurse SC
Zahida Nurse MERLIn
Dr Makbull MD LHW
Telephone Zacharias UNICEF Nut Specialist
TMK Bheru Lal ACF Depute Nut PM
Mohamad Ali ACF Nut data analyst
Mohammend Khan ACF Nut PM in Dadu
Abdul Shakar ACF Depute FSL PM
Badin Parez Ali CNV
M. Jumani CNV
M. Harif CNV
M Khan CNV
M Younis beneficiary
M Aslam beneficiary
Ali Imlan beneficiary
Rashid beneficiary
Abdul Hameed beneficiary
Musthaqua beneficiary
M Hasan beneficiary
Abdul Istif beneficiary
Sarfaz Khan Abro ACF Nut Team Supervisor
Misbrah Qambrani ACF Nurse
Salma Nizamani CNV Female teacher
Mehar-ul-Niza CNV
Bilgees Nizamani CNV
Sajan Das ACF Screener
Names of participant in Focus Group are not listed since it was considered not relevant
Annexe 5. qualitative data collection
Interviews to key informants Name; Position; Organisation;
Context 1. At national, health facility and donors level: current level of integration in
relation to nutrition policies/strategies within Ministries and related departments
Design 1. Main processes in relation to needs assessment and proposal design
2. In which way was the project strategy relevant and appropriate for the context
and for the target beneficiaries?
3. What could have been improved during the design phase?
Implementation Results achieved 1. What were the most effective activities of the Project?
2. What activities could have been improved?
3. Were there any delays in the delivery of project activities? If yes, why?
Staffing 1. Was the ACF staffing structure (e.g. number of staff, location, capacity, skills,
qualifications, management support) appropriate for building capacity of partners
and achieve Project implementation?
2.CNV are doing screening? defaulters tracing? follow ups?
M&E 2. Were appropriate indicators identified to measure outcomes and outputs of the
Project?
3. How was the Project monitoring conducted?
4. Were appropriate project revisions or timely actions taken based on the
monitoring system? How? (Examples)
Impact 1. In your opinion, in which way did the Project benefit to community volunteers,
communities, women and children and health system as a whole
2. How are the trained health workers participating in the project?
Collaboration &
Partnerships
1. What were the main partnerships challenges?
2. In which ways did the partnership strengthen the capacity of MoH and local
partners?
3. What recommendations will you have for future partnership models for Projects
the same or similar to this one?
Sustainability 1. What were the main processes for exit?
2. What results and activities appear to have a significant chance of continuing
now that the Project is completed?
3. Does it appear that local partners have the capacity to continue without funding
from ACF? If yes, how?
4. If no, why and what could have been done to ensure continuity?
Best Practices 1. Can you give an example of best practice from this Project?
Lessons Learned 1. Can you give an example of the main lesson learned in this Project?
Group Discussion: Community Leaders & Community Members
Permission and Information Greetings. Introduce evaluator and translator names. We are doing an evaluation of ACF
project and we really appreciate your participation. We want to ask about. This information
will help ACF and implementing partners to assess whether the Project was successful. Usually
this discussion takes time about 30 minutes. Any information that you give will be kept
confidential. This is a voluntary participation and you can freely decide not to answer. However,
we would appreciate your input. Do you have any questions about the discussion?
Topics to explore: Understanding of Project content
Level of participation (how; when)
Perceived benefit of activity (training; follow up; referral; counselling)
Most difficult or challenging activity (training; follow up; referral; counselling; collaboration
with health services)
Main perceived improvements or benefits (individual level, community level) – as a result of
this Project
Gender issues among CNV
Constraints and/or barriers to improvements in this particular community
Recommendations and/or suggestions for future ACF involvement
Group discussion: Community Volunteers
Topics to explore: Understanding of Project content
Organisation of training
Most interesting component/activity (training; follow up; referral; counselling)
Coverage, level od admissions among those sent to OTP
Most difficult or challenging component/activity (training; follow up; referral; counselling;
collaboration with health services)
Elements that receive particular emphasis during a follow up, home visit or counselling
session
Main perceived improvements or benefits (individual level, community level, health system
level) – as a result of this Project
Constraints and/or barriers faced during implementation of outreach activities
Exit plan, activities ongoing after leaving.
Activities that are/are not likely to continue after the Project ends. Why?
Recommendations and/or suggestions for future ACF involvement
Annexe 6 evaluation matrix
Evaluation
criteria
Key questions Hypothesis Sources of Information
Relevance/
appropriat
eness
How Adequately Did Acf Assess The Needs Of
The Population? appropriateness of the
objectives and results
How appropriate is the use of community
volunteers for case-finding ?
The design, scale and scope of the
Project were in line with the needs of
stakeholders & beneficiaries
CNV network is not well
implemented and they are not main
source of admissions
Proposal
Assessments
National surveys
Evaluation reports
Key Informant interviews
+
CNV FGD
Coherence How coherent are ACF’s activities with national
policy, ACF country strategy and other
humanitarian actors?
Did ACF adequately involve the community in
each stage of the programme cycle?
The Project was coherent and
complementary to actions of key
stakeholders and other ACF
interventions
Yes, ACF involve all the main actors
of the community
Key informant interviews
Direct observation
Narrative reports
Sustainabil
ity how can the partnerships been developed with
UNICEF, WFP be improved?
Assess the level of local-ownership and
financial sustainability of the programme
Did the project decrease the vulnerabilities of
the targeted population?
We doubt that the Project has
promoted the integration of
community health activities into the
health system
Yes the Project has decrease it
Key informant interviews
Focus Group Discussions
Narrative reports
Direct observation
Coverage How appropriate was the coverage of the
CMAM programme
The Project has reached all the
people equally within the designated
areas
Proposal
Assessments
National surveys
Evaluation reports
Key Informant interviews
Impact Integration with ACF’s WaSH and FSL
activities
Has ACF taken into account principles of ‘Do-
no-Harm’ and mitigated potential negative
environmental impacts?
Due to major constraints integration
is been difficult and limited
The Project produced long-term
benefits for the communities, the
final beneficiaries, the local
partners ... in accordance with ACF
principles
Survey reports
Key Informant Interviews
Focus Group Discussions
Direct observations
Effectiven
ness ACF’s ability to deal with foreseen and
unforeseen challenges
How effective were the sensitization and
awareness sessions given to the community
effective was the training in screening methods
for each chosen group (LHWs, community
volunteers etc.)?
The program has adapted to
challenges.
Identification of malnutrition and
knowledge of nutrition have
improved
They were reasonably satisfied with
the assistance that they got from the
Project.
M&E framework
Annual Reports
Survey reports
Focus Group Discussions
Key informant interview
Direct observations
Efficiency How could the current monitoring system be
strengthened in order to measure impact more
effectively?
What level of emergency preparedness should
be considered ?
To what extend could program cost have been
reduced?
Several indicators can be added to
the Information collected
ACF should be prepared for forssen
emergencies
The use of financial resources can be
optimized
Organogram
Job description
Key informant interviews
Budget
Financial reports/audits
M&E plan
Narrative reports
Cross
Cutting
issues
gender equality and sensitivity to other gender
issues
security of its employees and other stakeholders
Gender sensitivity should and can be
reinforced
ACF took effective steps to take
account of security
Project proposal
Key informant interviews
Focus group discussions
Direct observation
Annex 7. DAC-rating for project evaluation
Criteria Rating Rationale
1 2 3 4 5 Relevance X Good
Good assessment of needs from different point of views
Consultation with government, beneficiaries and NGOs
Good choice of activities
Insufficient detailed exit plan
Objectives too focused on CMAM and emergency. Not
addressing underlying causes
Coherence X Good
National protocol has been followed
good level of dialogue and communication with
government
Involvement of the community
Coverage X Moderate
Geographical distribution not well done
Mobile OTPS have greatly increased coverage and
reduce defaulters
Training coverage failed among health workers.
Beneficiary sessions too crowded. No refreshing
workshop for CNVs
No good integration with FSL
Sustainability X Very Poor
Only SC is operating
Exit plan is not being implemented
Health staff not trained. LHW not partcipated
Partnership with PPHI was not created
CNV stopped working once ACF was gone
not integrated with Health Structures
Impact X Moderate
OTP was greatly delayed
SC only opened in June 2012
Good number of admission in SFP and PLW
Overcrowding of session for beneficiaries
Failed to raise awareness
Few negative impacts
Efficiency X Moderate
concerning Health Structures have not been achieved
CMAM program was operational with the exception of
the SC
good at dealing with unforeseen challenges
Monitoring system can be improved but is on place
Efficacy X Moderate
trainings should be given when participation is assured
sensitazion sessions should be optimized while
beneficiaries are waiting and given to small numbers
even if shorter time
human resources should be well employed for their
defined tasks
avoid incentives but use them if necessary to implement
program
Annexe 8. Examples of best practices
Title of best practice 1. Mobile OTPs
Innovative Features &
Key Characteristics
CMAM is meant to be a community based management of
Acute Malnutrition but often outpatients services are
installed in health structures far away from beneficiary
households and the population hardly know about it.
When the PPHI decided no to allow ACF to work in the
Basic Health Unit a mobile approach was established. OTP
changed everyday and were never more than 10 km away
from beneficiaries. Tracking defaulters was easy since they
can be visited on the same they that are defaulting. Places
can be changed as the time goes by to meet beneficiary’s
needs.
Mobile OTPs require good planning and some extra
logistical challenges. For a start every team needs
transportation to different sites every day. Fuel must be
considered in the budget and every day the team must leave
the base with the entire equipment load in the vehicles: in
order no to loose valuable time all this equipment must be
loaded the day before.
Coordination is important not only with logistics, but
especially with security team since Nutrition workers are
moving around big areas and it is known that situation on
the field can change fast. Weekly updates about movement
and constant communication is vital.
It also requires a constant approach with local leaders and
communities to find out best places and days to installed
OTP sites. Take into account markets days, presence of a
school etc.
Practical/Specific
Recommendations for
Roll Out
Though it may seem a great community approach it has a
slightly negative side. Being away from Health Structures
there is no integration at all with existing Health facilities.
Once the program is finished sustainability is almost zero.
Therefore it should be combined with a classical static
approach, spending some days of the week within Health
Centres.
Teams must not forget other aspects of the community
approach now that OTPs are placed in the community:
community volunteer’s networks to identify malnourished
cases, sensitizing the community about malnutrition,
tracking defaulters etc. but these activities are easily
followed up since the teams are very close to patients and
households.
Mobile OTPs impose additional workload to teams. It is
advisable to let the teams rest in the base once per week,
conduct sessions and refreshment trainings to allow
different teams to meet and share experiences etc.