2. Introduction to reviewers This evaluation was conducted by Jigsaw Consult, a London-based social enterprise that
helps international development organisations to work more effectively.
The evaluation team was led by Dr Lydia Tanner; the fieldwork was conducted by Dr
Lydia Tanner and Ms Louise Randall. Mrs Sarah Voss provided research support and Dr
David Hollow provided input on the evaluation methodology and analysis. The views
expressed in this report are those of the authors and do not necessarily reflect the
official policy or position of Samaritan’s Purse, its partners or its funders.
3. Acknowledgements The evaluation team expresses its gratitude and appreciation to the project staff
members, care group volunteers, community members, local government
representatives and others who responded to our questions, offered their views and
opinions, and provided help.
The Samaritan’s Purse project and headquarter staff were a joy to work with: their hard
work and dedication to the project is impressive and the evaluation team appreciated all
the support they provided.
4. Abbreviations and acronyms CF Complementary feeding
DFID Department for International Development
EBF Exclusive breastfeeding
DHO District health officer
FGM Female genital mutilation
GPAF Global Poverty Action Fund
HP Health promoter
HS Health supervisor
IMNCI Integrated Management of Neonatal and Childhood Illnesses
IRC International Rescue Committee
KIMCH Karamoja Integrated Maternal Child Health (project)
LG Leader grandmother
LM Leader mother
LQAS Lot Quality Assurance Sampling
MCH Maternal and child health
MDG Millennium Development Goal
MMR Maternal mortality ratio
MoH Ministry of Health
NGO Non-governmental organisation
NW Neighbour women
SP Samaritan’s Purse
VfM Value for money
U5MR Under-five mortality rate
VHT Village health team
TBA Traditional birth attendant
www.jigsawconsult.com
Samaritan’s Purse Karamoja MCH evaluation
Page 3
5. Executive summary Samaritan’s Purse implemented a three-year maternal and child health (MCH) project in
Napak district in north east Uganda. The project was based on the care group model
which aims to increase knowledge and promote positive MCH behaviours. Health
indicators in Napak have historically been very poor: infant and maternal mortality rates
are high and there is chronic under-nutrition. The progress made as a result of the
KIMCH project should be viewed in relation to the challenging context.
Relevance
The project was designed to contribute to MDGs 4 and 5 in Uganda: to reduce child
mortality and to improve maternal health. The care group model is a proven model for
improving child health and is particularly relevant in Karamoja: in a baseline study, only
14% of women and 12% of men were able to name four preventative actions for
common childhood illness.
A notable strength of the project is the extent to which it identified and addressed the
specific needs, concerns and cultural barriers affecting improved MCH behaviour. At the
outset of each teaching module, a research technique called Barrier Analysis was
employed to gather information on perceived physical, physiological and social barriers
to behaviour change. All beneficiary groups interviewed were positive about the project
and felt it was relevant in addressing perceived health needs.
Effectiveness
Project staff were found to be effective in delivering the project, had a good
understanding of risk, and used robust financial processes. Non-financial risks well were
understood although only monitored informally. The activities were delivered on time
and in budget. The single exception was the distribution of insecticide treated nets (ITN)
which were delayed and then increased in cost. Although SP was able to reach a reduced
number of households, other organisations conducted ITN distribution to the rest of the
population in the same year.
Positive drivers for project delivery included the involvement of local Health Promoters
(HPs), a high level of local ownership, inclusion of community leaders and men,
promotion of peer-to-peer learning, and a positive response from care groups to a
Hepatitis E outbreak. A key challenge was introducing the idea of volunteering into the
community: the expectation of material benefit was a consistent frustration to Health
Supervisors (HSs) and affected how communities perceive the project.
Efficiency
In general, SP demonstrated a rigorous approach to management of costs, with strong
procedures for financial accountability and transparency. The project staff at all levels
were aware of the activity budgets and followed processes for procurement, performance
management and cost management.
The project achieved good value for money. It involved few capital costs and instead
focused on human resource development for a longer term impact. The project was
delivered at a cost of less than £10 per direct beneficiary over the 3 year lifetime of the
project. The evaluation team noted the SP had fostered efficiency through creating a
strong team, and building relationships with local government, health staff and
communities.
www.jigsawconsult.com
Samaritan’s Purse Karamoja MCH evaluation
Page 4
Impact
219 care groups were established for 2,130 elected leader mothers (LMs) and leader
grandmothers (LGs). The Karamojong culture has an oral tradition and LMs and LGs
responded well to oral learning methods; they enjoy the rhymes, sayings, simplified
anecdotes and stories and are quick to recount stories of Karamojong women from the
learning materials. Retention and turnout of LMs and LGs was high, with 85% passing
oral tests on teaching modules in 2015.
The care groups followed curriculum on four modules: nutrition, common disease
prevention and treatment, pregnancy and birth care, and hygiene and sanitation.
LMs identified lessons on hygiene and sanitation as the most important aspect of
their learning. This perspective was shared by local government, health workers and
is the area where the evaluation team saw the most significant direct impact.
Education had a positive impact on reduction of common childhood and maternal
illness in Napak district. The percentage of caregivers able to name 4 or more
actions to prevent childhood illnesses increased from 13% to 57%; the percentage
able to name 4 or more symptoms of common childhood illnesses rose from 21% to
67%. In addition, there was a pronounced increase in the number of caregivers
reporting that their children sleep under an ITN (30.0% in 2012 increasing to 97.1%
in 2015). The reported rate of treatment for diarrhoea increased from 72.2% to
82.2% and the reported rate of treatment for fever with malaria medication from
30% to 50%.
HPs identified teaching on exclusive breastfeeding (EBF) as one of the principle
benefits of the programme, noting a reduction in the use of water and cow’s milk for
babies. There was a large, statistically significant increase in the percentage of
women who reported practicing EBF for children aged 0-6 months, from 49.5% to
92.4%. Women understood messages relating to complementary feeding of children
aged 6-24 months and made some effort to prioritise a varied diet for pregnant
women and young children. However, changes in nutritional practice have been less
pronounced than in other care group projects globally. This is predominantly due to the harsh environment, high levels of poverty and poor access to varied foodstuffs.
In March 2012, only 40% of females could name at least three positive maternal
health actions; at the end of the project that figure had risen to 90%. The
percentage of women reporting to attend 4+ antenatal clinics rose from 60% to
90%; the percentage of women reporting to deliver with a skilled birth attendant
rose from 39% to 82%. Data from local clinics indicates a rise in access to maternal
health services, although the increase is less pronounced. However the survey
results represent an impressive impact on attitudes towards skilled maternal care.
In three focus groups with community leaders and men, participants expressed a high
level of respect for LMs and LGs, recognising them as carrying responsibility for health
teaching for the village, and as leaders of women in the community. Community
discussion groups are seen by the community as positive, but on their own are not
frequent enough to have a significant long-term impact. HPs and HSs identified greater
inclusion of men as the principal area for improvement in the project.
‘Improved continuum of care’ is the output for which the measurable outcomes of SP’s
involvement are least definable. The project has supported building the capacity of VHTs
through training, and it has facilitated dialogue between LMs and the community.
Sustainability
There is strong local ownership and women see care groups as their own; while formal
care group meetings are unlikely to continue in most locations without external support,
www.jigsawconsult.com
Samaritan’s Purse Karamoja MCH evaluation
Page 5
the structure of Karamojong societies means that leader mothers will continue to share
what they have learnt and exemplify good practice to their neighbours. Teaching
through storytelling, drama and songs has also ensured good retention of messages. The
care groups would therefore benefit from strengthened links to health clinics and local
government structures to support long term sustainability.
Recommendations
1. The curriculum materials would be improved by being less bulky and by making use of clearer diagrams throughout.
2. There are several recommended changes to the curriculum including: teaching
women to spot the early signs of labour; nuanced messaging for very rural
communities (where many women are delivering on the road on the way to the
clinic); revisiting nutrition training to be more relevant to the needs and limitations
that women perceive.
3. Strengthen relationships with health units, for example by facilitating midwives
outreach visits.
4. Strengthen links to local government.
5. Increase male involvement through Leader Fathers or further training on nutrition and hygiene.
6. Strengthen monitoring and evaluation processes.
6. Introduction
6.1. Purpose of evaluation
This document provides an evaluation of the Samaritan’s Purse (SP) Karamoja
Integrated Maternal and Child Health (KIMCH) project in north eastern Uganda. The
three-year maternal and child health (MCH) project was funded by the Department for
International Development (DFID) through the Global Poverty Action Fund (GPAF). It
was implemented in the Napak district of the Karamoja sub-region in north east Uganda
between 1 April 2012 and 31 March 2015.
The purpose of the evaluation was to assess the extent to which the KIMCH project
achieved its planned objectives and to identify lessons for future projects. The evaluation
had two objectives:
1. To independently verify (and supplement where necessary), the grantees’ record of
achievement as reported through Annual Reports and defined in the project’s log
frame.
2. To assess the extent to which the project was good value for money, which includes
considering:
a. how well the project met its objectives in relation to effectiveness, economy, efficiency and equity
b. what has happened because of DFID funding that wouldn’t have otherwise happened
c. how well the project aligns with DFID’s goals of supporting the delivery of the
MDGs.
6.2. Organisation context
SP is a leading Christian international relief organisation providing emergency aid to
victims of war, poverty, natural disasters, disease and famine around the world. SP has
www.jigsawconsult.com
Samaritan’s Purse Karamoja MCH evaluation
Page 6
had a presence in Uganda since the mid-1990s and has been delivering health projects
in Uganda since 2003. It began working in Karamoja in 2009, providing World Food
Programme assistance in the form of productive assets for communities and critical relief
to households experiencing food insecurity. This existing presence provided SP Uganda
with strong local knowledge and relationships that supported the KIMCH project
implementation.
6.3. Overview of GPAF-funded activities
The SP KIMCH project was implemented in the Napak district of the Karamoja sub-region
of Uganda. The project aimed to empower women with the knowledge, skills and
attitudes needed to improve their health and the health of their children. The KIMCH
project was implemented through the care group model, selected as a low-cost, effective
and sustainable model of promoting healthy behaviours and improving access to
services. The project included the following key project activities:
• Training health promoters (HPs) in participatory teaching methods, MCH knowledge and collection of data on births, deaths and illnesses.
• Bi-weekly meetings of 219 care groups where women receive lessons on nutrition,
prevention and treatment of common childhood illnesses, use of antenatal services,
skilled birth attendance, and hygiene and sanitation.
• Reaching out to 35,597 neighbour women (NW) by leader mothers.
• Training of community leaders and village health teams (VHTs) in knowledge of
Integrated Management of New-born and Childhood Illnesses (IMNCI).
• Community discussion groups and regular radio messages on MCH issues.
• Facilitating dialogue between health providers, care groups, VHTs and service users
on issues related to MCH.
Outcome: Improved MCH outcomes and increased utilisation of IMNCI actions in
Napak District of southern Karamoja region, Uganda.
Output 1: Care groups established and their capacity built to promote and demonstrate positive MCH behaviours.
Output 2: Pregnant women and caregivers of children under five aware of, and demonstrating, healthy behaviours.
Output 3: Community members, especially males, are supportive of initiatives to
improve MCH and provide enabling environments for their households to adopt positive behaviours and access services.
Output 4: A continuum of care is available for MCH services.
7. Evaluation methodology
7.1. Evaluation plan
This evaluation has followed the guidelines developed by Coffey International for GPAF
final evaluations. It was conducted by Dr Lydia Tanner, a monitoring and evaluation
specialist with a background in medical research, and Ms Louise Randall, a midwife and
sector specialist.
The evaluation is built on a contribution-based approach that attempts to provide a
www.jigsawconsult.com
Samaritan’s Purse Karamoja MCH evaluation
Page 7
‘plausible’ account of the difference that DFID’s funding has made to the effectiveness
and performance of grantees. It adopts a mixed-method approach that combines
qualitative data gathered in interviews and focus groups with quantitative data obtained
through surveys and government statistics. Qualitative data collection was used to
obtain information on the causes and drivers of change; quantitative data collection was
used to obtain information on the scale and type of results. Triangulation of data
collected through the two approaches enabled robust and reliable conclusions to be
drawn.
The developed methodology is based on internationally accepted guidance on
methodological structure from the Organisation for Economic Co-operation and
effectiveness, efficiency, impact and sustainability. It includes the following elements:
1. Document review
2. Focus groups with LMs, NW and men involved in the project
3. Interviews with SP programme and project staff
4. Interviews with other stakeholders
5. Observations of care group meetings
6. Quantitative analysis of survey and monitoring data
7.2. Qualitative data collection
The evaluation began with a background review of all documents relating to the KIMCH
project and a review of the literature on the care group model. The methodological
approach was shaped by findings of the desk review and the evaluators’ expertise in
delivering evaluations of health programmes. The documents reviewed are listed in the
Annexes.
A broad cross-section of stakeholders were interviewed, ensuring a wide range of
perspectives and giving voice to those involved in each element of the project.
Stakeholder interviews were conducted with:
one VHT
health staff in three clinics and one hospital
one district health officer (DHO)
one district community development officer
two sub-county chiefs
three HPs and three health supervisors (HSs)
representatives from Concern Worldwide and International Rescue Committee
(IRC)
six SP staff
The district includes seven sub-counties which have been divided into five supervision
areas; 185 village communities are included in the project. The evaluation team elected
to visit a sample of approximately 5% of participating communities from the five
supervision areas. A stratified random sampling scheme was used to select eight
communities. Communities were stratified by area, and communities were selected to
include a spread of supervision areas and population sizes. One additional community
was selected by SP to fit travel constraints. Communities already involved in the process
evaluation and internal impact evaluation (both in 2014) were excluded.
www.jigsawconsult.com
Samaritan’s Purse Karamoja MCH evaluation
Page 8
Location Supervision area Population
Lotutuwa 1 206
Naligoi 1 1,915
Natapar Apalemu 2 800
Matany Trading Centre West 3 1,262
Kanaura 3 211
Nakiceleet 4 643
Lojojora 4 1,642
Nakayot 5 2,911
Iriiri Trading Centre 5 1,733
Table 1: Location of focus groups
In each location the evaluation team conducted in-depth focus group discussions with
LMs, NW, community leaders, and other men from the community. In total the
evaluation team interacted with 161 women and 28 men in the communities visited.
Focus groups and observations were conducted with:
• focus group with nine groups of LM/GMs
• focus group with five groups of NW
• focus groups with three groups of community leaders and men
• observation of three care group meetings
The evaluation process also included a small survey of 35 mothers of children under two.
The survey was undertaken to enable the team to understand and verify findings of the
Lot Quality Assurance Sampling (LQAS) survey and to collect information on individual
behaviour, particularly in relation to pregnancy and birth care.
7.3. Interview and focus group methodology
The interviews were based on a semi-structured approach adopted to facilitate a detailed
and nuanced exploration of the issues pertinent in each context. The majority of
interactions in the communities visited were in the form of focus groups; the group
dynamic provided the opportunity for interaction between participants as well as directly
with the interviewee. The interviews lasted 30-60 minutes; the focus groups lasted 50-
60 minutes.
The interviews and focus groups used a semi-structured approach allowing the
interviewers to adapt to the responses and explore deeply the areas of most relevance
and interest. The interviews employed techniques such as Outcome Mapping and Most
Significant Change alongside semi-structured discussions in order to elicit information.
The objective of the interview templates was to go into as much depth as possible within
the time available. This means that the process depended on the interviewer asking
follow-up exploratory questions throughout: enquiring ‘why’ and ‘how’ in response to the
initial answers given by the respondents, so that the interaction moved from description
into analysis.
The evaluation team used a structured, evidence-based approach to analysing
qualitative data: the evaluators transcribed the interviews and focus group discussions
and then used a coding practice to link the specified output and outcome indicators.
Qualitative information was analysed using a three-step approach: first, responses from
the focus groups and other stakeholder interviews were summarised according to
emerging categories; next, the repeating themes were identified in statements from the
www.jigsawconsult.com
Samaritan’s Purse Karamoja MCH evaluation
Page 9
document review, observation data and focus group data; finally, impact statements
were drawn up. This approach engaged with the substance and weighting of qualitative
interviewee responses rather than relying solely on anecdotal feedback, thereby
imposing a structure on the analysis that ensures the data analysis is representative.
7.4. Quantitative data collection and analysis
The evaluation methodology relies on quantitative data analysis to provide insight into
the scale of results. The methodology relied on quantitative data collected by SP during
the project, including:
1. annual LQAS survey. A baseline and three subsequent annual surveys were
collected.
2. monthly monitoring data. Paper-based and Frontline SMS data collected from HPs
to monitor: care group attendance, the number of women met by each LM, births, child deaths, and maternal deaths.
3. barrier analysis. Four surveys were conducted at the outset of teaching modules
to inform care group curriculum design.
The baseline and end-of-project LQAS survey data was the primary evaluation data,
used to ascertain changes in the general level of health knowledge and health-seeking
behaviour among caregivers of children 0-23 months of age, as well as the health status
of children 0-23 months of age in Napak District. A description of the LQAS methodology
is provided in the Annexes.
The evaluation team carried out an independent analysis of the data. Quantitative data
from the LQAS survey and monitoring data was coded and entered into Excel
documents. The evaluation team coded equations for calculating statistical measures of
population change, including confidence intervals, p-values and statistical significance of
changing population parameters.
The evaluation also draws on quantitative data collected from the Napak district health
office on the use of health facilities in Napak over the past five years. The evaluation
team triangulated results of the analysis with data from the health clinics and
information obtained in the qualitative study.
7.5. Neutrality
In the course of field work, the values of the evaluation team impact their work as much
as the specific tools developed. Jigsaw Consult seeks to protect the dignity, rights and
welfare of all those involved in evaluation field-work and to promote high ethical
standards of research. To this end, the following principles informed this evaluation field
work: respect; do no harm; robust sample selection, including of the most marginalised;
rigorous recording of the data collected.
Throughout the evaluation, basic elements of good practice were maintained, including
remaining objective, offering empathy without advice, and practising active listening.
The researchers complied with SP child protection procedures at all times. The
researchers followed the advice of the field staff regarding the best way to obtain
consent for the interviews, and the appropriateness or otherwise of photography. Verbal
consent was obtained from the women and men interviewed. In the final report, real
names of beneficiaries have not been used.
7.6. Challenges encountered
Five challenges were encountered in the course of the evaluation. Firstly, time
www.jigsawconsult.com
Samaritan’s Purse Karamoja MCH evaluation
Page 10
limitations and long travel distances between villages in Napak meant it was only
possible to visit nine communities. This allowed the evaluation team to meet 5% of the
LMs engaged in the project but only a small proportion of NW.
Secondly, all focus groups required translation. Six translators were used (for different
regions), all of whom were independent of the project. In some instances it was
necessary to use male translators. It sometimes took longer for women to feel
comfortable expressing themselves through male translators.
Thirdly, it was more difficult to elicit detailed responses from men in the community on
attitude changes, particularly regarding relationships between men and women,
domestic violence and family planning. The evaluation team worked with male
translators in these scenarios and triangulated answers given with responses from the
HPs working alongside the communities.
Fourthly, due to the timing of the visit it was not possible to observe a community
discussion as planned.
Finally, the evaluation took place at the start of the ‘hunger season’, when food stocks
are running low. The timing of the evaluation is likely to affect responses to questions on
available foodstuffs and nutrition practices for women and children.
7.7. Strengths and weaknesses of selected design and research methods
The evaluation plan was followed and the team were able to meet the evaluation aims
and to conduct the majority of planned activities.
The qualitative data collected in this evaluation is based on a participatory
methodology that prioritised the voices of LMs, NW and their communities. The
methodology included a number of participatory activities that encouraged women to
rank their experiences and identify the most important parts of the project.
The evaluation could not assess the care group model against the counter-factual of the
absence of a care group. Nevertheless, interviews and focus groups allowed the
evaluation team to identify recurring narratives in the communities visited, such as the
importance of peer-motivation and peer-accountability as motivators.
There are five international NGOs and several community based organisations working in
the health sector in Napak district. A limitation of this study is the difficulty in attributing
causality for observed changes to a single factor or project. A strength of the project is
that it has promoted stronger links between communities, VHTs and health clinics: a
strength that makes attribution particularly challenging. Instead, this evaluation
highlights contributions of the project and attempts to trace the impact of those
contributions and to describe plausible attribution.
A third weakness of this approach is that it is difficult to obtain honest answers about
culturally sensitive subjects such as domestic violence and family planning. Interviews
with HSs, HPs, health clinic staff and sub-county chiefs were used to triangulate and
interpret focus group responses.
There are also limitations arising from reporting bias among the interviewees as they
may not always appreciate the external nature of the evaluation (though this was always
explained at the outset of the interviews), or they may consider positive responses to be
in their self-interest. The evaluation team created a warm and open atmosphere in focus
groups and explained that the exercise was intended to support SP in strengthening its
work.
www.jigsawconsult.com
Samaritan’s Purse Karamoja MCH evaluation
Page 11
The quantitative data used in this evaluation is based on LQAS, a rigorous technique
that allows for analysis of baseline and final datasets as well as regular monitoring of
project progress. It uses well defined indicators that can be easily measured throughout
the project lifetime. The data was collected by HSs and collated by the project M&E
coordinator: the evaluation team discussed data collection with those responsible and
did not identify any problems with the quality of the data other than those discussed
here.
A significant limitation of the quantitative methodology is the lack of control group: to
definitively attribute impact, a counterfactual that establishes what would have
happened in the absence of the intervention is required. The quantitative method design
did not include a control group, so the analysis relies on qualitative analysis to
understand the factors contributing to behaviour change.
Secondly, poor translation of the LQAS survey in the first year may have led to
misunderstanding of some questions. The evaluation team has looked at data from all
four years in order to identify any unusual patterns.
Thirdly, the LQAS survey uses a low sample size relative to the population being studied.
The methodology assumes that the sample is a random selection representative of the
population. The small sample reduces the size of the ‘confidence interval’ and means
larger statistical differences are necessary to demonstrate changes in attitude and
behaviour in the underlying population. The small sample size also means that it is not
possible to have evidence that targets have been achieved, instead, the method allows
for stating that there is ‘no evidence the target has not been achieved’.
Reporting bias is also a factor in the LQAS surveys: the proportion of women reporting
attendance at the health clinic, for example, is expected to be higher than the proportion
of women actually attending. Changes in LQAS responses over the four years will reflect
both behaviour change and knowledge transfer (i.e. more women now know the ‘right’
answer).
8. Overall results
8.1. Context
The Karamoja sub-region lies in north east Uganda, bordering Kenya and South Sudan.
Health indicators in Karamoja have historically been very poor: infant and maternal
mortality rates are high and there is chronic under-nutrition. The progress made as a
result of the KIMCH project should be viewed in relation to the challenging context.
Factors that contribute to poor health include the following:
• Food insecurity: Karamoja has suffered frequent droughts and food shortages;
many families rely on food distributions. Endemic foot-and-mouth disease in cattle
increases food insecurity. In 2014, the food security situation worsened and an
estimated 59 per cent of the population were experiencing stressed food security in early 2015.
• High population migration: In recent years, as insecurity has decreased, many
people have moved out of trading centres and large villages into smaller
settlements. Communities regularly resettle in response to food shortages. Many people live more than a 2 hour walk from the nearest health facility.
• Limited access to health centres: Poor infrastructure makes it difficult for many
women to reach health facilities. This is particularly challenging in the rainy season,
when bikes and motorbikes are unable to use dirt roads and tracks, and ambulances
www.jigsawconsult.com
Samaritan’s Purse Karamoja MCH evaluation
Page 12
struggle to reach emergency situations.
• Limited village health care: VHTs, who deliver basic health services at the village level, are small and receive limited training and supervision.
• Illiteracy: Low literacy among women and knowledge gaps result in a high
prevalence of traditional practices. For example, some women believe that a stranger’s presence during delivery can harm the baby.
• Aid dependency: Decades of aid have created dependency, with the result that
women and communities are not empowered to look for their own solutions to
health, nutrition and other needs.
There are five organisations engaged in health programmes in Napak district. A
UNICEF/CUAAM programme provides district level technical support and facilitates
nutrition programmes, capacity building of health workers, and support for the VHT
structure. The World Food programme provides nutrition support to pregnant women
and lactating mothers through the health clinics. Concern Worldwide supports capacity
building in health clinics; and IRC has provided training to VHTs alongside the MoH and
supplies of medicines to treat coughs, fever and diarrhoea.
8.2. Summary of findings
The evaluation team assessed the project against the results reported in the annual
reviews. It found that the reports provided a fair summary of strengths, challenges and
impacts.
Beneficiary groups Beneficiaries reached
Health promoters 25 health promoters were trained in participatory teaching
methods and MCH knowledge.
Leader mothers;
leader grandmothers
2,130 LMs were registered in care groups and met with the HP
each fortnight.
Neighbour women 35,597 NW were registered and met with a LM each fortnight.
Children < 5 28,3181 were supported through increasing MCH knowledge of
their caregivers.
Community leaders 900 community leaders attended MCH training.
VHT members 610 VHTs received training.
Community members 24,092 men and women attended community discussion
meetings.
Table 2: Project beneficiaries
1 The number of children under 5 was estimated using the Uganda Bureau of Statistics (UBOS) 2012 population demographic which found that 19.5% of the population is under 4. The population of Napak is taken from the 2014 census report to be 145.219.
www.jigsawconsult.com
Samaritan’s Purse Karamoja MCH evaluation
Page 13
The table above illustrates the beneficiary groups targeted and reached by the project.
Evidence from interviews and focus groups indicates that the project reached all of the
intended groups. All beneficiary groups interviewed were positive about the project and
felt it was relevant in addressing perceived health needs.
The project created a community-based health promotion system that engages an
extensive network of volunteer LM/LGs. This network has supported effective teaching
on IMNCI and has made steps in strengthening the link between communities and the
health system. The project has delivered its four outputs and there is evidence of
knowledge transfer and behaviour change, particularly among Leader Mothers and in the
area of hygiene and sanitation. There was some variation in effectiveness according to
location.
The evaluation team reanalysed all project data to verify log-frame values. It obtained
the same figures in the analysis of the baseline data but found differences in indicator
values (of up to 10%) in the end-of-year data for 2013, 14 and 15. The values used in
this report are those obtained by the evaluation team.
Overall, progress against quantitatively defined indicators has been impressive given the
size of the project area, the challenges presented by poor literacy and low health
knowledge, and the challenges of aid-dependency. 12 of the 16 output targets (for
2015) and 8 of the 9 outcome targets were achieved by the time of the final evaluation.
The majority of targets achieved were statistically significant (p<0.05).
9. Relevance
9.1. Supporting achievement of the Millennium Development Goals (MDGs)
The project was designed to contribute to MDGs 4 and 5 in Uganda: to reduce child
mortality and to improve maternal health. It also supported MDGs 3 and 6, promoting
gender equality among communities in Karamoja and helping to combat disease, in
particular malaria and other common diseases in children under five.
In 2010, progress toward reducing maternal and child mortality was accelerating in
many countries, yet it was found that most developing countries would take many years
beyond 2015 to achieve the targets for MDGs 4 and 5 (Lozano et al. 2011). A report by
DFID found that Uganda was behind its targets for reducing child mortality and seriously
behind in improving maternal health (DFID 2010).
Karamoja, a sub-region in north-east Uganda, performs particularly poorly on
development indicators. Cultural, political and economic factors have contributed to
elevated maternal mortality ratio (MMR) and under-five mortality rate (U5MR). At the
time of the most recent Demographic and Health Survey, in 2011, the U5MR stood at
90/1,000 live births and the MMR at 310/100,000 live births. The principal causes of
child death were: malaria (22%), neonatal complications (21%), diarrhoeal disease
(16%), pneumonia (14%) and other infections (14%).
The KIMCH project was a response to the Government of Uganda’s prioritisation of MCH.
The ‘Roadmap for Accelerating the Reduction of Maternal and Neonatal Mortality and
Morbidity in Uganda 2007’ focused on targets for achieving MDGs 4 and 5. It identified
key challenges, including the fact that women frequently lack adequate information and
knowledge concerning birth preparedness and emergency readiness. In particular, there
is limited understanding of danger signs during pregnancy, delivery, and immediately
after birth (Ministry of Health, Uganda 2007).
www.jigsawconsult.com
Samaritan’s Purse Karamoja MCH evaluation
Page 14
9.2. Rationale for care group model
The care group model was selected as a mechanism for empowering women and
increasing knowledge of MCH. The care group model has been used by at least 27 non-
governmental organisations (NGOs) across a total of 23 countries that trained more than
106,000 peer educators reaching an estimated 1.275 million households (CORE group
and USAID 2014). An expert review of the care group model found that globally care
group projects have, on average, doubled the reduction in U5MR in comparison to non-
care group projects (CORE group and USAID 2014).
A care group programme in Mozambique, for example, achieved high coverage for bed
net use (80%), oral rehydration therapy for children with diarrhoea (94%) and prompt
care-seeking for children with danger signs (Edward et al. 2007). Evidence from a
community-based surveillance system indicated a 66% reduction in infant mortality and
a 62% reduction in under-five mortality. A second study in Mozambique found the care
group model had reduced under-nutrition in children aged 0-23 months by between 8.1
and 11.5 percentage points (Davis et al. 2013).
Several factors made the care group model particularly relevant in Karamoja:
Karamojong women lacked knowledge on MCH issues. In a baseline study, only 14%
of women and 12% of men were able to name four preventative actions for common
childhood illness. The care group approach has been acknowledged to build
stakeholders’ capacities to contribute actively to their health and nutrition
development; it is noted as the best strategy for large-scale information sharing and skills transfer (Catholic Relief Services 2008).
The majority of families live in a ‘manyatta’, a small collection of households. The
care group model of identifying LMs to train their peers allows for identification of a
woman from each manyatta who can be an example to the NW living immediately around her.
The Government of Uganda report highlighted that male household heads are
frequently the decision makers; this can prevent women from taking positive actions
about their health (Ministry of Health, Uganda 2007). The care group model seeks to
create female knowledge sharing and to promote female leadership.
The roadmap highlighted seven strategies, including ‘Empower communities to
ensure a continuum of care between the household and the health care facility’. The
care group model facilitates improved interaction between communities and health
workers through empowerment of LM/LG support for the village health team (VHT),
and facilitating dialogue between health workers and leader women.
The KIMCH project provided teaching in four modules deemed important for promoting
positive behaviour and reducing maternal and child mortality: nutrition, common
childhood illnesses, antenatal and delivery care, and hygiene and sanitation. The
evaluation team found that the choice of teaching modules directly addressed underlying
causes of poor MCH in Karamoja.
1. Nutrition: Under-nutrition is the underlying cause of more than one-third (3.5
million) of all deaths in children under the age of five years. A Lancet review
concluded that breastfeeding counselling, appropriate complementary feeding, and
vitamin A and zinc provision have the greatest potential for reducing child deaths and
the disease burden globally. An extensive randomised trial reported that intensive
nutrition education significantly improves the status of moderately malnourished
children, with or without supplementary feeding (Roy et al. 2015).
2. Hygiene and sanitation: Improved hygiene and sanitation practices are important
in reducing common childhood illnesses and child deaths. Clean water is seen as
www.jigsawconsult.com
Samaritan’s Purse Karamoja MCH evaluation
Page 15
critical in fighting diarrhoeal diseases: education on hand washing and point-of-use
water treatment have both been found to reduce diarrhoea (Zwane and Kremer
2007).
3. Common childhood illnesses: Globally, diarrhoea and pneumonia are the leading
infectious causes of childhood morbidity and mortality. A Lancet review paper
observed that a high proportion of deaths occur during the first two years of life in
both diseases: 72% for diarrhoea and 81% for pneumonia (Walker et al. 2013). It
highlighted the potential role of shared risk factors, including under-nutrition, sub-
optimum breastfeeding, and zinc deficiency. The paper concludes that MDG 4 cannot
be achieved without increased focus on preventing and treating the two diseases in
low- and middle-income countries. Scaling-up of existing cost-effective interventions
could prevent 95% of diarrhoea deaths and 67% of pneumonia deaths in children
under five years by 2025.
4. Antenatal and delivery care: Increasing skilled birth attendance is important for
reducing maternal and neonatal mortality. Factors affecting health facility usage for
childbirth include: distance to the health facility, quality of care, access to antenatal
care and use of health care facilities in previous pregnancies (Gabrysch and Campbell
2009; Renfrew, Homer, and Downe 2014).
9.3. Beneficiary target groups
A community mapping exercise at the outset enabled the project to identify communities
within the district, arrange women into groups of approximately 21 on the basis of
location, and to facilitate the women in electing a leader mother.
SP elected to work with all women of reproductive age (defined as 15 to 49). The
evaluation team noted the value of this inclusive model: in the Karamojong context,
extended family members often hold responsibility for supporting girls in adolescence
and will share responsibility in caring for infants and young children. Grandmothers hold
similarly important roles: in the first year of the project, SP recognised that
grandmothers were playing a role in facilitating or preventing behaviour change at the
community level, particularly on issues relating to antenatal and delivery care. In
response, it piloted ‘Leader Grandmother’ care groups to share knowledge and skills with
grandmothers.
A notable strength of the project is the extent to which it identified and addressed the
specific needs, concerns and cultural barriers affecting improved MCH behaviour. At the
outset of each teaching module, a research technique called Barrier Analysis was
employed to gather information on perceived physical, physiological and social barriers
to behaviour change. The analysis was used to inform the curriculum, messaging and
imagery used in teaching resources: pictures and photographs show Karamojong women
demonstrating healthy behaviours; stories were used to tackle difficult cultural
messages; games and activities were designed to help women remember what they had
learnt.
Early in the programme implementation, SP recognised the importance of increasing
gender inclusion to prevent household-level conflict and improve support for mothers.
Women needed the permission of their husbands to be involved in care group meetings,
and many of the men expressed a desire to be more involved. Community mobilisation
meetings were implemented, but SP has recognised the possibilities for greater male
involvement in future projects and hopes to pilot male ‘Leader Father’ groups in future.
The evaluation team also note that while the project is inclusive it hasn’t consciously
identified women with physical disability or mental illness in its programming. This was
www.jigsawconsult.com
Samaritan’s Purse Karamoja MCH evaluation
Page 16
identified as a possible area for future work by SP staff (ID 7).
10. Effectiveness The evaluation team found that project tasks have been delivered effectively, most often
on time and within budget. The team has good capacity, and its members understand
their individual roles and know how their roles fit within the broader project.
10.1. Project drivers
The evaluation team identified four drivers to project delivery. Firstly, the selection of
local HPs and election of women as LMs has embedded the programme within the
community and fostered the idea of female leadership. HPs and HSs spend much of their
time in the community and engage in a way that has engendered respect and self-
reliance.
Secondly, giving the project a local name and developing songs, sayings and radio
messages promoted a sense of local ownership (see Section 15.2). In this way, the care
group model provided an effective way of engaging women in their own health problems,
developing peer-to-peer education, and ensuring widespread coverage of a community.
Thirdly, 2014 saw a hepatitis E outbreak in Karamoja. Hepatitis E disproportionately
affects women in the third trimester of pregnancy and the foetus (World Health
Organisation 2014). The hepatitis outbreak was a crisis for the Karamojong population.
However, SP was able to work through LMs to demonstrate the benefit and importance
of good hygiene practices and to support the government in its efforts to distribute
information. The disease has not yet been eradicated from Karamoja, and hygiene
practices remain poor in some areas. Nevertheless, women expressed a sense of
empowerment in being able to take practical action within their households and
manyattas in response to the disease.
Finally, the lack of material distribution has been both a strength and a barrier for
project delivery. The expectation of material benefit following involvement in NGO
projects has been a consistent challenge and has frequently affected how communities
perceive the project. LMs have received a number of incentives, including a t-shirt and a
bag, but regularly request greater compensation for their work. This was the sole
frustration raised by HPs when they were asked about their challenges (ID 10, 12, 14).
HSs also expressed frustration that the expectation of material benefit caused friction in
relationships with communities (ID 13). Nevertheless, the lack of material benefit has
introduced the idea of volunteerism, helped women take greater responsibility for their
own health, and minimised potential friction between LMs and their neighbours.
10.2. Project barriers
Four additional barriers to project delivery have been identified, several of which
demonstrate SP’s capacity for risk management. The project staff at all levels
demonstrated a good understanding of risks to the project and how risks can be
mitigated. Staff at the field level are empowered to take ownership for delivery of the
project and report that they feel supported by the management. There are strong
financial procedures in place; non-financial risks are understood but are monitored
informally (see Table 3).
Firstly, during the first year of the project there was a small outbreak of Ebola in the
region. The field team monitored the outbreak and provided weekly updates. SP
maintained regular communication with the grant manager at Triple Line in order to
www.jigsawconsult.com
Samaritan’s Purse Karamoja MCH evaluation
Page 17
anticipate and plan for the epidemic if it spread (ID 3).
Secondly, distribution of insecticide-treated nets was halted after an increase in theft of
nets from households. This imposed a delay on purchasing, which meant that the cost of
nets increased and SP was only able to purchase 16,000: not enough to supply the
whole population. When net distribution resumed, SP altered its approach to staggered
distribution: LMs were asked to identify the most vulnerable women, and nets were not
given as a ‘reward’ for participation in the programme (ID 3). Staff noted that the risk
could have been anticipated earlier, but once the most vulnerable women were identified
the response was effective.
Thirdly, there were cultural barriers to learning. The barrier analysis carried out during
curriculum design for each module was therefore fundamentally important in identifying
the cultural constraints and deep-rooted beliefs relating to MCH. For example, barrier
analysis identified that women who attend the clinic to give birth are more likely to
believe that their mothers, grandmothers and husband would approve of them attending
the health facility. The results of barrier analysis are described in Section 12.
Fourthly, population movement over the past three years has resulted in women moving
into the district without being included in the care group structure. This hasn’t impacted
on the delivery of the project but does mean that the level of inclusion across the district
is reduced.
Finally, poor infrastructure makes travel difficult, especially during the rainy season
when HSs struggle to visit more remote parts of the district.
Risk Mitigation
Financial
risk
There are varying approval levels for spending. Finance staff at field level
have a separate line-management structure that reports directly to Kampala.
An audit of systems and procedures was beyond the scope of this
evaluation. However, the project was observed to demonstrate a good level
of financial accountability and transparency.
Security
risk
The Karamoja region has stabilised within the last three years but still
suffers from some insecurity, raiding, and occasional violence. SP Uganda
closely monitored risks by attending working groups to understand the
potential impacts of cattle raiding, an outbreak of foot-and-mouth disease,
and food insecurity following poor harvests.
Political
risk
SP planned to implement the project in sub-counties of Moroto and Napak
districts. The project moved exclusively to Napak district following tensions
with the Moroto district leadership over World Food Programme distributions
that SP was implementing separately. SP recognised the risk of political
interference; it was mitigated through strong relationship building with
Napak district leadership (ID 1).
Household-
level
power
dynamics
SP noted the risk to project staff and LMs through the empowerment of
women within a male-dominated culture. SP noted the impact of material
distribution to LMs, particularly insecticide-treated nets, and sought to
understand local dynamics and vulnerabilities.
www.jigsawconsult.com
Samaritan’s Purse Karamoja MCH evaluation
Page 18
Aid-
dependent
culture
The care group approach is volunteer-based: it therefore requires
considerable local buy-in. A significant risk to the project impact was that
LMs would not perceive knowledge as sufficiently valuable to continue their
involvement; women also reported experiencing pressure from their
husbands when they did not return home with incentives. This was
recognised as a risk from the outset: it was mitigated through consistent and
continuous conversation on the benefit of volunteering and involvement for
improving community health.
Population
movement
The care group model requires a sedentary population. Karamoja has
traditionally been semi-nomadic, but an increasing number of people have
become agriculturists in the past decade.
Table 3: Risk and mitigation
10.3. Learning and evidence
SP has demonstrated willingness to learn and to apply learning. It had four systematic
means of learning:
Monthly monitoring data to assess progress and identify challenges at care group
level.
Module-level barrier analysis to identify cultural barriers to behaviour change.
Annual LQAS surveys to assess behaviour change and to compare progress being made in different parts of the district.
Feedback from Triple Line to the project annual report.
A number of community and focus group discussions have been organised to obtain
feedback from the community; a process evaluation was conducted to identify lessons
for implementation of other care group programmes. The most significant learning arose
from community discussions that highlighted the importance of including grandmothers
within the care group structure. This resulted in a pilot group of LGs and, ultimately, the
addition of 60 new LG care groups.
The process evaluation highlighted issues of family planning and contraception. The
KIMCH project promotes use of family planning and child spacing: through the process
evaluation the team learnt that people are afraid to take hormonal contraceptives
because they believe it will prevent them from conceiving in future. SP developed a radio
message and additional curriculum materials to engage LMs in the subject.
There is also evidence of SP using Triple Line feedback to the annual review to inform
and improve programming. For example, following feedback, SP undertook focus groups
with men to gain feedback on the project and to better understand men’s perspectives
on MCH issues. SP also provided teaching to community leaders on use of insecticide-
treated nets for mothers and children and gave additional teaching to care groups.
Other care group projects have used monitoring data collected from care groups to
inform curriculum design. In particular, following evaluation of six care groups Henry
Perry at Johns Hopkins University found that cause-of-death data could be used to
inform curriculum design. He observed that the design of care group programmes could
be improved by strengthening the use of vital events data. This has not been
implemented in Karamoja; the evaluation team note the difficulty of obtaining accurate
diagnostic information would present challenges.
www.jigsawconsult.com
Samaritan’s Purse Karamoja MCH evaluation
Page 19
10.4. Partnership
SP has a strong working relationship with district and sub-county government. SP
engage with local government through joint supervisions of care groups, supporting links
between care groups and VHTs, and joint training of VHTs. SP staff met with sub-county
chiefs and the district health officer quarterly and provide written reports of progress.
Government officers were positive about SP’s role in supporting government objectives,
training VHTs, and delivering health messages. HSs felt well engaged with local
government. There were isolated reports of a challenging working relationship between
SP and one sub-county chief. The DHO was positive about the care group model and the
impact of the project; however, the evaluation team felt that the project may be seen as
less significant than infrastructure or health system strengthening projects. SP should
spend more time showcasing care groups and other activities to district staff. NGO
partners also commented that the district level relationship would be improved if SP sent
representatives to more of the local government engagement meetings; the evaluation
team recognises this is difficult when staff are very busy delivering the programme.
LMs meet with health clinic staff quarterly; 64 LMs from different villages attend each
meeting. Midwives and nurses were positive about the meetings and about the care
group model. Midwives are undertaking separate outreach activities; SP should consider
working with midwives on outreach activities, which would improve the care groups’
relationships with clinic staff. Midwives were keen to be more involved with the care
group project.
SP has good relationships with other NGOs working in the health sector in Napak. They
have engaged in the following joint activities:
During the hepatitis E outbreak, SP worked with Concern Worldwide to distribute
information on prevention of the disease and tippy taps to promote hand-washing.
Concern Worldwide funded the jerry cans for tippy taps and delivered training to HSs
and HPs on Hepatitis E.
SP worked with CUAAM and IRC to provide training to VHTs. The training was funded
by SP and delivered by experienced trainers. CUAAM and IRC were very positive
about the partnership.
SP has worked closely with Concern Worldwide to prepare for the transition of care
groups into Concern Worldwide’s programme at the end of this project. The two
organisations also worked together to collect census data on the Napak population.
11. Efficiency
This evaluation has used cost-effectiveness as a measure of efficiency. Cost-
effectiveness is defined by DFID as: ‘A measure of how economically resources/inputs
are converted to results’. The evaluation provides an analysis of cost-effectiveness based
on the management of costs and measurement of outcomes.
In general, SP has demonstrated a rigorous approach to management of costs, with
strong procedures for financial accountability and transparency. The project staff at all
levels are aware of the activity budgets. The project managers closely track costs,
prioritise delivering activities within budget, and comply with reporting requirements of
the grant early and consistently.
Section 12 details the impact of the project activities; it illustrates that the majority of
planned outputs have been achieved and demonstrates the role SP has played in
www.jigsawconsult.com
Samaritan’s Purse Karamoja MCH evaluation
Page 20
achieving positive improvements in maternal and child health indicators in Karamoja. In
a budget analysis, the evaluation team found that budget allocation and scale was
appropriate for the range of activities delivered and results achieved.
11.1. Management
SP demonstrated good management of resources during the project implementation,
ensuring project delivery was cost effective. SP Uganda has processes in place for
procurement, performance management and cost management.
1. Procurement: SP uses a formalised procurement system based on set thresholds:
large purchases are determined by a bid committee that obtains a range of quotes
and considers the price and quality of each product as well as its appropriateness for
the target population (ID 1).
2. Performance management: SP commissioned a mid-term evaluation to assess the
delivery of activities and identify areas of learning.
3. Cost management: the project manager monitors budget expenses and is well
supported by management. She demonstrated a detailed understanding of key costs,
the drivers of costs, and a plan for delivering the project activities within budget. In
general, SP Uganda has experience of successfully delivering a variety of large,
complex programmes in different contexts and it demonstrates the management
procedures necessary to support delivery of these projects.
An analysis of the programme budgets and financial reporting revealed that SP was able
to deliver the vast majority of activities on budget and within the agreed time plans. This
was facilitated by predominantly favourable exchange rates. SP submitted minor
revisions to their budget: firstly, in Year 1 increased printing costs were met by marginal
savings across the rest of the budget; secondly, due to exchange rate changes in Year 3,
SP was able to transfer some money from staff salary costs into project activities. SP
used this funding for celebratory care group meetings to ‘solidify and celebrate the
achievement of LMs’.
The greatest challenge to efficient delivery of project activities occurred in year 2 with
the distribution of insecticide treated nets. Purchase of the ITNs was part of SP Uganda’s
25% contribution to the total project costs. An increase in the unit cost of nets meant it
was only possible to purchase 16,000 nets. Each LM was given responsibility for
identifying 7 of her 20 NW to receive nets based on prioritisation of pregnant women,
women with young children, and older women.
A second, minor challenge occurred in the delay to VHT training in year 2. This was
primarily due to the schedule of the partner organisation which meant that training
activities had to be postponed.
The evaluation team noted a number of ways that SP had endeavoured to reduce costs
and strengthen its results, most notably:
• Creating a strong team. SP has invested time in team-building and supporting
staff to understand and believe in the care group model. It has recruited local
staff who live amongst the target communities. As a result, HSs and HPs are
committed to their work and are invested in their communities. For example,
they regularly travel long distances on foot or bike in the rainy season to meet
with care groups.
• A focus on building relationships. HSs and HPs facilitate meetings between LMs
and the communities, health clinics, and local government at very low cost but
www.jigsawconsult.com
Samaritan’s Purse Karamoja MCH evaluation
Page 21
with valuable long-term benefits.
• Partnering. As illustrated in Section 10.6, costs were reduced by partnering with
other NGOs to leverage additional resources. The care group model facilitates
shared activities and provides a framework that other organisations or local
government can tap into to enhance delivery of health messaging and
interventions.
• Mobilising communities. SP has focussed on building local ownership of the
project and creating a spirit of volunteering. Good recruitment of LMs through an
election process has been important. LMs demonstrated commitment and
enthusiasm for their work.
11.2. Measurement
The KIMCH project focused on building the capacity of local staff to ensure project
activities were embedded in the community, to maximise local ownership and to
promote sustainability. The project involved few capital costs and instead focused on
human resource development for a longer term impact; because it is largely volunteer
based and predicated on a cascade model it allows for a wide reach at low cost.
The unit costs for this project are the cost per LM trained and the cost per NW reached.
Both demonstrate the cost-effectiveness of the KIMCH project. In the project proposal
submitted to DfID, SP quoted a cost of less than £10 per direct beneficiary over the 3
year lifetime of the project (including LMs, NW, children under 5, community leaders,
VHTs and community members attending discussion meetings). The project was
implemented within 219 communities with over 2000 volunteer LMs by a small team of
34 staff, as planned.
The quality of LM training was measured using questionnaires to assess whether LMs
had acquired knowledge from training. In Year 1, 90% of LMs passed short tests on
modules on nutrition; in Year 2 and 3, 79% and 85% respectively of LMs passed short
tests on modules on maternal care and common childhood illnesses.
In an interview, representative from Concern Worldwide who are also implementing care
groups in other districts of Karamoja, noted that SP have a lean staff, able to reach a
larger target population (35,000 for SP; 23,000 for CW) with a similar number of staff
and with minimal HR support, vehicle support and M&E support: she noted SP are
“having a big impact with few people” (ID 26). The evaluation concurs that the project
has been effectively delivered with a small project staff; it also notes that the project
manager works hard, meets significant reporting requirements, and may feel over-
stretch during busy periods.
12. Impact
12.1. Output 1: Care groups established and their capacity built
2,130 LMs were elected and placed into 219 care groups. The total number of registered
care groups, LMs, and NW is provided in Table 4.
The LMs describe their role as ‘teaching the neighbour women about health’, ‘setting an
example by attending the clinic’ and ‘showing women how to make their households
hygienic’. At the care group in Lojojoro village, one LM explained that ‘a leader mother is
a friend, a teacher and a voice’ (ID 36).
When asked what they most enjoyed about their roles as LMs, participants emphasised
www.jigsawconsult.com
Samaritan’s Purse Karamoja MCH evaluation
Page 22
teaching other women in the community, being respected within the community, the
ability to identify sick children, seeing a reduction in child malnutrition, and knowing how
to prevent the spread of communicable diseases. The perceived difficulties were limited
to lack of payment, occasionally poor relationships with the NW, and frustration that the
NW often asked for materials which the LMs could not provide.
Area Sub-county Number of CGs Registered LMs Registered NW
SA1 Lopei/Ngoleriet 61 570 8809
SA2 Lotome 25 250 3801
SA3 Matany 41 411 6652
SA4 Lokopo 37 384 6507
SA5 Iriiri 55 515 9828
Total 219 2,130 35,597
Table 4: Care groups, registered LMs and NW by sub-county (Feb 2015). There have been changes
of up to 5% in NW numbers since the outset due to population migration.
Care group meetings involve teaching a lesson from the module workbook. The HPs
observed were competent in following the lesson plan and using a variety of teaching
methods. This is a notable achievement because the Ugandan education system does
not often use participatory learning methods. Nevertheless, HPs interviewed understood
the value of the different forms of interaction. The Karamojong culture has an oral
tradition and LMs have responded well to oral learning methods; they enjoy the rhymes,
sayings, simplified anecdotes and stories and are quick to recount stories of Karamojong
women from the learning materials:
For example we heard a story where a child was given porridge before six
months and that child died. When the child reaches six months, introduce
other foodstuffs like porridge to that baby. (ID 30)
2012 2013 2014 2015 Total/
Change Significa
nt
# of Health Promoters who complete training and pass tests
0 25 25 19 19 NA
# of CGs established and meeting
on a biweekly basis. 0 204 219 219 219 NA
% of Leader Mothers (LMs) who complete training and pass test in
IMNCI.
0 94 79 85 85% NA
% of LMs outreaching their target households each month.
0 72% 71% 95% 95% NA
# of MCH messages developed in
collaboration with CGs and health
personnel.
0 2 5 0 5 NA
Table 5: Care group meetings and LM activities
Retention and turnout of LMs at each care group meeting is high. Table 5 illustrates care
group meetings, LM and NW attendance each month. In 2013 and 2014, approximately
71% of LMs reported that they met their target NW each month; that figure rose to 95%
in the final year of the programme.
www.jigsawconsult.com
Samaritan’s Purse Karamoja MCH evaluation
Page 23
The Napak DHO noted that population movement affects the success of care group
attendance. He observed that like the rest of the population, LMs frequently move to
‘resettlement areas’ away from trading centres where VHT support is greatest. The
monitoring data tracks the number of LMs present but does not record when women are
replaced. LMs described moving to pastoral areas for planting or to trading centres to
look for work; they ask another NW or their eldest daughter to take on their role for
several weeks or months. However, HPs and HSs say the rate of turnover is very low:
they recall the names of a handful of women who have left temporarily or permanently
in the past three years. In one area, for example, the HS reports that only one LM has
left the programme.
This high level of attendance can be attributed to a growing sense of responsibility and
motivation (ID 13). In observations, women were keen to learn and demonstrated a
sense of unity and commitment to change. The discussions were jovial, with a lot of
laughing and talking. Women responded very positively to learning through games, song
and drama. They were comfortable discussing barriers to change: for example,
explaining to the evaluators that some people believe that if a woman uses a latrine she
will become infertile (ID 28).
There is greater fluctuation in the number of NW reached each month. LMs report that
their neighbours are often busy collecting firewood, making food or cultivating their land
(ID 30). At harvest, for example, most LMs will attend the care group but they struggle
to meet with their neighbours (ID 8). HPs encourage the LMs to arrange meetings in the
evening when the women have all finished their daily work (ID 9). Gathering NW
together is particularly difficult in dispersed villages and during the rainy season, when
population movements are highest (ID 39). An LM in Iriiri told the evaluation team: ‘In
the beginning we met with many neighbour women but now it has reduced: the women
go to distant places to look for jobs and food’ (ID 45).
While HPs follow the lesson plans closely and use participatory teaching methods, they
do not train the LMs to do the same. The level of formality in the training of NW varies:
some LMs gather small groups of five or six NW and teach with the flipcharts, using
lesson pictures to recount messages and stories from memory; others simply go home-
to-home telling their NW what they have learnt.
HSs, HPs and LMs highlight the value of ‘women teaching women’ as something new that
has given women the opportunity to lead within their communities. One of the sub-
county chiefs observed that ‘the voice of women has been dignified and uplifted because
the programme is focusing on women whose voices have historically been ignored’ (ID
17).
The LMs demonstrate confidence in discussing MCH issues; while NW are able to discuss
what they have learnt, there is a notable difference in confidence levels between LMs
and NW in focus groups. There have been isolated incidences of friction between LMs
and their neighbours, and several LMs described being ‘chased’ by NW who expected
them to provide material goods (ID 45). Nevertheless, in general, LMs say they are
welcomed and well regarded, and that their standing in the community has increased.
There are numerous examples of LMs taking responsibility for health in their
communities. In Kanaura village, for example, the care group is saving communally in
order to provide loans to members of the community who need transport to clinics but
cannot afford it (ID 39). Others will walk pregnant women to the clinic so that they can
receive antenatal care or deliver. Health clinic staff emphasised the role LMs play in
setting an example to their neighbours in positive health-seeking actions (ID 23). The
DHO noted the difference between LMs and the rest of the community in ‘knowledge,
attitude and practices at household level’ (ID 15). He observed that LMs were influential
www.jigsawconsult.com
Samaritan’s Purse Karamoja MCH evaluation
Page 24
in the community and able to communicate their knowledge, and showed positive
health-seeking behaviour: ‘For example, in those households or villages where there is a
LM, it is unlikely that people will deliver in the home’ (ID 15). The LMs agreed, for
example:
We must be an example and a role model so that the neighbours will learn
that what I am saying is true. We shouldn’t just talk to the mothers about
having a latrine if we don’t first make one in our home. We must be an example (ID 28).
12.2. Output 2: Increased knowledge and capacity to exhibit healthy MCH behaviours
The care groups have followed curriculum on four modules: nutrition, common disease
prevention and treatment, pregnancy and birth care, and hygiene and sanitation. In oral
questionnaire-based exams, LMs have achieved a high pass rate (90% Year 1, 74% Year
2, 85% Year 3), indicating good knowledge sharing.
During focus groups, women were asked to select one to three flashcards that
represented the most important things they had learnt. The results are illustrated in
Figure 1; they illustrate that women considered a broad range of knowledge as
important with the greatest priority on hygiene messaging.
Figure 1. Focus group participants’ responses to the question 'What were the most important things that you have learnt and done?'
It is notable that when asked to describe the greatest health challenges in their
communities, LMs focused on the challenges of poverty, hunger, and a lack of
knowledge. They repeatedly described ‘good health care’ as constituting good hygiene,
sharing knowledge and learning.
16
15
10
18
6
3
15
4
6
21
26
www.jigsawconsult.com
Samaritan’s Purse Karamoja MCH evaluation
Page 25
Water, sanitation and hygiene
In focus group discussions, LMs identified lessons on hygiene and sanitation as the most
important aspect of their learning. This is also the area where the evaluation team saw
the most significant direct impact. In focus groups, both LMs and NW were able to
describe good hygiene practices. They also recognised the direct positive link between
hygiene and sanitation practices, disease prevention, and improved perinatal outcomes.
SP staff and LMs both agree that in the past, hygiene in villages in Napak district was
very poor, with little hand-washing and open defecation. LMs frequently commented that
communal areas were previously littered with children’s unburied faeces. The women
repeatedly described changes in hygiene practices as a result of the programme,
including hand-washing, household cleanliness, latrine use, waste disposal, bathing, and
cutting nails (ID 38, 46). For example:
I used to go and defecate and then straight away go and make food or breastfeed my child. But now I go and wash my hands. (LM, ID 32)
We didn’t use to make the children wash their hands. Now we and our
children wash our hands before and after eating… We know that there is
poverty here, but at least even when there is hunger, we know how to keep our children clean. (LM, ID 35)
My children used to be very dirty, and in the morning I would fetch firewood
straight away before cleaning the children and burying the faeces. Now the first thing I do is cleaning the household. (NW, ID 33)
Health clinic staff, HSs, HPs and the VHT also identified hygiene as a significant area of
change, reporting that homes are cleaner, faeces are not left in the open, and women
are bathing themselves and their children and using soap to wash their hands before
cooking (ID 13, 23, 25, 27). Government officers agreed that improved hygiene
practices have been a significant achievement of the project (ID 15, 16, 18, 19).
Changes in hygiene practices were seen as particularly vital during the hepatitis E
outbreak in 2014. Hospital and health clinic staff explained that LMs had been able to
distribute important health information and hygiene advice and to explain the warning
signs of hepatitis E. Hand-washing practices improved with the use of tippy taps during
the outbreak (ID 15). One HP reflected:
When there was the outbreak of hep E it was tragic, but we could tell people
how to respond to these diseases. Only one mother passed away. But in other
places, many people died. So we contributed to getting people informed about the danger and how to prevent the disease. (ID 12)
There have been significant improvements in hygiene practices, particularly among
LM/LGs. NW and men in the community all report that LMs are setting an example in
improved hygiene practice. Changes have been slowest in resettlement areas, where it is
difficult to access clean water (ID 16).
The evaluation recommends that SP seek opportunities to create partnership between
care groups and schools. According to the DHO, the average latrine ratio for girls is
1:140, with a slightly higher ratio for boys. HPs should seek out opportunities to share
hygiene messages in the school environment.
Care of common illnesses
In Uganda, around 75% of deaths in children under five result from pneumonia,
www.jigsawconsult.com
Samaritan’s Purse Karamoja MCH evaluation
Page 26
diarrhoea and malaria. These deaths are preventable through antibiotics, oral
rehydration therapy, good breastfeeding practices and use of insecticide-treated nets. A
WHO briefing identified the greatest determinants of childhood death as income, the
mother’s education level, and poor sanitation (World Health Organisation 2011).
The illnesses that most commonly affect women and children are: malaria, respiratory
tract infection, diarrhoea, malnutrition, and HIV AIDS (ID 15). A barrier analysis was
conducted to understand the knowledge and beliefs which influence a mother’s choice to
take a child with fever for treatment at a health centre. It was found that ‘doers’ were
3.6 times more likely to perceive fever as a ‘very serious’ problem and to have
supportive family members. Both doers and non-doers cited distance and cost as the
most significant determinants of whether they attend the health centre. The most
frequently cited disadvantage was a lack of medicine at the health facility. The analysis
noted that non-doers found it 20% more difficult to remember to take the child for
treatment.
In focus groups, LMs frequently described health problems as a lack of knowledge about
common diseases. Women recognised the role of hygiene and sanitation in disease
prevention, as well as good breastfeeding practice and good nutrition. All LMs and NW
were able to name symptoms of serious illness, preventative actions for common
childhood illness, and positive maternal health action. LMs were quick and confident in
responding to these questions; NW required more prompting but do have the underlying
knowledge.
Table 6 shows the number of caregivers able to name 4 or more actions to prevent
childhood illnesses and 4 or more symptoms of common childhood illnesses. In 2012,
14% of female caregivers and 12% of male caregivers could name preventative actions;
this increased to 59% and 55% respectively in 2015. In 2012, 20% of female caregivers
and 23% of male caregivers could name 4 or more symptoms of childhood illness; this
increased to 78% and 55% respectively in 2015.
2012 2013 2014 2015 Change Significant
% of caregivers of
children 0-23 months who can identify 4+ preventative actions
for common childhood illnesses
13.3%
(8.7–17.9)2
21.9%
(16.3-27.5)
47.1%
(40.0-53.8)
57.1%
(50.4-63.8) 43.8% Yes3
M F M F M F M F M F M F
12% 14% 22% 22% 40% 54% 55% 59% 43% 45% Yes Yes
% of caregivers of
children 0-23 months able to identify 4+ signs of serious illness in a child
21.4%
(15.9-26.9)
39.0%
(32.4-45.6)
59.5%
(52.9-66.2)
66.7%
(60.3-73) 45.2% Yes
M F M F M F M F M F M F
23% 20% 34% 44% 55% 64% 55% 78% 34% 58% Yes Yes
Table 6: Knowledge of preventative actions and symptoms of childhood illnesses
Women report that because of care group meetings, they feel confident in recognising
childhood illness and seeking early and appropriate treatment. Health clinic workers also
say there has been a reduction in use of roots and herbs as medicine (ID 26). All the
2 Throughout this report, values in brackets represent the confidence interval for each indicator 3 The change in an indicator value is said to be statistically significant if the confidence intervals of the two values do not overlap i.e. it represents a change in the underlying population parameter
www.jigsawconsult.com
Samaritan’s Purse Karamoja MCH evaluation
Page 27
women interviewed were using insecticide-treated nets for their children and recognised
these as valuable. LMs perceive that this new knowledge has helped to reduce diseases
such as hepatitis, diarrhoea and malaria (ID 43). For example:
Since this programme came in we have seen some changes. If you were to
come here in those days you would have seen faeces everywhere. Now you
see many families have a pit latrine, and hygiene has changed in the whole
village. Diseases like hepatitis, tetanus and diarrhoea have reduced because they are linked to hygiene. (LM, ID 38)
We continue to courageously keep talking to the neighbour women because
we believe it makes a difference to disease. The neighbour women now
appreciate the effect knowledge has in the community and are more receptive. (ID 39)
We couldn’t previously care for babies with diarrhoea but now we know how
to look for symptoms of sickness like weakness of body, diarrhoea. (ID 39)
The evaluation also demonstrates changes in behavioural indicators in Table 7 and 8.
Most pronounced is the increase in households reporting that their children sleep under
ITNs (30.0% in 2012 rising to 97.1% in 2015). There was a statistically significant
increase in the percentage of caregivers reporting treatment of fever within 24 hours,
from 30% to 50%. There was a non-significant increase in the proportion of children
treated with diarrhoea who received treatment, from 72% to 82%.
2012 2013 2014 2015 Change Significant
% of caregivers with children 0-23
months with at least two ITNs
12.3%
(7.9-16.8)
3.8%
(1.2-6.4)
1.4%
(0-3)
72.9%
(66.9-78.9) 60.6% Yes
% of children 0-23 months who slept under an ITN the previous night
30.9% (25-37.2)
21.4% (15.9-26.9)
31.4% (25.1-37.7)
97.1% (94.8- 99.8)
63.2% Yes
Table 7: Use of insecticide-treated nets
2012 2013 2014 2015 Change Significant
% of children 0-23 months who had an episode of diarrhoea who received ORT
72.2%
(65.7-78.0)
85.6%
(78.6-92.6)
67%
(57.8-76.2)
82.2%
(75.9-88.6) 10% No
M F M F M F M F M F M F
71% 73% 93% 77% 80% 58% 87% 78% 16% 5% Yes No
% of children 0-23 months with a fever who received any anti-malarial treatment within 24 hours
30%
(24.5 -35.4)
44.3% (24.4-54.2)
38.7%
(30.3-47.1)
50%
(39.9 - 60) 20% Yes
M F M F M F M F M F M F
28% 32% 47% 42% 41% 35% 33% 67% 50% 62% 20% 30%
Table 8: Rates of childhood diarrhoea in the two weeks preceding the LQAS survey
Overall, the evaluation findings indicate that education is having a positive impact on
reduction of common childhood and maternal illness in Napak district. Access to and use
of health facilities will be addressed in Section 12.4. The evaluation recommends a
continued focus on HIV education in teaching materials. The HIV rate in Napak district
has doubled in the past ten years, with growing risks including urban migration, alcohol
www.jigsawconsult.com
Samaritan’s Purse Karamoja MCH evaluation
Page 28
abuse and increased interaction with the outside world now that there is greater stability
(ID 15).
Nutrition
Good nutrition is fundamental to health: a study of child mortality found 52.5% of all
deaths in young children were attributable to under-nutrition, varying from 44.8% for
deaths because of measles to 60.7% for deaths because of diarrhoea (Caulfield et al.
2004).
A barrier analysis survey was conducted to understand the knowledge and beliefs which
influence whether a mother is a ‘doer’ or ‘non-doer’ of ideal breastfeeding practices:
practicing exclusive breastfeeding (EBF) to 6 months and appropriate complementary
feeding (CF) thereafter. The study found that doers of EBF were more likely to believe
that a child would become malnourished if not exclusively breastfed (33.5% vs. 22.7%),
that they could practice EBF with their current level of knowledge and skills (55.7% vs.
44%), and that traditional birth attendants (TBAs) and VHTs approve of EBF (12.7% vs.
6.5%).
Poor practice of appropriate complementary feeding (CF) of children age 6-23 months is
common in Karamoja. The barrier analysis found that non-doers of proper CF were less
likely to believe that proper complementary feeding is ‘very’ or ‘somewhat’ important.
They reported finding it difficult to remember to practice proper complementary feeding.
Both groups reported that the father is the person most likely to disapprove of good
breastfeeding practices; 80% of both groups also believed that it is easier to practice
proper EBF and CF when the family has sufficient food available.
The nutrition module addressed breastfeeding practice, appropriate complementary
feeding of children aged 6-24 months, and balanced diets within pregnancy. Teaching on
good breastfeeding practice focused on the importance of early-onset breastfeeding,
frequent feeding, and exclusive breastfeeding to six months. HPs identified teaching on
exclusive breastfeeding as a primary benefit of the programme and commented on the
reduction in use of water and cow’s milk (ID 10, 14). Focus group discussions
demonstrated that the importance of early-onset, frequent and prolonged breastfeeding
relationships is understood and reflected in attitude and practice. Table 9 illustrates
survey data on breastfeeding practice: there is a notable, statistically significant increase
in the percentage of women who report practicing exclusive breastfeeding for children
aged 0-6 months (49.5% to 92.4%).
Incidences of stunting are highest in the first two years of life, especially at 6-24
months. The majority of the population in Napak district survive on cereals which they
cultivate. The harvest in 2014 was poor: the rains were delayed and irregular and there
was just one cultivation season. At the time of the evaluation, vulnerability to under-
nutrition and starvation was high. Communities were beginning to turn to their coping
mechanisms, including urban migration, begging, consuming wild fruits and leaves, and
selling charcoal and firewood.
LMs were taught about complementary feeding as timely introduction of safe nutritional
foods in addition to breastfeeding. LMs were taught that a balanced diet is important for
a healthy pregnancy. Women have understood these messages and are making efforts
to prioritise a varied diet for pregnant women and young children. The survey suggests a
significant improvement in appropriate complementary feeding. The percentage of
caregivers reporting their children received an appropriate diet during the previous 24-
hours rose from 17% in 2012 to 55% in 2015. It is noteworthy that 100% of
respondents said their children were still breastfeeding. There was also a non-significant
decrease in the proportion of children suffering malnutrition (weight-for-age score <-2).
www.jigsawconsult.com
Samaritan’s Purse Karamoja MCH evaluation
Page 29
2012 2013 2014 2015 Change Significant
% of infants 0-5 months who are exclusively breastfed
% of children 0-23 months with weight for age scores <-2
21%
(16.9-24.7)
9.3%
(5.4-13.2)
12.1%
(7.8 - 16.5)
13.5%
(8.9-18.1) 7.5% No
M F M F M F M F M F M F
25% 17% 11% 8% 16% 7% 14% 13% 9% 4% No No
Table 9: Exclusive breastfeeding, complementary feeding practices and weight scores for children 0-24 months
Despite these positive survey results, the challenges of implementing appropriate
complementary feeding was a recurring theme in every focus group. Behavioural change
in nutrition practices has been limited by poverty and the availability of varied
foodstuffs. HSs and HPs commented:
How can we promote healthy eating when they have nothing to eat? Some of
the foods they are not able to get. They don’t know how to plant and grow them. (ID 10)
Now when you look at the foodstuffs we are supposed to teach them it is not
for the village level. It is a story to them. Some of them don’t even know the
types of food that are in the book. They see it as food that is for people who are rich. (ID 29)
Women in the trading centres, who often have smallholder businesses, find it easier to
access eggs, small fish and milk (ID 32). Other women explain that they are sometimes
able to access sunflower seeds, pawpaw, small fish, tomatoes, pumpkins, watermelon,
beans, eggs and milk. In future, focusing exclusively on these foodstuffs may improve
the impact of nutritional messaging.
A Concern Worldwide programme in other parts of Karamoja has introduced practical
demonstrations using oil, milk and some vegetables to provide women with first-hand
experience and is piloting kitchen gardens (ID 19). The evaluation team recommend an
analysis of the two approaches at the end of the Concern Worldwide project.
It should be noted that complementary feeding programmes implemented alongside
educational programmes have shown promising results. A systematic review of 17
nutrition interventions found that provision of appropriate complementary food +/-
nutritional education resulted in an average extra weight gain of 0.25kg (+0.18kg with
4 Complementary feeding is defined as the % of children who are breastfed and who have eaten 2+ food
groups if 6-8 months of age or 3+ food groups if 9-23 months of age during the previous 24 hours and who have eaten at least 2 (6-8 months) or 3 (9-24 months) times in previous 24 hours
www.jigsawconsult.com
Samaritan’s Purse Karamoja MCH evaluation
Page 30
nutritional education) and growth of 0.54cm (+0.38cm with nutritional education) in
children age 6-24 months (Imdad, Yakoob and Bhutta 2011).
Finally, two other factors contribute to poor nutritional status and should be considered
in future interventions. Firstly, alcohol consumption in Napak is high, including among
pregnant women. Alcohol is also perceived as a food source because local beers are
made from sorghum. The evaluation team were unable to establish the prevalence of
foetal alcohol syndrome. The programme has briefly addressed alcohol consumption
during pregnancy, but a thorough barrier analysis of the issue followed by an increased
focus in teaching is recommended.
Secondly, it is normally the male head of the household who determines what food will
be sold and bought in the market. It is recommended that men are included in nutrition
–based education as explained in Section 12.3.
Maternal and neonatal care
In the maternal and neonatal care module, LMs were taught the importance of antenatal
care and delivery at a health unit. Overall attitudes reflect that antenatal care and
birthing in the clinic improves perinatal outcomes and women are, with some important
exceptions, prepared to seek out this care.
Barrier analysis was conducted to understand the knowledge and beliefs which influence
whether a mother decided to give birth in a health centre (‘doer’) or elsewhere (‘non-
doer’). Doers were more likely to believe that their mothers would approve of them
attending the health facility (75% vs 45%), and that their grandmother and husband
would also be supportive. They were also more likely to believe that giving birth in the
clinic with the assistance of a doctor, nurse or midwife would help prevent complications.
By contrast, non-doers were more likely to believe that complications during delivery at
home were unlikely (14% non-doers; 2% doers), and that the presence of a doctor,
nurse, or midwife has little impact on prevention of maternal mortality.
Both groups said that barriers to attending the clinic were the distance to the clinic,
costs, lack of support from their husbands, and a fear to undress in front of health
workers. They also perceived disadvantages to delivering in the clinic as costs, fear of
episiotomy, fear of operations and rudeness by midwives and nurses. This corresponds
to findings of other research in Uganda which has noted a prolonged adherence to
traditional birthing practices and beliefs that pregnancy is a test of endurance and
maternal death a sad but normal event (Kyomuhendo 2003). The research
recommended community education on all aspects of essential obstetric care and
sensitisation of service providers to the situation of rural mothers.
2012 2013 2014 2015 Change Significant
% of caregivers able to identify 3+ positive maternal
health actions
39% (32.4-45.6)
61.9% (55.3-68.5)
80.5% (75.1-85.8)
83.3% (78.3-88.4)
44.3% Yes
M F M F M F M F M F M F
38% 40% 56% 68% 73% 87% 77% 90% 39% 50% Yes Yes
Table 10: Knowledge of positive maternal actions
Table 10 shows the changes in maternal health knowledge during the course of the
project. It demonstrates significant achievements in the information held by the
community: in March 2012, 40% of females and 38% of males could name at least three
positive maternal health actions; in January 2015, that figure had risen to 90% and 77%
respectively. The increase in health knowledge is statistically significant.
www.jigsawconsult.com
Samaritan’s Purse Karamoja MCH evaluation
Page 31
Table 11 illustrates progress on reported positive maternal behaviour. The percentage of
women reporting to attend 4 or more antenatal clinics rose from 60% to 90%; the
percentage of women reporting to deliver with a skilled birth attendant rose from 39% to
82%; and the percentage of women reporting to attend postnatal care rose from 43% to
91%. All increases were statistically significant and represent an impressive impact on
attitudes towards skilled maternal care.
2012 2013 2014 2015 Change Significant
% of pregnant women aged 15-49 years attended 4+ antenatal visits
60% (50.6-69.4)
74.3% (65.9-82.6)
75.2% (66.9-83.5)
90.4% (84.8-96.0)
30.4% Yes
% of births assisted by a skilled attendant
39.0% (29.7-48.4)
58.1% (48.7-67.7)
69.5% (60.7-78.3)
82.9% (75.7-90.1)
43.9% Yes
% of female caregivers of children 0-23 months who received a postnatal check within 24 hours of giving birth
43.8%
(34.3-53.3)
53.4%
(42.9-61.9)
51.4%
(41.9-60.9)
72.4%
(63.8-80.9) 28.6% Yes
Table 11: Rates of antenatal care, delivery with a skilled birth attendant and postnatal care
Antenatal care is increasingly seen as important for a safe and healthy delivery. Clinics
report that antenatal care appointments 1-3 are normally well attended; the fourth
appointment is less well attended. Health clinic staff believe that education and positive
examples of behaviour by LMs are improving attendance for antenatal care (and thus
delivery) (ID 22, 24 26). Other factors influencing women’s choices include HIV testing
and provision of food supplies, iron and insecticide-treated nets when attending
antenatal care. Health clinics report that when World Food Programme food is available
the number of women attending antenatal care clinics rises dramatically; yet even when
there is no food, attendance has improved compared to four years ago (ID 22).
The past five years have also seen an increase in women delivering at the health unit.
Women, particularly LMs, described care group education as a significant part of their
choice to deliver outside the home. LMs also believe that through setting an example
and supporting women in reaching the clinic they are impacting birth choices (ID 29, 30,
43). Women also note the reduction in traditional practices, such as insertion of locally
available fruits to stop bleeding after birth (ID 36).
A universal key indicator of effective attitudinal changes in pregnancy and birth care is
that a woman makes different health choices between pregnancies following health
education. Focus group discussions indicate this has been the case. In a small survey of
35 women, 30 women reported that they had delivered in the health unit. Of these, 15
(50%) said that they had delivered earlier children at home, and 11 (37%) attributed
their change in behaviour to care group education or the influence of LMs in the village.
For example:
The leader mother took me to the hospital; I had a caesarean. It was because
of knowledge from this programme. Otherwise I would have died. I would
have thought, I have delivered 5 here, I can deliver again. But the rest of the LMs took me and said you must go to the hospital (Survey 2).
Since I became a LM I got more knowledge about giving birth and that is why
I decided to go to the health centre (Survey 26).
www.jigsawconsult.com
Samaritan’s Purse Karamoja MCH evaluation
Page 32
Data collected by CUAAM from the district health units also indicate a positive increase in
maternal positive health behaviours5. Table 12 provides illustrative statistics from the
health units showing a notable increase in ANC attendance and deliveries in the health
units between 2010 and 2015; the most significant decrease is in the number of women
delivering with traditional (untrained) birth attendants. The corresponding increase in
maternal deaths and stillbirths shows some correlation with the increased number of
women attending the clinic.
Period ANC 1st
Visit ANC 4th
Visit ANC Total
Delivery in unit
Delivery with TBA
Still births
Maternal deaths
Jul 2010 to Jun 2011 5084 2493 1603 1694 11 1
Jul 2011 to Jun 2012 7145 3426 13659 2433 1805 32 8
Jul 2012 to Jun 2013 5886 3040 16753 2369 728 26 7
Jul 2013 to Jun 2014 5715 2828 16714 3309 313 29 22
Jul 2014 to Jan 2015 (6 months) 2840 1946 8699 2126 81 12 4
Table 12: District health unit data
Health clinic staff, midwives and the DHO recognised education through care groups as
an important factor in increasing attendance at the clinic for antenatal care and delivery
(ID 15, 22, 25, 26). For example, a nurse and representative from CUAAM commented:
We cannot reach the communities, so the LMs play an important role in
reaching out to the community with these messages. (ID 24)
In Napak we have tried to work with the LMs, and it is through their efforts
that it was possible to mobilise the women to start coming to the clinic. (ID 15)
Clinic staff in particular emphasised the importance of LMs helping other women to reach
the clinic to deliver and the example that LMs are setting in their communities. Other
influencing factors were the ‘Mama kits’ and ‘Baby kits’ provided by other agencies which
include soap, nets and food rations (ID 25), the increase in available transport services,
and the availability of birth cushions allowing women to deliver in a squatting position
rather than lying down.
Village ambulances were provided by SP and are seen as connected to the project. The
village ambulances are small carts that can be fixed to the back of a bicycle or motorbike
to transport people to the hospital. Perspectives on the village ambulances vary: HPs
report that they have helped women to reach the clinic (ID 9, 10), and several hospital
staff described them as invaluable for bringing women in labour or sick children (ID 26).
However, community members say they are not used because they are too expensive
(ID 24), because it still requires you to find a motorbike, and because the carriage is
‘shaped like a coffin so people are afraid’ (ID 15). The transport voucher scheme
recently introduced by UNICEF to improve referral of mothers to health facilities is likely
5 The magnitude of the increases in the number of women attending ANC clinics and delivering with a skilled health worked is significantly lower than the reported change in behaviour in the LQAS survey. This type of positive reporting bias is expected in behaviour surveys. Changes in the survey responses over the four years will reflect both behaviour change and knowledge transfer
(i.e. more women now know the ‘right’ answer).
www.jigsawconsult.com
Samaritan’s Purse Karamoja MCH evaluation
Page 33
to reduce use of the village ambulances further.
Overall, the message that birthing at health facilities is safest has been effective. Health
clinic staff say that maternal deaths in some villages have made women very receptive
to LM education on attendance at the health unit (ID 23). In focus groups LMs said they
go to the clinic because they fear miscarriage or death (ID 29). The rise in HIV has also
made traditional birth attendants more cautious about attending births.
Distance from the clinic is the main factor preventing attendance. Women tend to wait
until labour to move to the clinic: the majority of women do not know their estimated
delivery date (ID 25). In areas where transport is limited and distances are long, women
(most of whom are multiparous and likely to labour rapidly, particularly when spacing
between pregnancies is small) are setting out alone and on foot and often birthing by
the roadside (ID 21), increasing risk to them and the baby as well as creating
understandable distress. They return home without ever making it to the clinic for a
check-up.
This evaluation recommends that the message needs tailoring for individual remote
communities: there is a need for further health education for women on the importance
of knowing their estimated delivery date and being able to recognise early signs of pre-
labour to avoid strenuous journeys (this might include loss of the cervical plug, back
ache, period pain, disrupted bowels, and a ‘lightening’ under the ribs as the baby
descends further into the pelvis). LMs will sometimes fetch a birth attendant to provide
after care following a Birth Before Arrival: LMs and NW could also be educated in how to
recognise when a labour is progressing too rapidly to safely facilitate travel to the clinic.
This evaluation recommends increased education on post-natal care. It is unusual for
women to attend their six-day antenatal care appointment, although they will normally
attend the six-week check-up when the baby also receives the first vaccination (ID 24).
12.3. Output 3: Community members aware of MCH issues and provide enabling environment
The care group model is designed to provide health knowledge, not just for individual
women but for whole communities. Community tensions as a result of greater
empowerment of women were recognised as a risk at the outset of the project. In
general, there has been markedly positive support for the care groups from husbands,
community leaders and other men in the community. The project has used community
leader training, radio messaging and community discussion groups to promote MCH
messages in the wider community.
HPs report generally positive relationships with community leaders. There have been
challenges in some communities, particularly related to the lack of material distribution,
but in most communities the local leaders ‘are interested and they encourage us to
continue having such discussions because it is for the better of their area’ (ID 13).
The evaluation team saw evidence of a growing atmosphere of partnership on decision
making for health within families. In the three focus groups with community leaders and
men, the men expressed a greater respect for LMs, recognising them as carrying
responsibility for health teaching for the village, and as leaders of women in the
community. One LM’s husband commented: ‘We see them as leaders in the village… We
no longer overpower them in our own families: we try to treat them with respect’ (ID
47).
In focus group meetings LMs describe an increase in respect (ID 36, 45), unity (ID 45)
and harmony (ID 43), with some women noting a reduction in domestic violence (ID 36,
www.jigsawconsult.com
Samaritan’s Purse Karamoja MCH evaluation
Page 34
44). For example:
Our husbands have begun respecting us because of the health we have
introduced to the family. Sometimes the husbands used to beat the women
when the compound was dirty but these days they respect us. Our husbands encourage us to keep on what we are doing as leader mothers. (ID 35)
Our husbands have seen that since we became a leader mother we have
brought changes in our families: as soon as we get up in the morning we
wash the compound, we are eating some different foods. Our husbands like
the changes that we have brought to the families so they have given permission to continue. (ID 38)
The survey did not demonstrate a change in the percentage of men who report that
women are involved in decision making. However, there was a non-significant increase
in the proportion of females who said they are involved in decision making.
2012 2013 2014 2015 Change Significant
% of caregivers reporting that women are participating in health decisions
63.3%
(56.8-69.9)
61.9%
(55.3-68.5)
58.6%
(51.9-64.2)
67.6%
(61.3-73.9) 4.1% No
M F M F M F M F M F M F
47% 79% 32% 91% 43% 74% 45% 90% -2% 11% No Yes
Table 13: Reported rates of women involved in household decision making.
The community discussion group incorporates eight care group communities: the
meeting is attended by 80 women and 80 men. Community discussion groups are seen
by the community as positive, but on their own are not frequent enough to have a
significant long-term impact. Broadcast health messages air 33 times each month; talk
show broadcasts are relatively infrequent. Radio messaging helps target men in the
community (ID 10), but with many organisations involved in health messaging it is
difficult to trace attribution (although messages are branded). Nevertheless, in the 2015
survey over 60% of participants were able to recall KIMCH radio health messages.
2012 2013 2014 2015 Change Significan
t
# of community leaders with
improved IMNCI knowledge
0 296 612 900 900 NA
M F M F M F M F M F M F
0 0 240 56 454 158 602 298 602 298 NA NA
# of community members participating in
discussion groups
0 8,003 16,156 24,092 24,092 NA
M F M F M F M F M F M F
0 0 3376 4627 7050 9106 11265 12827 11265 12827 NA NA
% of caregivers
of children 0-23 months able to recall 1+ radio spot messages
0% 33.3%
(27.0-39.7)
58.1%
(51.4-64.8)
61.4%
(54.8-68.0) 61.4% Yes
Table 14: Community knowledge and radio messaging.
There is some evidence for behaviour changes among men in the community. LMs
reported that their husbands now help to bathe the children, wash clothes and assist
www.jigsawconsult.com
Samaritan’s Purse Karamoja MCH evaluation
Page 35
with household tasks when their wives are pregnant (ID 38, 43, 47). Women perceive
that their husbands have benefited from increased knowledge about women’s health. By
comparison to women, knowledge diffusion and behaviour change in men is limited.
When asked about the primary MCH issues raised through community discussion groups
or care groups, men most frequently mentioned family planning. Attitudes towards
family planning vary considerably: in some households it is becoming acceptable to use
natural planning methods (ID 25); in some areas there are existing cultural family by-
laws that require abstinence for a named period after birth; in others conception is
believed to be ‘up to God’ (ID 24).
A barrier analysis was conducted to understand the knowledge and beliefs which
influence whether a mother is a ‘doer’ or ‘non-doer’ of family planning practices. The
survey found that doers were more likely to believe that health staff approved, to attend
antenatal and family planning clinics, to be afraid of early pregnancy, and to perceive no
disadvantages. Doers were more likely to believe that child spacing would result in
healthier children (58% doers; 13% non-doers), help the family avoid malnutrition (73%
doers; 31% non-doers), and prevent complications. Nevertheless, 44% of both groups
believed that planning to delay or prevent pregnancy is not the will of God or gods.
During the evaluation focus groups women commented that traditionally the majority of
men have not approved of child spacing or family planning. Women believe that
attitudes among some men are changing as a result of education and an increasing
awareness of the health risks of closely spaced births (ID 44). Child spacing is normally
achieved through the husband taking a second, or third, wife. Women, who often
struggle to support multiple children, were interested in, and positive about, the use of
hormonal implants, which are now available through health clinics, stating that it would
give them greater control (ID 36, 37, 44, 45). Health clinics have seen an increase in the
demand for birth control support. The number of couples attending family planning
clinics for the first time rose from 102 in 2010-11 to 466 in 2013-14. Hospital staff
describe the teaching on family planning as ‘a real achievement’ that has ‘made a
difference’ (ID 26).
HPs and HSs interviewed all identified greater inclusion of men as the principal area for
improvement in the project. In focus groups, men emphasised a desire to be involved in
the programme (ID 47). Their suggestions include: establishing Leader Father Groups;
attending care group meetings; and specific training on nutrition and hygiene. The
evaluation team agrees that there should be greater engagement with men in nutrition
messaging. In Karamojong society, women are responsible for cultivation but men are
the decision makers and will determine how much of the cereal harvest to sell in order to
buy other food types or consumables. There is often little long-term planning, which can
result in vulnerability to under-nutrition later in the year. Men would benefit from
nutrition training that also addresses issues of resource allocation and annual resource
planning.
Finally, women agreed that greater inclusion of the men in their community would be
beneficial. Women will benefit from increased parity in addressing issues of family
planning, household decision making and domestic violence. In general there is a high
level of domestic violence, and women reported isolated examples of violence arising
from LM involvement in the programme without payment (ID 13, 37). Women
recognised the importance of male inclusion because of men’s role as decision makers
and felt that community leader engagement was not sufficient to affect household-level
decision making (ID 12). For example, one LM suggested:
It would be good to mix with men together so they can also learn some ideas,
which they have failed to hear from us their women… It would help us arrive
www.jigsawconsult.com
Samaritan’s Purse Karamoja MCH evaluation
Page 36
at consensus for contentious ideas such as family planning. The men should often come and learn with the women as well. (ID 33)
12.4. Output 4: Improved continuum of care available
‘Improved continuum of care’ is the output for which the measurable outcomes of SP’s
involvement are least definable. The project has supported increased demand through
health education; it has increased the capacity of VHTs through training, and it has
facilitated dialogue between LMs and the community. However, it does not impact on the
quality of services delivered; several other NGOs are engaged in health system
strengthening.
There is greater awareness and utilisation of health services for planned care. Women
are more likely to recognise the symptoms of common disease at an early stage, and to
seek care sooner. Basic care is provided by the VHT, who can prescribe zinc, oral
rehydration therapies and Amoxicillin. SP partnered with CUAAM, IRC and the DHO to
deliver refresher training for current VHTs and a comprehensive one-week training
package for new VHTs on IMNCI. This included information on basic treatments, referral
and emergency procedures such as sterilised cutting of the umbilical cord following
delivery in the home. In total 610 VHTs received training, including 222 new VHTs; as a
result, all villages in Napak have at least one trained VHT member, an achievement
given the high levels of population movement in the past five years. The VHTs appear
competent within the scope of their practice and in general have positive and
collaborative relationships with LMs and women in the community (ID 27).
Napak has one hospital, six health centres at level 3 (which includes services such as
antenatal care, maternity, nutrition, family planning); and seven health centres at level
2 (which provide limited to basic outpatient services). There appears to be good referral
pathways between primary health care settings and secondary health care settings;
referrals appear to be timely and appropriate.
Research conducted in western Uganda identified reasons for women’s reluctance to
deliver in health facilities or seek care for complications, including a lack of skilled staff
at primary health care level, complaints of abuse, neglect and poor treatment, health
workers' views that women were ignorant: these also explain the unwillingness of
women to deliver in health facilities and seek care for complications (Kyomuhendo
2003). Reluctance to attend health facilities was identified as a challenge at the outset of
the project. However, among women interviewed there was very little reluctance to seek
medical care from the VHTs or health clinics. Women associate better health outcomes
with attendance at health facilities. LMs, health centre staff and district health authorities
reported that there had been some perception and attitude change towards health care
services (ID 16, 23, 30). This is corroborated by Concern Worldwide, which has
developed a community scorecard to look at satisfaction with service provision by health
facilities: it has seen an improved and positive relationship between the health facility
staff and community since its project began six months ago (ID 19).
2012 2013 2014 2015 Total/
Change
Significa
nt
# of VHTs with improved IMNCI knowledge
0 0 610 390 1,000 NA
% of CG leaders in quarterly dialogues with health personnel
0 25% 52% 85% 85% NA
Table 15: Improving the continuum of care
www.jigsawconsult.com
Samaritan’s Purse Karamoja MCH evaluation
Page 37
Positive relationships have been facilitated by joint training for VHTs and LMs and
dialogue meetings between health clinics and care groups (see Table 15). The
discussions addressed issues of community-based care and strengthened the link
between the communities and the health centres. The IRC representative interviewed
noted that prior to the project there had been little interaction between health centres
and communities. He commented that women are more engaged in the health of their
communities and have increased capacity to work with the health centres.
There is scope for greater collaboration between midwives at the clinics and care groups:
this could include midwifery-specific teaching such as the importance of knowing your
due date, recognising the early signs of labour, and when not to attempt to attend the
clinic. Midwives already do some outreach work in the communities, but with limited
collaboration with LMs.
Other factors positively affecting the continuum of care include reduced insecurity,
improved road networks and the presence of clean water and ambulances in the
hospitals. Some gaps in midwifery knowledge and practice may be linked to poorer
longitudinal outcomes in child health: for example, cord clamping within a minute of
birth is normal practice but is now known to link with anaemia and reduced levels of iron
in infants. Cord clamping happens immediately in known HIV-positive mothers out of
misinformation that the cord blood that passes to the baby is maternal blood and
increases the risk of mother-to-child transmission.
An effective continuum of care, and the ability of women to change behaviour in
response to health knowledge, is dependent upon the stability of MCH services and
facilities. To ensure the sustainability of behaviour change, collaborative working with
partner organisations and local government should be continued.
13. Sustainability The care group model is easily leveraged: the model creates a community structure that
easily facilitates other health care intervention. This was most clearly seen through the
hepatitis E interventions in partnership with Concern Worldwide and the district
government. In future, it is hoped that district government and NGOs will make greater
use of the care group infrastructure to deliver community health interventions.
At the end of the project, in March 2015, the care groups in Napak will be absorbed into
a larger Concern Worldwide project that operates across Karamoja. There are some
differences in the two organisations’ approaches: the Concern Worldwide model only
includes women within 5 km of a health centre, only includes women actively caring for
a child under five, and links behaviour change to food aid. Concern Worldwide argues
that its approach is more streamlined (ID 19); it does not see the elder women in the
community as having significant impact on MCH choices.
As a result, some of the care groups, for example LG care groups, will not be
incorporated. There is evidence from other care group projects, most notably in
Mozambique, that LMs will continue to promote good health practices long after a
programme has formally finished. In focus groups, women believed they will ‘continue
with the knowledge that has been given to us’. There is strong local ownership and
women see care groups as their own; while formal care group meetings are unlikely to
continue in most locations without external support, the structure of Karamojong
societies means that women will continue to share what they have learnt and exemplify
good practice to their neighbours. Teaching through storytelling, drama and songs has
also ensured good retention of messages.
www.jigsawconsult.com
Samaritan’s Purse Karamoja MCH evaluation
Page 38
For us we will continue with what we have been told and we will continue
doing this. It is now our responsibility to continue this work. (ID 36)
If the programme ends, people will continue to promote the hygiene and
sanitation in the family, but maybe the group will not continue in the same
way. We will continue but we won’t have as much strength because people will not be coming to inform and encourage us. (ID 38)
Some care groups have started savings groups; they were given savings boxes as part
of their annual incentive from SP and were registered with the district authorities.
Saving groups will support ongoing cohesion of LM relationships and may help women
to meet more of their nutrition goals (ID 6).
In other contexts, care group support has been taken on by local government. A recent
publication on a Ministry of Health (MoH)-led Integrated Care Group model found that
similar health and nutrition outcomes were achieved in MoH-led models and NGO-led
models: this suggests that an MoH can implement and manage care groups on a par
with NGOs (USAID and Concern Worldwide 2014). The authors contend that integrating
the supervisory and managerial functions of care groups into MoH structures is a more
sustainable approach to harnessing community involvement for health than one that
relies solely on NGOs.
The care groups would therefore benefit from strengthened links to health clinics and
local government structures. The district and sub-county chief offices recognise and
value the role of care groups for dissemination of health messaging. In interviews they
suggested that links should be strengthened between LMs and VHTs: the VHTs are the
local government’s current community health mechanisms (ID 15). SP should
encourage local government and other NGOs to make use of the care group
infrastructure through complementary interventions; this would build capacity and help
cement the long-term future of the care groups.
Care groups would also benefit from stronger relationships with local clinics. There is
scope for better collaboration, for example through joint outreach sessions. While
nurses and midwives recognise the value of the project, they do not feel involved.
Finally, this evaluation advises greater clarity and consistency in the way the end of the
project is discussed. The HSs are concerned about the project finishing and in some
cases have been unclear and evasive in their communication with HPs and care groups.
14. Conclusions
14.1. Summary of achievements against evaluation criteria
This study had two primary objectives:
1. To independently verify (and supplement where necessary), grantees’ record
of achievement as reported through Annual Reports and defined in the project’s log frame;
2. To assess the extent to which the project was good value for money
The evaluation team found that SP had provided a realistic and proportionate record of
their achievements in the annual reports and log frame.
The evaluation analysed the project design, implementation and VfM based on the
OECD-DAC criteria: relevance, efficiency, effectiveness, impact and sustainability.
Relevance: The project directly addressed MDGs 4 and 5 and indirectly addressed
www.jigsawconsult.com
Samaritan’s Purse Karamoja MCH evaluation
Page 39
MDG 6. It targeted and reached women of reproductive age in Karamoja, a sub-
region of Uganda with low health indicators. The project used a care group model
that countered the prevailing culture of aid dependency by creating a strong
network of over 2000 volunteer women.
Effectiveness: The majority of activities were implemented effectively and SP used
evaluations, barrier analysis, and monitoring data to reflect on and improve its
work. Key drivers included SP’s focus on local ownership and local capacity building.
Key barriers included challenging environmental factors, delays to distribution of
insecticide treated nets, and cultural resistance to some teaching.
Efficiency: SP delivered the majority of activities on time and within budget (the
main exception was distribution of nets, part of the SP cost share, which was
delayed by to external mitigating factors). The project achieved good value for
money through rigorous procurement processes and a focus on knowledge transfer
with few capital costs.
Impact: The project delivered its four outputs. The project has achieved an
impressive level of knowledge transfer. There is evidence of behaviour change in
response to teaching, particularly in hygiene and sanitation practices, exclusive
breastfeeding at 0-6 months and attitudes towards maternal care. The relationships
between communities and health clinics have been strengthened. There are
opportunities for greater inclusion of men in knowledge transfer.
Sustainability: Outcome studies of the care group model in other settings have
demonstrated strong sustainability. SP has developed good relationships with other
NGOs in the district; there is a good example of the care group structure being used
by local government and Concern Worldwide in response to a hepatitis E outbreak.
There is opportunity to build stronger relationships at the health centre level.
14.2. Summary of achievements against rationale for GPAF funding
DfID provides funding to civil society organisations (CSOs) annually in line with its
overall strategy to alleviate poverty and promote peace, stability and good governance.
The GPAF fund provides CSOs with restricted funds for projects designed to deliver clear,
identifiable outputs and outcomes. The fund focuses on results that contribute to MDGs,
is relevant to target communities, and provides sustainable outcomes.
The planned outcome for the KIMCH project was designed to directly contribute to MDG
4 and 5 in Uganda. The evaluation has demonstrated a high level of MCH knowledge
among both LM/LGs and NW. There is also evidence that men have benefited from
increased knowledge on prevention and treatment of child illness.
There is strong evidence of behaviour change, particularly in hygiene and sanitation
practices, exclusive breastfeeding at 0-6 months and attitudes towards maternal care.
Behaviour change has been slower in response to nutrition messaging but this is largely
because of wider challenges of poverty. Positive maternal actions are being
demonstrated by LMs and it is believed that they will continue to set an example to other
women in the community; challenging social norms and traditional practices. This
behaviour change in NW is expected to be a medium to long term outcome: behaviour
change is slow, rather than resisted. However the results achieved within the three year
time frame are very positive.
The overall aim of the project was to contribute to a decrease in maternal and under-5
mortality rates in Napak district by 2015. At the time of the most recent Demographic
www.jigsawconsult.com
Samaritan’s Purse Karamoja MCH evaluation
Page 40
and Health Survey, in 2011, the U5MR stood at 90/1,000 live births and the MMR at
310/100,000 live births. It is not yet possible to determine if these rates have decreased
during the project lifespan.
14.3. Overall impact and value for money of GPAF funded activities
2012 2015 Target Target met6
% of children 0-23 months with weight for age scores <-2 (by sex)
21%
(16.9 - 24.7)
13.5%
(8.9 - 18.1) 11% No
M F M F M F M F
25% 17% 14% 13% 15% 7% Yes No
% of infants 0-5 months who
are exclusively breastfed
49.5%
(42.8 - 56.3)
92.4%
(87.2 - 97.4) 75% Yes*
M F M F M F M F
49% 50% 91% 94% 74% 76% Yes* Yes*
% of children age 6-23 months receiving appropriate complimentary feeding (by sex).
17% (11.6-21.7)
55.2% (45.7 - 64.7)
50% Yes
M F M F M F M F
15% 18% 53% 58% 48% 51% Yes Yes
% of children 0-23 months who have had an episode of diarrhoea in the last two weeks who received ORT (by sex)
72.2% (65.7 - 78.0)
79.2% (70.5 - 88.0)
85% No
M F M F M F M F
71% 73% 83% 76% 84% 86% No No
% of children 0-23 months with a fever in the last two weeks who received any anti-malarial treatment within 24 hours (by
sex)
30% (24.5 - 35.4)
50% (39.9 - 60)
60% No
M F M F M F M F
28% 32% 33% 67% 58% 62% No Yes
% of children 0-23 months who slept under an ITN the previous night
30.9%
(25 - 37.2)
97.1%
(94.8 - 99.8) 20% Yes*
% of births assisted by a skilled attendant
39.0% (29.7 - 48.4)
82.9% (75.7 - 90.1)
54% Yes*
% of female caregivers of children 0-23 months who received a postnatal check from a trained health care provider
within 24 hours of giving birth.
43.8% (34.3 - 53.3)
72.4% (63.8 - 80.9)
59% Yes*
% of pregnant women aged 15-49 years attended four or more
antenatal visits
60% (50.6 - 69.4)
90.4% (84.8 - 96.0)
75% Yes*
Table 16: Output and outcome indicator data.
6 The small sample size means that it is not possible to have evidence that targets have been achieved
for the underlying population. Instead, the method allows for stating that there is ‘no evidence the target has not been achieved’. “Yes” indicates that the target has been achieved for our sample group. The * indicates statistical significance – i.e. that there is no evidence the target has not been achieved within the wider population.
www.jigsawconsult.com
Samaritan’s Purse Karamoja MCH evaluation
Page 41
The project met 12 of the 16 output targets (for 2015) and 8 of the 9 outcome targets.
Table 16 illustrates the outcome targets achieved. Output achievements are given in the
Annexes.
Section 12 discusses the impact of the project and the extent to which each project
outcome can be attributed to the project. It indicates significant achievements in
knowledge acquisition by both LMs and NW. There has been notable behaviour changes
amongst LMs and, to a lesser extent, NW: survey data indicates increases in antenatal
care attendance, skilled birth attendance, breastfeeding practices, nutrition for children
aged 6-24 months and use of insecticide-treated nets.
The evaluation team analysed the extent to which VfM was achieved. It observed the
following:
The project demonstrated economy through low capital and infrastructure costs, and inexpensive training programmes.
Efficiency was demonstrated through focussing on mobilising communities: for
example, by ensuring local ownership of the care group and promoting volunteering.
The programme demonstrated effectiveness in meeting its objectives, most
frequently on time and within budget.
Equity is an intrinsic part of the care group model, which achieves broad
coverage of the entire population of mothers through establishing groups of
NW who elect their own LM.
VfM guidance also considers the extent to which the project aligns with the relevant
MDGs. This project was designed to contribute to MDGs 4 and 5 in Uganda: to reduce
child mortality and to improve maternal health. There have also been notable
achievements in support of MDGs 3 and 6: the project has promoting gender equality
among communities in Napak and is helping to combat disease, in particular malaria and
other common diseases in children under five.
15. Lessons learnt This section provides a summary of key lessons learnt from project design and
implementation. It is broken down into lessons at the policy, sector, GPAF fund and
organisational level.
15.1. Project level
Two lessons relating to project design, implementation and management were identified.
Firstly, HPs monitored attendance at care groups, the number of NW reached each
month, and births and deaths in the LMs’ communities. They entered data on a paper
chart and then submitted the information to HSs using a Frontline SMS mobile system.
The system was unnecessarily complex: it required long codes that led to inaccurate
data entry, and poor network coverage meant there were inaccurate and missing data
points. For example, between December 2012 and March 2013, 44% of SMS messages
submitted to the system were rejected. The HPs resorted to collecting and submitting
both SMS- and paper-based data. To improve data collection, the evaluation team
recommends use of a simplified electronic data collection system. SP should also
investigate use of a simple smartphone-based tool such as Open Data Kit (ODK).
Secondly, SP employed both male and female HPs. Both male and female HPs have
successfully delivered health knowledge to their care groups. Nevertheless, the
evaluation team found that LMs responded best to female HPs, who are better able to
www.jigsawconsult.com
Samaritan’s Purse Karamoja MCH evaluation
Page 42
understand challenges faced by women in Karamoja, and who can communicate
culturally sensitive information (ID 35, 36, 28). For example, traditionally, women in
Karamoja will hide their pregnancies and do not discuss labour or delivery with men as
they believe it will prolong labour. However, the evaluation team also noted that in focus
group discussions, community leaders and men responded well to male HPs and
translators.
15.2. Policy level
Two lessons have been identified relating to the wider beneficiary population. Firstly, a
chronic challenge to the implementation of any programme in Karamoja is promoting
local ownership. The Centre for Creative Leadership provided anthropological training to
SP staff and identified the importance of giving the project a local name that local people
would understand and see as their own. Decades of emergency relief have created a
perception that people must wait for aid agencies to intervene to create change. The
project was given a local name (Erot Ngolo Kitete-‘A New Way’), project volunteers were
established as leaders and not beneficiaries, and the idea that the project belongs to the
care groups was reinforced. The name ‘A New Way’ was chosen by staff to capture the
intent and spirit of the project - a new way of pursuing healthy families through
individuals taking ownership of their own health. As a result, LMs have shown willingness
to volunteer and have taken greater responsibility for the health of their communities
(ID 31).
Secondly, women are a marginalised population in Karamoja. SP identified male
resistance to changing power dynamics as a threat to the success of the project. Largely,
it has experienced the opposite: men have been eager to receive training, have attended
community discussion groups and have encouraged their wives to continue in the LM
role. Male inclusion has contributed to a positive learning environment and has improved
programme sustainability. It is believed that greater male inclusion is important for
addressing sensitive cultural issues as well as household decisions on nutrition and
resource allocation.
15.3. Sector level
Three relevant lessons for the sector were identified during the evaluation. Firstly, the
value of inclusion of LG care groups. Grandmothers and elderly relatives are respected in
Karamojong society and are influential in health decisions at the household level.
Grandmothers also expressed a strong desire to be involved and concern that if the
community is following a ‘new way’ they did not want to be left ‘in the old way’ (ID 11).
The piloted LG care groups were implemented and expanded to include other LGs. The
model has demonstrated the value of including elders who are opinion leaders
particularly in decisions on antenatal care and health clinic delivery. Health clinic workers
also emphasised the importance of including elder women and community leaders as
facilitators of behavioural change (ID 25).
Secondly, barrier analysis has provided an important understand of the underlying
cultural constructs that prevent behaviour change. SP has used the barrier analysis to
inform curriculum design. Staff have also learnt from other care group implementers: for
example seeing from other care group projects that inclusion of men is important.
Finally, the project has demonstrated the different responses to care groups by rural and
urban communities in Napak. In trading centres, for example, literacy rates are higher
and women have a better underlying understanding of health issues and have fewer
barriers to behaviour change. However, women in trading centres also often have their
own small businesses and therefore it is difficult to mobilise them or to create such a
www.jigsawconsult.com
Samaritan’s Purse Karamoja MCH evaluation
Page 43
strong sense of care group community (ID 9, 10).
15.4. GPAF funding level
GPAF provided a three-year grant for this project. The evaluation team note that
considerable impact has been achieved in the past three years. However, women in the
community would benefit from ongoing teaching and support in nutrition and maternal
health. SP staff felt that the three-year programme cycle was very short; the evaluation
team agrees that a five-year lifecycle is preferable for behaviour change programming in
complex settings.
16. Recommendations
16.1. Curriculum
SP have successfully implemented a participatory curriculum that relies on pictures,
songs, actions, games, stories and sayings. It would benefit from the following changes:
1. The LMs interviewed felt that the flipchart is too bulky and heavy. A separate
flipchart for LMs and HPs would allow SP to streamline the materials and to focus LM
flipcharts on clear and memorable imagery. A portable A5 book that women can
easily carry with them may increase the use of flipcharts outside of care groups.
2. Some of the cartoon images are confusing. For example, the evaluation team
attended a care group in which the HP and LMs misunderstood a cartoon of a
monster (to depict bacteria in water) as frogs. There was consequentially some
misunderstanding over the role of frogs in poor hygiene.
3. In some cases, the layout of pages does not make it clear which story refers to
correct behaviour and which refers to incorrect behaviour.
It was noted that story telling was a very effective means of communicating health
messages within Karamojong communities. It is recommended that SP use LMs more
frequently in radio messaging to talk about their experiences and tell stories about the
changes in their households as a result of improved MCH practices.
16.2. Health teaching
The content of the curriculum is good and responds to issues raised in barrier analysis.
The evaluation recommends that SP consider inclusion of the following issues.
1. Increase antenatal and labour education: for example, teach women how to spot
early signs of labour, how to identify which stage of labour a women is in.
2. Domestic violence: This affects many women in Karamoja. Women asked to discuss
issues of beating and domestic violence. Clearly, careful consideration would be
required in designing discussions and empowering the community to respond to and
reduce violence.
3. Revisit messaging for individual communities: SP should work with the HPs to
identify where messaging is having unanticipated consequences. In Napak, the
unexpected consequences were women delivering on the way to the clinic when
travelling from remote communities; and in three villages, unique soil composition
leading to frequent severe flooding of latrines causing a worse hygiene scenario (ID
14). HPs should be assisted to know how to deliver nuanced messages in these
instances.
www.jigsawconsult.com
Samaritan’s Purse Karamoja MCH evaluation
Page 44
4. Revisit nutrition training: this training module should be re-examined and SP should
look at how to support women in taking steps toward more nutritious diets for
themselves and their children.
5. FGM is practiced in the Tipeth communities on the outskirts of Napak; consider ways
of including teaching on the impact of FGM on pregnancy and birth in Tipeth and
neighbouring communities.
6. Consult health workers to identify messaging on neglected, endemic diseases in the
region. In Napak, health workers emphasised the prevalence of Jiggers and lack of
local action.
16.3. Strengthen relationships with health units
Health workers were positive about the role of LMs and care groups in improving
community level health. However they did not feel deeply involved in the project. SP
should work to strengthen links with health units, and specifically midwives. For
example, SP could facilitate outreach visits from midwives into the relevant
communities.
16.4. Strengthen links to local government
SP should work to demonstrate the value of the care group to local government by
showcasing the achievements of care groups and encouraging government to use the
care group infrastructure to support VHT and other initiatives. One way of achieving this
is increasing links between care groups and schools. This would reinforce positive health
messaging to young people.
16.5. Increase male involvement
Men would benefit from teaching on nutrition and planning and management of family
resources. This could include establishing Leader Father groups, encouraging men to
attend care group meetings, or providing specific training on nutrition and hygiene. Men
would particularly benefit from training that addresses allocation of resources, annual
resource planning, family planning, decision making at the household level and domestic
violence.
16.6. Monitoring and evaluation
Firstly, frontline SMS has not been used effectively for data collection. SP should
consider simplifying the Frontline SMS system: it is possible to implement an interactive
SMS system where HPs respond to specific questions with single data points.
Alternatively consider a simple smart-phone tool such as ODK that provides simple
questionnaires and stores the data on the telephone until there is sufficient phone signal
for transmitting the responses.
Secondly, the tools for monitoring data should be consistent for the lifetime of the
project. In this project, changes to monitoring tools mean it is very difficult to compare
LM attendance at the outset and end of the project.
Thirdly, the evaluation team felt that HPs experienced pressure to report positive
monitoring data for fear of disappointing donors. This may lead to HPs being reluctant to
report low attendance and LMs to over-report their reach to NW in the community. SP
should work with the HSs to explain the value of monitoring data and to ensure they are
not placing unhelpful pressure on HPs. Finally, other care group programmes have used
community-based vital registration and health information to facilitate routine
www.jigsawconsult.com
Samaritan’s Purse Karamoja MCH evaluation
Page 45
surveillance of childhood illnesses and maternal and child deaths. A similar system would
allow SP to monitor incidence of illness and death and use this information to inform the
project curriculum and design of additional interventions. However, reliable data would
require additional M&E support and a more formal data collection than is currently
practiced. It would also require HSs to verify data, for example, through random
sampling of households to check the data provided by LMs. Cause of death for infants
and children under 5 years can be determined using standard verbal autopsy procedures
described by the WHO (1999).
17. References Caulfield, Laura E., Mercedes de Onis, Monika Blössner, and Robert E. Black. 2004. “Undernutrition as an Underlying
Cause of Child Deaths Associated with Diarrhea, Pneumonia, Malaria, and Measles.” The American Journal of Clinical Nutrition 80 (1): 193–98.
CORE group, and USAID. 2014. Care Groups: Implications of Current Innovations, Scale-up and Research. Summary Report of a CORE Group Technical Advisory Group meeting. http://www.coregroup.org/storage/documents/meeting_reports/Care_Group_TAG_Meeting_Summary_2014_Final_August_11_2014.pdf.
Davis, Thomas P., Carolyn Wetzel, Emma Hernandez Avilan, Cecilia de Mendoza Lopes, Rachel P. Chase, Peter J. Winch, and Henry B. Perry. 2013. “Reducing Child Global Undernutrition at Scale in Sofala Province, Mozambique, Using Care Group Volunteers to Communicate Health Messages to Mothers.” Global Health, Science and Practice 1 (1): 35–51. doi:10.9745/GHSP-D-12-00045.
Edward, Anbrasi, Pieter Ernst, Carl Taylor, Stan Becker, Elisio Mazive, and Henry Perry. 2007. “Examining the Evidence of under-Five Mortality Reduction in a Community-Based Programme in Gaza, Mozambique.” Transactions of the Royal Society of Tropical Medicine and Hygiene 101 (8): 814–22. doi:10.1016/j.trstmh.2007.02.025.
Gabrysch, Sabine, and Oona M. R. Campbell. 2009. “Still Too Far to Walk: Literature Review of the Determinants of Delivery Service Use.” BMC Pregnancy and Childbirth 9: 34. doi:10.1186/1471-2393-9-34.
Imdad, Aamer, Mohammad Yawar Yakoob, and Zulfiqar A. Bhutta. 2011. “Impact of Maternal Education about Complementary Feeding and Provision of Complementary Foods on Child Growth in Developing Countries.” BMC Public Health 11 Suppl 3: S25. doi:10.1186/1471-2458-11-S3-S25.
Kyomuhendo, Grace Bantebya. 2003. “Low Use of Rural Maternity Services in Uganda: Impact of Women’s Status, Traditional Beliefs and Limited Resources.” Reproductive Health Matters 11 (21): 16–26.
Lozano, Rafael, Haidong Wang, Kyle J. Foreman, Julie Knoll Rajaratnam, Mohsen Naghavi, Jake R. Marcus, Laura Dwyer-Lindgren, et al. 2011. “Progress towards Millennium Development Goals 4 and 5 on Maternal and Child Mortality: An Updated Systematic Analysis.” Lancet 378 (9797): 1139–65. doi:10.1016/S0140-6736(11)61337-8.
Ministry of Health, Uganda. 2007. Roadmap for Accelerating the Reduction of Maternal and Neonatal Mortality and Morbidity in Uganda. http://www.nationalplanningcycles.org/sites/default/files/country_docs/Uganda/uganda_mnh_roadmap_2007-2015.pdf.
Renfrew, Mary J, C S E Homer, and S Downe. 2014. “Midwifery: An Executive Summary for The Lancet’s Series.” The Lancet.
Roy, S. K., G. J. Fuchs, Zeba Mahmud, Gulshan Ara, Sumaya Islam, Sohana Shafique, and Syeda Sharmin & Chakraborty Akter. 2015. “Intensive Nutrition Education With or Without Supplementary Feeding Improves the Nutritional Status of Moderately-Malnourished Children in Bangladesh.” The Journal of Health, Population and Nutrition 23 (4): 320–30. Accessed February 24.
USAID, and Concern Worldwide. 2014. Shifting the Management of a Community Volunteer System (Care Groups) from NGO Staff to Ministry of Health Staff in Burundi. Operations Research Brief.
Walker, Christa L. Fischer, Igor Rudan, Li Liu, Harish Nair, Evropi Theodoratou, Zulfiqar A. Bhutta, Katherine L. O’Brien, Harry Campbell, and Robert E. Black. 2013. “Global Burden of Childhood Pneumonia and Diarrhoea.” Lancet 381 (9875): 1405–16. doi:10.1016/S0140-6736(13)60222-6.
World Health Organisation. 2011. Maternal and Child Health: Uganda. World Health Organisation.
———. 2014. Hepatitis E Fact Sheet No. 280. http://www.who.int/mediacentre/factsheets/fs280/en/.
Zwane, Alix Peterson, and Michael Kremer. 2007. “What Works in Fighting Diarrheal Diseases in Developing Countries? A Critical Review.” The World Bank Research Observer 22 (1): 1–24. doi:10.1093/wbro/lkm002.
www.jigsawconsult.com
Samaritan’s Purse Karamoja MCH evaluation
Page 46
18. Annexes
18.1. Summary of findings against OECD-DAC criteria
Criteria Findings
Relevance The project directly addressed MDGs 4 and 5 and indirectly
addressed MDG 6. It targeted and reached women of
reproductive age in Karamoja, a sub-region of Uganda with
low health indicators. The project used a care group model
that countered the prevailing culture of aid dependency by
creating a strong network of over 2000 volunteer women.
Effectiveness The majority of activities were implemented effectively and
SP used evaluations, barrier analysis, and monitoring data to
reflect on and improve its work. Key drivers included SP’s
focus on local ownership and local capacity building. Key
barriers included challenging environmental factors, delays to
distribution of insecticide treated nets, and cultural resistance
to some teaching.
Efficiency SP delivered the majority of activities on time and within
budget (the main exception was distribution of nets which
was delayed by to external mitigating factors). The project
achieved good value for money through rigorous procurement
processes and a focus on knowledge transfer with few capital
costs.
Sustainability Outcome studies of the care group model in other settings
have demonstrated strong sustainability. SP has developed
good relationships with other NGOs in the district; there is a
good example of the care group structure being used by local
government and Concern Worldwide in response to a hepatitis
E outbreak. There is opportunity to build stronger
relationships at the health centre level.
Impact The project delivered its four outputs. The project has
achieved an impressive level of knowledge transfer. There is
evidence of behaviour change in response to teaching,
particularly in hygiene and sanitation practices, exclusive
breastfeeding at 0-6 months and attitudes towards maternal
care. The relationships between communities and health
clinics have been strengthened. There are opportunities for
greater inclusion of men in knowledge transfer.
www.jigsawconsult.com
Samaritan’s Purse Karamoja MCH evaluation
Page 47
18.2. Achievement of output targets
Output 1: Care Groups (CGs) established and their capacity built to
promote maternal child health in their communities
2012 2015 Target Target met7
# of Health Promoters who complete training and pass test in the Integrated Management of Newborn and Childhood Illnesses (IMNCI).
0 19 25 No
# of CGs established and meeting on a biweekly basis.
0 219 180 Yes
% of Leader Mothers (LMs) who complete training and pass test in IMNCI.
0 85 90 No
% of LMs outreaching their target households each month.
0 95 82 Yes
# of MCH messages developed in collaboration with CGs and health personnel.
0 0 0 Yes
Output 2: Pregnant women and caregivers of children <5 years with
increased knowledge and capacity to exhibit healthy MCH behaviours.
2012 2015 Target Target met
# of caregivers/pregnant
women educated by LMs on prevention, treatment of infections, ANC and nutrition (by sex)
0 42,385 47,600 No
M F M F M F M F
0 0 11,867 30,518 12,600 35,000 Yes No
% of caregivers of children 0-23 months who can identify four or more
preventative actions for common childhood illnesses (by sex).
13.3%
(8.7 - 17.9)
57.1%
(50.4 - 63.8) 55% Yes
M F M F M F M F
12% 14% 55% 59% 53% 57% Yes Yes
% of caregivers of children 0-23 months able to identify four or more signs of serious
illness in a child (by sex).
21.4%
(15.9 - 26.9)
66.7%
(60.3 - 73%) 60% Yes
M F M F M F M F
23% 20% 55% 78% 60% 60% No Yes*
% of caregivers with
children 0-23 months with at least two insecticide treated nets (ITN).
12.3%
(7.9 - 16.8)
72.9%
(66.9 - 78.9) 20% Yes
% of caregivers able to identify three or more
positive maternal health actions (by sex).
39%
(32.4 - 45.6)
83.3%
(78.3 - 88.4) 60% Yes*
M F M F M F M F
38% 40% 77% 90% 59% 61% Yes* Yes*
7 The small sample size means that it is not possible to have evidence that targets have been
achieved for the underlying population. Instead, the method allows for stating that there is ‘no
evidence the target has not been achieved’. “Yes” indicates that the target has been achieved
for our sample group. The * indicates statistical significance – i.e. that there is no evidence the
target has not been achieved within the wider population.
www.jigsawconsult.com
Samaritan’s Purse Karamoja MCH evaluation
Page 48
Output 3: Community members aware of MCH issues and provide enabling environments for their households to adopt positive behaviours
and access services.
2012 2015 Target Target met
# of community leaders with improved IMNCI knowledge (by sex)
0 900 900 Yes
M F M F M F M F
0 0 602 298 600 300 Yes No
# of community members participating in at least two church-based discussion groups in 12 months (by sex)
0 24,092 24,000 Yes
M F M F M F M F
0 0 11,265 12,827 12,000 12,000 Yes Yes
% of caregivers of children 0 - 23 months able to recall one or more radio spot messages.
0 61.4%
(54.8 - 68.0) 45% Yes
% of caregivers of children
0-23 months reporting that women are participating in household decisions regarding when to seek health care services.
63.3%
(56.8-69.9)
67.6%
(61.3-73.9) 79% No
M F M F M F M F
47% 79% 45% 90% 70% 87% No Yes
Output 4: Improved continuum of care available for maternal child health services
2012 2015 Target Target met
# of VHTs with improved IMNCI
knowledge 0 390 600 No
% of CG leaders in quarterly dialogue with health personnel
0 85 75 Yes
18.3. Evaluation methods
Method Participants Objective
Document review N/A To understand the project aims, activities and outcomes, and to understand progress of the project against indicators and challenges faced.
Interviews with SP UK programme staff
Skype interviews with SP staff involved in programme design, management and
monitoring
To collect information on project objectives, strategy, management and challenges, and to agree priorities for the evaluation.
Interviews with SP Uganda staff
SP in-country staff
To collect information on achievements, impact and difficulties faced during implementation. This includes training, management, monitoring and reporting.
Qualitative focus At each location: To collect information on training, including what
www.jigsawconsult.com
Samaritan’s Purse Karamoja MCH evaluation
Page 49
groups with LMs/LGs and NW
focus group with 10-12
LMs/LGs and 10-12 NW
has been learnt and what has been applied; to understand challenges, successes and impact of the project on community health. (We will also review training records to triangulate findings.)
Qualitative focus group with other community stakeholders
At each location, including: community leaders health workers men who have
received training
To collect information on training in MCH, including what has been learnt and applied; to understand challenges, successes and impact of the project on community health.
Interviews with other stakeholders
District health officials Sub-county chiefs International NGO
partners
To collect information on attitudes and perceptions of officials and partners towards the programme.
Observations Observation of training and other relevant project activities
To observe training activities, to collect information on contextual challenges and to observe relationships between stakeholders.
Survey 40 LM and NW To collect data on patterns and motivations for
individual behavior change.
Review of quantitative data records
Monthly monitoring
data
Annual survey data
Barrier analysis data
To collect information on the impact of the project on the identified health indicators.
18.4. List of people consulted
ID Date Individual interviews Location Area
ID 1 Chris Blackham, SPUK Head of Programmes Skype
ID 2
Keren Massey, former Project Manager Skype
ID 3
Judy Samuels, former Institutional Funding Manager SPUK Skype
ID 4 27/01/2015 Dan Enarson, SP Uganda Country Director Kampala
ID 5 27/01/2015 Bettina Baesch, SP Uganda Deputy Country Director Kampala
ID 6 27/01/2015 Kristen Gunther, KIMCH Project Manager Kampala
ID 7 27/01/2015 Woubishet Mengistu, Programme Manager for Health and Social Protection Kampala
ID 8 02/02/2015 Lokiru Joseph Emmy, M&E officer Moroto
ID 9 28/01/2015 Health Supervisor Kangole sub-office SA1
ID 10 28/01/2015 Health Promoter Kangole sub-office SA1
ID 11 31/01/2015 Health Supervisor Matany sub-office SA4
ID 12 31/01/2015 Health Promoter Matany sub-office SA4
ID 13 05/02/2015 Health Supervisor Iriiri sub-office SA5
ID 14 05/02/2015 Health Promoter Iriiri sub-office SA5
ID 15 02/02/2015 DHO and MCH assistant officer Napak
www.jigsawconsult.com
Samaritan’s Purse Karamoja MCH evaluation
Page 50
ID 16 02/02/2015 Community Development Officer Napak
ID 17 29/01/2015 Sub-county chief Lopeei sub-county office SA1
ID 18 30/01/2015 Assistant Community Development Officer Lotome sub-county office SA2
ID 19 02/02/2015 Health Programme Manager and two advisors, Concern Moroto office
ID 20 02/02/2015 International Rescue Committee Moroto office
ID 21 29/01/2015 Midwife Lopeei Health Centre SA1
ID 22 29/01/2015 Nurse Lopeei Health Centre SA1
ID 23 30/01/2015 Health Assistant Lotome Health Centre SA2
ID 24 30/01/2015 Midwife Lotome Health Centre SA2
ID 25 05/02/2015 Midwife and Clinical Health Officer Iriiri Health Clinic SA5
ID 26 04/02/2015 Health worker representative Matany hospital SA3
ID 27 04/02/2015 VHT Matany Trading Centre SA3
ID Date Focus group Location Area M F
ID 28 28/01/2015 CG Observation Naligoi village, Ngoleriet SA1 0 10
ID 29 28/01/2015 Leader Mothers Naligoi village, Ngoleriet SA1 0 10
ID 30 29/01/2015 Leader Mothers Lotutuwa village, Lopeei SA1 0 10
ID 31 29/01/2015 Neighbour Women Lotutuwa village, Lopeei SA1 0 13
ID 32 30/01/2015 Leader Mothers
Natapar Apalemu village,
Lotome SA2 0 15
ID 33 30/01/2015 Neighbour Women Natapar Apalemu village, Lotome SA2 0 11
ID 34 31/01/2015 CG Observation Nakiceleet village, Lokopo SA4 0 10
ID 35 31/01/2015 Leader Mothers Nakiceleet village, Lokopo SA4 0 10
ID 36 31/01/2015 Leader Mothers Lojojora village, Lokopo SA4 0 10
ID 37 03/02/2015 Neighbour Women Nakayot village, Iriiri SA5 0 10
ID 38 03/02/2015 Leader Mothers Nakayot village, Iriiri SA5 0 12
ID 39 04/02/2015 Leader Mothers Kanaura village, Matany SA3 0 10
ID 40 04/02/2015 CG Observation Kanaura village, Matany SA3 0 10
ID 41 04/02/2015 Neighbour Women Kanaura village, Matany SA3 0 11
ID 42 04/02/2015 Men Kanaura village, Matany SA3 5 0
ID 43 04/02/2015 Leader Mothers Matany TC SA3 0 12
ID 44 04/02/2015 Men Matany TC SA3 9 0
ID 45 05/02/2015 Leader Mothers Iriiri SA5 0 10
ID 46 05/02/2015 Neighbour Women Iriiri SA5 0 17
ID 47 05/02/2015 Men + Community Leaders Iriiri SA5 12 0
www.jigsawconsult.com
Samaritan’s Purse Karamoja MCH evaluation
Page 51
18.5. List of supporting documentary information
List of background documents including in the document review and of quantitative data
