“Infant and Young hild Feeding (IY F) Programme in Syrian ... · LRFP-2015-9121419# #IYCF...

UNITED NATIONS CHILDREN’S FUND (UNICEF) - JORDAN

“Infant and Young Child Feeding (IYCF) Programme in Syrian Refugee Camps and Host Communities in Jordan” Evaluation

Final Report November 15, 2016

Presented by: Dolly Bassil (Team Leader) Linda Shaker Berbari, Dr. Ali Shalak, Dr. Hala Ghattas, Dr. Omar Obeid, Dr. Nabil Kronfol (Team members)

LRFP-2015-9121419# #IYCF Programme Evaluation – Final Report – November 7, 2016 2


TABLE OF CONTENTS

List of Acronyms ............................................................................................................... 7

Executive Summary .......................................................................................................... 8

Background ............................................................................................................ 14

Object of Evaluation – The Project .......................................................................... 18 2.1. Infant and Young Child Feeding Programme (IYCF-P) .............................................................. 18 2.2. Programme Theory of Change ................................................................................................. 23

3. Purpose, Objectives, Scope, Evaluation Criteria, and Use of Evaluation ................... 25 3.1. Purpose .................................................................................................................................... 25 3.2. Objectives ................................................................................................................................. 25 3.3. Scope ........................................................................................................................................ 26 3.4. Evaluation Criteria and Questions ........................................................................................... 26

4. Evaluation Methodology ........................................................................................ 27 4.1. Evaluation Approach ................................................................................................................ 27 4.2. Data Collection Method and Tools........................................................................................... 30 4.3. Evaluation Process ................................................................................................................... 34 4.4. Strategy for Ensuring Data Quality for All Deliverables ........................................................... 35 4.5. Strategy for Ensuring an Ethical Approach .............................................................................. 36

5. Limitations ............................................................................................................. 37

6. Evaluation Matrix ................................................................................................... 38 6.1. Team Members and Their Responsibilities .............................................................................. 38

7. Results and Findings ............................................................................................... 38 7.1. Results and findings based on OECD criteria ........................................................................... 38

Relevance .......................................................................................................................................... 38 1) Whether the IYCF Programme and its expected results are aligned with the MOH National Strategy 2015-2017 and relevant in the context of Health Sector Reform in Jordan ....................... 38 2) Whether the IYCF Programme and its expected results are aligned with the regional and national plans for the Syria response (namely National Resilience Plan 2014-2016, Jordan Response Plan 2015, Regional Response Plan and Regional Resilience and Response Plan). .......... 42 3) Whether UNICEF’s support to the IYCF Programme is in line with its organizational mandate. 43 4) Whether the IYCF Programme and its expected results are aligned with international policies and guidance related to IYCF-E ......................................................................................................... 44 5) To what extent is the Programme relevant to needs and gaps identified through assessments and baseline evaluations? ................................................................................................................ 46 6) Whether and to what extent the Programme design, including activities and expected outputs, is consistent with the intended results at the outcome and impact levels of IYCF/IYCF+? 48 7) Whether and to what extent is the Programme design appropriate and accepted by the target community? ........................................................................................................................... 49 8) To what extent the design took into account gender issues? .................................................. 51 9) How is the Programme different from other approaches inside and outside of Jordan? ........ 53 10) Is the Programme design able to reach the most vulnerable, such as refugee mothers, informal and formal sector working mothers, to promote exclusive breastfeeding practices? ....... 56 Effectiveness ..................................................................................................................................... 58


11) To what extent the Programme demonstrated expected results at all levels (i.e. inputs/outputs level indicators, knowledge, behaviour change and coverage of Program)? .......... 58 12) Did the Programme identify and reach out to the most vulnerable? What was the difference in reach between those in camps and those in host communities, especially in the hard-to-reach areas? Whether and to what extent special attention has been made to reaching out to those beneficiaries who are hard-to-reach? Any difference between outreach (one-to-one) versus caravan (fixed-site provision of health education)? ......................................................................... 60 13) Whether and how has the integrated approach, IYCF+ contributed to the expected results? 61 14) Whether and to what extent the technical support was provided to the implementation team to address issues and ensure achievement of results? ..................................................................... 62 15) How has the quality of Programme been ensured, especially with respect to the Global Operational Guidance on Infant Feeding in Emergencies and other standards? ............................. 63 16) What are the major factors (internal and external, any issues related to gender) influencing the achievement or non-achievement of the results? ...................................................................... 66 17) Were there components of the Programme that worked well or did not work well? Why so, and why not? .................................................................................................................................... 66 18) What are the areas of strengths and weaknesses? Are there other approaches that will help overcome the weaknesses, if any? ................................................................................................... 66 Efficiency ........................................................................................................................................... 70 19) To what extent did the actual or expected results justify the costs incurred (considering the difference of approach and Programme design for camps and host communities)?....................... 70 20) Have the resources been used and has the Programme been implemented as efficiently as possible? (Resources here refer to those allocated for this Programme, as well as any other external resources to enhance efficiency such as referral to other services) ................................... 70 21) What is the cost of the response per unit of aggregation as compared to the cost being incurred by other IYCF Programmes implemented in Jordan? ......................................................... 70 22) Whether and how has the integrated IYCF+ approach influenced the efficiency of the Programme? ..................................................................................................................................... 87 23) Did UNICEF and partners explore the possibility of integrating IYCF in other relevant interventions (e.g. immunization referral or back to school campaigns)? Are there areas of improvement? ................................................................................................................................... 87 24) Whether and to what extent the technical and other human resource capacities have been utilized and were appropriate to achieve results?............................................................................ 88 25) What have been the roles and contribution of volunteers in the results? ............................... 90 26) Were achievements made on time? If not, why? ..................................................................... 91 27) How have the IYCF Programme activities been coordinated with different stakeholders and their similar programmes, such as other UN agencies, INGOs, NGOs, CBOs, and Ministries to achieve overall objective? ................................................................................................................. 92 Likelihood of Impact ......................................................................................................................... 93 28) What are the key short term and long term (and lasting) changes produced by the Programme (positive or negative, intended or unintended) as perceived by the stakeholders (PLW, front line workers, government, implementing partners including CBOs)? What are the key factors behind these changes? ..................................................................................................................... 93 29) Whether and to what extent the desired changes have been experienced by PLWs and children as a result of this Programme (as perceived by end users). Did the integration of other components (as IYCF+) make any difference in those changes? ...................................................... 94 Sustainability .................................................................................................................................... 97 30) Whether measures have been in place to ensure the sustainability of achievements after the withdrawal of external support or in the context of the anticipated decrease in donor funding (taking into consideration the specificity of the emergency context)? ............................................. 97


31) Whether and how the Programme identified and built on existing national, local, civil society, government capacities? What new capacities within services or communities have been established or restored that can contribute to sustainability? " ...................................................... 97 32) Extent to which IYCF has been integrated within MOH institutions and whether there are plans to institutionalize the Programme .......................................................................................... 97 33) What are the possibilities for scale-up or replication? ............................................................. 98

7.2. Results and Findings Based on the Theory of Change ........................................................... 103

8. Conclusions .......................................................................................................... 107 8.1. Relevance ............................................................................................................................... 107 8.2. Effectiveness .......................................................................................................................... 108 8.3. Efficiency ................................................................................................................................ 109 8.4. Likelihood of Impact ............................................................................................................... 110 8.5. Sustainability .......................................................................................................................... 110

9. Lessons Learned ................................................................................................... 111

10. Recommendations ............................................................................................... 113

11. Annexes ............................................................................................................... 122 11.1. Appendix A: IYCF Programme Target Areas and Number of Beneficiaries ............................ 122 11.2. Appendix B: Summary of PCAs extensions and amendments ............................................... 122 11.3. Appendix C: List of IYCF Programme Partners ....................................................................... 122 11.4. Appendix D: Evaluation Matrix .............................................................................................. 122 11.5. Appendix E: Key Informant Interview Questions ................................................................... 122 11.6. Appendix F: List of IYCF Programme Key Informants ............................................................. 122 11.7. Appendix G: Topic Guide for Focus Group Discussion with Service Providers ...................... 122 11.8. Appendix H: Topic Guide for Focus Group Discussion with End-Users .................................. 122 11.9. Appendix I: KAP Assessment Questionnaire .......................................................................... 122 11.10. Appendix J: KAP Assessment Results ..................................................................................... 122 11.11. Appendix K: Timeline Events .................................................................................................. 122 11.12. Appendix L: Summary of Skills and Capacity of Evaluation Team .......................................... 122 11.13. Appendix M: Quotes .............................................................................................................. 122 11.14. Appendix N: Table Matching National Strategy with IYCF Programme ................................. 122 11.15. Appendix O: Programme Log Frames Across Implementation Period .................................. 122 11.16. Appendix P: Mapping of nutrition services ............................................................................ 122 11.17. Appendix Q: Key Beneficiaries, Staffing and Non-Staffing Costs for Camps and Host Communities ....................................................................................................................................... 123 11.18. Appendix R: Unit Costs (JODvisit) for Camps and Host Communities ................................... 123 11.19. Appendix S: Key Beneficiaries and Staff Costs per Year of Implementation and Camp ........ 123 11.20. Appendix T: Unit Education and Counselling Unit Costs (JODvisit) by Target and Year of Implementation .................................................................................................................................. 123 11.21. Appendix U: Proportions of IYCF and non-IYCF cost by non-staff cost item ......................... 123 11.22. Appendix V: Organizational chart for IYCF Programme and number of staff ........................ 123 11.23. Appendix W: Theory of Change ............................................................................................. 123 11.24. Appendix X: Case Study Report .............................................................................................. 123


LIST OF FIGURES Figure 1 - The Programme management structure also found in Appendix V ........................ 19 Figure 2 - Schematic presentation of the IYCF-P Theory of Change – the results chain ......... 24 Figure 3 - Multiple case study design with embedded units of analysis ................................. 28 Figure 4 - Budget outlays over the period of the Programme evaluation .............................. 71 Figure 5 - Total number of PLW’s counselled (one-on-one) over the duration of the project72 Figure 6 - Programme structure at the field level ................................................................... 89 Figure 7 - Structure of community IYCF support in Indonesia................................................. 90 Figure 8 - Schematic presentation of the IYCF-P Theory of Change – the results chain ....... 104

LIST OF TABLES Table 1 - Prevalence of early initiation of breastfeeding, exclusive breastfeeding under 6

months, introduction of solid food after 6 months, and continuation of breastfeeding at 2 years among Syrians and Jordanians since 2006 .......................................................... 16

Table 2 - IYCF Programme activities per type, date of initiation, and target area .................. 22 Table 3 - Application of evaluation criteria.............................................................................. 27 Table 4 - Targeted geographic areas ........................................................................................ 29 Table 5 - Focus group participants with services providers .................................................... 31 Table 6 - Focus group participants with end users .................................................................. 33 Table 7 - Jordan National Strategy indicators .......................................................................... 39 Table 8 - Summary of international initiatives ........................................................................ 54 Table 9 - Number of volunteers working in each target area ................................................. 73 Table 10 - Key beneficiaries, staffing and non-staffing costs for camps and host communities

.......................................................................................................................................... 77 Table 11 - Unit costs (JOD/visit) for camps and host communities ......................................... 79 Table 12 - Key beneficiaries and staff costs per year of implementation and camp .............. 80 Table 13 - Unit education and counselling unit costs (JOD/visit) by target and year of

implementation ............................................................................................................... 81 Table 14 - Proportions of IYCF and non-IYCF cost by non-staff cost item ............................... 85


List of Acronyms BFHI Baby Friendly Hospital Initiative BMS Breast-Milk Substitute CBO Community Based Organization DAC Development Assistance Committee EPI Expanded Programme on Immunization ERG Evaluation Reference Group FGD Focus Group Discussion IFRC International Federation of Red Cross IMC International Medical Corps IOM International Organization for Migration IYCF Infant and Young Child Feeding JHAS Jordan Health Aid Society JPFHS Jordan Population and Family Health Survey KAP Knowledge, Attitude and Practice KII Key Informant Interview MoH Ministry of Health OECD Organization for Economic Co-operation and Development ORS Oral Rehydration Solution PLW Pregnant and Lactating Women RSTC Rabaa Al-Sarhan Transit Centre SC-J Save the Children Jordan ToC Theory of Change TT Tetanus Toxoid UNEG United Nations Evaluation Group UNFPA United Nations Population Fund UNHCR United Nations High Commissioner for Refugees


Executive Summary UNICEF Jordan commissioned an evaluation to independently assess the Infant and Young Child Feeding (IYCF) Programme in Syrian Refugee Camps and host communities and conduct a systematic and impartial examination of the relevance, effectiveness, efficiency, sustainability, and likelihood of impact of the Programme. This evaluation aims at informing decision-making for future programming, in the context of Jordan transitioning from humanitarian response to the Syria refugee crisis to resilience building. It has been increasingly recognized that systems strengthening is essential to build the resilience of national institutions and communities against the protracted regional crisis as well as any future acute shocks. Primary users of the evaluation are management and programme staff of UNICEF Jordan Country Office, the Government of Jordan’s Ministry of Health (MoH), other institutions and partners working in the nutrition sector in Jordan, regionally and globally. Evaluation Objective Optimal infant and young child feeding is crucial for child survival and health, especially among those living in dire conditions. Poor IYCF practices were evident among Syrians and Jordanians prior to the Syria Crisis. The protracted Syrian crisis and its impact has further exacerbated maternal and child health in general. In 2012, UNICEF Jordan initiated the Infant and Young Child Feeding (IYCF) programme in partnership with Save the Children Jordan (SCJ). This was initially an emergency response in Zaatari refugee camp, with an annual target of 40,000 pregnant and lactating Syrian women (PLWs) and children aged 0-59 months. It has incrementally evolved as a national programme covering urban centres and towns (host communities) outside refugee camps. In August 2013, the IYCF programme expanded to host communities, targeting an additional 40,000 PLWs and 65,000 children under five years of age from Syrian and Jordanian populations. Activities aimed at addressing misconceptions related to breastfeeding by providing one-to-one counselling sessions including home visits, group education sessions, maternal and child health and nutrition referral services (such as routine and supplementary immunization, antenatal and postnatal care), and providing regular maternal nutrition advice through follow-up visits with PLWs. The programme has diversified strategic partnerships and developed the capacity of Government and local partners, such as the MoH, Ministry of Social Development, Community-Based Organisations, private hospitals and companies. This approach has been important under the protracted nature of the Syria Crisis and in the interest of developing resilience in Jordan. The programme has trained government health care providers (nurses and midwives), SCJ staff and IYCF educators selected from Syrian refugees on optimal IYCF practices, counselling and education methods, and the code of marketing Breast Milk Substitutes.


Evaluation Methodology and Analysis Considering the IYCF programme design, the objectives and scope of the evaluation, as well as a lack of baseline data, the evaluation adopted a qualitative methodology. Central to this was the multiple case study design with embedded units of analysis. The evaluation team collected information about the programme from key stakeholders, service providers, and end-users (pregnant and lactating mothers). Since the programme has expanded both at geographic and activity levels, the multiple cases encompassed units of analysis in view of different geographic areas, different durations of implementation, and different contexts. Data collection methods were identified to respond adequately to evaluation questions, developed as per the OECD DAC Criteria for Evaluation. These included: (1) desk review; (2) semi-structured interviews with 25 key informants; (3) five focus groups discussions with 40 service providers; (4) 10 focus group discussions with 133 end-users benefiting from the Programme at different stages; and (5) e-questionnaire targeting beneficiaries of capacity development activities. Key informant interviews were also conducted with six non-beneficiaries. Data analysis consisted of analysing quantitative data from the questionnaires using frequencies, and a thematic analysis was conducted on the four types of qualitative data. Throughout the analysis, the evaluation team also examined ways in which the programme is progressing against the Theory of Change, developed for the purpose of this evaluation at the inception phase. Main Findings and Conclusions The evaluation found that the IYCF Programme is relevant while it can be further enhanced through a stronger alignment to national priorities and systems. The Programme is partially effective in a sense that it has achieved results at activity and output levels, yet fell short of achieving outcome level results. The programme design must explicitly consider the roles of duty bearers, such as men and fathers, and strengthen approaches for capacity development of service providers, influencing policies and addressing social norms. The evaluation also found that the Programme is efficient. However, integrated approaches and engaging volunteers can further enhance not only efficiency but also the effectiveness. Likelihood of impact was noted in term of changes at individual level. Changes at institutional level or nutritional status of children could not be determined while assessment against the Theory of Change indicated potential of contribution towards impact. Sustainability of the programme in its current modality and approach is questionable, and there must be an articulated and phased plan for integration and institutionalization of the programme, capitalizing on the achievements and strengths.


Specific findings against the evaluation criteria are as follows: Relevance The IYCF Programme is broadly aligned with national priorities that cover a wide spectrum of services under primary and secondary health. Although stated as a priority by the MoH, the evaluation team could not draw clear conclusions on their readiness to exercise ownership and take the leadership in implementation. The programme can be further tailored to parallel the emphasis placed by the Government on quality of care and institutionalization of activities. At the same time, the Programme responds to the Government of Jordan’s Response Plan for the Syria Crisis and is considered a high priority by UNICEF with emphasis on improved sustainability and efficiency. The Programme is also based on international IYCF guidelines and comprehensive in nature, and has been adequately contextualized in terms of language and culture to the Jordanian setting. The Programme responds to knowledge and practice gaps. There is a clear indication that the Programme is well accepted by and appropriate to PLWs. Therefore, it is regarded as well received through its adopted mode of operation, approach and design. The design is similar to a number of other international initiatives. The Programme targets the most vulnerable groups in refugee camp settings while there is no evidence to show that the demographic profile of those targeted in host communities represent the most vulnerable. In its current mode, gender issues are not explicitly addressed, i.e. the role of men and fathers in supporting adequate IYCF. There was an indication that men were welcomed during the sessions in case they were present, and this was validated in the attendance sheets. Yet this practice started recently. The Programme was designed initially in response to the emergency situation resulting from the Syrian refugee crisis. There was no baseline established nor clear expansion plan as the time progressed and context evolved. Instead of scaling-up based on needs assessments and clear expansion plan, decisions to modify service package and expand the Programme came often from UNICEF and based on consultations with the partner organisation, it was implemented . Effectiveness The Programme was able to implement almost all planned activities, yet fell short to achieve in full and to the desired extent all of its outcomes. Results were more evident at the level of increasing awareness of the target groups and provision of services to the PLWs. Achievements were less obvious in terms of increased knowledge and capacity of service providers, and inducing a major change in the policy framework and social norms. Gaps were also identified in the reporting of some indicators and the matching of indicators to actual activities. The integration of additional activities within the IYCF+ proved to be useful, and it was apparent that providing a comprehensive package is of added value to beneficiaries.


Evidence shows that UNICEF and SC-J provided sufficient technical assistance and support to programme staff. The programme staff has accumulated technical capacity through extensive and continuous training, as well as on communication skills. A number of strengths were identified, namely: (1) Outreach efforts and the presence of local or Syrian community mobilizers who are mothers; (2) Engaging men, though it started much later in the programme phase, was seen as a positive approach to facilitating the necessary behaviour change among mothers, especially when men play the role of enablers; (3) The service provision organization’s name, and the connotation its name gives in Arabic (that it “saves children”) contributed to the perceived value of the Programme; (4) The one-on-one counselling and follow up provided to mothers which was seen as most effective in changing behaviour; and (5) The contribution to enforcing the Code and the inadequate support given to artificial feeding. Efficiency Evidence shows that the Programme was relatively cost-efficient as compared to similar programmes in other countries. The unit costs for counselling are on the lower side of counselling costs referred to in the available literature, and tend to corroborate the cost-efficiency of the intervention. The financial management of the Programme lacks some clarity and proper documentation of disbursements as per budget line items and activities. This could have strengthened the cost analysis. Opportunities exist for institutionalization of activities within the services of the MoH. Nevertheless, it was noted that the Programme is being implemented separately from existing services despite the claimed efforts for integration. The evaluation team noted that IYCF+ could be considered as a cost effective intervention that contributed to increasing access to services with minor cost modifications. The evaluation found that there is added value to engage volunteers, however, it is premature to draw on the extent to which their contributions affected efficiency. The Programme has had delays in implementation, thus the implementation rate did not match the plans. This was partly due to waiting time for approvals from the Government, which required several extensions and amendments. Still, the Programme was able to function within planned budgets. Likelihood of Impact A perceived change was noted in knowledge, capacity, and practices among service providers and end-users such as pregnant and lactating women. Other perceived changes related to positive changes in social interaction and cost savings by practicing breastfeeding. Most changes were attributed to the counselling, follow-up and education activities that were implemented by the Programme. The programme might have contributed to some policy


changes, including the endorsement by parliament of the code for marketing breast milk substitutes. The assessment of the Programme against the theory of change also indicates potential of the Programme contributing towards impact. With this said, the Programme’s contribution towards improved nutrition status of children cannot be confirmed until quantitative data is analysed1. It should also be noted that achievement at impact level will be dependent upon the Government’s willingness to take ownership and responsibilities around the IYCF Programme. Sustainability Although sustainability was taken into consideration in the design of the Programme, there is no evidence of a clear and gradual exit plan. As the Programme moved forward, some efforts to build partners’ capacity were made, but these fell short of what is needed to ensure sustainable quality services. The MoH and other partners indicated their willingness to integrate IYCF-P, yet the preparedness level varied across structures. Some level of knowledge related to IYCF has been transferred to MoH health care providers. At the same time, the evaluation indicates that trained staff would not be able to provide adequate and quality advice to mothers at this point in time. Other barriers included: the lack of current prioritization of IYCF indicating inappropriate timing for phasing out the UNICEF supported programme implemented by SCJ; limited technical capacity on the part of the MoH; lack of funding; and the power of businesses marketing infant formula. For integration of the Programme within MoH systems, a capacity assessment will be required to review the MoH’s current structure, related systems and resources. In camp settings, the Programme is being implemented in coordination with other partners, and services are being provided separately from other health activities. Efforts have been made to incorporate newborn care including vaccination and hygiene promotion within the services provided by the Programme, which proved to be useful. In camps, the Programme can be also integrated into existing early childhood development activities. Community members can be more effectively engaged to enhance the sustainability of the Programme. Recommendations Recommendations based on this evaluation are divided into short term (more operational in nature) and long term (more strategic in nature). These are also stratified by context, e.g. camp settings and the host communities.

1- Short-term recommendation in camps (#1) Review the Programme scope for an integrated and comprehensive Primary Health Care Programme covering IYCF, Early Childhood Development, newborn and child care, antenatal and postnatal care, and mental health. Consider focusing on the first 1,000 days in a child’s life with essential services that would ensure a healthy start.

1 The evaluation team is analyzing the data from the Interagency Nutrition Survey conducted in 2016.


2- Short-term recommendation in camps (#2) Enhance the Programme design to influence outcome-level results such as: 1) providing mother-to-mother support that contributes to sustainability of the Programme; 2) Incorporating a “men’s component” into the Programme where men will be targeted separately; 3) conducting regular capacity building and sensitizing activities targeting camp service providers including NGOs.

3- Short-term recommendation in camps and host communities Address gaps in monitoring, data and financial management. The current system does not capture all the necessary information to track achievements against the stated results. The Theory of Change developed for this evaluation can be used to refine the results framework and monitoring system.

4- Short-term recommendation in host communities Develop an exit strategy with concrete and phased actions, including capacity building, coaching and mentoring of Syrian volunteers and partner staff. SC-J can conduct capacity assessment to identify existing centres with the necessary and adequate capacity. The exit strategy should be accompanied with the recommended sustainability measures.

5- Medium-term recommendation in host communities (National/strategic) Conduct a thorough assessment of bottlenecks in existing structures and health systems within MoH to improve effective IYCF service coverage. Accordingly devise a plan to strengthen IYCF within existing Primary Health Care system platforms.

6- Long-term recommendation in host communities (National/Strategic) (#1) Integrate Infant and Young Child Feeding into the National Health Strategy and support the implementation. This evaluation provides knowledge and learning for strengthening the national strategy and its implementation.

7- Long-term recommendation in host communities (National/Strategic) (#2) Develop a plan for Infant and Young Child Feeding activities to be part of an integrated package of Primary Health Care services for children aged 0-2 years of age, encompassing newborn, child health, early childhood development and nutrition.


Background Jordan is a Middle Eastern country with a population of around 9.5 million, of which around 6.6 million are Jordanians and 2.9 million are non-Jordanians. According to the Jordan population and housing census of 20152, Syrians constitute about 13.2 per cent of the overall population. A report by the United Nations High Commissioner for Refugees (UNHCR) published in August 2016 states that while the political and civil unrest in Syria completed its fifth year in March 20163, the Government of Jordan estimates hosting up to 1.3 million Syrians4, including around 656,198 registered refugees. While 79% of refugees had settled in urban host communities within Amman and the northern governorates, the remaining poured into refugee camps; mostly in Zaatari and Azraq camps. Of the total refugee population, children between 0 – 4 years of age attribute to 8.3% and 7.8% of males and females, respectively5. The conflict in Syria has severely strained economic and social systems and institutions in Jordan. In 2015, a humanitarian and development planning process led by the Jordanian Government has resulted in the Jordanian Response Plan (JRP), which aims to coordinate and mainstream responses from humanitarian and development actors into two main categories; refugee response and resilience. The Jordanian Response Plan (JRP) has replaced the United Nations-led Regional Response Plan (RRP) and has been further developed into a biennial plan (2016 – 2018)6. The JRP’s health sector objective includes providing equitable access, uptake and quality of primary, secondary, and tertiary healthcare for Syrian women, girls, men, and boys, as well as for vulnerable Jordanian populations in highly impacted areas, indicated through healthcare services catered at maternal and child health within all levels of service provision7. In addition, with an overall approach aiming at strengthening resilience within institutions, systems and communities, the Jordanian National Resilience Plan (NRP) 2014 – 2016, which was later (2015) replaced by the JRP, indicates a capacity building approach of MOH Medical and Managerial staff on the integrated nutrition Infant and Young Child Feeding (IYCF) among proposed interventions within the health sector8. Optimal infant and young child feeding is crucial for children’s survival and health. It has been recently confirmed by The Lancet series that more than 800,000 infant deaths could be prevented with optimal feeding including breastfeeding 9 . During emergencies, ensuring

2 http://www.unicef.org/jordan/media_10894.htm, accessed on August 5, 2016 3 http://www.unocha.org/syria, accessed on August 5, 2016 4 http://www.unicef.org/jordan/media_10894.htm, accessed on August 5, 2016 5 UNHCR Syria Regional Refugee Response: http://data.unhcr.org/syrianrefugees/country.php?id=107, accessed on August 27, 2016 6 http://www.jrpsc.org, accessed on August 27, 2016 7 http://static1.squarespace.com/static/522c2552e4b0d3c39ccd1e00/t/56b9abe107eaa0afdcb35f02/1455008783181/JRP%2B2016-2018%2BFull%2B160209.pdf 8 http://faolex.fao.org/docs/pdf/jor145353.pdf 9 The lancet series – January 2016

http://www.unicef.org/jordan/media_10894.htm

http://www.unocha.org/syria

http://www.unicef.org/jordan/media_10894.htm

http://data.unhcr.org/syrianrefugees/country.php?id=107

http://www.jrpsc.org/

http://faolex.fao.org/docs/pdf/jor145353.pdf


optimal infant and young child feeding becomes even more important given the dire living conditions of families The extended war situation in Syria has had negative impact on maternal and child health in general and on nutrition in particular. The already poor infant and young child feeding (IYCF) practices that existed before the war within the region in general and in Syria and Jordan in particular (Nasreddine et al., 2012)10 have been also aggravated. IYCF indicators have differed among both Syrians and Jordanians before and after the crisis, knowing that all reported numbers are low with rates of Jordanians showing poorer practices than Syrians as depicted in table 1. In Syria, before the crisis, the percentages of early initiation of breastfeeding (within the first hour of life) and exclusive breastfeeding for 6 months were 32.4% and 28.7% respectively (MICS11). Rates were higher as reported by UNICEF, compiling data from 2008 to 201212 where the prevalence of early initiation increased was 45.5% and exclusive breastfeeding 42.6%. In Jordan, IYCF indicators among Jordanians were shown to be even lower. Reports 131415 showed that early initiation varied between 34.6% and 38.8% and the percentage of exclusive breastfeeding between 21.8% and 22.7%. After the Syria crisis, two assessments of IYCF practices among Syrian refugees in Jordan took place in 2012 (pre-implementation) and 2014 (during-implementation). Data showed that in 201216, 36.8% and 26.7% of the Syrian refugee mothers living in Zaatari camp and within the host community, respectively, do not introduce solid food to their babies after 6 months. The assessment performed in 2014 on the same populations17, revealed percentages of 57% and 46.4% for early initiation of breastfeeding and exclusive breastfeeding among Syrian refugees residing in Zaatari camp and 48.7% and 36%, among Syrian refugees living within the host community, respectively. Table 1 is a compilation of existing studies showing IYCF practices amongst Syrians and Jordanians pre and post crisis. It summarizes four IYCF indicators since 2006.

10 L. Nasreddine, M.N. Zeidan, F. Naja, N. Hwalla. Complementary feeding in the MENA region: Practices and challenges. Nutrition, Metabolism and Cardiovascular Disease. 2012 11 Multiple Indicator Cluster Survey (MICS), Syrian Arab Republic, 2006 12 UNICEF, Syria, Statistics, available at http://www.unicef.org/infobycountry/syria_statistics.html, accessed on October

12, 2016 13 World Health Organization. Indicators for assessing infant and young child feeding practices: part 1: definitions: conclusions of a consensus meeting held 6-8 November 2007 in Washington DC, USA. 14 UNICEF, Jordan, Statistics, available at: http://www.unicef.org/infobycountry/jordan_statistics.html, accessed on October 12, 2016 15 IBFAN, 2014, REPORT ON THE SITUATION OF INFANT AND YOUNG CHILD FEEDING IN JORDAN 16 Interagency Nutrition Assessment Syrian Refugees in Jordan Host Communities and Za’atari Camp, 2013 17 Interagency Nutrition Survey on Syrian Refugees in Jordan, 2014

http://www.unicef.org/infobycountry/syria_statistics.html

http://www.unicef.org/infobycountry/jordan_statistics.html


Table 1 - Prevalence of early initiation of breastfeeding, exclusive breastfeeding under 6 months, introduction of solid food after 6 months, and continuation of breastfeeding at 2 years among Syrians and Jordanians since 2006

Name of the study/report

Year Population Breastfeeding Indicator

Early initiation of breastfeeding

Exclusive breastfeeding under 6 months

Introduction of solid food after 6 months

Continuation of breastfeeding at 2 years

In Syria

Pre-Crisis MICS18† 2006 Syrians 32.4% 28.7% 36.5% 16.3% UNICEF stat19‡ 2008-

2012 Syrians 45.5% 42.6% - 24.9%

In Jordan

Pre-Crisis WHO20‡ 2007 Jordanians 34.6% 21.8% - 10.9% UNICEF stat21¥ 2008-

2012 Jordanians 38.8% 22.7% 83.5% 12.9%

IBFAN, 201422‡ 2008-2012

Jordanians 38.8% 22.7% 83.5% 12.9%

The Population and Family Health Survey23†

2012 Jordanians 19% 23% 91% -

UNICEF Summary Data24‡

2009-2013

Jordanians 19% 23% 92% 13%

During-Crisis Inter-Agency Nutrition Assessment25†

2012 Syrian Refugees in Zaatari camp

- - 36.8%* 9.5%#

Syrian Refugees in host community

- - 26.7%* 19.8%#

Inter-Agency Nutrition Assessment Report26†

2014 Syrian Refugees in Zaatari camp

57% 46.4% 42.1% 15%

Syrian Refugees in host community

48.7% 36% 36.4% 23.8%

18 Multiple Indicator Cluster Survey (MICS), Syrian Arab Republic, 2006 19 UNICEF, Syria, Statistics, available at http://www.unicef.org/infobycountry/syria_statistics.html, accessed Oct 12, 2016 20 World Health Organization. Indicators for assessing infant and young child feeding practices: part 1: definitions: conclusions of a consensus meeting held 6-8 November 2007 in Washington DC, USA. 21 UNICEF, Jordan, Statistics, available at: http://www.unicef.org/infobycountry/jordan_statistics.html, accessed on October 12, 2016 22 IBFAN, 2014, REPORT ON THE SITUATION OF INFANT AND YOUNG CHILD FEEDING IN JORDAN 23 Department of Statistics [Jordan] and ICF International. 2013. Jordan Population and Family Health Survey 2012. Calverton, Maryland, USA: Department of Statistics and ICF International 24 UNICEF 2015, Statistical Tables, Economic and social statistics on the countries and areas of the world, with particular reference to children’s well-being. 25 Interagency Nutrition Assessment Syrian Refugees in Jordan Host Communities and Za’atari Camp, 2013 26 Interagency Nutrition Survey on Syrian Refugees in Jordan, 2014

http://www.unicef.org/infobycountry/syria_statistics.html

http://www.unicef.org/infobycountry/jordan_statistics.html


The Jordan Family Health Survey (2012)27 emphasized the influence of socio-economic and cultural factors in Jordan on supporting IYCF. Results from the survey also revealed lack of a monitoring system for the implementation of the Code of Marketing of Breast-milk Substitutes, which had negative influences across Jordan. These figures pinpoint the existence of poor preventive health and nutrition behaviours associated mainly with IYCF practices, in both refugee and host populations in Jordan. As previously described, the prevalence of exclusive breastfeeding (for infants under 6 months of age) and other IYCF practices in Jordan were lower than in pre-war Syria. After the conflict, Syrian refugees living in camps maintained better IYCF practices (table 1) as compared to those residing within host communities. With time, in 2014, the percentages of Syrian refugees with proper IYCF practices further decreased. Infant and young child feeding practices need to be protected during emergencies, since infants who are not breastfed are at higher risk of morbidity and mortality than breastfed children (Gribble & Berry, 2011)28. At the same time, poor complementary feeding has been shown to be a primary cause for malnutrition29. In emergency situations, adequate and safe IYCF practices are less probable than under stable conditions. Bottle-feeding occurs with increased risks associated with poor water quality, inability to sterilize the bottle/nipple, artificial ingredients in breast-milk substitutes (BMS), and lack of availability of BMS. This situation can lead to poor nutrition and health conditions among infants dependent on BMS. Infant and young child feeding in emergencies (IYCF-E) aims at supporting infant and young child feeding in emergency situations and refugee settings, where breastfeeding and complementary feeding are jeopardised by existing suboptimal conditions. IYCF-E Programming responds to the immediate needs of infants and young children who are most vulnerable and includes frontline support. IYCF-E is considered a life-saving intervention and focuses on responding to IYCF needs in emergencies and difficult situations. It has been shown that for an IYCF intervention to be implemented effectively, it is best for IYCF to be mainstreamed within other sectors targeting mothers and infants. Several policies and guidance have been put in place to uphold optimal infant and young child feeding during emergencies. These include the WHO Code of Marketing of Breast-milk Substitute and the Operational Guidance on Infant and Young Child Feeding that aim to support and protect infant and young child feeding. In view of the prevailing situation in Jordan as an impact of the Syrian crisis, the IYCF Programme, supported by UNICEF, was initiated and implemented by Save the Children Jordan (SC-J). The Programme started in November 2012 and continues to date, aiming at maintaining and improving infant and young child feeding among populations in Jordan that are affected by the Syrian crisis.

27 Department of Statistics [Jordan] and ICF International. 2013. Jordan Population and Family Health Survey 2012. Calverton, Maryland, USA: Department of Statistics and ICF International. 28 Gribble and Berry. 2011. Emergency preparedness for those who care for infants in developed country contexts. International Breastfeeding Jouran. 29 Wenfang Yang, 2012, Zhou H, 2012, Mishra 2014.


Since its launch in November 2012, the Programme has witnessed an expansion in scope and area of implementation; including camps and host communities. UNICEF’s Programme requirements include performing an external evaluation while the Programme implementation is ongoing, in order to determine whether or not the Programme is achieving its stated objectives and continues to serve the evolving needs of the targeted communities. The current report presents the findings of the Evaluation Study of the ‘Infant and Young Child Feeding (IYCF) Programme in the Camp and Host Communities in Jordan.’ The Evaluation Study has been executed by an independent team of consultants, selected by the Programme Evaluation Reference Group (ERG) to carry out the assignment, with technical oversight and quality assurance provided by UNICEF. The report provides the ERG and UNICEF with an overview of the Evaluation Team’s approach to this evaluation. It presents results and findings relevant to the purpose, scope, and objectives of the evaluation as set out in the Inception Report. The report is structured as follows:

Section 2 presents the object of evaluation Section 3 sets out the purpose, objectives, scope, evaluation criteria, and use of

evaluation Section 4 describes the evaluation methodology Section 5 presents limitations Section 6 presents the evaluation matrix Section 7 presents results and findings Section 8 presents conclusions Sections 9 and 10 present lessons learned and recommendations Annexes are within section 11 of this report

Object of Evaluation – The Project Context and Programme This section presents the programme design of the ‘Infant and Young Child Feeding Programme’ (IYCF-P), which includes IYCF in emergencies and the integrated IYCF+ packages of services. The section also includes the Programme’s package of services, target beneficiaries, model and progress.

2.1. Infant and Young Child Feeding Programme (IYCF-P)

Core Package of IYCF services: Infant and young child feeding (IYCF) Programming includes activities aiming at supporting, promoting and protecting optimal infant and young child feeding amongst infant and young children aged 0 to 2 years of age. Activities in support of the IYCF Programming include education, promotion, counselling, peer support, as well as the setup of guidance and policies


to protect IYCF from the harms of marketing of breast-milk substitutes. The initial core package of services and interventions provided through IYCF-P are listed below:

1. Awareness on optimal feeding practices through education sessions administered in

IYCF caravan and targeting families and mothers of children under 5 years of age. 2. Counselling and support for lactating mothers through home visits. 3. Outreach and community mobilization through community volunteers to administer

basic messages on IYCF and encourage families to visit the IYCF caravan. 4. Monitoring of WHO Code of Marketing of Breast-milk Substitutes (BMS)

implementation and addressing Code violation, including uncontrolled distribution of infant formula.

5. Implementing bottle-cup amnesty during which families exchange a bottle for a cup. 6. Distribution of in-kind incentives for mothers, including breastfeeding shawls and

nutrition supplements. Beneficiaries of the IYCF Programme: The Programme targets mothers and children. Appendix A shows the number of mothers and children targeted by geographical areas and other related Programme characteristics. Management of the Programme: The programme is implemented by Save the Children Jordan (SC-J), led by the organizations’ Health and Nutrition Manager, who supervises four officers including the IYCF Community Officers (in charge of the programme in host communities), the IYCF Camps Officers (in charge of the programme in the camps), and the Supplementary Feeding Programme (SFP) and Anaemia officers. The latter supervise coordinators in the field who in turn supervise counsellors, educators and community mobilizers. The Programme management structure is depicted in the Figure 1 below. All staff are SCJ staff in addition to the community mobilizers, who are Syrian refugees. In addition to the Programmes structure, the Programme is linked to the Operations Management, which includes Logistics and procurement as well as Finance.

Figure 1 - The Programme management structure also found in Appendix V

Health & Nutrition Manager

IYCF Community

Officer

IYCF community coordinator

IYCF Amman team leader

7 IYCF counselors

4 IYCF educators

10 syrian mobilsers

7 IYCF counselors

7 IYCF educators

10 IYCF syrian mobilisers

Anemia officer

nutritionist labtechnician 4 syrian

mobilisers (all locations)

SFP officer

SFP team leader

SFP assisstant

5 syrian mobilisers

(all camps)

IYCF Camps officer

zaatari IYCF coordinator

Zaatari IYCF team leader

15 IYCF counselors

5 IYCF educators

21 IYCF mobilsers

and 4 caravan ass.

Azraq IYCF coordinator

6 IYCF counselors

4 IYCF educators

1 midwife

10 syrian mobilisers &

2 caravan assiss.

EJC IYCF coordinator

2 IYCF Counselors

2 IYCF educators

1 midwife 4 syrian

mobilisers

RS team leader

4 nurses

Camps mobilisers work for 4hours/day for 5 days/week with 1.5JoD/hour.

Community mobiliser work 3days/week with 10 Jod/day.

Zaatari caravan assistants include base camp cleaner

Mobilisers for anemia should be coordinated within the same location.

RS team (team leader & nurses) are under the same line on the budget.

IYCF Team leaders are available in locations with more than 10 staff to support the coordinator

with technical follow up.

Position in red will remain empty till further notice.


The total Programme budget ranges from $1.5 million (1st year), $5 million (2nd year), and $3 million (3rd year) as summarized in Appendix B.

Programme History: In 2012, the IYCF-P was initiated as a Do No Harm IYCF-E initiative focusing on ensuring the least damage to IYCF practices amongst Syrian refugees in Jordan. The Programme was launched in response to the urgent IYCF need in the beginning of Syria crisis. An IYCF-E array of activities was developed and planned to respond to the immediate needs of mothers, infants and young children in the first camp that hosted Syrian refugees in Jordan; Zaatari camp. Since January 2014 in an attempt to engage national organizations (including the Ministry of Health) and contribute to sustainability, SC-J expanded the Programme to encompass capacity building activities for partners, including government and local partners such as the Ministry of Health (MOH), Ministry of Social Development (MOSD), Community Based Organizations (CBOs), and the private health sector hospitals and companies. Within this context, more than 75 government health providers (nurses and midwives), 100 SC-J staff and 100 IYCF community mobilizers (Syrian refugees) attended trainings on optimal IYCF practices, counselling and education methods, as well as on the implementation of the code of marketing of Breast-Milk Substitute (BMS). Within IYCF-P, skilled support is provided through a holistic approach of one-on-one counselling sessions, group education sessions, home visits, referrals to multiple services, and regular follow-ups of all cases to improve and sustain nutrition behaviour change and best childcare practices of the affected population. As the Programme progressed, IYCF-E activities integrated other nutrition and health interventions within what was called IYCF+. Additional packages of IYCF+ include: malnutrition screening (MUAC: Mid-Upper Arm Circumference Screening) for children 6-59 months of age and Pregnant and Lactating Women (PLW); Supplementary Feeding Programme (SFP); and promoting diarrhoea prevention and management through Oral Rehydration Therapy (ORT) corner, referring to vaccination, and promoting hygiene behaviour. The IYCF-P started in 2012 in Zaatari camp with only one caravan, and extended to three caravans by the end of June 2013 to meet the needs of the Zaatari camp expansion. Throughout the Programme, more caravans in different areas were added subsequently. In July 2013, one caravan was added to the Emirati Jordanian Camp (EJC). Afterwards, in the beginning of 2014, the programme was expanded to Azraq camp, where 2 caravans were established. In addition, in May 2014, IYCF activities were introduced at Rabaa Al Sarhan (RS) site, where Syrian refugees settled first before moving to other camps or host communities. The IYCF-P also reached host communities covering different areas starting with the southern governorates of Jordan; namely Aqaba, Maan and Karak in August 2013, Amman in October 2013, and Jordan Valley (South, Center, and North) in January 2015. The design of the Programme in host communities targeting both Syrian refugees and Jordanians differed from the one implemented in camps, which aimed at targeting refugees only. In August 2014, remote areas in the south of the country were reached through the IYCF mobile caravan. By


the end of 2015, the programme successfully established 3 caravans in Zaatari camp, 1 in Emirati Jordanian Camp (EJC), 2 in Azraq camp, 1 in Rabaa Al Sarhan (RS), and 7 in host communities, in addition to the mobile station in the South. Programme activities are implemented in the different areas in partnership with different stakeholders including CBOs and health sector institutions (Appendix C). At the beginning of the Programme in 2013, a partnership was established with all existing and new health service providers in the camp for Save the Children Jordan Infant and Young Child Feeding Educators. During the same period, networking with CBOs in the South and other governorates, as well as coordination with hospitals, the International Organization for Migration (IOM), the United Nations Population Fund (UNFPA), and the Jordan Health Aid Society (JHAS) to follow up on breastfeeding newborns started to take place. It is during this period of time that the Breast-Milk Substitute (BMS) code was also yielded. Moreover, the IYCF team started monitoring formula distribution in RS, Zaatari, and EJC camps, and developed and maintained a referral mechanism for children to be utilized by health care providers. A new mechanism was launched with the Saudi clinic by the end of October 2013. Between April and December 2013, the IYCF team has recruited 30 counsellors and 20 mobilizers. By the end of 2014, the number of counsellors and mobilizers reached 38 and 36, respectively. All existing connections with different CBOs and NGOs were sustained to further improve the IYCF-P. During the first trimester of 2014, new collaborations with IMC and UNHCR were established in order to begin the monitoring of infant formula prescription at Azraq camp. By May 2014, the control of infant formula was settled in the three (3) above-mentioned camps. During this month, there was an expansion of the coordination with the Ministry of Health and the antenatal care centres. This was paralleled with the recruitment of counsellors and employment and training of staff (midwives, nurses, and health professionals). In 2014, the IYCF team incorporated training sessions and awareness campaigns in the programme. During the first five months of the year, the IYCF-P took part in three (3) education campaigns and provided a training session for twelve (12) midwives and nurses working with the MOH. Awareness sessions were also held in all twelve (12) governorates by the end of August 2014. In the last semester of the year, orientation sessions for International Federation of Red Cross (IFRC) health professionals in Azraq camp were conducted and three (3) workshops were planned for the upcoming months. Between December 2014 and February 2015, the IYCF team, in partnership with MOH and UNICEF, launched the national breastfeeding campaign. A closer coordination with the MOH was activated after the inauguration of the BMS code. After this date, advocating and monitoring of the implementation of the BMS code took place. In parallel, a 4-day technical training for all 72 IYCF staff was delivered and collaboration with hospitals, schools, health centres, and CBOs, etc. exceeded the number of 230. Between May and June 2015, the IYCF team established three (3) CBOs in the North, Center, and South (in addition to the 3 previous ones in Karak, Maan, and Aqaba). Around 300 CBOs, schools, hospitals, and health centres were working together by this time. Partnerships with ACTED and International Medical Corps (IMC) at Zaatari camp were reported and numerous training sessions were attendant in June 2015. Two (2) trainings and three (3) workshops were conducted for 34 midwives working with MOH and health centres and for 47 participants


from CBOs community staff, respectively. In June 2015, the monitoring of infant formula distribution started at the RS camp. More educational sessions were held in the second half of the year. As between July and October 2015, IYCF staff was trained in different regions. Also, the training of health professionals was extended and covered 14 private hospitals. Follow-ups and counselling during this period have reached 82 MOH centres in all locations. By the end of 2015 (October to December 2015), a new cooperation with Nour Al-Hussein Foundation (NHF) at Zaatari camp was established during a session about breast cancer awareness. This was followed by the provision of two (2) sessions in NHF about the importance of breastfeeding in the prevention of breast cancer. The IYCF team conducted trainings in all hospitals which accept referrals and 9 orientation sessions in camps with a total attendance of 105 partners. Table 2 below provides a description of the conducted activities by type, date of initiation, and target area.

Table 2 - IYCF Programme activities per type, date of initiation, and target area

Type of Activity Date of Initiation Geographic Area

IYCF-P in camp (Counselling/awareness/monitoring of BMS/MUAC screening)

December 2012 Zaatari camp


July 2013 EJC camp


May 2014 Azraq camp

IYCF-P in centre (Counselling/awareness/monitoring of BMS/MUAC screening)

May 2014 RS transit centre

IYCF-P in host community (Counselling/awareness/monitoring of BMS/MUAC screening)

Oct 2013 Amman


Aug 2013 Karak


Aug 2013 Aqaba


Aug 2013 Maan


Jan 2015 Jordan Valley

Capacity building for MOH and CBOs Jan 2015 National


2.2. Programme Theory of Change Though the IYCF-P does not have an explicit Theory of Change (ToC) to guide and monitor

progress towards results, a preliminary outline of such a theory has been developed at the

inception phase for this evaluation (Appendix W – Figure 2). While conducting the evaluation,

a better understanding of the Program and its associated elements and processes allowed the

evaluation team to better visualise the initially developed TOC to show linkages among the

different Program results (Figure 2). The theory presented herein was used as a theoretical

framework to define and/or validate the following aspects associated with the Programme as

well as the related evaluation questions.

a. The nature and scope of the IYCF-P (relating mainly to the relevance, effectiveness,

and sustainability of the Programme):

This was achieved through examining: (i) the Programme aim and strategic direction; (ii) the Programme-promoted joint collaboration and partnerships in order to reach established objectives; and (iii) the Programme’s governance structure and mechanisms.

b. The chain of results and associated causal pathways of the IYCF-P (relating mainly to the effectiveness, efficiency, and likelihood of impact of the Programme):

This was achieved through examining: (i) key inputs leveraged to help implement the

Programme’s activities allowing to achieve the outputs and outcomes of the Programme and

ultimately its likelihood of impact; and (ii) the delivery modalities that the Programme relied

upon to achieve its intended results.

c. The conditionality and assumptions that underline the achievement of results

(relating mainly to the effectiveness, efficiency, and sustainability of the Programme):

This was achieved through examining/validating: (i) the conditions that have been built into the IYCF-P structure; (ii) the Programming principles and expected stakeholder contributions; (iii) the assumptions that reinforce the proposed chain of results; and (iv) the risks associated with the assumptions for the transition between and across levels of results that could have potentially undermined Programme achievements.


Figure 2 - Schematic presentation of the IYCF-P Theory of Change – the results chain


3. Purpose, Objectives, Scope, Evaluation Criteria, and Use of the Evaluation

3.1. Purpose As specified in the Terms of Reference (ToR), the purpose of this evaluation is to independently assess the IYCF-P, supported by UNICEF and implemented by Save the Children Jordan in the target communities (camps and host families). In particular, the evaluation aims at conducting a systematic and impartial examination of the Programme’s performance, including the relevance, effectiveness, efficiency, and sustainability of the Programme. This evaluation is primarily intended to support the decision-making process of the UNICEF Jordan Country Office and the Ministry of Health to examine Programme’s performance and inform future health and nutrition programmes in Jordan. The secondary audience includes relevant institutions working in the nutrition sector in Jordan, including SC-J, nutrition and health partners, as well as donors.

3.2. Objectives The specific objectives include:

1. To assess the relevance of the IYCF-P (with integrated IYCF+ services for emergency response and resilience components) in the current context in Jordan and its national priorities, including those in the context of the health sector reform. To assess the relevance of IYCF-P design to achieve the expected results, taking into account the appropriateness for and social acceptability by children, PLWs, caretakers and communities.

2. To assess the effectiveness of the IYCF-P and to measure to what extent the Programme has achieved its stated objectives and any intended and unintended effects. These include behaviour change among PLWs in terms of initiation of breastfeeding, exclusive breastfeeding, continued feeding and introduction of safe and appropriate complementary foods from 6 months of age. To also assess the Programme equity and targeting.

3. To assess the efficiency of the IYCF-P and to m e a s u r e t o what extent the Programme has used resources (human, financial, and others) in an efficient manner, and if the achieved results justify the resource. To also assess if UNICEF and partners pursued options to achieve efficiency, e.g. integration with other relevant interventions (e.g. referrals for immunization, back to school campaigns).

4. To assess the sustainability of the IYCF-P and its results, considering the likelihood of

Programme sustenance in the absence or reduction of ongoing UNICEF and donor


support. This is to be done by identifying the degree to which the IYCF-P has built on existing institutional and local capacities, and a potential exit strategy.

In the end, the evaluation will articulate lessons learned on the strategic direction of the Programme and operational and technical Programme aspects, so as to improve future Programming of activities at the national level. It is important to note that the ToR specifies the need for the evaluation to measure the extent to which the Programme has had an impact on the target population. However, given the lack of baseline data and further to related discussions with UNICEF officials during the inception period, the current evaluation did not attempt to measure the Programme impact but rather examine the likelihood of impact. This was achieved through assessing the enablers and contributing factors associated with the Programme’s success. The data to be generated from the upcoming Interagency Nutrition Survey (data collection will be completed 1st week of October, preliminary result is available on October 12) would add value to this study in documenting the Programme impact, provided that it includes additional Programme-related questions and targets a representative sample of mothers with infants less than 2 years of age. The Interagency Nutrition Survey, with disaggregation between beneficiaries who received IYCF services from the program and those who did not, will provide information about IYCF practices and therefore be able to highlight the impact of the programme.

3.3. Scope The evaluation covers the full range of activities implemented within the context of the IYCF-P, supported by UNICEF and implemented by Save the Children Jordan for the period extending from December 1, 2012 to December 31, 2015, in the targeted geographic areas as specified in Table 2.

3.4. Evaluation Criteria and Questions The evaluation is structured around the five standard Organization for Economic Co-operation and Development's (OECD) Development Assistance Committee (DAC) evaluation criteria. Table 3 below shows how these Criteria applied within this evaluation:


Table 3 - Application of evaluation criteria

Relevance The planning, design and implementation processes of IYCF- P in relation to responsiveness and alignment with national priorities and needs, as well as UNICEF –in-house strategies, policies and International guidance and policies related to IYCF and IYCF-E (including IFE Core group OPS, WHO global strategy UNICEF programming guide etc.).

Effectiveness The success or otherwise of IYCF-P in achieving its stated objectives and any intended or unintended effects. Associated strengths, weaknesses, and evidence of innovation. Extent of equity achieved by the Programme.

Efficiency The extent of resources allocated and utilized to achieve the desired (stated) objectives. The degree of control exercised over the quality and quantity of outputs. Efforts made at the national level to leverage pre-existing results, partnerships, synergies and approaches.

Likelihood of Impact

The IYCF-P enablers and factors contributing to its long-term effects.

Sustainability The extent to which the results of the IYCF-P have generated effective partnerships and strengthened national capacity. The extent of institutionalization of the Programme within the Ministry of Health.

Based on these evaluation criteria, the evaluation questions and sub-questions mentioned in Appendix D were considered. These questions and sub-questions guided the data collection and analysis work throughout the evaluation process and were also used to test and fine tune the Programme Theory of Change developed in the inception phase (Refer to Section 2.2).

4. Evaluation Methodology The overall design of this evaluation is based on the use of the Programme’s emerging Theory of Change (ToC), as articulated in the previous section. The evaluation uses the Programme theory to assess: i) how the different contexts where the IYCF-P operates affect the Programme’s performance; ii) whether or not the proposed logic of results holds; and iii) whether the assumptions made in terms of external factors and conditions needed to enable and sustain change are valid, and if not, how such discrepancies affect performance.

4.1. Evaluation Approach Given the nature of the Programme, objectives, scope, roll out schedule and reach; the evaluation has adopted a general qualitative methodology. The research approach is the case study methodology, where the IYCF-P is evaluated using the multiple case study design with embedded units of analysis. This approach is valuable in health research and Programme evaluations as it joins rigor and flexibility at the same time (Baxter & Jack, 2008)30, especially

30 Baxter and Jack 2008. Qualitative Case Study Methodology: Study Design and Implementation for Novice Researchers


that the nature of the Programme is unique and measurement of impact is currently not possible for reasons mentioned above. Through this methodology, the team worked on collecting information about the case (defined as the IYCF-P in Jordan implemented by Save the Children Jordan, supported by UNICEF) from different perspectives including those of key stakeholders, service providers, and end users within the context of the Syrian refugee crisis. Since the Programme has witnessed expansion both at the geographic and the activity level, and in an attempt to overcome the challenge of capturing information that is representative of the different geographic areas and different stages of the Programme, the multiple cases encompassed different units of analysis to capture different geographic areas, different duration of implementation, and different contexts. Figure 3 below is a schematic representation of the study design with the embedded units of analysis. As shown in Figure 3, the IYCF-P is examined in two contexts; camps and host community. Multiple units of analyses are used for each context as follows:

1. For the camp context, the Programme has started at different times in each of the targeted camps and each camp was considered as a unit of analysis by itself.

2. For the host community context, the Programme is implemented in different geographic areas and has started at different times within each of the target communities. Two comparative units of analysis were selected with similar geographic characteristics and time of implementation, and another single case / unit of analysis was selected with different geographic characteristics and time of implementation (please refer to the section on geographic area below for details on selection criteria).

For each unit of analysis, the Programme theory of change (ToC) was tested and fine-tuned. Figure 3 - Multiple case study design with embedded units of analysis31

31 Adapted from http://www.sagepub.com/sites/default/files/upm-binaries/41407_1.pdf

Case: IYCF Programme

Context 2: Host Community

Context 1: Camp

Case: IYCF Programme Embedded unit of analysis 1:

Maan (Programme initiation: Aug 2013)

Embedded unit of analysis 1: Programme running for more than 2 years (Zaatari)

Embedded unit of analysis 2: Karak (Programme initiation: Aug 2013)

Embedded unit of analysis 3: Jordan Valley (Programme initiation: Jan 2015)

Embedded unit of analysis 2: Programme running for less than 2 years (Azraq)


The case study approach was also applied while planning data collection which included multiple sources of evidence including: (1) records and documents reviewed through a desk review; (2) semi-structured interviews with key informants; (3) focus groups discussions with service providers; (4) focus group discussions with end-users benefiting from the Programme at different stages; and (5) questionnaire targeting beneficiaries of capacity building activities. Targeted Geographic areas The areas targeted by the Programme are shown in Table 4 below (as per the Programme documents provided by SC-J). Areas are described depending on the context (camp or host community), Programme start date, and extent to which the area fits the selection criteria. Primary criteria for the selection of the geographic areas include:

1. Area is either a camp or a host community 2. Programme started before December 2015

Table 4 - Targeted geographic areas

Geographic area Context Programme start Date

Fitting selection criteria

Zaatari camp Camp Dec 2012 Yes Azraq camp Camp May 2014 Yes EJC camp Camp Jul 2013 Yes RS transit centre Transit May 2014 No Amman Host community (urban) Oct 2013 Yes Karak Host community (urban/rural) Aug 2013 Yes Aqaba Host community (urban/rural) Aug 2013 Yes Maan Host community (urban/rural) Aug 2013 Yes Jordan Valley Host community (rural) Jan 2015 Yes

In addition to above criteria, the following areas were excluded based on following reasoning:

1. EJC camp: Given that this camp is funded by UAE and hosting a much smaller refugee population as compared to other camps, the context of EJC camp is thus considered to be particular.

2. Aqaba: The number of refugees in this community is small and the characteristics of the area are also considered to be particular (touristic area).

The particular condition of these two geographical areas was perceived to limit the possibility of generalizing or comparing the Programme evaluation findings. The remaining areas included the four host community sites (Karak, Maan, Jordan Valley and Amman). Out of these four sites, three (Karak, Maan, and Jordan Valley) were selected based on the following reasoning:

a. Karak and Maan as two sites that are similar in nature; both have the same socio-demographic profile and rural arid nature and would be considered as two comparable areas.


b. Jordan valley as another site that has different characteristics (agricultural-rural), where Programme activities were launched at a later stage and would also serve as a unique case.

Amman was excluded because the time of the Programme initiation in Amman is very close to that of the two other areas selected (Karak and Maan). Therefore, the selected areas include: Context 1: Camp Area 1: Zaatari Area 2: Azraq Context 2: Host community Area 1: Maan Area 2: Karak Area 3: Jordan valley

4.2. Data Collection Method and Tools As mentioned above, data and information was collected using a combination of the following methods targeting the different project counterparts/stakeholders: (1) records and documents reviewed through a desk review; (2) semi-structured interviews with key informants; (3) focus groups discussions with service providers; (4) focus group discussions with end-users benefiting from the Programme at different stages; and (5) e-questionnaire targeting beneficiaries of capacity building activities.

a. Documents Review

A thorough review of relevant plans, strategies and documents, global standards, and Programme reports was conducted as described in Appendix D. Documents included Programme agreements, reports, national plans, regional plans, progress reports, scientific literature, and grey literature. Content analysis was adopted to capture the essential information from the documents in order to answer the research questions. For each of the OECD criteria and research questions, the evaluation team attempted to provide an answer based on documents reviewed, as relevant. Key Informant Interviews (KII)

Semi-structured interviews (face-to-face and/or by telephone) were conducted with selected key informants to collect information concerning key questions and sub-questions of the evaluation. Appendix E includes the interview questions. Key informants were selected in a non-random purposive manner based on information provided by UNICEF and Save the Children Jordan. A snowball sampling method was adopted whereby key informants that were recommended by other informants were considered and


interviewed. The list of stakeholders that were interviewed is provided in Appendix F. In total, 25 key informants were interviewed from different organizations including UNICEF, Save the Children Jordan, Ministry of Health, other UN agencies, partner agencies, and CBOs. Focus Group Discussions (FGDs)

Another main data collection method consisted of conducting focus group discussions (FGDs) with service providers and end users. As part of this method, data and information was collected about each context and unit of analysis, as follows: Focus group discussions with service providers: Within each of the selected geographic areas, one focus group discussion was conducted with service providers. Service providers are considered to be staff members of entities that are providing the IYCF service as part of a team in a certain area. Five (5) focus group discussions were conducted with a total of 40 participants as indicated in Table 5 below. Participants included coordinators, counsellors, educators, and community mobilizers. Each focus group included 3 to 11 participants. Information that serves to answer questions in Appendix G was collected to assess the service providers’ perceptions about the Programme’s relevance, effectiveness, efficiency, likelihood of impact and sustainability. The following selection criteria were applied to select participants of focus group discussions:

Service provider from Save the Children Jordan team (field coordinator, counsellor, educator, mobilizer)

Service provider from CBO/local partner/health care provider/hospital who is part of the team providing support in that particular geographic area

Service provider who is willing to participate

Service provider who has been engaged in the Programme since at least June 2015 (6 months before the end of the scope of this evaluation)

As mentioned earlier, one focus group per geographic area/unit of analysis was conducted for a total of five (5) FGDs in the five (5) areas targeted by the evaluation. Appendix G includes the topic guide for the focus group discussion with service providers. Table 5 below shows the number of participants in the focus group discussions with health care providers and their positions. Table 5 - Focus group participants with services providers

Focus Group

Location Date # of

interviewees # of Camp

officers # of

coordinators

# of team-

leaders

# of counsellors

# of educators

# of mobilizers

1 Azraq Camp

May 17, 11 1 0 0 4 3 3

2 Zaatari Camp

May 21 12 0 1 1 5 2 3

3 Maan May 26 9 0 1 0 2 2 4


4 Dar-Alla June 4 3 0 0 0 0 1 2

5 Al Karak June 22 5 0 0 0 1 1 3

Total 40 1 2 1 12 9 15

Focus group discussions with end users: Focus group discussions were conducted with end-users that benefited from the Programme in the selected areas. The FGDs targeted both Syrian and Jordanian beneficiaries (Appendix H). Mothers/end-users who have participated in any of the Programme activities including counselling, education, provision of nutrition support, or provision of in-kind incentives such as the shawls, were invited to participate in the focus group discussions. Ten (10) focus group discussions (two in each of the selected geographic areas/units) were conducted with end-users that benefited from the Programme. End-users that benefited both fully and partially from the different activities of the Programme were invited. This is to ensure that different levels of interventions (awareness, counselling etc.) were captured in the focus group discussions. A list of beneficiaries was requested from Save the Children Jordan, of which end-users were selected randomly and invited to attend. Eligibility criteria for participants in the noted focus group discussions include:

a. Mother of infant(s) less than 24 months of age at the time of the execution of the Programme;

b. Mother who benefited partially from the Programme (education). It is understood that information is not available about mothers who have attended education sessions, therefore, the following steps were followed to ensure women who have partially benefited are included: i. Camps: Given the assumption translated by SC-J that 100% of women are covered

in the camp, the selection of women was made randomly and through the camp focal point/outreach worker.

ii. Host community: Given that no electronic lists exist for mothers who have participated in education sessions, the evaluation team relied on SCJ staff to go over the paper-based attendance sheets, randomly identifying and inviting women who have participated in an education session. The evaluation team acknowledges the limitation of this process but also realizes the importance of including such mothers.

c. Mother who benefited fully from the Programme (counselling, education, visits etc.). d. Mother who participated in the Programme in the last 6 months of the year 2015. In

order to avoid recall bias, mothers who have been engaged in the Programme for the last 6 months of the year 2015 (which is the end date for the scope of the evaluation) were prioritized and invited.

e. Mother who has been living in the same area for the last 6 months of the year 2015. f. Mother who is willing to participate.

Although it is understood that the Programme has targeted community members other than mothers, the evaluation focused on collecting information from mothers as the primary beneficiaries of the Programme. Invitations were extended to members of the family that are


considered influencers of decisions related to infant and young child feeding. These included fathers, mothers, and other members of the family that may have an impact on a mother’s decision to feed her infant/child. However, as evident from attendance sheets, no males attended the discussions; only female influencers accompanied the mothers. In total, 133 beneficiaries, with an average age of 34 years, participated in 10 focus group discussions as shown in Table 6. Seventy-seven (77) (58%) of the mothers who participated in the focus group discussions were breastfeeding at the time of the interview. Appendix H includes the topic guide for the focus group discussions with mothers. In addition, and in order to capture information from non-beneficiaries including their perception of the Programme, access, and need, the team conducted a few (6) Key Informant Interviews with mothers who have not benefited from the Programme in Maan and Karak, as shown in Appendix F. It was not possible to conduct focus group discussions with non-beneficiaries, as it was very difficult to have access given that the evaluation was being conducted in areas targeted by SCJ. Therefore, it was more convenient to reach out individually to non-beneficiaries than to invite them to focus group discussions. Table 6 - Focus group participants with end users

Knowledge, Attitudes, Practice (KAP) Assessment Targeting Capacity Building Beneficiaries To assess the effectiveness of the capacity building component of the IYCF-P, a KAP assessment was conducted targeting CBOs and MOH staff that benefited from the capacity building trainings. A questionnaire was devised and addressed to relevant staff that participated in a capacity building training conducted by SC-J. A list of the training participants was requested from SC-J and the KAP questionnaire was addressed to all listed trainees by e-mail and telephone. Reminders were sent by e-mail and through telephone, as needed. The KAP questionnaire (Appendix I) tested the core components of the capacity building activities. Its development was based on the training material provided by SC-J. Informed consent was obtained from all respondents either by telephone or electronically. No names or other identifiers were collected and questionnaires were anonymous. The questionnaires were completed and data was entered on excel worksheet for analysis. Descriptive statistics were conducted, and differences in KAP across analytical units were explored as depicted in Appendix J and under the ‘Results’ section below.

# of interview

Location Date # of mothers interviewed

Average age of mothers

interviewed

Average # of

children

# of women who are currently

breastfeeding

1 Azraq Camp May 17, 2016 28 30.96 years 3.52 13

2 Zaatari Camp

May 21, 2016 38 29.45 years 3.66 31

3 Maan May 26, 2016 27 39.52 years 4.19 11

4 Al Karak May 29, 2016 16 41.56 years 4.00 7

5 Dar-Alla June 4, 2016 24 33.50 years 2.91 15

133


4.3. Evaluation Process The evaluation included three phases, as follows:

a. Design Phase This phase included:

A desk review of all relevant documents available

Stakeholder mapping – This includes the mapping of the main stakeholders of the IYCF-P to gain a better understanding of the relationships between them

The development of the Programme Theory of Change

The finalization and refinement of the list of evaluation questions

The development of the data collection tools

The development of a data collection and analysis strategy as well as a concrete work plan for the field phase

b. Field Phase

After the design phase, collection of the data required was undertaken in order to answer the evaluation questions as agreed upon at the design phase. Appendix K describes the actual timeline for data collection.

c. Analysis and Synthesis Phase

The evaluation analytical plan consisted of analysing quantitative data from the KAP questionnaires and four (4) types of qualitative data: (i) documents data; (ii) key informant data; (iii) focus group data; and (iv) structured interview data. Throughout the analysis, an explanatory approach was adopted to describe and explain the way in which the case (the IYCF-P) is functioning using the generated theory of change. For each context and analytical unit, the case was described using the theory of change which was tested and then compared within the different contexts and units of analysis. Within each unit of analysis, the analysis followed an iterative process where information was slowly built and validated. For example, following data collection in the first locality, the evaluation team conducted preliminary analysis generating preliminary patterns describing the case and its theory of change. Data from FGDs within the first locality were triangulated/validated with other data sources including KIIs, the KAP survey, direct observation, and the document review. This process aimed to generate preliminary patterns for that particular case described through the theory of change and answering the OECD criteria. Questions were added/modified further to the results of the analysis. Results of the analysis from the different contexts and units of analysis were compared within the same context (camp/host community). To analyse the quantitative data associated with the questionnaire, frequencies were calculated and tabulated. Descriptive statistics were computed.


For the analysis of the qualitative data, the thematic approach was adopted. The process consisted of the following steps:

i. Data Review

Before starting the analysis, data gathered from the different sources was read thoroughly to gain a better understanding of the content. Initial impressions and remarks were noted and referenced as per the data source.

ii. Data Organization

To make the collected data manageable and easy to navigate, data was grouped by data collection method, evaluation measures and questions asked, and responding groups.

iii. Data Coding

Under each of the research questions, preliminary categories/themes were identified and coded prior to data analysis. These categories/themes and related codes were then reviewed and refined while analysing the data, taking into consideration the common trends, patterns, and ideas that appeared repeatedly throughout the data.

iv. Data Interpretation

Data analysis and data interpretation was based on the theory of change (ToC) and OECD criteria as described under (c) above. The team conducted analysis simultaneously as data was being collected and information was generated progressively. As described above under section (c), the team started with analysis for the first case under one context and unit of analysis and moved forward for each case progressively to describe the case using the theory of change and answering the OECD criteria. Using the thematic approach, the analysis process consisted of completing the list of the key categories and themes, followed by the identification of similarities and differences in responses from the different target groups. The relationships between the different categories/themes were examined to determine how they are connected and how they influenced the IYCF-P implementation and achievements as guided by the theory of change (Refer to Section 2.2). After preliminary analyses were completed, a debriefing presentation on the preliminary results of the evaluation was provided to the IYCF-P officials, with a view to validating preliminary findings and testing tentative conclusions and/or recommendations.

4.4. Strategy for Ensuring Data Quality for All Deliverables To deliver credible and quality evaluation deliverables, the following was conducted:


1. Developing standardized protocols for data collection 2. Training data collectors to administer Key Informant Interviews (KII) and Focus Group

Discussions (FGDs) according to the protocols 3. Recording interviews where possible 4. Coding all data and storing it on a password-secured central drive 5. Following a standardized procedure for data transcription 6. Conducting random checks to ensure accurate transcription and translation 7. Triangulating data across multiple interviewers, multiple data collection points, and

across methods 8. Validating main findings with key stakeholders including UNICEF, MoH, and SCJ at the

end of data compilation and analysis

4.5. Strategy for Ensuring an Ethical Approach The evaluation was implemented in line with the following ethical principles and standards: UNICEF Procedure for Ethical Standards in Research, Evaluation, Data Collection and

Analysis (CF/PD/DRP/2015-001) The Belmont report on “Ethical Principles and Guidelines for the Protection of Human

Subjects of Research” developed by the National Commission for the Protection of Human Subjects of Biomedical and Behavioural Research in 1978

United Nations Evaluation Group (UNEG) Ethical Guidelines for Evaluation (UNEG, 2008) Throughout the evaluation, the main principles emphasized while conducting interviews, meetings, and focus group discussions with the concerned participating stakeholders include: 1. Being sensitive to beliefs, manners and customs of participants 2. Acting with integrity and honesty with participants 3. Ensuring a respectful communication and contact with participants 4. Protecting the anonymity and confidentiality of individual information 5. Obtaining informed consent of each concerned individual to be interviewed through

providing him/her with a printed statement that will inform about the following: a. The conduction of an evaluation; b. The purpose of the evaluation; c. The procedures to be followed during the interviews; d. The data and information are collected solely for the purpose of the evaluation and

will be presented in a way that will not allow linking a specific piece of information to an individual;

e. That the participant is free to ask questions and may refuse to participate; and, f. That the participant has the right to end the interview at any point in time without

any implications.

To emphasize, within this context, that three members of the evaluation team have formal training and certification in the responsible conduct of human research (focusing on social and behavioural research) through their research at the American University of Beirut (AUB), and by having served on the AUB’s Institutional Review Board (IRB). They have completed the


CITI Collaborative Institutional Training Initiative course for IRB members and have experience evaluating research protocols for the IRB. They ensured that high ethical standards are applied in the present evaluation.

5. Limitations The main limitations associated with the evaluation design included: The inability to assess Impact in the strict OECD/DAC sense of the term in view of the

absence of the needed baseline data; the Programme likelihood of impact was investigated through examining related enabling and contributing factors.

The inability to cover all geographical areas targeted by the Programme in view of the large scope of the Programme and the limited timeframe and resources available for the evaluation; specific geographical areas were selected to be targeted by the evaluation based on criteria described in Section 3.4.

In addition, the following challenges were encountered during the evaluation: Limited collaboration of a few key stakeholders; however, these were replaced by other

key stakeholders that were deemed of equal relevance. The lack of participation from males in focus group discussions. During focus group

discussions with end-users, participants were all females and although influencers including husbands were invited, however no males were part of the focus group discussions.

The bias of having information from mothers that were willing to participate in the focus group discussions. Although the evaluation randomly selected mothers benefiting from the Programme (both partially and fully), however, not all those invited attended and therefore this might have implied that those who were present are those who were more engaged in the programme. The participation of these particular mothers may have skewed results related to Programme likelihood of impact.

The challenge of finding non-beneficiaries given the lack of centres that are not targeted by the Programme in the same geographic region considered within the context of the evaluation.

The challenge of retrieving financial data to help in the cost efficiency evaluation. The evaluation team found it very hard to compile financial information in a format that is analysable including information organized by year and by area. There was no financial data about support cost that is stratified by area and, therefore, the evaluation team working on the cost efficiency had to use assumptions and formulas to run cost-analysis and draw conclusions on cost efficiency by area. In addition, the timeliness of providing the data by SCJ was a challenge.


6. Evaluation Matrix The matrix in Appendix D provides information about the evaluation questions and sub-questions, data sources, data collection methods associated with each question, indicators/ standard or measure by which each question was evaluated, and methods for data collection.

This matrix formed the basis for the narrative of findings, conclusions and recommendations in the evaluation report.

6.1. Team Members and Their Responsibilities The evaluation team was composed of 5 main experts, in addition to research assistants and data collectors. In addition, a support expert member for the Programme efficiency was later summoned to conduct further cost analysis. Appendix L includes a summary of the skills and capacities of the team that worked on this evaluation.

7. Results and Findings This chapter presents the results, findings, and conclusions derived from data collection using the case study methodology described above. Appendix X includes the Case Study Report. In the first section of this chapter, the case – defined as the Infant and Young Child Feeding Programme implemented by Save the Children and supported by UNICEF – is examined and evaluated based on the different OECD criteria in two different contexts; camp and host community, and at different implementation time-points. In the second section of this chapter, the case will be examined against the theory of change that was developed at the initiation of this evaluation within and across the two different contexts. The entirety of this chapter will serve as a basis for the conclusions and recommendations.

7.1. Results and findings based on OECD criteria In order to assess the relevance, effectiveness, efficiency, likelihood of impact and sustainability of the Programme, data from Focus Group Discussions with beneficiaries and heath care providers, Key Informant interviews, document review, KAP survey and economic analysis were analysed. These are triangulated and analysed by context, including camp and host community. Appendix M includes a selection of quotes from key informants and focus group discussions.

Relevance

1) Whether the IYCF Programme and its expected results are aligned with the MOH National Strategy 2015-2017 and relevant in the context of Health Sector Reform in Jordan

• Does the MOH National Strategy 2015-2017 include objectives related to IYCF?


• If so, how do they relate to the IYCF Programme objectives? • Does IYCF feature in the Health Sector Reform in Jordan? • If yes, how do they relate to the IYCF Programme objectives? • Has there been an attempt to meet and incorporate the Government guidelines in the Programme? (SC-J) • Do the IYCF Programme intended results contribute to the aim of strategic documents?

The evaluation examined the extent to which the Programme was relevant in terms of alignment with the MOH strategy and in the context of the Health Sector reform. Nutrition and IYCF as a priority The MOH National Strategy 2015-2017 includes an institutional objective which emphasizes reproductive health, family planning and child health services. Under this objective, the strategic document includes a specific objective related to nutrition: “Enhance the nutritional status of pregnant women and children,” which also includes nutrition specific indicators32 that are listed in Table 7 below. Table 7 - Jordan National Strategy indicators

Indicator Values in 2012 Target in 2017 Programme activities

1. Prevalence of anaemia among women of childbearing age

34% 30% 1. Community-based nutrition programmes: Add iron and vitamins to flour, Iodine to salt. 2. Distribution of vitamin A capsules to children under two years of age

2. Prevalence of anaemia among children under five years of age

32% 28%

3. Vitamin A deficiency rate among children under five years of age

18.3% 15%

4. Exclusive breastfeeding rate for infants under six months of age

23% 25% Women’s Health Project

5. Postpartum Services utilization rate

17.3% 25%

The Strategy does not specify the mode of reaching these objectives except by naming the department involved and stating the name of the initiative, such as “Women’s health project”. Under the institutional objective focusing on improving the quality of health care services and ensuring their continuity, the strategic document includes “improving infrastructure of primary and secondary health care institutions” with a defined indicator related to IYCF: “Number of MOH hospitals participating in Baby-friendly hospital initiatives33’’. The MOH National Strategy 2015-2017 sets new targets for “the baby-friendly hospital initiative”. In 2012, only 18 MOH hospitals took part in the “baby-friendly hospital initiative”. The goals for 2017 are to include all MOH hospitals in the initiative and to collaborate related activities with the World Health Organization (WHO). The Programme subject of this evaluation does not tackle this objective specifically, except for the limited

32 MOH National Strategy page 27. 33 MOH National Strategy page 29.


capacity building activities that have been initiated targeting hospital staff amongst others.

The National Strategy for Health Sector in Jordan 2015 – 2019 also mentions similar priorities. Appendix N includes a table that matches the different objectives/indicators included in the two strategic documents with the IYCF-P activities.

Increasing access to reproductive health services, family planning, and child health is a priority for MOH as it is mentioned in almost all recent health related ministry reports (MOH National Strategy 2015-2017, National Health Strategy 2006-2010, Health Systems Research Strategy 2011-2015, National Reproductive Health/Family Planning Strategy 2013–2017). The MOH National Strategy 2015-2017 has set specific targets for 2017 for several indicators including infant mortality rate, mortality rate of children under 5, maternal mortality rate, etc.

“The Health Systems Research Strategy 2011-2015” report mentions that mother and child’s health is a major priority. The report specifies two priorities related to the perinatal phase; mainly the perinatal death cases and prematurity and low birth weight. In the nutrition section, the report highlights the importance of the prevention of micronutrient deficiencies (vitamin B12, D, and A, iron, and iodine) without setting any goals. This report emphasizes the importance of the collaboration and coordination within the health sector between partners under the umbrella of the Ministry of Health. As part of the community based initiatives (CBI), the reports also stresses on developing child-friendly houses and communities, especially in poor societies and regions.

Reproductive Health reports, namely the National Reproductive Health/Family Planning Strategy 2013-2017, focus on increasing commitments and sustaining financial support to National Reproductive Health/Family Planning issues in order to reach the Demographic Opportunity. Other priorities include reinforcing the role of policies in enabling support for Programme implementation. The only component that could be considered related to IYCF that has been mentioned in the National Reproductive Health/Family Planning Strategy 2013-2017 and that contributes to improving mothers’ and children’s health is offering Family Planning counselling and services in mother and childcare centres.

Interviews with MOH reveal that the latter perceive IYCF as a priority but not the highest priority in comparison to other health issues to address within the limited financial and human resources available. MOH is working towards improving service delivery in general including primary health care. They are looking at furthering the capacity of the Ministry at different levels and regard IYCF as part of this effort which is being supported by UNICEF. MOH welcomes SC-J activities although there are indications that some entities within the Ministry do not regard the Programme as an MOH programme, but rather as a separate programme being implemented by a separate organization (SC-J) and supported by UNICEF. According to SC-J, the initiation of the Programme was based on international priorities and the acute need of the humanitarian response. However, after the first year of the Programme implementation, and upon expanding its activities to include IYCF in host communities, the Programme started taking into account the Government priorities and actively engaging the MOH. By then, MOH ensured the provision of guidance to the Programme and “was on board in all steps” according to SC-J.


Capacity building, policy development, and institutionalization Within the strategic document (National Strategy 2015-2017), a number of objectives and indicators emphasize capacity building of both primary and secondary services including expanding on staff, capacity, and efficiency. Emphasis is also placed on access and comprehensiveness of services. As indicated and further elaborated upon under ‘Sustainability’ below, the Programme includes capacity building components, however, these activities are not fully embedded within a strategic capacity plan contributing to the MOH National Strategy. The MOH strategy document also looks at ensuring capacity improvement in both technical capacity and number of staff. Within this context, the Programme has only mildly contributed to sensitization and capacity building of staff.

Considerable sections of the strategic documents (National Strategy 2015-2017, Ministry of Health Strategic Plan 2013-2017, and others) focus on information management and institutionalization, including capacity building and expansion.

The National Strategy also touches upon development of health related policies and includes a specific objective on “ensuring continuity in the provision of subsistence foods according to nutritional requirements”. Policy has been one key component of the Programme although it is not clear whether the Programme was contributing to policy setting or policy enforcement alone.

Emergency preparedness The MOH strategic document (National Strategy 2015-2017) includes emergency preparedness where one indicator mentions the “existence of an updated contingency/emergency plan at each health directorate including hospitals”. This relates to the Programme since it started off as an emergency IYCF-P.

Contribution to the strategy / Documenting contribution The MOH has developed a two-year strategy called “Jordanian National Strategy of Breastfeeding, 2013-201534” that details actions related to breastfeeding and IYCF practices and emphasizes on introducing baby-friendly hospitals as a core strategy for the coming couple of years. The strategy was based on a field data collection relevant to IYCF and according to the results found, objectives were set. Because of the high needs, the MOH has established a technical committee, which has participated in launching the strategy and introducing the capacity building. The strategy booklet focuses on the following strategic objectives and specifies related activities and targets:

o Capacity building o Coordination o Development of material o Media and communication o Research o Monitoring and Evaluation o Review of policies and guidelines o Awareness

34 The Jordanian National Strategy of Breastfeeding, 2013-2015


SC-J has developed training and educational materials that have been reviewed and approved by MOH. Materials are regarded as relevant and appropriate by MOH staff. The prepared materials are in line with MOH IYCF strategy and guidelines. On the other hand, there is no indication that the materials developed – although in line with MOH guidance – are being used or are regarded as part of MOH materials. It seems that the materials are still considered as “owned” by SC-J. SC-J indicated willingness to give the material to MOH and for the material to be considered as MOH material provided that the name of SC-J is acknowledged. According to Programme documents, the Programme claims to feed into the Ministry’s strategic objectives; however, there is no evidence of any attempt to include results within the Ministry’s own reporting system. SC-J highlights that a key point in supporting IYCF in host communities is to contribute to improvement in breastfeeding and support the Government in improving breastfeeding practices. However, there are no set reporting mechanisms from the Programme to the MOH, such as numbers of PLWs counselled – except in few centres and upon the request of the latter (the centres). The number of beneficiaries receiving assistance is only reported by SC-J to UNICEF with occasional general reports to MOH. There was no indication that achievements were documented by the Ministry as its own achievement and therefore, it is not evident how contribution to MOH strategy is documented other than through SC-J and UNICEF reports. It is evident from the above that although the Programme is aligned with Government priorities, and activities have been shared and discussed with the Ministry prior to implementation, this alignment is stronger in host communities and the MOH lacks programme ownership. It is also worth noting that governmental priorities encompass much more than nutrition and IYCF; they include a spectrum of services under primary and secondary health including reproductive and child health. The Programme is therefore relevant, however, further emphasis needs to be put on the priority needs of the Government including quality of care and institutionalization of services and information.

2) Whether the IYCF Programme and its expected results are aligned with the regional and national plans for the Syria response (namely National Resilience Plan 2014-2016, Jordan Response Plan 2015, Regional Response Plan and Regional Resilience and Response Plan).

• What are the objectives related within the RRPs, NRPs, and 3RPs that include IYCF / maternal and child health / primary health / nutrition? • To what extent do Programme activities and objectives match those included in the RRPs, NRPs, and 3RPs?

The evaluation team found that the IYCF-P in both the camps and host communities responds to the national plans for the Syria response. Both IYCF and nutrition are key outputs in the plans. The Regional Response Plan includes two main outputs that are relevant to the Programme: Output 1.4: Appropriate infant and young child feeding practices promoted.


Output 4.3: Community management of acute malnutrition programmes implemented and monitored.35

“Improvement of equitable access, quality and coverage to secondary and tertiary health care for Syrian refugees” is also included in the RRP and relates to the IYCF capacity building and counselling in hospitals. RRP indicators include “number of women counselled for IYCF” and “number of children screened by MUAC”. Anaemia is also mentioned under reproductive health. The National Resilience Plan (NRP) 2014 – 2016, indicates ‘Training of MOH Medical and Managerial staff on the integrated nutrition Infant and Young Child Feeding (IYCF) approach’ among proposed interventions within the health sector which was not fully adopted by the Programme as shown in section X below.

Within its health sector objectives, the JRP refers to equitable access, uptake and quality of primary, secondary and tertiary healthcare for Syrian WGMB as well as vulnerable Jordanian populations in highly impacted areas, indicating provision of healthcare services catering to maternal and child health within all levels. The Livelihoods and food security sector refers to the need for nutritional support and education to ensure consumption of safe, nutritious, and diversified food promoted among WGMB Syrian refugee populations and vulnerable Jordanian host communities. Maternal, neonatal, and child health workforce were highlighted as needs for a well-performing health workforce to be responsive to achieve the best health outcomes possible.

In a number of key informant interviews, it was apparent that there has been considerable advocacy to include IYCF in the response plan. More emphasis was seen on MUAC in the camp setting but not in host communities. In the latter, focus was seen to be on improving breastfeeding rates through primary health care. It is evident that the SC-J programme responds to the refugee response plan of which IYCF is regarded as an integral part. Compared to other response plans such as those in Iraq 36 Lebanon37, Syria38 and Turkey39, this is consistent where nutrition is often included as a priority during emergencies, however, often times it is hidden under nutrition or primary health.

3) Whether UNICEF’s support to the IYCF Programme is in line with its organizational mandate.

• Does UNICEF include IYCF in its organizational mandate in Jordan? • Does the UNICEF current strategic plan prioritize infant and young child feeding? Why is UNICEF supporting the Programme? • What are the main outcomes related to infant and child health that UNICEF prioritizes?

35 Regional Response Plan 36 http://www.3rpsyriacrisis.org/wp-content/uploads/2016/01/Iraq-–-Regional-Refugee-Resilience-Plan-2016-2017.pdf 37 http://www.3rpsyriacrisis.org/wp-content/uploads/2016/01/20151223_LCRP_ENG_22Dec2015-full-versionoptimized.pdf 38https://www.humanitarianresponse.info/en/system/files/documents/files/2016_hrp_syrian_arab_republic.pdf 39 http://www.3rpsyriacrisis.org/wp-content/uploads/2016/02/Turkey-2016-Regional-Refugee-Resilience-Plan.pdf

http://www.3rpsyriacrisis.org/wp-content/uploads/2016/01/Iraq-–-Regional-Refugee-Resilience-Plan-2016-2017.pdf

http://www.3rpsyriacrisis.org/wp-content/uploads/2016/01/20151223_LCRP_ENG_22Dec2015-full-versionoptimized.pdf

http://www.3rpsyriacrisis.org/wp-content/uploads/2016/01/20151223_LCRP_ENG_22Dec2015-full-versionoptimized.pdf

https://www.humanitarianresponse.info/en/system/files/documents/files/2016_hrp_syrian_arab_republic.pdf

http://www.3rpsyriacrisis.org/wp-content/uploads/2016/02/Turkey-2016-Regional-Refugee-Resilience-Plan.pdf


There is confirmation from UNICEF that IYCF is considered a key intervention and priority for UNICEF in Jordan where the IYCF-P has been regarded by top management as the “flagship programme” for UNICEF. As documented, the office addresses Adolescent and Youth participation, Education, Child protection, Health and Nutrition and WASH to ensure children’s general wellbeing. The mandate of UNICEF includes supporting the government in ensuring that children get the best start to life through supportive policies, quality health services and information for children and families40. As per UNICEF Jordan’s Country Programme Document 2013 – 2017, one of the components of the Programme is “Young child survival and development,” which addresses child mortality, particularly neonatal mortality, as well as child health services and early childhood development. 41 Breastfeeding or IYCF practices were not emphasized; however, their outcomes were indirectly stated in terms of providing technical support to ensure “the best start to life”. In 2014, and following the initiation of the process for the development of the Jordan Response Plan by the Jordanian Government, UNICEF Jordan initiated a Strategic Moment of Review and Reflection42. It was confirmed in the document that general health and nutrition are a priority especially for mothers and children living in vulnerable conditions. Outputs state the needs to provide awareness and promote utilization of child health and nutrition services by parents and care providers. They also stress that the health system should meet high quality newborn, child health, and nutrition services. More emphasis was made on the importance of sustainability, reaching the most vulnerable, cost effectiveness and efficiency amongst others. Therefore, this is indicative that UNICEF is looking at supporting sustainable development and shifting from rapid direct response to integration. This was also confirmed by key informants from UNICEF. There is evidence that IYCF is considered a high priority within UNICEF country office, however, there is also emphasis on ensuring sustainability and efficiency. The latter is an importance consideration to take while evaluating the IYCF-P as it seems the Programme still has the structure of an emergency response in both camp and host settings. Please refer to the section below on sustainability.

4) Whether the IYCF Programme and its expected results are aligned with international policies and guidance related to IYCF-E

• What are the main programmatic activities and their expected outputs and outcomes? • What are recommended activities, outputs, and outcomes as indicated in international guidance and policies related to IYCF-E (including IFE Core group OPS, WHO global strategy, UNHCR SOPs for IYCF, UNICEF programming guide etc.)?

Appendix O includes the list of activities and their expected outputs and outcomes throughout the period of the Programme. A review of the activities and comparisons with

40 http://www.unicef.org/jordan/activities.html 41 UNICEF Jordan’s Country Programme Document 2013-2017 (pg 7 & 8) 42 Strategic Moment of Review and Reflection UNICEF Jordan Country Programme, 2014

http://www.unicef.org/jordan/activities.html


other international guidance show that in what relates to IYCF-E, Programme activities are in line with international guidance including the Operational Guidance for IYCF-E. The Programme design clearly centres around assimilating optimal IYCF practices as recommended by WHO within the community; early initiation of exclusive breastfeeding within the first hour of birth, exclusive breastfeeding during the first 6 months of life, complementary feeding introduced after 6 months until 2 years of age, and continuing breastfeeding until age of 2. The Programme design has been adapted to operate at various programmatic levels, including advocacy and legislation, health service provision, as well as community level action; aligning with UNICEF guidance for creating a comprehensive IYCF strategy43. In terms of advocacy and legislation, the Programme is compliant with the International Code and relevant World Health Assembly (WHA) Resolutions44, particularly at the camp level. In camps, Programme design ensures that required breast-milk substitutes are purchased, distributed and used according to strict criteria45, upholding the provisions of the Operational Guidance on Infant Feeding in Emergencies (IFE)46, and in line with UNHCR SOPs for IYCF.47 At the health service provision level, the Programme has been designed to provide training and capacity building to operational partner and MOH nurses and midwives, as well as staff working in CBOs, to build capacity and ensure unified accurate IYCF messages are delivered to mothers48. The Programme also includes educational material that has been contextualized to the Jordanian context. Counselling cards (UNICEF) have been translated and are being used in Arabic. At the camp level, coordination mechanisms and referral systems are established between partners although these need further improvement (please refer to question #27). Referrals are provided to primary health care centres as needed. Furthermore, counsellors are said to be present at delivery wards in hospitals in both camp and the host community to ensure early initiation of exclusive breastfeeding and provide further education and follow-ups. Programme documents report that the education and counselling components of the Programme take into consideration a behaviour change communication approach by providing mother and child friendly spaces at the community level; facilitating group sessions and providing demonstrations; opening grounds for success story sharing; and providing one-on-one counselling and close follow-ups49. Furthermore, the Programme provides means to support intended behaviour assimilation; for example, cups, breast pumps, and shawls are provided to support mothers to breastfeed. Community mobilizers from within the targeted

43 UNICEF Programming Guide, page 30 44 Latest guidance from WHA: Resolution 63.23 (May, 2010) 45 Table 1, activities 8 and 12 46 Infant Feeding in Emergencies Core Group (2007). Infant and young child feeding in emergencies. Operational guidance for emergency relief staff and programme managers. Version 2.1. Oxford, Emergency Nutrition Network. http://www.ennonline.net/pool/ les/ife/ops-guidance-2-1-english-010307- with-addendum.pdf 47 UNHCR SOPs for IYCF 48 Table 1, activities 15 and 16 49 table 1, activities 2,3,5, and 11


population are utilized on a voluntary basis to identify beneficiaries and provide support with necessary follow-ups. Within camp settings, the Programme structure functions at grassroots level eliciting a preventive aspect to infant and child risk of diseases and infection. The Programme therefore also integrates a CMAM structure, bringing a curative component to address cases with MAM, and further refers them to emergency programmes for the management of SAM, in line with UNICEF Programming Guide on creating a comprehensive IYCF strategy50. The particular strength of the SC-J programme is the comprehensive approach used. The IYCF-E toolkit was used as a reference for the programme which benefited from the IYCF framework. This means that there was an attempt for the Programme to be integrated with education and WASH health sectors, and related integration efforts are thus initiated. Given the above, the evaluation team concludes that the Programme has been based on international IYCF guidance, is comprehensive, and has been adequately contextualized in terms of language and culture to the Jordanian context.

5) To what extent is the Programme relevant to needs and gaps identified through assessments and baseline evaluations?

• What are the main needs and gaps identified through baseline assessments and other nutrition assessments? • What are the main objectives, outputs and outcomes of the Program? • To what extent do objectives, outputs and outcomes match gaps identified through baseline assessments (or progress during Programme implementation) and other nutrition assessments? • What was the evidence used initially to propose the initial IYCF Programme, since there was no baseline assessment? Has the information been confirmed as the Programme went along? Was the degree of malnutrition as expected- or was it lower or more severe? Same about the prevalence of breastfeeding. Have the needs been confirmed? • Would you consider a different approach and targets if you were to re-engineer the Programme, given the experience and information gained from the Programme implementation?

The Programme was initiated as a result of the Syria crisis and the establishment of the Zaatari camp in Jordan. In the midst of the emergency and refugee crisis, IYCF was deemed a priority given the problems with hygiene, diarrhoea and rates of breastfeeding as indicated by key informants. Qualitative data was supported by quantitative data from the interagency nutrition assessment conducted in 2012, which showed low breastfeeding rates. The Interagency Nutrition Assessment in 2012 highlighted sub-optimal Infant and Young Child Feeding (IYCF) practices; 49.6% of children born in the last 24 months at Zaatari camp and 42.7% of those born in the host community continued to be breastfed. In both communities, more than 50% of the children were being breastfed for up to 1 year (65% in host communities and 80.9% in Zaatari camp) and much less than 50% of the mothers breastfeed their children for up to 2 years (19.8% in host communities and 9.5% in Zaatari camp)51. Field reports also highlighted gaps in IYCF practices where early field assessments and observations showed that both Jordanians and Syrians had serious misconceptions about the benefits of

50 UNICEF Programming Guide, page 30 51 (Inter-Agency Nutrition Assessment Report, 2013).


breastfeeding, especially with the definition of exclusive breastfeeding. The lack of education and knowledge was noticed among mothers, caregivers, and even health professionals and doctors52. All pre-crisis data revealed poor IYCF practices in both Syria and Jordan which indicated the need to intervene. The Programme subject of this evaluation aimed at addressing the above-mentioned gaps through a comprehensive IYCF intervention. As the Programme has evolved, the needs have been confirmed and the objectives of the Programme were proven right. Between April and May 2014, another assessment was performed at Zaatari camp and within host communities. The study showed that 46.4% of women practice exclusive breastfeeding in the camp and only 36% have the same practice within the host community. Around 24.2% of those continued to breastfeed for 1 year in the camp versus 39.5% in the host community. Taking the confidence interval of each indicator into consideration, no statistical difference was detected between Syrians living in camps and those living within the host community53. When expansion to host communities took place, a vulnerability assessment was conducted and areas that are most vulnerable were identified. The assessment showed that Northern and Eastern Jordan have the highest proportion of highly and severely vulnerable refugees. Although host community areas selected by SC-J did not fall within those most vulnerable; it was noted that SC-J was guided by MOH to target host communities in the South including Maan and Karak. MOH confirms that the Programme is reaching vulnerable and needy areas as per their guidance. In addition, and according to SC-J, the latter received guidance from UNICEF to target the Southern part of Jordan which was not receiving adequate support. No baseline data was used at the start of the Programme, however, as indicated by SC-J, data from the 2012 JPFHS survey were used as an indicator for the rates of breastfeeding, but the data was not used to assess anything else. The JPFHS survey is planned to be repeated in 2017 and IYCF indicators will be included. The assumption is that since the IYCF Programme implemented by SC-J is only in the Southern part of Jordan, indicators on IYCF from JPFHS for this particular area could be compared with those in other areas and might be indicative of an impact of the Programme. These assessments done highlight major gaps that are reflected by low rates of IYCF practices (exclusive breastfeeding, early initiation of breastfeeding, introduction of complementary feeding, breastfeeding up to the age of 2 years, etc.) in both Syria and Jordan before the crisis. It was evident to think about targeting Syrian refugees and providing them with education, counselling and assistance and implementing IYCF-P in refugee camps. Nevertheless, based on the preliminary database, gaps found initially among the host community have contributed in setting wider objectives. In addition to the assessments that showed a gap in infant feeding (breastfeeding), focus group discussions with mothers showed that the Programme filled a knowledge and practice gap /need illustrated by mothers. Women explained that they benefited from the Programme

52 Field article, Fänder & Frega, 2014 53 Inter-Agency Nutrition Assessment Report, 2014


(and hence that the Programme was actually needed), as their prior knowledge and practices were incorrect. Focus group meetings with health care providers, mothers and key informant interviews showed that the Programme is filling a gap in terms of service provision. In camps for example, it was noted that in case the Programme was not implemented by SC-J, another organization would be needed to step in to fill the gap. Similarly, in host communities, staff indicated a gap in knowledge amongst beneficiaries as well as a gap in services. For example, health staff in hospitals would be inquiring about the attendance of SC-J staff highlighting a need to follow up with mothers, hinting that there would be no one else to follow up with mothers. Although the Programme responds to gaps and needs as identified by assessments, no baseline assessment was conducted to actually be able to see the difference in the impact and gaps that the Programme was able to fill. There is evidence to say that nutrition and IYCF presented gaps within the Syrian refugee crisis in Jordan through the interagency nutrition assessment. There is also evidence to show that breastfeeding rates were low in Jordan according to the DHS. The qualitative data compiled as part of the evaluation confirms a need that was filled by the Programme. The Programme was also based on existing indicators and evidence for gaps in IYCF. At the same time, as the Programme itself did not collect any quantitative data at baseline, it was not possible to estimate the extent to which the Programme contributed to filling that gap.

6) Whether and to what extent the Programme design, including activities and expected outputs, is consistent with the intended results at the outcome and impact levels of IYCF/IYCF+?

• What are the main programmatic activities? • What are the main expected outputs of the Programme? • What are the Programme intended results at the outcome and impact level? • To what extent do outputs and activities match the intended results? • Could you explain the reasons behind your decision to incorporate additional components in the IYCF+ that were not included in the initial program? (Was it to encourage acceptance by the community? An attempt to reduce cost? An initiative to make it more acceptable to the public sector professionals?)

Examining the Programme logical framework, it was challenging for the evaluation team to devise a unique comprehensive framework that summarizes all Programme activities and outputs, as well as outcomes for the entire period of implementation. This is because, as indicated under question #11 below and in Appendix B, the Programme underwent three amendments and two PCA54. The amendments and extensions were done through requests from SC-J with justifications often related to time and addition of activities or target areas. Throughout the amendments, and despite the addition of activities to the original set, amendments #1 and #2 did not have any logical framework attached. The team had to examine three logical frameworks over three separate periods in order to evaluate the Programme design. Appendix O includes all three logical frameworks combined for the three periods. An analysis of the indicators for each of the logical frameworks is also included in Appendix O.

54 Programme Cooperation Agreements


Results from the analysis show that Programme activities seem to be aligned with Programme outputs and outcomes; however, the degree of alignment is somewhat unclear. All logical frameworks included outputs and outcomes as well as indicators, however, the activities examined on the ground did not fully match those documented (discussed further under question #11). Within the identified results frameworks, it was noted that a number of activities including those under the IYCF+ initiative were missing. Specifically, for IYCF+, there were no indicators measuring for example activities related to vaccination or referrals. To understand the design of the Programme, the team resorted to examining the list of activities within the text of Programme documents and included this information in the Programme theory of change, which describes the flow between the different levels of results and is discussed further under section 2 of this chapter. Therefore, there was a gap between Programme logical frameworks and actual Programme activities. For instance, missing data include reports on activities, indicators for vaccination, indicators related to improved awareness, etc. In what relates to IYCF+ activities, in most focus group discussions and key informant interviews (except with SC-J and UNICEF), the terminology of IYCF+ was not known to interviewees. However, when asked about other activities integrated within the IYCF-P, respondents listed vaccination, referrals for newborn care, hygiene promotion and anaemia. Both newborn care and hygiene promotion were missing from the results frameworks included in the project documents. Through SC-J, it was highlighted that vaccination messaging was added initially when “UNICEF saw a potential in the outreach” of SC-J. According to SC-J, and as confirmed by UNICEF, the latter saw that it would be an opportunity for integration with other services given that SC-J is able to reach mothers in different ways. Activities under IYCF+ started with immunizations and referral for Tetanus (TT), Oral Rehydration Solution (ORS). Expanded Programme on Immunization (EPI) and vitamin A were later added, followed by hygiene promotion. As indicated by UNICEF, this was also a way to decrease cost and improve efficiency. These findings were identified through qualitative data collection conducted by the evaluation team; however, there is no documentation for the above in any of the results framework documents. This raises the issue of data reporting and the image that the Programme portrays. Current Programme documentation indicates that the Programme is focused solely on IYCF whereas, in reality, qualitative data show an expanded array of activities, which include the activities under IYCF+. Compared to other results frameworks examined from different sources, the Programme could use more clarity and flow in the documentation of activities, outputs, and outcomes.

7) Whether and to what extent is the Programme design appropriate and accepted by the target community?

• What is the Profile of the different target groups? • Do key informants perceive the Programme design as appropriate and accepted? • Do service providers perceive the Programme design as appropriate and accepted? • Do end users perceive the Programme design as appropriate and accepted?


The Programme targets pregnant and lactating women and children under 5 in the targeted areas as mentioned in the Programme description (section 2 above). SC-J clarifies that in camps, the target is all PLWs and children under 5 and the aim is to reach 100% of the population. Whereas in host communities, SC-J has adopted the strategy of targeting the geographic area that is considered poor or “pockets of poverty” hosting a vulnerable population including Syrian refugees. Within each of the host communities and the hosting partner or agency, S-CJ aims at targeting all PLWs and children under 5 that visit or are hosted by the partner agencies. Interviewed mothers receiving services perceived the Programme as useful and appropriate. The overall impression from focus group discussions with beneficiaries showed that the Programme is well accepted and appreciated by mothers. Mothers saw value within camps given the absence of social support systems and the vulnerability of the camp setting. The mode of implementation of the Programme was seen as appropriate when mothers described the frequency and mode of visits received by SC-J team. Mothers described the Programme as providing awareness and counselling. They described the Programme as consisting of regular visits and provision of awareness focused on 6 topics. Mothers perceived the Programme as providing a comprehensive package. They described how attendance to sessions gradually increased due to mothers’ interest. Emphasis was made on the “quality” of the Programme and the professionalism of staff. The Programme was also seen to be aligned with religious beliefs and local cultures and norms. Acceptance is evident by the perception that mothers (both Syrian and Jordanian) travel from far to attend and perceive the Programme as unique in providing a comprehensive package. Mothers compared their present with previous experiences and described the present as pleasant and filled with care. There was some indication that the 6 awareness topics55 were somewhat repetitive although in most cases repetition was considered useful and women were well aware of the information provided. In addition, in host communities where focus group discussions were conducted, some mothers mentioned that follow up was done by phone or not done at all and recommended increasing the follow up or using other methods to ensure follow up such as media outlets and mobile messaging. Interviews with non-beneficiaries (although limited) showed that the Programme had a potential of being accepted when explained to them. Service providers have a clear understanding of the Programme structure and equally regarded the Programme design and mode of operation as appropriate and accepted by the community. SC-J staff mentioned the importance of having a well-equipped team and the presence of community volunteers within the Programme structure. Programme teams were regarded as assets and contributors to the acceptance of the Programme by Syrian refugees. Key informants were well aware of the Programme and expressed admiration and acceptance of the Programme both by beneficiaries and partners. This is aligned with the literature56 57

The 6 topics were: Breastfeeding, Pregnancy Care, Complementary Feeding, Food Allergy, Anaemia, Newborn Care 55

during illness. 56 UNHCR, UNHCR: Syrian refugees at model school in Lebanon, 2013, http://yalibnan.com/2013/05/23/unhcr-syrian-refugees-at-model-school-in-lebanon/ accessed on October 9, 2016 57 UNHCR. Learning for a Future: Refugee Education in Developing Countries, 2001

http://yalibnan.com/2013/05/23/unhcr-syrian-refugees-at-model-school-in-lebanon/

http://yalibnan.com/2013/05/23/unhcr-syrian-refugees-at-model-school-in-lebanon/


which shows that counselling and education programmes are usually well regarded and appreciated by beneficiaries. Activities are considered an opportunity for mothers to be heard and be provided with health advice. Mothers perceive the Programme activities, including education and counselling, as “pleasant” occasions to develop healthy and friendly relationships with SC-J staff. The comfortable atmosphere in which these activities were implemented contributed to their success. Similarly, Programme staff spoke highly of the Programme and appeared to be proud of their achievements. As mentioned in section X, there was an indication that part of the value of the Programme was linked to its name (inkaz al toufl; save the child). Women believed that the activities they were participating in were likely to “save” their children. Key stakeholders evaluated the Programme as being needed and designed in an acceptable way. There were indications by some that the design could be more efficient and integrated. However, this was not consistent across key informants. Specifically, in camps, the Programme was seen as highly needed and designed in a way that provides consistent and needed services. In host communities, the Programme design was seen as appropriate by health care providers, but perceptions of key informants varied. Focus on sustainability and integration was a key point amongst some key informants including those in camps. Comparing the camp and host contexts in terms of appropriateness and acceptability of the Programme, the evaluation did not note any significant difference at this level from the target population. In host communities, where focus group discussions were conducted, interviewees did, however, note some gaps in follow up, but there was no indication of a difference in acceptance of the Programme. In both contexts, the Programme was well regarded and accepted. There was also no evidence of a difference between the two camps; Zaatari and Azraq, in terms of acceptability by the target population. There was some indication by SC-J that the Programme initiation was smoother in Azraq given the lessons learned from Zaatari camp. For the latter, it was difficult to launch activities that focused on protection from marketing of BMS and it took SC-J great efforts to enforce the BMS Code in Zaatari. Once the Programme was established in Azraq, SC-J worked on sensitization of refugees before implementing the activities. In addition, SC-J noted that refugees are now provided with education about the BMS Code at Programme entry point; however, the evaluation team could not verify this fact. The evaluation team found enough evidence to show that the Programme is well accepted and appropriate for the target population; however, the design of the Programme needs to incorporate efforts for some sort of continuity for the activities.

8) To what extent the design took into account gender issues?

• Is gender mentioned and taken into consideration in the Programme description? •Are males also targeted by the Programme? If yes, how? • To what extent key informants perceive that the Programme design took into account gender issues? • To what extent service providers perceive that the Programme design took into account gender issues? • To what extent end users perceive that the Programme design took into account gender issues?


SC-J programme description does mention gender issues through an amendment attached to the final PCA of the Programme (Appendix B). It states that the IYCF-P targets children regardless of gender. The document also mentions that although IYCF caravans aim to provide privacy for mothers to breastfeed when appropriate, fathers are reached with the key IYCF messages through sessions conducted in clinics and community centres, and are always encouraged to participate in tent counselling sessions. It is also emphasized that the Programme attempts to empower women by giving them the resources and information necessary to make informed decisions concerning the long-term health of their families58. However, this was not confirmed through FGDs and most key informants. SC-J confirms that at the beginning, the Programme targeted PLWs only; excluding men. Starting 2014, there was greater focus on gender in the sector working group and encouragement to involve men in the sessions. SC-J indicated attempting to encourage men to accompany their wives to education and counselling sessions, adding that there are plans to report on the number of male beneficiaries in the future. The general impression from beneficiaries, service providers, and key informants is that the Programme did not directly target men. The Programme was mainly targeting women and children, although willingness to do so was noted by the Programme staff. Also, recently, there is indication that men were welcomed during the sessions in case they were present and their attendance was noted on the attendance sheets. This practice is only recent. The importance and value of targeting men was highlighted by interviewees, especially in what relates to providing support for mothers and ensuring adequate infant feeding practices. SC-J staff noted the role of male caravan guards in keeping the company of fathers, conversing with them, and sometimes relaying key messages, although guards were not formally trained. The latest barrier analysis59 implemented by IMC in Jordan and Lebanon showed that one of the main enablers for breastfeeding amongst the Syrian refugees were husbands and men in general. It was shown that acceptance by men of the idea of breastfeeding and awareness about the importance of breastfeeding played a key role in the success of breastfeeding. Given cultural specificities, it was seen as most appropriate to target men separately. Women cited examples of lack of acceptability of breastfeeding by men, and of mothers not being allowed to attend activities in case other men were present. SC-J indicated that there was an attempt to recruit male staff to target men; however, the idea was not taken forward. There was some evidence that the Programme did not fully take into consideration the gender issue. Men are considered an important part of decision-making about infant feeding and contribute to change in behaviour. It would therefore be of value for the Programme to further incorporate men as a target beneficiary in the programme; although this would need to be done separately from women to ensure cultural acceptability. Gender is insufficiently addressed in the Programme and is not taken into consideration in the design. It is also not embedded in the design of the different activities.

58 PCA Amendment 5 June 2015 until June 2016: Section 2 : Programme Description Section 2.3 on page 4 59 Barrier analysis on IYCF – IMC – unpublished report.


9) How is the Programme different from other approaches inside and outside of Jordan?

• What are the activities and outputs of the Programme? • What are activities and outputs of existing programmes on IYCF (if applicable)? • What are examples of outputs and activities of IYCF programmes implemented in other countries? • What is the difference in the approach between the implemented Programme and other programmes?

The Programme offers a bouquet of services that revolve around improving IYCF practices. The Programme includes activities that support nutrition for children as well as newborn and PLW health, including vaccination and ORT. The mode of implementation of the Programme differs between camp and host communities, whereby in camps, the Programme is established as a standalone IYCF+ Programme implemented through SC-J caravans and staff. There is indication that Programme activities are coordinated with other organizations through sub-sector working groups in order to avoid duplication in camps, as shown during interviews conducted with key stakeholders. On the other hand, in host communities, the Programme is implemented using external partners’ outlets and venues, where SC-J staff provides education and counselling services on IYCF+. In camp and host communities alike, outreach activities occur through Syrian refugee community mobilizers. Further details about the Programme outputs and activities is included in Appendix O and described above under section 2. The evaluation team conducted a comparison of some of the existing activities related to IYCF in Jordan and found that the IYCF services vary across providers; from education to sensitization to some counselling. Geographic coverage also varies. Organizations that have been found to be engaged in IYCF include Medair, TDH, JHAS, ACF, and IMC. IFH was also found to conduct IYCF activities; however, these are currently being supported by SC-J. Also, during focus group discussions, the names of some centres providing IYCF services integrated within a comprehensive antenatal package were mentioned. For example, Nour el Houda centre was described as providing antenatal care, including breastfeeding messaging. Appendix P includes a matrix of nutrition related activities including IYCF. Compared to existing programmes in Jordan, the IYCF-P implemented by SC-J provides consistent one-on-one counselling which is quite time-intensive. The counselling is home-based or facility-based. This is different from JHAS which also provides counselling; however, in different areas and only through facility-based counselling. The other organizations provide education and some one-on-one counselling; however, it was not shown that the service was as consistent as that of SC-J Programme. It seems that the support provided by other programmes was less structured and more ad-hoc. For education, there were some indications that other organizations were providing considerable incentives, including transportation for women, to attend sessions, which highlights the importance of the effort made by SC-J that provides minimal incentives (snacks during sessions).


It was also found that in most cases, for other organizations, IYCF was seen as integrated within activities – such as a comprehensive package of primary health in the case of JHAS and Nour El Houda. The importance of one-on-one counselling and support has been highlighted as a most effective way to provide support for breastfeeding and lead to change in behaviour. So, the SC-J’s Programme component on counselling is regarded as a key element and contributor to success. Awareness is indeed essential, however, contributes to improved knowledge which does not necessarily lead to a change in behaviour. The integration with other primary or antenatal care services adopted by some other organizations might seem like a good approach that would contribute to reduced cost; however, the quality of IYCF services is yet to be evaluated. In terms of geographic coverage and as depicted in Appendix P, SC-J seems to have the largest reach; however, gaps are still evident. Looking at other interventions and initiatives in the region and internationally (Table 8), the evaluation team found that for camp settings, the modality of implementation of the IYCF-P resembles interventions that are implemented in other emergency settings. Table 8 - Summary of international initiatives

Programme Setting/context Modality Description

North Western Tanzania

Refugee camp Standalone IYCF-P

Establishment of breastfeeding corners

Counselling and education SFP programme Artificial feeding support

Haiti Acute Emergency Standalone IYCF-E Programme

Establishment of baby tents Counselling and education SFP Media and awareness campaigns

Turkey Refugee camp Standalone IYCF-P

Establishment of IYCF tents Counselling and education Artificial feeding support Protection (upholding the code)

Greece Informal settlements

Standalone IYCF-P

Establishment of baby tents Counselling and education Artificial feeding support

Lebanon Refugee setting and protracted emergency

Semi-integrated IYCF-P

Counselling and education through primary health centres, hospitals, and within communities

Capacity building for health care providers and BFHI

Protection (upholding the code) Artificial feeding support

Ethiopia Refugee setting / acute emergency

Fully integrated IYCF-P

Integration of IYCF services within the Productive Safety Net Programme (PSNP) including education and counselling through antenatal care


Community follow up and mobilization

For example, in refugee camps of North Western Tanzania, one programme was set around establishing breastfeeding corners around health facilities within the camp setting, to provide awareness sessions as well as breastfeeding counselling. It was also similarly attached to a targeted supplementary feeding programme for both the mother and child. Infant formula was prescribed where a wet nurse could not be identified for the infant. Referrals or health consultations were provided through the programme, and a capacity building component was implemented training health care staff and community health workers utilized as breastfeeding promoting agents (Machibya, 2007) 60. In Haiti too, a similar programme included baby tents for awareness raising, breastfeeding counselling, and was attached to a targeted supplementary feeding programme. It also had a media component promoting breastfeeding through radio broadcasts, leaflets and community events. A blanket supplementary feeding programme was provided in this context as a prevention measure (Pantella , 2001)61. Other examples include programmes in Turkey and Greece, where IYCF support occurs through breastfeeding tents where counselling and education services are provided. Artificial feeding support and supporting the implementation of the Code is also one component that is usually incorporated in programmes. In some cases, such as in Haiti, ready-to-use-infant-formula is provided by the agency itself, whereas in the case of the Jordan Programme, SC-J only ensures that it is properly prescribed. Therefore, for what relates to an emergency or refugee context, a number of similar modalities were found. On the other hand, also in camps, a few examples were found where IYCF is being incorporated within health programmes, specifically RH programmes or even mental health programmes62. Such modalities portray an example of integration which is currently being advocated for in a number of IYCF publications63. IYCF is being seen as most effective when it is streamlined with existing interventions. It is evident that the SC-J Programme in camps, as it stands, uses the former approach with some attempts to integrate and incorporate other activities within the IYCF-P such as the case of IYCF+. For programme activities implemented in host communities, modalities differ between countries and contexts. In Lebanon for example, the IYCF-P was set up in healthcare centres, hospitals, and within the communities. Referrals to lactation consultants for BMS prescription were made if needed. Mothers were identified through community outreach activities.

60 Machibya (2007). Breastfeeding support in the refugee camps of North Western Tanzania. Field Exchange 31, September 2007. p31. www.ennonline.net/fex/31/refugeecamps 61 Pantella (2011). Save the Children’s IYCF Programme and linkages to Protection, Food Security and Livelihoods in Haiti. Field Exchange 41, August 2011. p64. www.ennonline.net/fex/41/save 62 IMC programme – unpublished report. 63 IYCF-E friendly framework – add reference + add other references.


Capacity building for healthcare staff was provided on a national level, and further, a baby friendly hospital initiative (BFHI) was supported through training and capacity building (Bariani & Shaker Berbari, 2015)64. Similar to the SC-J Programme in host communities, the programme in Lebanon provided IYCF services in health care centres through an IYCF counsellor that provided the service separately from other services provided by the centre. Examples from Ethiopia show IYCF integrated in the Productive Safety Net Programme (PSNP) that is under a Federal Food Security Programme. There, health workers were supported to carry out education sessions, screening for growth promotion, as well as provide supplementary feeding. IYCF education in this context was provided alongside other relevant topics such as improving food production. IYCF counselling was also provided to mothers during pregnancy and after delivery at the hospital, PHC and community levels. Community follow-ups were provided as necessary, and referrals were provided to pregnant women encouraging iron supplementation during pregnancy as well as vitamin A during postnatal period. A community mobilization approach involving influencers at different levels of the community was designed and centred on a training-of-trainers approach. Other activities targeted fathers, religious leaders, and girls’ after school clubs (Fox, 2012)65. Other examples ranged from providing IYCF services as a separate activity implemented by the supporting organization to programmes that were fully integrated within existing structures. The extent of integration varied depending on the length of the crisis and the extent to which the country had existing IYCF support structures. The SC-J IYCF-P in Jordan provides activities that are similar to other initiatives internationally; however, consideration needs to be given to the context of this implementation and the extent to which it is sustainable, particularly in this protracted refugee setting. Other organizations in Jordan are attempting to provide integrated IYCF services and therefore can be used as models to test different approaches.

10) Is the Programme design able to reach the most vulnerable, such as refugee mothers, informal and formal sector working mothers, to promote exclusive breastfeeding practices?

• Who are the target population reached by the Programme and by each kind of activity? • What is the demographic profile of beneficiaries reached by the Programme and by each kind of activity? • What are the approaches taken to reach out to hard to reach population? Are the adopted approaches sufficient?

The Programme targets pregnant and lactating women as well as children under 5 in camps and host communities. The Programme uses different modes of targeting as listed below: Camps:

o Caravans o Camp/community outreach o New arrival area

Host communities:

64 Darjani and Shaker Berbari (2015). Infant and young child feeding support in Lebanon: strengthening the national system. Field Exchange 48, November 2014. p20. www.ennonline.net/fex/48/infant 65 Fox (2012). Integrating Infant and Young Child Feeding and the Productive Safety Net Programme in Ethiopia. Field Exchange 44, December 2012. p60. www.ennonline.net/fex/44/infant


o Partner centre o Mobile caravan (only 1) o Community outreach (volunteers) – CBOs, schools and other community venues o Hospital outreach o MOH centres

In camps, the Programme strives to reach 100% of PLWs and children under 5. For targeting, six IYCF caravans are established (3 in Zaatari camp, one in EJC camp, and 2 in Azraq camp) for mothers who seek needed support. Likewise, an IYCF caravan is established in Raba’a Al Sarhan arrival centre, providing areas for mothers to breastfeed. In order to conduct outreach within camps, the Programme also uses Syrian Refugee volunteers as community mobilizers to identify potential beneficiaries. In host communities, SC-J uses outlets such as health centres, CBOs, etc. to target beneficiaries reached by these venues. It was indicated that in host communities, beneficiaries are targeted through those outlets and no filtering happens. Thus, all clients that visit the service provider (CBO, health centre, etc.) are considered beneficiaries who will benefit from the IYCF-P. SC-J noted that partner centres have been selected in most disadvantaged and vulnerable areas based on vulnerability assessments. This was also mentioned by MOH senior management, which confirmed that centres have been chosen in most disadvantaged areas. Given this fact, SC-J considers that that the target population receiving the service is also most vulnerable and, therefore, no further targeting occurs except during education sessions. During awareness sessions, mothers are asked about their needs and interests, and the sessions are tailored and topics are chosen accordingly. As noted before, SC-J conducts an assessment for the centre and selects the centre based on an evaluation of the economic profile and vulnerability of the population served by the centre. No further details were given on the process. SC-J acknowledges that although the venues are chosen in vulnerable areas, the targeted population – for example in private hospitals – would still not be consistent in terms of vulnerability. For example, women from different socio-economic status would be targeted. SC-J still values this kind of targeting which includes all beneficiaries in the same area, as it assumes that it would contribute to improvement in IYCF indicators at the National Level. This method of targeting might be valid given the difficulty in filtering beneficiaries in this context. At the same time, this general targeting through blanket provision of services to all clients visiting a centre in a vulnerable area may not be ideal given that other more vulnerable beneficiaries may be missed by the Programme. Both in camps and in host communities, the Programme uses Syrian Refugee volunteers as community mobilizers to identify potential beneficiaries. In addition, a mobile caravan was later added during the project implementation period to reach mothers in remote areas in the South66.

66 Time 4 and Time 5 in Appendix O


Outreach and identification of beneficiaries happens at different levels as indicated by SC-J staff and beneficiaries:

1. Through hospitals where staff visits hospitals and counsels mothers. Hospitals also share the list of mothers who have delivered to be followed up by SC-J at the community level.

2. In health centres, CBOs and schools. 3. Through the newly established mobile caravan. Informal tented settlements are

mapped and a schedule is set to visit the settlements once a week (to be verified). Despite large outreach efforts by SC-J, there were indications that access was difficult for mothers with more than one child and mothers living far. Recommendations given by beneficiaries and Programme staff to improve reach included increasing the number of mobile units and using media and communication outlets. On the other hand, compared to other programmes, where beneficiaries are given incentives such as transportation to attend and access programme services such as awareness sessions, it is noted that the Programme was able to reach beneficiaries without providing any considerable incentive. The evaluation team found that the Programme targets the most vulnerable in camp settings; however, there was no evidence to show that the demographic profile of those targeted in host communities represent the most vulnerable. Targeting was mainly based on areas rather than individual targeting.

Effectiveness

11) To what extent the Programme demonstrated expected results at all levels (i.e. inputs/outputs level indicators, knowledge, behaviour change and coverage of Program)?

• What are inputs, outputs, and outcome indicators of the Programme?

• What are reported actual results as indicated in Programme reports? •To what extent the Programme demonstrated expected results at the output and outcome level? • What is the perceived impact of the Programme as reported by key informants, service providers and end-users? • Did the Programme identify new benefits that accrued from its implementation? Did the contacts with the beneficiaries as well as the stakeholders provide for new information, especially as concerned capacity building and public education? • What additional improvement was noticed from better hygiene and contacts with health professionals – beyond nutrition and breastfeeding?

As indicated in questions #6 above, it was challenging for the Programme team to examine the different versions of the Programme logical framework that exist given the time frames of implementation. Still, the team worked on three frameworks and examined the extent to which Programme results and outputs were achieved.


The three analysed frameworks are as follows67: Framework #1 covering the period from January 1, 2013 to December 31, 2013 Framework #2 covering the period from January 1, 2013 to June 30, 2015 Framework #3 covering the period from July 1, 2015 to December 31, 2015 Target indicators were plotted against actual indicators for each of the outputs and results. For some outputs, new indicators without targets were added just to provide an idea about the numbers reached. For example, number of counselling visits was added to the tables. Targets were retrieved from the log frames themselves, whereas figures and information for actual results reached were retrieved from the Programme tracker as well as Programme quarterly progress reports. For indicators that were reported in both the tracker and Programme narratives, they were included in the tables and a comparison was conducted. An analysis of the results was conducted for each of the indicators as shown in Appendix O, which include the logical frameworks and an analysis of the demonstrated achievements at the different levels. Below is a summary. Consistency and change within frameworks over time Frameworks were modified by the Programme based on modifications of the Programme activities, however, as noted above, for two of the amendments and despite a change in the activities, the log frames were not modified. Indicators were modified throughout the implementation period mainly to reflect the change in the areas targeted and the kind of activities implemented except for changes made in amendments #1 and #2, which were done later. The main activities and indicators that were added throughout the period of implementation include those related to capacity building (for the first period of implementation, i.e. PCA 1 – Jan 1, 2013 to June 30, 2015). A new framework was devised as of July 1, 2015 which includes indicators for outputs such as: partnerships with CBOs, orientation sessions on IYCF, and indicators related to the SFP programme. On the other hand, a number of indicators were missing throughout the frameworks as noted under before. Although there were activities related to IYCF+ which included the integration of vaccination and other health related messaging, however, the frameworks did not capture this information except until the last framework, where attempts were made to revise the log frame and reflect integration of newborn care. Analysis of targets and indicators In general, the analysis of the different frameworks showed that targets that were reported on and that were related to number of beneficiaries were achieved in most of the areas and during most of the periods as shown in Appendix O. SC-J reported through narrative reports on a quarterly basis and tracked changes of some indicators on an excel sheet on monthly basis using quantitative data. The reports of SC-J did not include log frames but some of them have summarized numbers of beneficiaries throughout a certain period of time. However, there was a clear discrepancy between the numbers calculated from the tracker and the numbers reported in the quarterly reports. Numbers from the tracker sometimes exceeded those in the quarterly reports; at other times they were less. This might be because the

67 To note that the overlap in dates between framework #1 and #2 is due to the fact that these are two frameworks part of one PCA. The evaluation team used the same frameworks that are included in the amendments.


reporting period was not clearly defined in reports; however, it also gives the indication that there is a need for clear and unified reporting mechanism on indicators. For targets related to MUAC assessment, it was difficult to retrieve the figures as this information was not available in the tracker but was reported in the quarterly reports on an ad-hoc basis. The analysis showed that targets related to number of trained individuals including MOH and CBO staff were reached although a difference in numbers was noted. Numbers reported in the training attendance sheets shared by SC-J with the evaluation team were different from those in the quarterly reports submitted by SC-J to UNICEF.. It was difficult to analyse targets that were not quantified such as those related to policies and knowledge change amongst women who participated in awareness sessions. No information about these two indicators was available in the tracker. Information about policy was retrieved from Programme reports and showed that indicators were achieved; however, no data could be retrieved about the knowledge change within PLWs attending awareness sessions. The indicator about the percentage (%) of women able to recall three messages as a result of the awareness session was not reported on. This does not mean that the Programme did not achieve this indicator because results from focus group discussions did show that women were able to remember a number of messages from awareness sessions and this was consistent throughout the focus group discussions. However, it shows a gap in reporting on this indicator. In summary, findings show that the targets related to establishment of caravans and provision of training sessions, workshops, and awareness campaigns are reached in all areas. This was in line with the targets set for PLW having access to IYCF services. Targets were not set for around 20% of the indicators, which made the interpretation of their results difficult. Most of the data from the trackers were not in line with the ones from SC-J reports.

12) Did the Programme identify and reach out to the most vulnerable? What was the difference in reach between those in camps and those in host communities, especially in the hard-to-reach areas? Whether and to what extent special attention has been made to reaching out to those beneficiaries who are hard-to-reach? Any difference between outreach (one-to-one) versus caravan (fixed-site provision of health education)?

• Which areas are categorized as hosting the most vulnerable according to National assessments? • How the Programme identified the most vulnerable and based on which criteria? • What are the approaches adopted by the Programme to reach camps and host communities? • What are the approaches adopted by the Programme to reach out to hard to reach groups & who are the hard to reach groups? • What is the number and geographic distribution of beneficiaries in camps and host communities targeted by each of the activities? • What is the number of beneficiaries reached through the caravan and through outreach? Their profile & what are the similarities and differences/Achievements and challenges between the two approaches?"

According to the SC-J trackers and as shown in Appendix O, a cumulative of 52,493 and 143,994 (beneficiaries both PLWs and children) were reached in camps (through caravans) and in host communities (through outreach) respectively, between January 2013 and


December 2015. These numbers are the result of the summation of the numbers of PLW and U5 in Zaatari, EJC, Azraq, and RS on the first hand and host community on the other hand. No specific numbers about the beneficiaries reached through the mobile unit in the South were reported by SC-J. Segregation by type of activity and by indicators was also shown in the log frame sheets (Appendix O); however, differences and similarities or criteria of identification were not mentioned anywhere. The 2015 national vulnerability assessment68 showed that Northern and Eastern Jordan have the highest proportion of highly and severely vulnerable refugees, and that Amman and the South have the highest proportion of ‘least vulnerable’ families, relative to the total population. SC-J confirmed that despite targeting least vulnerable areas, still, within these areas, SC-J chose the most vulnerable sub-regions including Maan, Karak and Jordan Valley. At the same time, and when discussed with MOH, the latter confirmed that they approve of the selection given that the centres chosen where reaching to most vulnerable population.

The approaches adopted to reach targets were described in question #10 above.

13) Whether and how has the integrated approach, IYCF+ contributed to the expected results?

• What are the programing changes that occurred as a result of the integration and their implications? • What are the changes in the Programme results, if any, after the integration of IYCF+? • To what extent did key informants and service providers perceive integrating IYCF within other Programme activities as important and of added value? • Would additional “integration” provide an added-value? What programmes could be considered? Were health promotion and preventive measures augmented as a consequence of the IYCF program?

Initially, upon the introduction of IYCF+ within the Programme, IYCF+ consisted of including messaging about immunization and referrals for TT. ORS was then added as well as hygiene promotion. In the latest programme PCA69, SC-J indicated integrating “an evidence-based package for newborns” into IYCF activities (IYCF+) and through the existing modality of home visits to mothers and newborns. The package included home visits, counselling and referrals. The purpose of this integration as explained in the documents is to improve the capacity of community health workers in providing home-visits for newborn care services and promote best practices through Infant and Young Child Feeding plus (IYCF+) approach. Despite the integration of these services, Programme indicators were not fully modified and indicators on number of mothers receiving counselling on newborn care were not fully integrated until the last log frame. When asked about IYCF+, most interviewees were not aware of the terminology of IYCF+ although service providers were aware of the activities under this Programme. Activities such as channelling vaccination campaigns through the IYCF-P were mentioned through FGDs with healthcare providers. It was indicated that health care providers inform beneficiaries about necessary vaccinations for mother and child and mothers are referred to clinics upon need. Staff takes necessary information of newly arrived mothers and follow-ups on child vaccinations and clinic referrals with the beneficiaries, verifying referrals at the clinic

68 Jordan Refugee Response - Vulnerability Assessment Framework Baseline Survey May 2015 69 PCA (1/7/2015- 30/6/2016)


premises. Although hygiene promotion was included in the package as documented, however, this was not confirmed in Jordan Valley, where health care providers only mentioned vaccination and anaemia as complementary activities. This is probably because staff was mostly newly employed in Jordan Valley (few months). In terms of contribution to Programme results, such services are bound to have a role in improving health outcomes. A number of mothers in more than 3 focus group discussions mentioned benefiting from information on vaccination and anaemia. The information was consistent throughout the focus group discussions. This was also mentioned by SC-J staff. It was apparent that providing a comprehensive package is of added value. However, the Programme team was not able to verify the contributions at the impact level. Assessment of the impact necessitates a time lapse between the intervention and noted changes among the Programme beneficiaries as a result of this intervention. It also requires the adoption of an experimental evaluation approach not adopted within the context of this evaluation.

14) Whether and to what extent the technical support was provided to the implementation team to address issues and ensure achievement of results?

• What is the composition/profile/affiliation of the team providing technical support? • What is the frequency and nature of meetings between the technical support team and the implementation team? • How many trainings were conducted for the implementation team? • What are the subject areas covered by the training conducted for the implementation team? • What are the areas of the Programme implementation tackled by the Technical Support provided and related results? "

The technical capacity of the team implementing the Programme is highly regarded by the different stakeholders including mothers, service providers themselves, key informants including MOH and other coordinating partners. The team providing the services is composed of counsellors, educators, coordinators and outreach workers/volunteers. According to Programme documents confirmed by SC-J staff, technical support provided to field staff and volunteers is comprised mainly of a 5-day training at the commencement of the Programme, as well as refresher training as required, to ensure ongoing capacity building70. The training material was reviewed by the evaluation team and was found to comply with international and national guidance. In addition, as indicated by SC-J, the training is repeated on a yearly basis and the information is updated upon need. Refreshers are conducted on a monthly basis to include revisions as well as weekly ad-hoc tasks as mentioned by SC-J; however, the evaluation team could not verify this for lack of clear related documentation. As indicated by SC-J, coordinators go through a full assessment for all counsellors on a monthly basis in each of the locations. Training of staff was mentioned in a few recent SC-J reports; however, no further elaboration was found on frequency or staff assessments conducted. According to SC-J, all training material is based on the UNICEF/WHO training manual for IYCF. It is clear from FGDs with mothers that the technical information retained by mothers is in line with WHO recommendations.

70 PCA 1


To note that in Jordan Valley for example, some SC-J team members were very new (employed for a few months only). Still, they showed commitment and knowledge about the Programme objectives and functionality. UNICEF has contributed to providing essential technical capacity to SC-J through guidance during the PCA development process and then throughout the implementation of the Programme. SC-J management team also receives technical assistance related to nutrition and IYCF, however, less frequently and upon need, from the SC technical advisor. UNICEF also provides ad-hoc assistance upon request from SC-J management team. UNICEF’s technical assistance was mentioned to be more concentrated around the development of the PCA and activities at the start of the project cycle. UNICEF has also been present later during implementation; however, the role was more seen as active during the development phase. There is evidence to show that there was enough technical assistance and support for Programme staff, which was apparent from the positive perception that prevailed amongst interviewees including Programme beneficiaries. It is evident that Programme staff received extensive and continuous training on IYCF as well as on communication skills.

15) How has the quality of Programme been ensured, especially with respect to the Global Operational Guidance on Infant Feeding in Emergencies and other standards?

• What are the main IYCF-E guidance notes adopted by the Programme including the Operational Guidance and Sphere? • Are there National policies on IYCF or IYCF-E? What are they? • How are the Programme activities aligned with the national and international standards? • What are monitoring tools that are used? • To what extent did activities abide by recommendations related to international and national guidelines on IYCF-E?

In 1996, the MOH released instructions promoting breastfeeding and alerting people about breast-milk substitutes (BMS). Despite this procedure, breastfeeding rates were still declining and BMS use was still increasing, and this was attributed to the lack of knowledge and education among the population. In response, a local national code, known as the “Jordanian code for protecting and promoting breastfeeding”71 that restricts and controls the general promotion of BMS was issued in 2009. Afterwards, other supplementary materials have been released. Details about the national code and other IYCF supporting laws can be found in Box 1 below.

71 Jordanian code for protecting and promoting breastfeeding, 2009


Box 1: Jordanian Code information and IYCF supporting laws in Jordan

Source Detail

Jordanian code for protecting and promoting breastfeeding

The Code includes the following details: Promoting breastfeeding especially exclusive breastfeeding Endorsing good breastfeeding practices Detecting promotions for BMS Elaborating the health care system Restrictions related to infant milk production (promotion forbidden,

free sample distribution forbidden, etc.)

Law 47/2008 Promotes healthy lifestyle and breastfeeding, forbids media promotion of BMS, etc.

Law 25/2007 Forbids the imprisonment of pregnant women

Law 80/2001 States the criteria to register medications, supplements, BMS, etc. It also mentions the right to form a committee responsible for controlling the composition of BMS. People who promote BMS will be subjects to legal actions (going to jail or paying fees).

Law 8/1996 Preserves the right of pregnant women at work (the right of maternity leave (10 weeks), the right to take days off starting the second trimester, the right to take an unpaid leave for up to one year after delivery, etc.). This law obligates having a baby-friendly work environment.

Law 61/1976 The mother who delivered is forced to breastfeed her baby if the father doesn’t afford to pay for a wet nurse.

Law 2/1972 Defines the maternity leave as a paid leave that does not exceed the 90 days.

Law 35/1966 States the right for pregnant women working in the military field to have a maternity leave.

Law 38/1965 Defines the maternity leave as a paid leave that does not exceed the 70 days.

Other than the breastfeeding code, supplementary material was created and distributed to support proper IYCF practices. Such activities include guidelines about ‘Guidance Notes on Appropriate IYCF Practices in the Current Refugee Emergency in Jordan’ and Standard Operating Procedures on ‘Donations, Distribution, and Procurement of Infant Formula and Infant Deeding Equipment in Syrian emergency in Jordan’ developed by Nutrition Sub-Working Group and approved by MOH (initial in 2012 and updated 2014). Additionally, in refugee settings within the camps, IYCF policy facilitates IYCF support in this context (Rashid


et al., 2016)72. The Camp Management Committee (CMC) assures that the labour law in Zaatari camp is being implemented and this stands for allowing mothers who breastfeed to take a one-hour break within their working hours. Furthermore, organizations (such as UNHCR) work to adhere to the SOPs that forbid the general distribution and optimize a proper handling of BMS. In 2015, a joint statement on IYCF73 for Syria crisis (in all neighbouring countries including Jordan) was issued and endorsed by Nutrition stakeholders/cluster consisting of UN agencies and NGOs to protect against “unnecessary” illness, mainly related to diarrheal infections that lead to mortality, especially in malnutrition cases. The target of the statement is to promote breastfeeding as “lives saviour” since it reduces all risks of infection associated with poor hygiene and sanitation in the Syria emergency context. Assistance and support are provided for all breastfeeding mothers and cases of psychological trauma and depression are referred to specialists. HIV positive mothers should also be encouraged to exclusively breastfeed. For children who have lost their mothers or who are separated from them, wet nursing should be explored. When BMS is the only solution, milk should be administered with a cup and this should be accompanied by training for safe preparation. The complementary feeding after 6 months of age is also mentioned in the statement. Similar to all documents supporting breastfeeding, the joint statement strongly discourages the uncontrolled general distribution and use of BMS. SC-J Programme documents indicate that compliance with the local code, other guidance, as well as the International Code and relevant World Health Assembly (WHA) Resolutions74 are addressed at the camp level, where Programme design ensures that required breast-milk substitutes are purchased, distributed and used according to strict criteria, as proposed through Programme activities. SC-J also works on upholding to the provisions of the Operational Guidance on Infant Feeding in Emergencies (IFE)75. Basic and skilled support is provided to mothers and caregivers utilizing various channels; community based, primary and secondary healthcare services, and outreach, to ensure timely and appropriate feeding support, aligning with Sphere Standards requirements76 and the UNICEF/WHO programme guide for IYCF. To ensure the quality of the Programme, SC-J conducts regular capacity building and training to Programme staff as indicated under question #15. SC-J senior staff (coordinators and team leaders) conduct follow up visits and assessments to their staff on a regular basis to ensure information provided is up-to-date. In addition to the follow up with staff, SC-J indicated conducting random checks / calls to mothers, where feedback is received about the services provided. Also, each counsellor has a monitoring form for all the mothers where number of follow ups is documented. The monitoring tools are based on existing international tools

72 Rashid et al. IYCF-Friendly Framework pilot in Jordan, Bangladesh and Kenya, retrieved from www.ennonline.net/fex/51/iycfpilotjordanbangkenya, on September 10, 2016 73 UNICEF, 2015, Joint Statement on Infant and Young Child Feeding, Nutrition stakeholders call for appropriate feeding of infants and young children in the Northern Syria 74 Latest guidance from WHA: Resolution 63.23 (May, 2010) 75 Infant Feeding in Emergencies Core Group (2007). Infant and young child feeding in emergencies. Operational guidance for emergency relief staff and programme managers. Version 2.1. Oxford, Emergency Nutrition Network. http://www.ennonline.net/pool/ les/ife/ops-guidance-2-1-english-010307- with-addendum.pdf 76http://www.spherehandbook.org/en/infant-and-young-child-feeding-standard-1-policy-guidance-and-coordination/

http://www.ennonline.net/fex/51/iycfpilotjordanbangkenya

http://www.spherehandbook.org/en/infant-and-young-child-feeding-standard-1-policy-guidance-and-coordination/


including the ‘simple assessment form’ and ‘in-depth assessment forms’ suggested in the UNICEF IYCF-E programming guide. Samples of the monitoring forms have been examined by the evaluation team and consist of thorough collection of information related to mothers and infants. Unfortunately, the forms do not yet have a system for data compilation and, therefore, it was not clear how the information would be used other than by field staff that follow up on each of the mothers.

16) What are the major factors (internal and external, any issues related to gender) influencing the achievement or non-achievement of the results?

• What are the operational factors influencing the progress of the Programme? •What are the technical factors influencing the progress of the Programme? • What are the environmental, including socio-cultural factors, influencing the progress of the Programme? • What are other factors influencing the progress of the Programme (barriers and opportunities for implementation)? • Could a twinning of the SC-J Programme public with the sector outlets and staff be beneficial to the effectiveness of the IYCF Programme? • Was there an attempt to “piggy bag” on these outlets and facilities? If so, how beneficial was it? If not, why?

17) Were there components of the Programme that worked well or did not work well? Why so, and why not?

• What are the activities within the Programme that are perceived as most effective? and why? What are the activities within the Programme that are perceived as least effective? and why? • What are recommendations or lessons learned from the Programme as perceived by key informants, service providers and end users? • Which outputs were achieved and which outputs were not achieved and why? • Could one have achieved similar results with different interventions that may be less costly?

18) What are the areas of strengths and weaknesses? Are there other approaches that will help overcome the weaknesses, if any?

• What are the perceived areas of strength and weaknesses by Programme staff, key informants, service providers and end users? • What are recommendations or lessons learned from the Programme as perceived by Programme staff, key informants, service providers and end users related to overcoming challenges and weaknesses? • Is there a mechanism to capture weaknesses & strengths (monitoring findings or field-level data) and feed them into the Programme design?

Factors influencing achievements Results from focus group discussions with beneficiaries, health care providers and key informant interviews generated a list of enablers and disablers that contributed to Programme achievements. These are listed below. Staff capacity, technical and communication skills came through as strong enablers and strengths of the Programme. Solid and strong technical skills of staff were mentioned by a number of stakeholders including mothers and partner agencies. Staff was described as heavily committed and passionate about their work. Staff communication skills and friendliness seem to play a role in acceptance of messages and attendance of sessions. Mothers indicated being motivated by the staff to attend and alluded to the birth of friendships between mothers and the staff.


Team work was also referred to as a main enabler for the smoothness of the work as indicated by health care providers/SC-J staff. The outreach efforts and presence of local/Syrian community mobilizers that are mothers was perceived as contributing to the strength of the Programme and a component that was perceived as working well. FGDs with mothers showed that being Syrian facilitated the process of approaching mothers, especially that mobilizers themselves are mothers most of the time. On the other hand, in host communities, the presence of Syrian outreach workers was mentioned during one focus group meeting as hindering outreach to Jordanian mothers. This could not be verified in other areas. Also, the reach of the team was mentioned as an enabler to enrolling as many mothers as possible. This was further facilitated by the partnerships with a large number of CBOs which provided an opportunity to access mothers. The mobile van was also given as an example to improve access and reach. Engaging men was seen as a positive approach to facilitating the change in behaviour of mothers, especially given the role that men play as enablers in the family. Examples were given of mothers being motivated to change behaviour because their husbands “approve” it. There was disagreement about the mode of engagement; in camps it was less acceptable for men to attend with mothers, whereas in host communities, it was suggested that men accompany women to sessions. The name of the organization and its connotation in Arabic that it “saves children” were seen as an enabling factor contributing to the value of the Programme. As previously mentioned, mothers linked the name to the benefit it gives to their children which contributed to further acceptance of the Programme. The mode of operation of the Programme in host communities is subject to improvement and more integration within existing services. The Programme is implemented through centres and CBOs where SC-J staff provides IYCF counselling and education. This approach of the Programme which consists of using partner venues to provide services may be creating a parallel system and might contribute to dependence and therefore to harming the existing systems. The attempt to piggy bag on existing public systems was not clear in the approach taken. Further elaboration on efforts for integration within host communities is found under section 7.1.d below. Programme components most and least effective One of the main Programme activities that was valued as strength provider to the Programme and contributor to Programme outcome is the one-on-one counselling and follow up provided to mothers. Mothers and health care providers valued this activity as one leading to a change in behaviour through the troubleshooting that occurs and the support follow up that is provided on a one-to-one basis. Staff highlighted the amount of time that is spent with mothers which ranged from 30 to 45 minutes and which was perceived as vital for mothers and the Programme. Although the educational sessions were also mentioned as contributing to a change in knowledge, the counselling was seen as most effective in changing behaviour.


Another Programme activity considered as contributing to the outcome and achievement of the Programme is the work done on enforcing the Code and the support for artificial feeding. It was noted during FGDs and KIIs that the perception of mothers has changed regarding artificial feeding whereby it became custom not to be provided with infant formula. SC-J staff reported facing difficulties in enforcing guidelines related to the Code at the beginning of the project at Zaatari camp. However, as the initiative progressed, it was apparent that guidelines were more accepted. One lesson learned that staff mentioned was the importance of ensuring sensitization about the Code for mothers and heath care providers in the camp. On the other hand, in host communities, staff noted the lack of implementation of guidance related to the Code, which was mentioned as a drawback. The education sessions are noted to be working well as evidenced by the amount of related information acquired and shared during focus group meetings. Mothers were well aware of the topics that were taught and kept repeating them. At the same time, mothers recommended increasing the variety of topics, given that the sessions were seen as repetitive. One main challenge that was mentioned by staff and beneficiaries was the difficulty accessing services (mainly educational sessions) for families living far from the centres and for those with more than one child. Distance and having more than one child were factors that contributed to beneficiaries’ decreased attendance ability. However, these factors were counteracted by recommendations to ensure access through provision of mobile services and child care activities during sessions. Factors hindering achievements: (operational, environmental, technical, etc.) Standard Programmatic Reports submitted by SC-J to UNICEF on a regular basis include sections on “Lessons Learned”, “Constraints” and “Suggestions for Programme improvement” that SC-J captured from the field. These sections incorporate information about challenges faced during the implementation of the Programme and state the changes made to address them. In addition, through focus group meetings and discussions, the evaluation team noted factors that contributed to hindering achievements. These include the following: The role of hospitals and the challenge of lack of enforcement of the Baby Friendly

Hospital and other policies on BMS marketing were noted as hindrance in host communities, which might be contributing to poor IYCF practices. Interviewees noted that since those policies are mostly enforced in camps, a variation was noted in host communities where the policies are not enforced. Therefore, extent of policy enforcement was seen as an environmental factor in the system affecting Program outcome.

Program challenges as perceived by mothers and health care providers included access to services, where it was more difficult for those living farther to access the Program mainly in host communities. In addition, mothers with small babies and more than one child found it more difficult to attend given that they needed to have child care to be able to attend the sessions. Heat was also mentioned as a barrier. Suggestions related to providing entertainment for children and some transportation for both mothers and staff for security reasons were mentioned. Mothers also mentioned the repetition of topics as a limitation. In addition, diversified mode of awareness provision was mentioned including use of media and communications outlets. Mothers in host communities


recommended use of mobile services, media outlets, IEC material, more marketing of the Programme, text messaging, and travel incentives.

The evaluation noted some differences in the one-on-one follow up with mothers between camps and host communities. In host communities, follow up was sometimes less consistent and often conducted remotely with mothers, which might be considered as a factor contributing to or hindering achievement. This is possibly due to the distance that mothers or counsellors have to travel. Suggestions related to use of media and mobile technology were mentioned as a mending for this challenge.


Efficiency As explained under the section 5 on limitations, this section aims to undertake a cost efficiency evaluation of the Programme over the course of the evaluation scope. The below results are based on the limited available data.

19) To what extent did the actual or expected results justify the costs incurred (considering the difference of approach and Programme design for camps and host communities)?

• What is the cost of implementation of each activity in each of the target areas per number of target groups per approach adopted? • What is the relative weight (in terms of resources) of each activity? i.e. how much do the resources expanded improve the feeding of one infant compared with the encouragement of breastfeeding? • Did the resources expanded vary between camps and host communities within each period? Within camps? Within host communities? • What lessons could be derived from this cost difference, if any? •To what extent is the cost considered justifiable for each of the activities? •Are there any economies of scale enhanced throughout the different periods of Programme implementation? • Has the cost of each intervention changed over time? • It is noticed that the Programme support and the indirect cost constitute the largest component of cost as compared to the actual provision of food supplements and other materials to the beneficiaries. What measures have been taken to reduce this imbalance and decrease the overhead costs?

20) Have the resources been used and has the Programme been implemented as efficiently as possible? (Resources here refer to those allocated for this Programme, as well as any other external resources to enhance efficiency such as referral to other services)

• What was the burn rate of the Programme? • How did Programme stakeholders manage resources (human, financial, information) to ensure efficiency? • Were the Programme staff members concerned about the cost? Were there measures taken to reduce costs? What were these measures, if any? • How did the recruitment of volunteers from the community contribute to the reduction of costs? If so, would you consider a greater involvement of the volunteers? What are the advantages and disadvantages – and their impact on cost reduction? • Did the staff consider measures that could yield the same results yet at a lower cost? • Did the Programme benefit from the services of other concerned entities? if yes, how? What are the related mechanisms adopted? How efficient are they?

21) What is the cost of the response per unit of aggregation as compared to the cost being incurred by other IYCF Programmes implemented in Jordan?

• Have there been other similar programmes to the IYCF that have been implemented by SC-J in the past few years? If so, have there been lessons learned from these programmes that have influenced the current Programme? How can this current Programme compare in terms of efficiency, costs and effectiveness? • Have there been similar programmes to the IYCF that have been implemented by SC-J in similar situations outside Jordan, say Lebanon, Turkey, Syria? Any lessons learned? Any impact on the current Programme? • Have there been similar programmes conducted by other NGOs or INGOs and/or the public sector that have yielded improvement to the current Programme? • Under more optimal conditions, how could the Programme have been more effective and efficient? Were there hurdles that affected performance? Could they have been overcome? If so, what were the challenges and the obstacles that were faced?


An examination of expenditure throughout the last two years of implementation, shows clearly an increase in expenditure was incurred as the Programme moved forward (Fig. 4).

Figure 4 - Budget outlays over the period of the Programme evaluation

The increase has been in large part driven by expansions in the scope of the intervention including additional breastfeeding caravans and geographic expansion to cover host communities. In May 14, 2014, additional activities were included such as the acquisition of 2 more caravans as well as a mobile caravan for remote areas. Other activities include the operation in Azraq camp and the introduction of the new geographical locations such as Jordan Valley, the undertaking of new activities such as capacity building for MOH and CBO staff and the conduction of sensitization workshops to MOH, and a breastfeeding media campaign. Starting Q3 in 2015, there was a decrease in expenditure at the start of the new project possibly due to (a) lessening focus on supplies and additional structural cost after 3 years of initial start-up implementation period, and (b) reduction in SCJ contribution. Against this background, the number of beneficiaries (PLWs counselled) has been steadily increasing over the whole duration of the project for all target areas, whether camps or host communities, as attested in Figure 5.

0

100000

200000

300000

400000

500000

600000

2014 Q12014 Q22014 Q32014 Q42015 Q12015 Q22015 Q32015 Q4

Quarterly Expenditure

Expenditure


Figure 5 - Total number of PLW’s counselled (one-on-one) over the duration of the project

The increase in expenditure discussed above seems to be justified in light of this steady increase in geographical coverage. In terms of overall budget, there was a decrease in overall budget after mid 2015 due to the shifting of the Programme from initial 3 years of high level capital (human, supply related costs) investment period to a period of recurrent and continuation of project. Cost efficiency and productivity gains in the implementation of the various activities of the IYCF intervention throughout the project period assessed under this evaluation require consideration of capital investment cost and recurrent cost. Cost efficiency analysis – Method of analysis 1: including staff and non-staff cost To probe into the cost efficiency and efficiency gains of the IYCF intervention, an examination of staff, non-staff, and total costs of implementing activities in camps, host communities and both was undertaken. An analysis of the cost of each of the main activities per beneficiary was conducted for each of the areas (when possible), using total cost and then staff cost alone. Below are details about how the analysis was conducted and based on what information and data. In terms of staff costs, the evaluation team had access to the staff salaries paid towards activities including education and counselling for only two years (2014 and 2015). Staff cost included staff engaged in direct implementation or support staff or staff pertaining to the work of volunteers. Although the program started in 2013, however, no staff costs could be obtained for this year with the needed details (stratified by area) from SCJ finance section and, accordingly, year 2013 was omitted from the analysis. Therefore, information on staffing salaries used was from 2014 and 2015.

0

5000

10000

15000

20000

25000

30000

35000

Zaatari Azraq EJC RS HC Total

24710 256

546

32732861

734 348

4152

8095

5059

2245 8850

12438

20627

10391

29791489

17136

31995

2013

2014

2015

Total


In addition, stipend paid to volunteers were not readily available and those had to be computed as follows:

nr. wks/yr = 365/7 nr. working days/wk = 5

nr. working hrs./day = 4 hourly pay = JOD1.5/day

a aSV N

Where the subscript a denotes the target area (camp or host community) and N denotes the number of volunteers in a certain target area as tabulated below in Table 9. Table 9 - Number of volunteers working in each target area

RS Azraq EJC Zaatari HC

0 12 4 5 18

For non-staff costs, information was not readily available over the whole length of these two

years, and had to be imputed based on the IYCF audit reports. This meant that for 2014, the

figures pertained to PCA1 expenses (start of the Programme to June 2015), while for 2015

they pertained to PCA2 expenses (July 2015 – December 2015). The various non-staff cost

items were not readily assigned by camp and host community, but rather aggregated over all

these. In some instances, some items could be logically assigned to either camps or host

communities (e.g. breastfeeding caravans). In other instances, the expense item applied to

both, in which case it was split between the two in proportion to the total number of

education and counselling visits/sessions in each of these locations over the 2014-2015

period, that is 35% for camps and 65% for host communities.

In terms of outputs (i.e. numbers of beneficiaries, education and counselling visits etc.), the figures and numbers were compiled over the years from the tracker shared by SC-J and referred to in sections above. In terms of division, the information was reported on by camps and host communities, where the total expense items were split between IYCF activities – counselling and education – and non-IYCF activities. To that end, the proportion of the expense item assigned to non-IYCF activities was first calculated as follows for each area:

staff salaries

staff salaries

years non IYCF

non IYCF non IYCF

years non IYCF IYCF

p a

Where a is an indicator variable that equals 1 if the expense item relates at least in part to non-IYCF activities (e.g. SFP) and 0 otherwise, and p denotes a proportion. Needless to say, the value of pnon-IYCF was set to one if the expense item went solely to non-IYCF activities (e.g. ‘screening – diaspect/lancet/cuvet’). Next, the proportion assigned to counselling was derived as follows:


1 if the IYCF expense item was dedicated solely to counselling;

0.5 1 if the expense item was split between counselling and education;

0 if the

counselling non IYCF

counselling non IYCF

counselling

p p

p p

p expense item went solely to education.

Finally, 1education counselling non IYCFp p p .

A list of these proportions is tabulated by cost item in Table 14 below. According to these figures, the proportion of the IYCF costs to the overall non-staff cost of the Programme is 92%. These proportions were then used to apportion non-staff expense items to counselling and educational IYCF activities. Table 10 below tabulates key indicators of beneficiaries for the years 2014 and 2015 as well as the imputed and aggregated counselling and education staff and non-staff salaries. Finally, and to further interrogate these data, unit costs for counselling and education activities were derived. The idea is to try and derive how much of the money expended on each education and counselling visit/session can be attributed to staff, volunteers or support, or an aggregate of all those, and likewise for non-staff costs. To this end, for each year and target area, the number of education and counselling

beneficiaries was derived from Table 10 as follows:

1. Education: The sum of the number of first time daily visits to caravans of women who

are:

a. Pregnant,

b. Lactating, and

c. Other/indirect

2. Counselling: The sum of mothers receiving initial counselling with children aged:

a. 0-6 months,

b. 6-12 months, and

c. 12-24 months

in addition to:

d. Number of follow-up visits to tents.

Thereafter, counselling and education salaries were computed as follows:

1. Education: The sum of the salaries paid to:

a. Education staff,

b. Volunteers, assumed to be 50% of the total salaries paid to volunteers (the

other goes to counselling), and

c. Support staff, assumed to be 50% of the total salaries paid to support staff

(coordinator, administrator, midwife, health-nutrition manager, programme

manager, nurse and lab technician) and going towards IYCF (excluding SFP or


anaemia officer). It is worth noting that the total sum of staff salaries going

towards to IYCF activities was allocated in proportion to the sum of salaries

going towards IYCF education, counselling and volunteers; the remaining going

to SFP and anaemia officer.

2. Counselling:

a. Counselling staff,

b. Volunteers, assumed to be 50% of the total salaries paid to volunteers (the

other goes to education), and

c. Support staff, assumed to be 50% of the total salaries paid to support staff (the

remainder is assumed to go to education).

To these were added the non-staff costs.

Once all these numbers were derived, the unit costs were then computed by dividing any staff

or non-staff figure of interest by its corresponding number of beneficiaries. Per-unit-costs are

tabulated in Table 11 below.

Discussion of analysis 1 – Including staff and non-staff cost

These figures show that for education and counselling, for both staff and non-staff, cost

increased substantially at the level of overall cost (Table 10). However, the unit cost on

education decreased substantially (half) from 2014 to 2015 in host communities, while it

increased in camps (18.65 to 26.72) (Table 11). As for education unit costs, staff unit costs

decreased only for host communities, while they can be seen increasing for camps, such that

the total unit cost for camps also increased. The total cost on Counselling also increased

substantially in both host communities and camps, but unit cost on counselling decreased in

both camps and host communities, which suggest an increase in the number of counselling

beneficiaries through efficiency improvement of the Programme (Table 11). Finally, the

analysis found that unit cost for activities in camps had increased, mainly due to an increase

in staff cost for camps.

These figures exhibit gains in productivity and economies of scale between 2014 and 2015

mainly in host communities, but not in camps. This finding also suggests that the Programme

was able to reach more beneficiaries at a lower cost in host communities through more

efficient use of human and other resources at the community. This is complemented at the

demand side by the fact that, with the passage of time, early beneficiaries would inform by

word of mouth or other means other women about the usefulness of the programme, a

process facilitated by social, kinship or other kinds of networks which might have contributed

to increased access and utilization of services by non-beneficiaries.


These findings support what has already been hinted in the beginning of this analysis that

proportional increase in new beneficiaries between 2014 and 2015 was greater than increase

in overall cost in this time period. Improved economies of scale and productivity gains in the

host community programme was particularly notable. Therefore, there is evidence that

overall, the project implementation has indeed become more cost-efficient in host

communities.

Method of analysis 2: Including staff cost only

To further investigate the reasons behind the staff cost increase over the years for camps,

the figures disaggregated over camps were investigated. Below are key indicators of

beneficiaries for 2014 and 2015, in addition to tabulated counselling and education staff

salaries (Table 12). Recall that only staff costs were available for each camp, so the analysis

was restricted to this category only and non-staff costs were ignored. It is worth noting that

for the purpose of the analysis, no great loss of information will result from the omission of

non-staff costs (e.g. transportation, indirect costs etc.), as these are less likely to witness

productivity gains to the extent as human labour, at least not in the relatively short-term of

the project implementation. Moreover, these costs constitute human resource, variable,

costs of implementing the intervention, and as such would capture virtually all of the time

trends inherent in the evolution of unit costs (e.g. economies of scale, productivity gains etc.)

and hence would allow to make an informed judgment as to cost efficiency and productivity

gains. The per unit staff cost by camp and year figures are listed in Table 13 below.

Discussion of analysis 2: Including staff cost only

While no 2015 data are available for RS and all unit costs exhibit a decrease between 2014

and 2015, except for Zaatari camp where unit cost for Education session substantially

increased from 8.5 JD/visit to 55 JD/visit; the picture is different for all three camps. In the

case of EJC, unit costs for support staff increased for both counselling and education. This is

due to a drastic increase in the support staff costs (especially on coordinator and midwife)

without a commensurate increase in the number of beneficiaries. As for Zaatari, a dramatic

increase in the unit cost of education staff and volunteers as well as support (while all

counselling unit staff costs decrease, if mildly so) is noted. This is mostly due to the fact that

the number of visits to caravans has decreased dramatically between 2014 and 2015 for this

camp (i.e. from 8,208 to 2,153) while the salaries for education staff have increased

dramatically as a reflection of major drive in the recruitment of education staff that seemed

to have failed to project the volume of activity in 2015.


Table 10 - Key beneficiaries, staffing and non-staffing costs for camps and host communities

All Camps Host communities Total

2014 2015 2014 2015 2014 2015

# of (children under 5) 44,055 123,523 18,143 40,138 62,19

8 163,661

# of first time daily

visits to caravans

Pregnant 4,302 1,340 8,102 8,024 12,40

4 9,364

Lactating 3,456 2,912 7,517 18,738 10,97

3 21,650

Other/indirect 5,962 6,895 9,958 25,244 15,92

0 32,139

Total 13,720 11,147 25,577 52,006 39,29

7 63,153

children u5 7,753 9,623 16,187 46,853 23,94

0 56,476

# of mothers receiving

initial counselling

sessions

0-6 3,241 4,511 4,020 8,272 7,261 12,783

6-12 376 490 124 128 500 618

12-24 326 333 8 30 334 363

Total 3,943 5,334 4,152 8,430 8,095 13,764

children u5 4,106 9,520 4,289 14,805 8,395 24,325

# of Follow up visits to tents 11,685 21,476 3,189 10,197 14,87

4 31,673

# of education or support sessions in the caravan 4,867 9,531 3,983 7,818 8,850 17,349

# of education or support sessions in partners

locations 5,079 7,165 0 800 5,079 7,965

Education expenses

(JOD)

Staff

Education staff 93,395 120,643 39,486 63,178

132,8

81 183,821

Volunteers 16,425 16,425 14,079 14,079

30,50

4 30,504

Support staff 75,149 99,974 54,794 71,563

131,7

80 181,612

Total - staff 184,969 237,042 108,359 148,820

295,1

64 395,937

Other Training 1,135 1,135 2,215 2,215 3,296 3,296

Training For IOU Staff 0 0 0 0 0 0

Transportation &

storage costs

16,223 17,263 31,668 33,698 47,11

4

50,134

Supplies

41,175 40,955 62,991 47,597 103,1

12

88,470

Indirect costs

12,318 2,220 24,044 4,333 35,77

2

6,446

Total - other

70,851 61,572 120,919 87,843 189,2

94

148,345

Counselling expenses

(JOD)

Staff

Counselling staff 133,721 218,419 51,109 105,021

184,8

30 323,440

Volunteers 16,425 16,425 14,079 14,079

30,50

4 30,504

Support staff 69,804 112,789 54,794 71,563

131,7

80 181,612

Total - staff 219,950 347,633 119,982 190,663

347,1

13 535,556


Other Training 1,135 1,135 2,215 2,215 3,296 3,296

Training For IOU Staff 0 0 0 0 0 0

Transportation &

storage costs

16,223 17,263 31,668 33,698 47,11

4

50,134

Supplies

41,175 42,369 62,991 50,224 103,1

12

92,511

Indirect costs

12,318 2,220 24,044 4,333 35,77

2

6,446

Total - other

70,851 62,986 120,919 90,469 189,2

94

152,386

This table can be also found in Excel sheet in Appendix Q


Table 11 - Unit costs (JOD/visit) for camps and host communities

All Camps Host communities

2014 2015 2014 2015

Education Staff 13.48 21.27 4.24 2.86

Non-staff 5.16 5.52 4.73 1.69

Total 18.65 26.79 8.96 4.55

Counselling Staff 14.07 12.97 16.34 10.24

Non-Staff 18.61 15.32 32.82 15.09

Total 27.56 34.23 20.58 13.10

All (education +

counselling)

Staff 9.70 7.87 21.20 6.55

Non-staff 18.65 26.79 8.96 4.55

Total 14.07 12.97 16.34 10.24

This table can be also found as excel sheet in Appendix R


Table 12 - Key beneficiaries and staff costs per year of implementation and camp

RS Azraq EJC Zaatari

2014 2015 2014 2015 2014 2015 2014 2015

No. of(children under 5) 5,552 4,152 4,813 53,53

5 5,366 7,250

28,32

4

58,58

6

No. of first time daily visits to

caravans

Pregnant 577 508 146 467 237 57 3,342 308

Lactating 1,531 1,197 468 1,112 54 138 1,403 465

Other/indirect 1,552 3,296 781 1,879 166 340 3,463 1,380

Total 3,660 5,001 1,395 3,458 457 535 8,208 2,153

children u5 4,079 4,152 1,123 3,630 73 305 2,478 1,536

No. of mothers receiving

initial counselling sessions

0-6 0 0 374 1,059 275 344 2,592 3,108

6-12 0 0 206 261 16 70 154 159

12-24 0 0 154 147 57 133 115 53

Total 0 0 734 1,467 348 547 2,861 3,320

children u5 0 0 978 2,638 223 851 2,905 6,031

No. of Follow up visits to tents 0 1,528 5,240 2,343 3,410 7,814 12,82

6

No. of education or support sessions in the caravan 237 406 461 3,944 985 1,106 3,184 4,075

No. of education or support sessions in partners

locations 0 129 936 2,376 1,487 1,745 2,656 2,915

Education expenses (JOD)

Education staff 2,717 0

17,83

5

27,67

1

34,94

3

17,34

4

37,90

0

75,62

8

Volunteers 0 0 9,386 9,386 3,129 3,129 3,911 3,911

Support staff 5,345 0

32,17

6

38,15

8 9,899

22,31

7

27,72

9

39,49

8

Total - staff 8,062 0

59,39

6

75,21

5

47,97

1

42,79

0

69,54

0

119,0

37

Counselling expenses (JOD)

Counselling staff 0 319

15,13

3

46,86

6

18,70

2

22,14

1

99,88

6

149,0

93

Volunteers 0 0 9,386 9,386 3,129 3,129 3,911 3,911

Support staff 0

12,81

6

32,17

6

38,15

8 9,899

22,31

7

27,72

9

39,49

8

Total - staff 0

13,13

5

56,69

4

94,41

0

31,73

0

47,58

7

131,5

26

192,5

02

This table can be also found in excel sheet in Appendix S


Table 13 - Unit education and counselling unit costs (JOD/visit) by target and year of implementation

RS Azraq EJC Zaatari

2014 2015 2014 2015 2014 2015 2014 2015

Education Staff Education staff 0.74 - 12.78 8.00 76.46 32.42 4.62 35.13

Volunteers - - 6.73 2.71 6.85 5.85 0.48 1.82

Support staff 1.46 - 23.07 11.03 21.66 41.71 3.38 18.35

Total Total 2.20 - 42.58 21.75 104.97 79.98 8.47 55.29

Counselling Staff Education staff - - 6.69 6.99 6.95 5.60 9.36 9.23

Volunteers - - 4.15 1.40 1.16 0.79 0.37 0.24

Support staff - - 14.22 5.69 3.68 5.64 2.60 2.45

Total Total - - 25.06 14.08 11.79 12.03 12.32 11.92

All Staff Education staff 0.74 - 19.48 14.99 83.41 38.01 13.97 44.36

Volunteers - - 10.88 4.11 8.01 6.64 0.84 2.06

Support staff 1.46 - 37.29 16.72 25.34 47.35 5.98 20.79

Total Total 2.20 - 67.64 35.83 116.76 92.01 20.79 67.21

This table can be also found in excel sheet in Appendix T


Overall discussion

With a different contextual and programmatic approach, data was obtained from an

emergency IYCF-P implemented for Syrian refugee mothers on a short-term stay on a Greek

island by an NGO called NPI which uses volunteers for its implementation. In this programme,

200 mothers are counselled for short durations (usually a month or so) with the help of 12 to

16 volunteers working pro bono in addition to 4 staff who are paid a salary of EUR400/month

each and housing benefits amounting to an aggregate of EUR3,000/month over all staff

members. This puts the staff costs at 3000+400×4 = EUR 5,400/month, or approximately

JOD4,340 of mostly counselling visits. If divided by the number of beneficiaries (200) and then

multiplied by 12 months/year, the unit cost of counselling would amount to an equivalent of

around JOD260/beneficiary/year.

A few caveats are warranted here. This figure is per beneficiary rather than visit, as in our

case; if multiple visits are affected per mother, the unit cost figure could be substantially

lower. Also, it should be noted that this Greek programme is a high emergency one, with a

very quick turnover of both women (who usually stay for a short while on the island before

moving to mainland Europe) and volunteers, which means that the transaction costs

attendant to constantly recruiting new volunteers and counselling new women would be high

and yet are unaccounted for here.

Still, our overall counselling unit cost of JOD15.32/visit/year in 2015 compares very favourably

to the Greek figure, and shows that it is highly efficient while the Greek one is highly

unsustainable. Even if recalculation is done for the counselling unit cost based on the number

of women counselled (as proxied by the number of mothers receiving initial counselling

sessions – 13,764 women; (see last column of Table 10), the figure becomes JOD50.56

(inclusive of staff costs) for 2015, an order of magnitude lower than the Greek one.

Further evidence from the literature77 finds unit costs, inclusive of both staff and non-staff

costs, of peer counselling ranging between US$74 to US$233 (JOD52 to JOD165). The low

value was obtained from a scenario analysis performed in a Ugandan study looking into the

cost of individual peer counselling for the promotion of exclusive breastfeeding78 while the

high value comes from a similar study estimating the costs of peer counselling in Zambia (data

available on request). A range of US$ 2.2 to US $6.4 was found for costs of health facility

promotion services, representing the lowest and highest costs identified in the literature

77 Chola, L., Fadnes, L. T., Engebretsen, I. M., Nkonki, L., Nankabirwa, V., Sommerfelt, H., ... & PROMISE-EBF Study Group. (2015). Cost-effectiveness of peer counselling for the promotion of exclusive breastfeeding in Uganda. PloS one, 10(11), e0142718. 78 Chola, L., Nkonki, L., Kankasa, C., Nankunda, J., Tumwine, J., Tylleskar, T., & Robberstad, B. (2011). Cost of individual peer counselling for the promotion of exclusive breastfeeding in Uganda. Cost Effectiveness and Resource Allocation, 9(1), 11


(Levin et al., 2003 and Orach et al., 2007).7980 In another study (Holla-Bhar et al., 2015)81,

counselling about breastfeeding (and fortification) is found to have the greatest potential to

reduce the burden of child mortality and morbidity (Bhutta et al., 2008)82. In this regard, this

study estimates that breastfeeding programmes cost from US$ 100 to US$ 200 per death

averted, making them equally or more cost- effective than measles and rotavirus vaccination.

All these findings establish the fact that the IYCF-P in Jordan was relatively cost-efficient as

compared to similar programmes implemented in other countries. The Programme

counselling unit costs are on the lower side of published counselling costs, and clearly

corroborate the cost-efficiency of the intervention.

In response to the evaluation sub-question, the following is to be noted:

Answers to the cost of implementation of each activity in each of the target areas per number

of target groups per approach adopted are found tabulated in Tables 11 and 13 and further

discussed above.

Regarding the relative weight (in terms of resources) of each activity i.e. how much the

resources expand to improve the feeding of one infant compared with the encouragement of

breastfeeding; no cost data mapped to individual activities were made available to the

evaluation team in order to carry out this analysis.

As for the question on whether the resources expanded vary between camps and host

communities within each period and within context, the unit costs presented in Table 11

differ considerably between host communities and camps; in both 2014 and 2015, education

costs are higher in camps, while counselling costs are higher in host communities in 2014, but

decrease much faster than camps and hence become lower in 2015. As for differences across

the years, the general trend is for a decrease in unit costs with time, with an exception of

Zaatari camp. Finally, between the camps (see Table 11), the unit costs vary widely and are

highest in EJC, followed by Azraq and then Zaatari (which, however, displaces Azraq for

second place in 2015). The unit staff costs are lowest for RS in 2014; as for 2015 and non-staff

costs, there is no data available to complete the analysis at this level.

In terms of lessons that could be derived from this cost difference, the intervention should be

better planned and attempt to more rigorously forecast the number of beneficiaries and their

79 Levin, A., Dmytraczenko, T., McEuen, M., Ssengooba, F., Mangani, R., & Van Dyck, G. (2003). Costs of maternal health care services in three anglophone African countries. The International Journal of Health Planning and Management, 18(1), 3-22 80 Orach, C. G., Dubourg, D., & De Brouwere, V. (2007). Costs and coverage of reproductive health interventions in three rural refugee affected districts, Uganda. Tropical Medicine & International Health, 12(3), 459-469 81 Holla-Bhar, R., Iellamo, A., Gupta, A., Smith, J. P., & Dadhich, J. P. (2015). Investing in breastfeeding–the world breastfeeding costing initiative. International Breastfeeding journal, 10(1), 12 82 Bhutta, Z. A., Ahmed, T., Black, R. E., Cousens, S., Dewey, K., Giugliani, E., ... & Shekar, M. (2008). What works? Interventions for maternal and child undernutrition and survival. The Lancet, 371(9610), 417-440


growth (or lack of it) throughout the years of implementation in order to better and more

efficiently deploy resources. On the other hand, it has been noted that the situation is very

dynamic and has been affected by the unpredictability of the number of beneficiaries

exhibited by SC-J given the sudden opening and closing of the borders.

Regarding whether there were any economies of scale enhanced throughout the different

periods of Programme implementation, Tables 11 and 13 show that in general, economies of

scale grew between 2014 and 2015 with the exception of some camps. Here, it is worth

noting that the time horizon may be too short for the full economies to materialize.

The section above discusses at length the general decrease in the unit cost of counselling and

education services except for some camps (see Tables 11 and 13).

In terms of measures taken to reduce this imbalance and decrease the overhead costs, Table

10 shows that non-staff costs (‘other’) decreased significantly between 2014 and 2015 in both

camps and host communities. This would suggest that the growth of the number of

beneficiaries and attendant economies of scale has enhanced the unit productivity and

efficiency of the support resources.

Regarding volunteers’ effect, these generally have considerably lower unit costs than regular

staff (see Table 13). This means that their overall contribution would be to lower the overall

unit cost of counselling and education services, and therefore enhance their efficiency. Yet,

their lower level of education could mean that considerable resources need to be deployed

to train them to become competent workers, something that could be further exacerbated

by their high turnover.

LRFP-2015-9121419# #IYCF Programme Evaluation – Final Report – Draft 3 – Oct 15, 2016

Table 14 - Proportions of IYCF and non-IYCF cost by non-staff cost item

Camps HC All

IYCF IYCF IYCF

Education Counselling Non-IYCF Education Counselling Non-IYCF Education Counselling Non-IYCF

Training 0.451 0.451 0.097 0.474 0.474 0.051 0.459 0.459 0.083

Training For IOU Staff 0.451 0.451 0.097 0.474 0.474 0.051 0.459 0.459 0.083

Transportation & storage costs 0.451 0.451 0.097 0.474 0.474 0.051 0.459 0.459 0.083

Supplies

Breastfeeding Caravans 0.500 0.500 0.000 0.500 0.500 0.000 0.500 0.500 0.000

Electrical Connection plus Generator 0.500 0.500 0.000 0.500 0.500 0.000 0.500 0.500 0.000

Caravan operating costs 0.500 0.500 0.000 0.500 0.500 0.000 0.500 0.500 0.000

Furniture for caravan 0.500 0.500 0.000 0.500 0.500 0.000 0.500 0.500 0.000

Fencing/Gravat 0.500 0.500 0.000 0.500 0.500 0.000 0.500 0.500 0.000

IYCF/counselling cards and flip charts 0.500 0.500 0.000 0.500 0.500 0.000 0.500 0.500 0.000

Posters, leaflets, Roll ups. 0.500 0.500 0.000 0.500 0.500 0.000 0.500 0.500 0.000

IYCF-E forms and reporting formals 0.500 0.500 0.000 0.500 0.500 0.000 0.500 0.500 0.000

Nutrient dense snacks for BF caravan beneficiaries 0.500 0.500 0.000 0.500 0.500 0.000 0.500 0.500 0.000

Water for BF caravan beneficiaries 0.500 0.500 0.000 0.500 0.500 0.000 0.500 0.500 0.000

Screening (diaspect/lancet/cuvet) 0.000 0.000 1.000 0.000 0.000 1.000 0.000 0.000 1.000

MUAC tapes 0.451 0.451 0.097 0.474 0.474 0.051 0.459 0.459 0.083

Cups/Spoons for bottle exchange 0.451 0.451 0.097 0.474 0.474 0.051 0.459 0.459 0.083

Tent to tent visit kits 0.000 1.000 0.000 0.000 1.000 0.000 0.000 1.000 0.000

Camp office furniture 0.451 0.451 0.097 0.474 0.474 0.051 0.459 0.459 0.083

Communication 0.451 0.451 0.097 0.474 0.474 0.051 0.459 0.459 0.083

Laptops 0.451 0.451 0.097 0.474 0.474 0.051 0.459 0.459 0.083

Projector 0.451 0.451 0.097 0.474 0.474 0.051 0.459 0.459 0.083

Stationary 0.451 0.451 0.097 0.474 0.474 0.051 0.459 0.459 0.083

Cleaning Supplies 0.451 0.451 0.097 0.474 0.474 0.051 0.459 0.459 0.083


Visibility (banners, signs, T shirt-etc.) 0.451 0.451 0.097 0.474 0.474 0.051 0.459 0.459 0.083

Apartment & internal cost 0.451 0.451 0.097 0.474 0.474 0.051 0.459 0.459 0.083

Media 0.451 0.451 0.097 0.474 0.474 0.051 0.459 0.459 0.083

World breastfeeding campaign 2015 0.500 0.500 0.000 0.500 0.500 0.000 0.500 0.500 0.000

Mobile caravan (new line) 0.451 0.451 0.097 0.474 0.474 0.051 0.459 0.459 0.083

Printers 0.451 0.451 0.097 0.474 0.474 0.051 0.459 0.459 0.083

Printing 0.451 0.451 0.097 0.474 0.474 0.051 0.459 0.459 0.083

IT upgrade Hardware and software 0.451 0.451 0.097 0.474 0.474 0.051 0.459 0.459 0.083

Capacity Building For SCJ Staff 0.451 0.451 0.097 0.474 0.474 0.051 0.459 0.459 0.083

Staff Vaccinations 0.451 0.451 0.097 0.474 0.474 0.051 0.459 0.459 0.083

Breast pump 0.000 1.000 0.000 0.000 1.000 0.000 0.000 1.000 0.000

Indirect costs 0.451 0.451 0.097 0.474 0.474 0.051 0.459 0.459 0.083

This table can be also found as excel sheet in Appendix U


22) Whether and how has the integrated IYCF+ approach influenced the efficiency of the Programme?

• To what extent integrating IYCF within other activities affected the unit cost of IYCF activities? And how? • How the IYCF+ affected the performance of the human resources? • What was the unit cost for each additional activity in the revised IYCF+ Programme? • Would additional activities help in reducing the costs? To what extent and what are these potential activities? Are they being considered – or rather have they been considered – in order to improve cost efficiency?

23) Did UNICEF and partners explore the possibility of integrating IYCF in other relevant interventions (e.g. immunization referral or back to school campaigns)? Are there areas of improvement?

•To what extent did the Programme investigate the possibility of integrating IYCF in other programmes? And how? • Were IYCF activities implemented along with other activities such as health, food security or mental health? • What are recommendations for integrating IYCF in other relevant interventions?

As elaborated under the Sustainability section, the Programme is being implemented separately from existing services; SC-J provides the services independently from any other existing health service, be it provided by an existing NGO (in camps) or a health centre (in host communities). However, this evaluation did note efforts perceived by SC-J and some key informants as efforts for integration such as: Coordination with other sectors including protection and education and facilitating

referrals (quote on complaint box – protection). The Programme as it stands is perceived to complement services relevant to health, food security, and WASH sectors, as generally reported by key informant interviews.

Incorporating messaging related to newborn care, immunization, hygiene promotion, SFP, and anaemia (although the latter two may not be considered as fully incorporated) under the IYCF+ initiative. Education sessions promote dietary diversity through items provided within the food basket received by WFP, and hygiene promotion has been integrated to awareness topics provided to beneficiaries, including demonstrations.

Conducting capacity building training for CBOs and MOH staff in an attempt to engage staff and incorporate IYCF messaging within staff’s duties, particularly given the lack of Programme ownership by MOH. However, as noted previously, there is a need for more extensive capacity building efforts.

Collaborating with CBOs and other outlets such as schools to implement activities related to the Programme. The Programme works through CBOs at the field level as well as MOH clinics and hospitals, and private hospitals. The outreach component is channelled through public service sectors; schools, CBOs, and charities, for awareness sessions and further identification of beneficiaries.

The extent to which this attempt for integration contributed to Programme efficiency was examined through two methods; (1) looking at the extent to which the integration affected cost and (2) examining the extent to which the integration affected the quality of the work. It was not possible to discern the cost of the IYCF+ services through the cost analysis since the same staff tasked to provide IYCF services were also providing the additional IYCF+ services. The integration of IYCF+ happened through the integration of messaging that occurred via


existing staff that SC-J had already trained on IYCF. Additional training occurred on IYCF+ that staff used during their field work and service provision. At the same time, the analysis did not note a considerable change in the unit cost between the period where IYCF was implemented as is and the time of integration of IYCF+. Therefore, it may be assumed that the integration of IYCF+ is considered as a cost effective intervention that contributed to increased access to services with minor cost modifications; however, this could not be verified. To note that there was no evidence to show that this integration affected the quality of the work. On the contrary, as noted by mothers, staff, and SC-J, the provision of additional messaging under IYCF+ was considered of added value, where mothers were given the opportunity to have access to a reference point service provider. The process of integration is noteworthy. SC-J was reluctant at first to include activities other than IYCF within the Programme. SC-J saw a need for the staff to be highly focused on the IYCF messaging given the importance of the intervention. The wish to integrate additional services came from UNICEF as a way to take advantage of the reach of the IYCF-P and include additional services. With time, and as the system became well established, the perception of SC-J changed. There is now consideration from the part of SC-J to further expand and integrate additional services including newborn care, growth monitoring in camps, and support for actual immunization and antenatal care.

Other efforts that were mentioned but not taken forward included integration with Early Childhood Development, which was mentioned by mothers as a recommendation. Both SC-J and UNICEF mentioned such a possibility. SC-J had incorporated the cost of setting up ECD spaces for children into one of the proposed budgets for an activity in camps. However, it was not approved due to budget constraints as indicated by UNICEF, so it was not fully integrated due to a required budget increase. In addition, results from key informant interviews showed that some meetings took place between SC-J and UNICEF to discuss opportunities to integrate with ECD; however, these were not taken forward. In addition, as noted by staff managing the ECD intervention at UNICEF, discussions about options and modalities for integration happened between the Health and Nutrition section and ECD staff took place but did not materialize. One of the main opportunities for integration is the institutionalization of activities within MOH services, which is discussed under the Sustainability section.

24) Whether and to what extent the technical and other human resource capacities have been utilized and were appropriate to achieve results?

• What is the number and profile of staff that worked on the different Programme components? Who is doing what? To whom? and How? • Was it feasible to recruit staff with lower qualification – yet capable of good performance – in order to effect savings? • Any personnel performance review? • Any technical follow up? If yes, how?

The organizational structure of staff working on this Programme is included in Appendix V. The chart is based on a structure where in each locality a counsellor and educator is


accompanied by two Syrian outreach / community mobilizers, in addition to the Jordanian support staff such as caravan assistant (in a camp) and driver. The field staff mentioned above is supported by a Programme Coordinator who is supervised by the Programme Manager. To simplify and compare with other programmes, the evaluation depicted the structure as per Figure 6 below. Figure 6 - Programme structure at the field level

The evaluation conducted a rough calculation on the number of beneficiaries reached through each of the position and found the following: Each counsellor accompanied by an outreach / community mobilizer reaches around 7

women per day. Each education officer accompanied by an outreach / community mobilizer reaches

around 100 women per day.

Provided there are around 4 hours of actual field work per day, and taking into consideration average commuting time, it was calculated that around 40 minutes is spent with each mother for counselling (one-on-one). This was confirmed by SC-J where it was indicated that the counsellor spends between 30 to 60 minutes with each mother for counselling. The structure and load of work is consistent with other programmes where educators and lactation specialists (counsellors) work together along with community mobilizers. However, in some structures, one counsellor would work with more than one educator and mobilizer. In Indonesia for example, one counsellor/specialist oversees the work of 4 educators who work with mother support groups as shown in Figure 7 below.


Figure 7 - Structure of community IYCF support in Indonesia

Also, mother support groups are usually composed of mothers and supported by an education officer/peer educator. In the case of the Programme subject of this evaluation, although mothers support groups are mentioned as an activity, however, these are only composed of occasional testimonies conducted by mothers for other mothers. As mentioned under question #14 above, the quality and technical skills of the staff working within the Programme have been seen as high. Staff receives regular training and follow up. Further elaboration on this subject can be found in the question mentioned above.

25) What have been the roles and contribution of volunteers in the results?

• Were there any volunteers that contributed to the Programme? If yes, what was the nature of their contribution per their numbers? • What was the nationality of the volunteers and where did they live? • To what extent did volunteers contribute to the Programme results? • Were incentives considered or used to improve the commitment and efficiency of the volunteers? • Was there consideration for training the volunteers to perform some of the responsibilities that devolved to the Programme staff? • Were volunteers assigned to seek and bring forward the “non-users” of the Programme?

The evaluation found that there is added value for the volunteers within the Programme; however, it is premature to draw on the extent to which their presence affected efficiency. There are around 43 community mobilizers at the time of writing this report divided on all Programme activities as detailed in Appendix V. These are considered volunteers since they do not receive what is considered a salary but rather stipends to cover for transportation and communication. Including volunteers in the structure was initially a requirement as noted by SC-J, which saw value in this addition in terms of security and outreach. However, as the Programme moved along and volunteers became more engaged, SC-J found that the presence of volunteers is actually a necessity and something to take pride of. This was verified through focus group discussions where mothers and SC-J staff confirmed that the presence of community mobilizers had facilitated a lot of the outreach; providing an entry point to refugees and contributing to the acceptance of the Programme since these use the “same language”. In fact, it was mentioned by SC-J that they now often rely on volunteer community mobilizers to provide messaging; however, this was not verified during focus group discussions. It may be premature to draw a conclusion on whether volunteers have contributed to messaging or not. On another hand, there seems to be a positive impact of the Programme on the volunteers themselves, including improved capacity as well as receiving benefits at the social level (acknowledgement etc.) There was indication that throughout the implementation period of the Programme, volunteers’ role moved from just being an obligation to being a necessity for the proper functioning of the Programme. Based on the role played by volunteers, SC-J indicated striving to provide more capacity building and more responsibilities to volunteers. However, this was not observed during focus group discussions with volunteers, who only indicated that they


are tasked with accompanying counsellors and educators; with no elaboration on taking up more responsibilities.

26) Were achievements made on time? If not, why?

• What is the action plan of the Programme (planned timeline)? • What is the actual implementation timeline? • To what extent did actual implementation match planned timeline? • What were barriers that affected the implementation rate? • What are some of the recommendations to avoid delays and improve the timeliness of the Programme?

There is evidence to show that the Programme deliverables and Programme activities were not implemented on time. In an attempt to evaluate the extent to which Programme deliverables were implemented on time, a review of the PCA list of activities against Programme reports was conducted. Although there were action plans in a few of the PCAs, however the latter was not updated regularly and, therefore, the evaluation team had to rely on narrative reports and PCAs to conduct the analysis. As described in Appendix B, the first PCA was drafted and signed between SC-J and UNICEF for the IYCF-P in January 2013. It was meant to cover the whole year; from January to December 2013. This PCA joint proposal covered the Zaatari camp with possible extension to Azraq camp. The main tasks were to: (1) recruit, form, and train the IYCF team to support PLW, (2) develop and maintain a referral mechanism, (3) establish 2 caravans, (4) provide counselling and establish breastfeeding support groups, (5) raise awareness, (6) monitor and promote the BMS code, (7) set up the bottle exchange for mothers, and (8) develop a team that monitors malnutrition in children less than 5 years of age. A first amendment of this PCA took place in June 2013 and this included and 1) increment in budget, 2) addition of one caravan in Zaatari camp (to have a total of 3 caravans), 3) expansion to host community in Maan, Aqaba, Jordan Valley, Amman, 4) include EJC and Azraq, and 5) incorporation immunization messaging. A second amendment comprising a further budget increment was made in November 2013. Four caravans were then planned in Zaatari, in addition to a caravan in Rabah Sarhan and a mobile IYCF-E specialist. In terms of implementation, by the end of 2013, IYCF supporting teams were developed and trained along with a referral mechanism and the bottle exchange system. The work on the BMS code has been started. Awareness on the immunization and follow up on cases of malnutrition were also done. However, not all planned caravans were established. Two no-cost-extensions were requested in March and April 2014, and no changes in activities were noted. These were followed by the third amendment that included a raise in budget and added new caravans to the plan. The new adjustment introduced new geographical and remote areas (Azraq camp and Tafileh). Coordination (especially with MOH) and capacity building were introduced in addition to sensitization workshops, breastfeeding campaigns,


distribution of breastfeeding shawls and others. A final no-cost extension for this PCA was signed in May 2015. In total, the first PCA underwent six modifications including five no-cost extensions and the agreement duration was concluded on June 30, 2015 instead of January 15, 2013, which is a Programme extension of 2 years. For each modification, there was an expansion of the Programme deliverables, still, the fact that the Programme included at least three no-cost extensions is indicative of slow implementation rate and limited financial management. This was mentioned by UNICEF, and indications that challenges related to finance were faced were seen during the process of this evaluation. At the same time, amendments may also mean that lessons learned were integrated and modifications were requested to accommodate the Programme dynamism. Rather than waiting for the Programme to end before improving it, SC-J proceeded with those amendments. When reasons for slow implementation rates were investigated, the evaluation team was only able to find justifications related to slow process for approvals of activities. Activities needed to be approved by different entities including MOH and the Ministry of Planning amongst others. Improved planning and management including financial management of the Programme are recommended for proper running of the Programme.

27) How have the IYCF Programme activities been coordinated with different stakeholders and their similar programmes, such as other UN agencies, INGOs, NGOs, CBOs, and Ministries to achieve overall objective?

• What type of IYCF related coordination mechanisms the Programme was/is part of (health working group, nutrition sub-working group etc.)? • Who were the coordinating parties? And what is the nature of coordination with the Programme? • What was the frequency of coordination meetings? • Is there a 3W matrix for activities targeting women, infants and young children? And what was the role of each of the partners? Was there duplication of efforts? • What are the main achievements and challenges of such coordination? • Did the Programme attempt to improve coordination and collaboration with the public sector? With other NGOs and INGOs? If so, what are the specific measures that have been proposed by the Programme as well as by the other parties? What were the challenges faced? Were there recommendations to adopt?"

The Programme includes coordination and partnerships with other operational partners in the field as well as the Government. In the host communities, the Programme works through CBOs at the field level, MOH clinics and hospitals, as well as private hospitals. The outreach component is channelled through public service sectors; schools, CBOs, and charities, for awareness sessions and further identification of beneficiaries. On the camp level, this is channelled through other operational partners. Referrals to specialized services within both camp and host community settings are provided as needed through pre-identified coordination mechanisms between SC-J and operational partners.


The Programme as it stands is perceived to complement – to a certain extent – services relevant to health, food security, and WASH sectors, as generally reported by key informant interviews. BMS prescriptions are received through partner clinic pharmacies, education sessions promote nutrition diversity through items provided within the food basket received by WFP, and hygiene promotion has been integrated to awareness topics provided to beneficiaries, including demonstrations. The Protection sector has been referred to through Jordan Valley healthcare provider focus group discussion, with specific reference to a feedback box where beneficiaries may report and file any kind of abuse. The general integration of services has been reported by health care providers in Azraq to be of added value to beneficiary satisfaction and utilization of service, presenting a comprehensive approach towards health. Further coordination refers to SC-J’s key role in co-chairing the Nutrition working group, facilitating a coordination mechanism between key players, ensuring that no overlapping in service provision takes place. The role of other service provider entities is reflected through referrals provided to specialized services; secondary and tertiary service providers including MOH, as well as CMAM structure integration in relation to managing SAM cases through IMC and JHAS clinics at Azraq and Zaatari camps, respectively. Further to the avoidance of overlapping services, as noted by JHAS key informant on JHAS’ clinic-based IYCF-P provided in the Central (Amman, Salt, and Zarqa) and Northern (Mafraq and Irbid) governorates, while SC-J implements its IYCF-P within camps and southern governorates. Furthermore, abiding by the Guidance Note on IYCF in Emergencies in camps and its associated BMS administering criteria is vital towards progressing towards the Programme’s intended impact, especially among partners within the food security sector. As reported by WFP key informant, donations received for children’s milk had been discussed with SC-J and UNICEF; these would be provided as a school snack to school children.

Likelihood of Impact

28) What are the key short term and long term (and lasting) changes produced by the Programme (positive or negative, intended or unintended) as perceived by the stakeholders (PLW, front line workers, government, implementing partners including CBOs)? What are the key factors behind these changes?

• What are the main perceived changes by key informants, service providers and end-users? • What are the main activities that are perceived as key contributors of change? • What are the contributing factors to the success of the Programme as perceived by key informants, service providers and end-users? • Did the stakeholders and other service providers adopt some (or all) of the Programme ingredients? Have they adapted any for their own services? Any consideration of this development? Was experience shared to improve impact and sustainability? • Any recommendations to improve the impact of this Programme? Any modifications to that end?


29) Whether and to what extent the desired changes have been experienced by PLWs and children as a result of this Programme (as perceived by end users). Did the integration of other components (as IYCF+) make any difference in those changes?

• To what extent do end-users perceive the Programme as having an impact on their behaviour related to IYCF? • What are the main Programme activities that were perceived as helpful by end-users? • How did the integration of other components (IYCF+) influence the desired changes?

The below information is complemented by section 7.2 of this chapter which looks at the theory of change and extent to which results match the developed theory of change. Findings are also complemented by questions 16,17, and 18. The evaluation team found evidence from qualitative data that shows a perceived change by key informants, service providers and end-users relating to a change in knowledge, capacity, practices, as well as perceived change in the health and nutritional status of children and pregnant and lactating women. Perceptions varied as related to changes in mother and child health. Similarly, other changes were perceived such as those related to changes in economic status and social interaction. Most changes were attributed to the counselling, follow up and education activities that were received and implemented by the Programme. Policy change was also noted as a result of the Programme. Question #18 above discusses Programme enablers and disablers. Improved learning and knowledge (Immediate outcome) Focus group discussions with beneficiary mothers showed that the latter perceive the Programme as contributing to learning about different key messages related to infant nutrition, as well as health. Family planning and child spacing was also mentioned. Topics that were listed by beneficiaries included learning about breastfeeding, the importance of breastfeeding, the early initiation of breastfeeding, exclusivity of breastfeeding including limiting the consumption of teas and other liquids for infants, techniques about breastfeeding, avoidance of artificial nipples, the importance of colostrum. Mothers also perceive the Programme as contributing to learning about complementary feeding, pregnancy health (nutrition, vaccination, fluids, and iron), diarrhoea (prevention and management), and food allergies, in addition to correction of misconceptions. Learning was also perceived by staff themselves who emphasized how the Programme contributed to improving their own knowledge and correcting misconceptions they had held. Knowledge transfer was also perceived to occur from mothers to other mothers, from mothers to other family members, and from volunteers to other mothers. Although the majority of examples given demonstrated accurate knowledge about IYCF, still, in a few instances, mothers mentioned incorrect information that relates to the frequency and timing of feeding. This shows the importance of continuity and sustainability of the messaging. There is evidence to show that the Programme has contributed to a change in knowledge related to IYCF key messages amongst beneficiaries.


In addition, the program has contributed to a change in knowledge amongst trained stakeholders including MOH staff. Knowledge of MOH staff was assessed by the KAP survey (Appendix I) and results are presented in Appendix J. The average of correct answers to knowledge of all participants was 74%. The analysis of KAP results helped in defining the main gaps in knowledge that were represented by an average of correct answers of 50% or less. These are mainly related to 1) the best timing for introducing complementary food to the baby (17% of MOH participants provided a correct answer), 2) the type of food that should not be introduced to a 9 month old baby (50%), 3) the role of oxytocin and prolactin hormones (42%), and 4) the cause of uterine contractions and a rush of blood during feeds in the first week (8%). The percentage of correct answers to the rest of the questions varied between 58% and 100%. Change in behaviour (Intermediate outcome) Health care providers highlighted that a change in behaviour is a long process and documenting such change is challenging. However, all interviewed stakeholders drew on perceptions that the Programme is contributing to a change in behaviour related to infant nutrition and health practices. Mothers gave examples of different behaviours they are adopting as a result of the Programme. Behaviours included limiting the use of artificial nipples, ensuring the provision of colostrum to their babies, ensuring exclusive breastfeeding and the proper introduction of solids, etc. Change in behaviour is evident from the examples given by mothers who describe their experiences comparing current child with previous children and emphasize how they changed their practices from one child to the other. Other examples included those mothers who successfully went through a re-lactation process, where after a few months of stopping breastfeeding, they successfully went back to breastfeeding. These were also mentioned by health care providers and key informants, who also mentioned the perception that there is an increase in exclusive breastfeeding at the camp level. There are also indications of improved feeding practices observed through focus group discussions carried out by a designated UN agency. The study conducted in 2016 indicated that great efforts were made by SC-J to improve IYCF practices and despite the fact that mixed feeding is still seen, behaviour change was evident in improvement in exclusive breastfeeding83. There were few examples where mothers indicated having to revert back to artificial feeding and, therefore, did not experience the change in behaviour as described by their peers. The evaluation team noted that these are few examples of all the mothers that participated in the FGDs and the majority of examples were indicative of a positive change. At the same time, it is possible that those who attended the FGDs are those that were the most positive about the Programme and were most likely to have benefited, which might indicate to selection bias. The Programme, as emphasized by different stakeholders, has contributed to a change in behaviour related to IYCF. However, the extent of the Programme contribution cannot be measured except through the collection of associated quantitative data.

83 Newborn Health Baseline Assessment JORDAN – UNHCR – 2016


Improved health (Likelihood of impact) In addition to the perceived change in behaviour related to feeding, specifically breastfeeding, mothers also attributed prevention of ailments and improved health of children to the change in feeding behaviour that occurred as a result of the Programme. Mothers gave examples of positive changes in health status, such as absence of allergies and diarrhoea as noted below and prevention of complications perceived to result from improved feeding from the Programme assistance. Mothers also gave examples of children being “calmer” as a result of breastfeeding and referred to the positive bond between themselves and their children, a fact which is strongly supported by the literature showing the role of oxytocin on the mental health of both mothers and babies (Britton et al., 200684; Else-Que et al., 200385; Jansen et al., 200886). Concerning examples of ailments among children, varied between camps and host communities, it was noted that those in camps were more related to acute illnesses such as infections, diarrhoea and others. However, in host communities, examples were more related to non-communicable diseases such as allergies.

Enforcement of IYC policy (Intermediate outcome) One mentioned change that was perceived by the different stakeholders about the Programme’s administered impact is the contribution to improved national policy enforcement by the Government related to IYCF. This was noted by a number of stakeholders and recognized through the examples given on mothers returning cans of milk and reporting on violations. Other outcomes There were some indications that the Programme has helped reduce social tensions between Syrians and Jordanians through bringing them together. Examples were also given on the role of the Programme in contributing to cost saving as a result of resorting to breastfeeding and saving on costs of formula milk. Staff benefit was also noted where the latter profited from advancement in capacity by participating in national and international trainings through this Programme.

Factors contributing to the likelihood of impact A key activity that was mentioned frequently by mothers was the “visit” that is regularly conducted by SC-J staff, during which mothers receive counselling and advice. The counselling and follow up activities, either face to face or by phone, were of value to mothers and contributed to troubleshooting and decreasing drop outs among them. The one-on-one support is also noted to be key for successful breastfeeding. Programme reports show that the Programme has reached its targets. Qualitative data show that both beneficiaries and service providers perceive the Programme as beneficial and

84 Britton, John R., Helen L. Britton, and Virginia Gronwaldt. "Breastfeeding, sensitivity, and attachment." Pediatrics 118.5 (2006): e1436-e1443. 85 Else-Quest, Nicole M., Janet Shibley Hyde, and Roseanne Clark. "Breastfeeding, bonding, and the mother-infant relationship." Merrill-Palmer Quarterly (1982-) (2003): 495-517. 86 Jansen, Jarno, Carolina de Weerth, and J. Marianne Riksen-Walraven. "Breastfeeding and the mother–infant relationship—a review." Developmental Review 28.4 (2008): 503-521.


affecting both knowledge and behaviour. Further analysis about the extent to which the Programme demonstrated expected results is illustrated in the following section, where the Case (the Programme) is analysed against its theory of change. In terms of unforeseen effects, the evaluation noted a sort of ‘dependence’ that the Programme has created, especially in host communities, by beneficiaries and service providers. Given that the SC-J staff focused on providing IYCF services to mothers present in centres that did not normally provide such a service, it was noticed that this has created a “need” that did not previously exist. Both mothers and key informants (for example in hospitals) gave examples of instances when mothers “ask for the specialist” or “look for her” in order to get the service. This is both a positive and negative effect. It is positive since it has created a ‘culture’ of providing IYCF support and asking for it. It is negative since the service is provided separately from existing services and, therefore, may not be maintained indefinitely. This raises the question of the “do-no-harm” effect and whether the Programme was able to maintain this principle or not.

Sustainability

30) Whether measures have been in place to ensure the sustainability of achievements after the withdrawal of external support or in the context of the anticipated decrease in donor funding (taking into consideration the specificity of the emergency context)?

• What is the sustainability plan within the Programme document? How much was achieved from this plan? • Any established/integrated related systems within the concerned main stakeholders mainly within the MOH? Are they operational? • Which department at the Ministry is responsible for IYCF and was the department engaged in the implementation of the Programme? • What are key informants' perceptions related to the sustainability of the Programme? • Was sustainability highlighted in the Programme document? Was it discussed with the Government? Were there any recommendations? Were these followed over the time of the project? Have some of the recommendations been implemented? If not, what were the challenges faced? • How long do you estimate the need for this Programme (IYCF+)? • Have exit plans been elaborated – or is there an effort to that effect?"

31) Whether and how the Programme identified and built on existing national, local, civil society, government capacities? What new capacities within services or communities have been established or restored that can contribute to sustainability? "

• Who are the main national stakeholders (list of partners and description)? • What is the mandate of each party in relation to IYCF? • How were they identified? • How did the Programme complement the existing national IYCF initiatives? • Did the Programme provide capacity building activities to concerned stakeholders? What type of CB activities? • To what extent do key informants and service providers perceive the Programme as contributing to improved capacity?

32) Extent to which IYCF has been integrated within MOH institutions and whether there are plans to institutionalize the Programme

• Is there any structured unit with dedicated personnel in charge of IYCF activities within MOH? • What are IYCF activities/services that are provided by MOH institutions? • Does the Ministry have a budget for IYCF integrated within its budget?


33) What are the possibilities for scale-up or replication?

• What are key stakeholders' perceptions about scaling up the initiative or replicating it? • What are recommendations for expanding the Programme? • Are there specific objectives within the Programme to improve the capacity of the public sector staff to replicate the Programme in other facilities and locations/ Has there been an effective sharing of information? Would you say that the cooperation with Government is most optimal – or can it be improved? • Are there demonstrated results that could make a wider-scale likelihood of impact? • Are findings relatively generalizable so the Programme can be replicated out of the original context?"

Sustainability is noted to have been faintly considered in the design of the IYCF Programme. But to emphasize, those efforts towards ensuring the Programme sustainability have been invested as the Programme moved forward. There is no evidence of a clear sustainability plan or strategy for SC-J’s Programme articulating the steps to ensure smooth gradual phasing out of the Programme. Despite anecdotes about staff collaborating with partners, there is no documented evidence to show plans or elements that would ensure sustainability within the design of each of the activities. SC-J sustainability plans are included within the last SC-J agreement with UNICEF87 . In camps, plans included engaging Syrian mothers as the main and only stakeholders to assume the role of promoting IYCF practices. This measure is perceived to improve cost-efficiency. It is mentioned in the last agreement with UNICEF that handing over the work to Syrian mothers in camps will be tested through different modalities. Syrian mothers will be empowered and trained to become IYCF mother peer leaders, working in the camps and host communities as IYCF advocates for sustained behaviour change. However, this had not happened and there was limited indication that SC-J planned on training lead mothers. The evaluation team found only that Syrian refugee mothers were recruited as volunteer community mobilizers, who accompany counsellors and facilitate communication with and outreach to Syrian refugees. In host communities, sustainability plans focused on capacity building. Documents focus on expanding the IYCF training hub and increasing training sessions for various stakeholders that include the Ministry, CBOs, private hospitals, and other organizations, not only for Jordan but also for the Middle East region. Future capacity building and supervision support to the Ministry and various hospitals were suggested to be part of a gradual exit strategy that builds empowerment of key actors in health and nutrition to ensure the sustainability of recommended health and nutrition practices in Jordan. The training for MOH and CBOs was validated and reports and qualitative data show that it was conducted. However, it is clear from the KAP survey mentioned under question #29, that one training session is not sufficient to support Programme sustainability. Results showed that overall, MOH participants did better than CBO staff, which was expected; however, knowledge was lacking. For example, more than half the participants (50%) did not have the exact information about complementary food (when and what to introduce to the baby).

87 PCA (1/7/2015-30/6/2016) in the “Program Description/ Rationale- Justification”


According to Perez’s gear model (Pérez et al., 2012)88, sustainability of IYCF programmes are based primarily on institutionalization of IYCF activities within existing systems such as government systems. Pre-requisites for integration include but are not limited to; 1) political will, 2) existing policies and legislations, 3) capacity building and programme delivery, and 4) advocacy. The above documented plan only includes capacity building, which may not be sufficient as an exit strategy or sustainability plan. On another hand, the evaluation noted work through the Programme on moving policy and advocacy forward. Qualitative data show that the Programme is being implemented as a silo and there is no evidence to show the execution of a sustainability plan. Data collected through key informants and focus group discussions indicate that especially in host communities, the Programme is being implemented in a vertical mode and is parallel to existing structures. SC-J partners with CBOs, hospitals and health care centres and an agreement is made to provide IYCF services within the partnering agency’s premise. SC-J staff visits these outlets either on a daily basis or on a schedule to reach women who visit the centres and provide counselling and education. SC-J staff provides highly technical and consistent advice and counselling to mothers on infant and young child feeding in addition to other newborn care (through IYCF+). This activity has been seen as beneficial as noted earlier in this report, however, there was no indication that staff are providing any on-the-job training or support for existing staff within those centres to be able to eventually provide the service themselves. Similarly, there is no evidence to show that there is a clear plan for reporting and sharing information between SC-J and the hosting organization; whether in a MOH centre or a CBO centre. Data sharing is limited to SC-J and UNICEF with occasional ad-hoc and on-demand exchange of information between SC-J and MOH central or MOH centres. This is indicative that despite the work that is being done on improving IYCF services through the centres, there is no clear indication of how such work is being documented as part of the hosting structures or how it is contributing to an overall strategy. Documentation does however exist within SC-J programme documents albeit not in a systematic way. No documents were reviewed that show evidence of supervisory visits between MOH and SC-J. Regarding the engagement of mothers and establishment of mother peer support (the plan mentioned in the project documents), there is no indication that this has been moved forward except that community mobilizers consist of trained mothers. SC-J indicated no further plans to activate any mother to mother support groups, however, it mentioned the utilization of mothers as education agents through testimonies given during education sessions. In terms of existing structures with potential for ensuring sustainability, the evaluation team looked at the capacity and structure of MOH and other partners. Within MOH health centres, a team consisting of a midwife, nurse, and a doctor are present. According to key informants at MOH, this team is mandated to provide IYCF messaging, however, they would refrain from doing so and would instead rely on the SC-J staff. This was validated by health care providers during focus group meetings, where it was noted that IYCF

88 Pérez-Escamilla, R., Curry, L., Minhas, D., Taylor, L., & Bradley, E. (2012). Scaling up of breastfeeding promotion programs in low-and middle-income countries: the “breastfeeding gear” model. Advances in Nutrition: An International Review Journal, 3(6), 790-800.


is not fully established or prioritized within health centres like immunization is, and that nurses and midwives give priority to the latter. In some centres – mainly hospitals – it was mentioned that when the SC-J staff is not in the hospital, the hospital staff conducts the counselling. At the same time, other examples showed that centre staff would eagerly wait and ask for SC-J staff when faced with a situation that requires counselling. In a number of instances, it was mentioned that given the lack of capacity and load of work (lack of enough staff), hosting centres’ staff are not providing the required services. In centres that are supported by SC-J, the service is almost only provided through the Programme. Therefore, it is questionable whether these will be able to take on these tasks when SC-J phases out. In fact, SC-J conducted capacity building activities targeting CBO and MOH staff, however, results of the KAP survey administered to assess the capacity of trainees showed little evidence that staff would be capable of taking on the appointed task at the same quality level. The training was based on adopted IYCF guidance including those of MOH. Forty-seven (47) staff were trained from CBOs and 45 from MOH, including health centres and hospitals. Results from the KAP survey targeting trained staff revealed that although staff has some knowledge about IYCF, the acquired knowledge did not translate into appropriate practices. Thirty-one per cent (31%) out of 103 targeted trainees responded to the questionnaire which was divided into three sections: 1) knowledge, 2) attitude, and 3) practice. Under knowledge, correct answers ranged from a maximum of 97% correct answers to a minimum of 25% correct answers on some more technical questions. The majority of the respondents perceived the trainings as beneficial although most of the CBO staff that responded to the questionnaire responded that the training was moderately relevant. The results of the IYCF practices among the trained staff revealed that although there might be attainment of some level of knowledge related to IYCF, however, this is not indicative that trained heath care providers would be able to provide adequate and quality advice to mothers once faced with the need to do so. In fact, it has been confirmed by MOH and key informants that staff in health centres need more capacity building support to be able to deliver quality IYCF counselling and education. There are indications that staff need constant follow up, monitoring, and regular refreshers in order to be able to match the existing services that SC-J is providing. Obviously, one training is not enough to build the capacity of staff and equip them for the provision of IYCF services. Within the MOH, the Directorate of Maternal and Child Health which includes infant nutrition is an independent structure under the Department of Primary Health and functions to support mother and child health. A key role that this Department plays is to ensure that IYCF is supported. SC-J has been closely coordinating activities in host communities with this Department including supporting the capacity building of staff (training) and development of awareness material (media campaign). Discussions have taken place between MOH and SC-J with the Department of Mothers’ Health as well as other departments including the Directorate of Planning and that of Primary Health. Challenges of communication have been noted by SC-J, however, there are indications that MOH perceives the collaboration as positive.


Also, it is important to note that there is interest and willingness of existing structures including MOH and other partners to integrate and contribute to sustainability of the Programme, yet the preparedness level varied among these existing structures and some pre-requisites were identified. At the level of the MOH, key officials perceive the need to integrate and institutionalize IYCF within the MOH structures and that there is willingness to do so. However, barriers for such a move include: Inappropriate timing for phasing out of the services. A key informant at MOH who plays

a key role in strategic planning at the national level, indicated that despite the importance of integrating IYCF activities implemented by the Program within MOH, it is not the appropriate moment to completely phase out. The MOH is currently working towards the adoption of a new strategy aiming at building public-private partnerships, and it would be counterintuitive to proceed with phasing out the Programme at this particular time. However, this does not negate the interest and willingness of the Ministry to “smoothly” integrate the Programme services. Integration could be initiated further to “thorough assessments of Ministry capacity and needs”, followed by the development and setting of the required systems and processes within the MOH allowing the success of such integration.

Limited capacity at MOH – both at the central and field level. Some key informants mentioned that having only one person at MOH is not sufficient to support IYCF and move it forward. This could not be verified with the MOH. In addition, it was consistent throughout the key informant interviews and FGDs that there is limited capacity from staff within health care centres to be able to deliver the needed IYCF services and sustain the Programme. Emphasis was put on the need to not only extensively build the capacity of staff but also ensure continuous monitoring and quality assurance of their work. The limited capacity also encompasses limited person-time of existing staff at the MOH. The KAP survey further supports this gap in technical capacity.

Lack of funding and other resources to sustain the Programme was also mentioned, in addition to the importance of UNICEF funds for at least the coming few years, to assist MOH in setting up a base for institutionalization of the services.

The power of marketing of infant formula was mentioned by several stakeholders as a hurdle to moving forward with the implementation of existing policies, including the enforcement of the Code. This is usually a main challenge in countries where a policy has been recently established to uphold the Code. It takes political will and prioritization of the enforcement in order to move forward with establishing implementation committees and penalty processes.

In terms of integration of IYCF activities within MOH structure, the evaluation team saw that there is a need and willingness to do so, however, it is still premature to conclude what detailed pre-requisites are needed to proceed with this step. A thorough assessment of the capacity of MOH’s structure and systems needs to be conducted. When asked about scenarios where SC-J would withdraw from the centres, key informants – mainly stakeholders representing partner agencies indicated willingness to take on the


activity, provided some capacity building is conducted to their staff. As noted earlier, the Programme is regarded as highly valuable and the evaluation team saw willingness from partner agencies (in camps and host communities) to sustain it. A key pre-requisite for sustainability includes the establishment of national IYCF strategies and policies (Pérez et al., 2012)89. Through conducted key informants and the document review, it was apparent that policies are being put in place with the latest Jordan Code approved in 2015. On the other hand, key informants confirmed that the IYCF policy / code90 which was translated from the international Code cannot, in any way, be considered enforced since it is still in the early stages; there are steps to be taken including the creation of a committee and the development of penalty legislations to accompany the policy. The evaluation team could not verify whether the development and issuing of the policy documents are actually the result of the IYCF-P since each of the interviewed stakeholders related the effort to their own initiative and advocacy effort, whereas MOH related this to their own effort. On the other hand, the fact that there is a legal document indicates that it is a step towards sustainability regardless of who was behind the effort. In addition, results from FGDs and key informants showed that the Programme helped enforce the policy at least within the camp settings as indicated in the above section on impact. However, violations to the policy were still cited as common in host communities including in heath care centres, hospitals and doctors’ clinics.

Challenges and barriers against the implementation of the IYCF policy were highlighted by key informants including political will and the prioritization of implementation. Interviewees expressed the importance of prioritizing the implementation of the policy and related the delay in implementation to lack of prioritization of IYCF or lack of will. Multiple key informants spoke about “some reasons” that are keeping MOH from proceeding with the implementation, and some referred to the power and politics surrounding the marketing of infant formula. As seen in many other countries and contexts, the marketing of infant formula and the lobbying of companies is considered a large barrier towards the advancement of breastfeeding support at the National level. The evaluation team could not properly verify this fact in Jordan except that it was noted that distribution of infant formula was still common in hospitals. Furthermore, the impact of the pressure put by infant formula companies on the actions taken by MOH to properly monitor the implementation of the Code could not be verified despite being mentioned by a couple of key stakeholders. It is evident from the above that despite some effort to ensure sustainability, more strategic planning needs to be incorporated within individual activities to sustain the Programme and its effect. Drawing on existing integration initiatives in a number of contexts, the following potential opportunities for ensuring sustainability were identified:

89 Pérez-Escamilla, R., Curry, L., Minhas, D., Taylor, L., & Bradley, E. (2012). Scaling up of breastfeeding promotion programs in low-and middle-income countries: the “breastfeeding gear” model. Advances in Nutrition: An International Review Journal, 3(6), 790-800. 90 Jordanian code for protecting and promoting breastfeeding, 2009


Integration of IYCF services within MOH existing structures: Although it was noted that the timing may not be ideal, some ground work may start in preparation for institutionalization and integration such as (1) conducting a capacity assessment for MOH that highlights the gaps in systems and policies related to IYCF, (2) devising a phase out plan that includes capacity building but also extensive on-the-job training and coaching for staff within centres, (3) establishing a monitoring and reporting system that includes services related to IYCF, and (4) allocating a budget for IYCF within MOH to support a clear IYCF Plan.

Identification of potential local partners that would integrate IYCF within their systems. The evaluation team noted that JHAS may be such an entity that could champion IYCF programming and support from the local community level given that it is a local organisation and may potentially contribute to sustainability as opposed to other INGOs that might not be as sustainable.

Scaling up the IYCF-P to include and encompass a comprehensive newborn care package that would be incorporated within existing structures using the approach described above.

7.2. Results and Findings Based on the Theory of Change The following section examines the Programme /Case against the Programme revised Theory of Change (Refer to section 2.2).


Figure 8 - Schematic presentation of the IYCF-P Theory of Change – the results chain

Inputs Activity Outputs Outcomes Impact

A1: Establ ishing mother-baby friendly breastfeeding

spaces (caravans , mobi le units , and others )

A2: Recruiting, tra ining and supporting an IYCF team to

screen for IYCF di fficul ties , refer as necessary and

provide appropriate bas ic IYCF-E messaging and support

to PLW

`

A3: Providing one-on-one IYCF Counsel l ing for mothers

with chi ldren under 2 years of age

O1: Increased avai labi l i ty of qual i ty IYCF counsel l ing

services to target groups (Change in availability of

Quality services)

Out1: Target groups correctly

identi fy, enrol l and

participate/uti l i ze services at the

IYCF program (Change in access to

services)

A4:Monitoring of infant formula dis tribution in camps

and providing ski l led support for arti ficia l feeding

A5: Conducting health education sess ions through IYCF

caravans and at cl inics

O2: Increased knowledge and ski l l s and confidence

on IYCF practice among mothers (Change in

individual/beneficiary knowledge and skills)

Out2: Improved key IYCF practices

among targeted chi ldren,

caregivers and communities

(Change in individual behaviors)

Improved health and

nutri tional s tatus of chi ldren

A6: Bui lding capaci ty of MoH centers and concerned CSOs

on IYCF-E

O3: Increased and reta ined knowledge and ski l l s

among IYCF s taffs in the government and CSOs, to

identi fy and respond to IYCF and IYCF-E needs

(Change in knowledge and skills of providers)

A7: Advocating and coordinating with other sectors to

monitor and promote the BMS code

O4: Increased and reta ined knowledge about BMS

code

Out3: Changes in pol icy framework

and legis lation to support and

promote IYCF (Change in policy

frameworks)

A8: Supporting the Minis try of Health to develop chi ld

nutri tion pol icies and plansO5: Chi ld nutri tion pol icies and plans developed

A9: Conducting mass campaigns and implementing

campaign activi ties to promote IYCF and change socia l

norms supportive of IYCF practices .

O6: Increased socia l awareness on IYCF and i ts

va lue

Out 4: Changes in socia l norms

which promotes IYCF practices in

the community (changes in social

norms)

A10: Provide socia l and behaviora l change

communication to address socia l norms

A11: Providing supplementary feeding to moderately

malnourished chi ldren

Money

Staff

Volunteers

Suppl ies

El igible

Beneficiaries

Conducting

evidence-

based

research

Assumptions: • Openness from communities to receiving information

and behavior change messages• Existence of minimum level of technical capacity

among service providers that can be supported• Service providers are able to continue and respond to

and use the knowledge and information made available to them.

• Partners incorporate IYCF policies into sector policies and plans

Assumptions: • Government resources and capacity are able to respond to the humanitarian crisis.• Political leaders continue to support efforts to scale-up children’s nutrition• Progress in other child health intervention areasAssumptions: • Stable security situation in Syria and Jordan

• Assumptions: • Continuous use of IYCF

services by end-users.• Resources and capacity

remain sufficient among donors and implementing partners for IYCF.

1

3

4

6

2

5


The IYCF-P used different inputs to implement its planned activities. The used inputs include the financial resources (UNICEF funds), guidance and follow-up provided by UNICEF, the technical expertise of SC-J staff, volunteers’ time in support of the Programme’s activities, and eligible beneficiaries including PLW, MOH and CSOs staff, and duty bearers including the targeted communities and caregivers. The Programme did not conduct research to establish baselines and provide additional evidence on needs and current trends. Yet, the Programme relied on secondary information sources to provide needs evidence. IYCF-P used the above noted inputs to conduct a set of activities including service provision to mothers and children; awareness raising of PLW; capacity building of service providers; public campaigns; and advocacy. Almost all planned activities were implemented except for the establishment of breastfeeding support groups (mother to mother support group). The achievement of the IYCF-P results and associated causal chains are explained as follows (shown in Figure 8 and Appendix W): Activities focussing on awareness raising and service provision activities (A1 to A5 and A11) induced a positive change relating to the availability of quality IYCF services (Output 1) and to individual/beneficiary IYCF knowledge and skills (Output 2) as evidenced by results of this evaluation. Subsequently, by increasing the availability of quality services (Output 1), PLW are more likely to have a better access to IYCF services (Outcome 1); and by improving IYCF knowledge and skills among PLW (Output 2), the latter improved their IYCF practices (Outcome 2), as also evidenced by the evaluation results. The capacity building activities targeting the IYCF staff in the government and CSOs (A6) contributed to a minor change in IYCF knowledge and skills among service providers (Output 3) as evidenced by the KAP survey. Given the remaining limitations in skills of service providers (other than SCJ staff), there is evidence to say that Output 3 had a limited contribution to improving IYCF practices among target groups (Outcome 2). The advocacy activities associated with the promotion of the BMS code (A7) were planned to increase the awareness of MOH and other concerned on the value of promoting and monitoring the BMS code (Output 4). These advocacy activities contributed to processes leading to the endorsement of the BMS code, but the extent of its contribution is yet to be investigated (Outcome 3). Furthermore, the support provided to MOH through the IYCF-P towards the development of nutrition policies and plans (A8) succeeded to increase the awareness of MOH at this level (Output 5) and to initiate a process of change in policy frameworks but did not lead to a total change at this level (Outcome 3). Conducting mass promotion campaigns (A9) and the provision of social and behavioural change communication to address social norms (A10) is assumed to be contributing also to some positive change in social awareness on IYCF and its value (Output6) and inducing some change in social norms to promote IYCF practices (Outcome 4) despite the limited targeting


of men through the Programme activities during the initial phases of IYCF-P. While mass promotion campaign and social and behavioural change communication may contribute to some extent to achieve a positive change in social awareness and in social norms, it is evident that issues are not holistically addressed to establish this causal link. The causal links between all activities, in general, and the stated outputs is strong in what relates to the technical capacity and communication skills of the concerned IYCF staff, the technical support provided to them by UNICEF in addition to the guidance provided by the SC international team with global experience in IYCF. The collaboration between SC-J and other IYCF actors contributed also to reinforcing these causal links, mainly through the coordination role assumed by SC-J in the nutrition sub-working group. The Programme design flexibility that took into consideration the socio-economic and geographical specificities as well as the operating health systems of the targeted regions (camps and host communities) contributed also to the strength of these causal links. This is more relevant with the approaches adopted with service provision and awareness. In the camps, mothers were the direct target of the Programme activities. SC-J created the needed structure for the IYCF service delivery. Whereas in the host communities, mothers were reached through the concerned CBOs and MOH centers. In addition, in order to reach the hardest to reach population, in the host communities, SC-J has opted for the mobilization of a mobile caravan to cover the Southern part. Engaging Syrian community mobilizers and peers contributed also to reinforcing the causal between service provision and the achievement as well as the likelihood of sustainability of related outputs and outcomes. Yet, the reliance of the IYCF-P on intensive financial and technical resources in addition to the limited availability of the resources and capacity among implementing partners including the MOH health centres, in host communities, and concerned CBOs staff, weakened the causal link between the Programme activities and outputs. The limited availability in technical and financial capacities, influence subsequently, the continuous availability and use of quality IYCF services. The witnessed delays in the Programme’s implementation contributed also to weakening the causal links between the Programme’s activities and outputs (Assumptions 1). The examination of the causal links between outputs and outcomes reveals that the link between Outputs 1 and 2 and Outcomes 1 and 2 (arrows #1 and #2) is somehow strong in what relates of the openness shown by the targeted communities to receiving information and behaviour change messages, more so at the camps level. The link between Output 3 and Outcome 2 (arrow #3) is somehow weak in view of the limited existence of a minimum level of technical capacity among service providers within the targeted CBOs and more so the MOH centres that are supported by SC-J interventions. The limited abilities of service providers to continue to respond to and use the knowledge and information made available to them in view of the limited follow up activities ensured by the Programme staff at this level contributes to the weakness of this link. As for the link between Output 4 and Outcome 3 (arrow #4), it is stronger in the camps in comparison to the host communities since there is evidence that the BMS code has been better enforced in the camps. The link between Output 5 and Outcome 3 (arrow #5) is still weak since the incorporation of the IYCF policies into sector


policies and plans still necessitate additional reinforcement mainly at the level of the MOH, despite the expressed interest within this context. The link between Output 6 and Outcome 4 (arrow #6) was not directly but indirectly assessed through this evaluation. An assumption was made that the Programme effect on the target groups would ultimately be propagated to the general public (Assumptions 2). At the impact level, and despite the lack of a baseline to quantitate the extent of changes in nutritional status of children, with emphasis on the most disadvantaged, the IYCF Programme had the potential to contribute, to the extent of its achieved outputs and outcomes, to the improvement of the nutritional status of children among Syrian refugees and host communities in Jordan with emphasis on the most disadvantaged. Yet, the causal link between the Programme outcomes and its likelihood of impact is greatly influenced by the Programme reliance on the government willingness to assume ownership and having the financial and technical capacity to take over and scale up IYCF related activities. This link is strong in terms of expressed interest by key officials to support efforts to scale up children’s nutrition but not sufficiently strong in view of the technical and overall material challenges currently prevailing at the ministry level. The prevalence of other highest priority health intervention areas, such as immunization, infectious diseases, non-communicable diseases, requesting the MOH and donors support contribute to weakening this causal link. In addition, this causal link could also be weakened further if a political deterioration in the situation in Syrian prevails leading to an increase in refugee influx to Jordan (Assumption3).

8. Conclusions The following are the main conclusions of this evaluation and their relevance to each of the OECD criteria.

8.1. Relevance The planning, design and implementation processes of IYCF-P in relation to responsiveness and alignment with national priorities and needs, as well as UNICEF in-house strategies, policies and International guidance and policies related to IYCF and IYCF-P (including IFE Core group OPS, WHO global strategy UNICEF Programming guide etc.).

Although the IYCF-P was planned and implemented based on several theoretical frameworks, the development of a Theory of Change framework at the conceptualization phase of IYCF-P could have sharpened the relevance and planning of the Programme through a clear intervention logic. The Programme is aligned with Government priorities; however, the National priorities include a broader spectrum of services under primary and secondary health including reproductive and child health, within which nutrition and IYCF are embedded. Although stated as a priority, the evaluation could not draw clear conclusions on MOH readiness to implement this as priority. The alignment of the Programme with Government priorities is


stronger in host communities. The Programme activities have been shared and discussed with the Ministry prior to their implementation; however, related achievements were not documented by MOH as their own achievement. The MOH therefore lacks Programme ownership, particularly in camps. It is not evident how Programme contribution to MOH strategy is documented other than by SC-J and UNICEF. The Programme is therefore relevant, however, it could be further tailored to parallel the emphasis placed by the Government on quality of care and institutionalization of activities and information. The IYCF-P in both camps and host communities responds to the national plans for the Syria response. Both IYCF and nutrition are key outputs in the plans. IYCF is considered a high priority by UNICEF and there is emphasis on ensuring its sustainability and efficiency. However, the Programme continues to have the structure of an emergency response and running in parallel to existing primary health care structure, in both camp and host community settings. The Programme has been based on international IYCF guidance, is comprehensive in nature and has been adequately contextualized in terms of language and culture to the Jordanian setting. The Programme fills a knowledge and practice gap; however, in view of the absence of baseline quantitative data, it was not possible to estimate the extent to which the Programme contributed to filling that gap. There is a clear indication that the Programme is well accepted by, and appropriate to, the target population of vulnerable PLW. Therefore, it is regarded as well received through its adopted mode of operation, approach and design. The design is similar to a number of other international initiatives; however, consideration needs to be given to the context and the extent to which it is sustainable, particularly in the protracted refugee setting. The Programme targets the most vulnerable in camp settings; however, there is no evidence to show that the demographic profile of those targeted in host communities represent the most vulnerable. In its current mode, the Programme does not sufficiently recognize the role of men in supporting adequate IYCF. The Programme design has not embedded men into the plan of different activities. In terms of planning and filling IYCF needs and related gaps, the Programme was designed initially in response to the emergency situation resulting from the Syrian crisis. However, as the Programme timeline progressed, there were no clear expansion plans. Instead of scaling up based on continuous needs assessments, many of the decisions to modify and expand the Programme came as a result of guidance from UNICEF.

8.2. Effectiveness The success or otherwise of the IYCF-P in achieving its stated objectives and any intended or unintended effects. Associated strengths, weaknesses and evidence of innovation. Extent of equity achieved by the Programme.

The Programme was able to implement almost all planned activities, yet fell short to achieve in full and to the desired extent all of its outcomes. Achievement in results was more


prevalent at the level of increasing awareness of the target groups and provision of services to the PLWs. It was less obvious in terms of increasing the capacity of service providers among the concerned staff of MOH and CSOs, and inducing a major change at the policy framework and social norms levels. Inducing changes in policy frameworks and social norms necessitate a longer Programme’s engagement and an increased national commitment (Refer to the ToC section). Gaps were also identified in the reporting of some indicators and the matching of indicators to actual activities. The integration of additional activities within the IYCF+ proved to be useful and it was apparent that providing a comprehensive package is of added value to beneficiaries. Evidence shows that there was enough technical assistance and support for SCJ Programme staff by SC-J and guidance from UNICEF, particularly during the development of the Programme’s PCAs. The Programme staff has accumulated technical capacity through extensive and continuous training on IYCF as well as on communication skills. There were a number of strengths that were identified within the Programme including: (1) outreach efforts and presence of local/Syrian community mobilizers who are mothers; (2) engaging men (although at a later stage of the Programme) which was seen as a positive approach to facilitating the change in behaviour of mothers, especially when men play the role of enablers; (3) the name of the organization and the connotation it gave in Arabic (that it “saves children”) contributed to the perceived value of the Programme; (4) the one-on-one counselling and follow up provided to mothers which was seen as most effective in changing behaviour; and (5) the contribution to enforcing the Code and the inadequate support given to artificial feeding.

8.3. Efficiency The extent of resources allocated and utilized to achieve the desired (stated) objectives. The degree of control exercised over the quality and quantity of outputs. Efforts made at the National level to leverage pre-existing results, partnerships, synergies and approaches.

Evidence shows that the IYCF-P in Jordan was relatively cost-efficient as compared to similar programmes in other countries. The Programme counselling unit costs are on the lower side of counselling costs referred to in the available literature, and tend to corroborate the cost-efficiency of the intervention. On the other hand, it was noted that the Programme is being implemented separately from existing services despite the note that efforts for integration were made. These included: (1) Coordination with other sectors; (2) Incorporating messaging related to newborn care, immunization, hygiene promotion, supplementary feeding programme, and anaemia prevention; (3) Conducting capacity building training for CBOs and MOH staff; and (4) Collaborating with the MOH, CBOs and other outlets such as schools to implement activities related to the Programme. The quality and technical skills of the staff working within the Programme has been seen as high. Staff receives regular training and follow up. The financial management of the


Programme lacks some clarity and proper documentation of disbursements by budget line items and activities. The evaluation found that there is added value of the volunteers to the Programme; however, it is premature to draw on the extent to which their presence affected efficiency. The evaluation team noted that the integration of IYCF+ can be considered as a cost effective intervention that contributed to increasing access to services with minor cost modifications. Other efforts were mentioned but were not taken forward, including integration with Early Childhood Development. One of the main opportunities for integration is the institutionalization of activities within MOH services, which would contribute greatly to further efficiency and sustainability. The Programme has had delays in implementation partly due to waiting time for approvals from different Ministries, which required the Programme to undergo several extensions and amendments. Still, the Programme was able to function within planned budgets at different time points; however, implementation rate did not match set plans.

8.4. Likelihood of Impact A perceived change in knowledge, capacity, and practices among service providers and end-users, as well as in the health and nutritional status of children and pregnant and lactating women is noted. Other changes were perceived such as those related to changes in economic situation and social interaction. Most changes were attributed to the counselling, follow up and education activities that were received and implemented by the Programme. Policy change was also noted as a result of the Programme. The assessment of the Programme against the theory of change also indicates potential of the Programme contributing towards impact. With this said, whether the Programme has contributed to the impact, which is improved nutrition status of children, cannot be confirmed until quantitative data is analysed. It should also be noted that achievement at impact level will be dependent upon the Government’s willingness to take ownership and responsibilities around the IYCF Programme.

8.5. Sustainability Although sustainability was taken into consideration in the design of the IYCF-P, there is no evidence of a clear sustainability strategy and gradual exit plan. As the Programme moved forward, some efforts to build partners’ capacity were made, but these fell short of what is needed to ensure sustainability. Qualitative data show that the Programme is being implemented as a silo. In host communities in particular, the Programme is implemented in a vertical mode parallel to existing structures. Despite the work that is being done to improve IYCF through the centres, there is no clear indication of how such work is being integrated and institutionalized as part of IYCF services offered by hosting structures. Similarly, it is not clear how this contributes to an overall strategy.


Existing structures with potential for ensuring sustainability include centres within the MOH, however, there is little evidence that centre staff would be capable of implementing the needed tasks at the same quality level. Although some level of knowledge related to IYCF has been transferred to MOH staff, the KAP survey shows that trained heath care providers would not be able to provide adequate and quality advice to mothers once faced with the need to do so. Although there is willingness of existing structures including MOH and other partners to integrate the Programme, the preparedness level varied across structures. Various barriers were identified for this integration including: the lack of current prioritization of IYCF indicating inappropriate timing for the phasing out of the UNICEF supported programme implemented by SC-J; limited technical capacity within the MOH; lack of funding; and the power of businesses marketing infant formula. Additional in-depth assessment is still needed to conclude the particular pre-requisites needed to proceed with the integration of the Programme within MOH systems, including a thorough assessment of the MOH’s current structure, related systems and capacity. Despite some effort towards sustainability, more strategic planning needs to be incorporated within individual activities to sustain the Programme and its effects. Potential opportunities for ensuring sustainability include integration of IYCF services within MOH existing structures and the identification of potential local partners that would integrate IYCF within their systems. In camp settings, the Programme is being implemented in coordination with other partners; however, services are still being provided separately from other health activities. Efforts have been put to incorporate newborn care including vaccination and hygiene promotion within the services provided by the Programme, which proved to be useful. However, opportunities to improve sustainability include building on community members as well as existing partners already providing health services. Other opportunities for integration within the camp settings include integration within ECD activities.

9. Lessons Learned The evaluation drew main lessons learned that should be taken into consideration while planning similar Programmes and for future developments: A baseline assessment needs to be incorporated at the initiation phase of any similar

future Programmes. This is essential to measure the impact of such Programmes. Longitudinal tracking of change in breastfeeding rates among PLW can give an indication

about the impact of the IYCF-P. The establishment of a monitoring and tracking system as well as implementing representative longitudinal surveys contribute to assessing the impact and capturing the effects of the Programme.

Sufficient quantitative indicators need to be reported throughout the Programme to ensure reporting and measuring impact of the Programme. Although the Programme did not have a baseline assessment, still, throughout the three years of implementation, there could have been some modifications in the indicators to include some form of evaluation


of the Programme impact including rate of breastfeeding mothers or other IYCF indicators. Despite the availability of such data on paper through monitoring forms, they are not systematized in a manner that enables reporting on progress. Having a clear log frame that includes IYCF indicators and targets linked to the theory of change readily available would have facilitated the process of evaluating the impact of the Programme and therefore further highlighted its need.

Stronger capacity building and mentoring components needed to be incorporated from the beginning. It was apparent that from its beginning the Programme was designed as an emergency intervention.

Stronger engagement of the MOH from the beginning is needed to ensure sustainability. As the Programme progressed to target host communities, no attempt was made to provide a sustainability plan or at least documentation for the gaps needed to be filled to provide appropriate IYCF services through a sustainable approach. In addition to the capacity building, there should have been a stronger engagement of the MOH from the beginning to be able to brainstorm about sustainability plans and measures.

Better financial quality assurance was needed. The evaluation team noted gaps in financial management from the part of the implementing partner (SC-J). Attempts are being made by SC-J to improve financial quality, and these need to be taken forward.

Further communication and guidance from UNICEF needs to be provided throughout the project cycle. UNICEF consistently provided feedback and guidance to SC-J during the PCA development stage; however, this guidance appeared to be less during the implementation period.


10. Recommendations The below recommendations are based on the findings and results concluded from this evaluation. Recommendations have been discussed with key stakeholders including MOH, UNICEF, and Save the Children Jordan, and are divided into Short term (more operational in nature) and Long term (more strategic in nature). Recommendations are also stratified by context; those for the camp settings are different than those in the host communities. Strategic / Long-term recommendations relate to National level interventions. The schematic presentation below summarizes the main recommendations according to context and time.

Short - term

Camp

•Recommendation #1:

Scale up and build on strenghts

•Recommendation #2:

Develop Programme activities

•Recommendation #3: Address gaps in data

Host

•Recommendation #4: Pilotintegration plan

Medium - term

National/Host

Recommendation #5: Assessment of structures and systems

Long-term

National

Recommendation #6: Integrate the IYCF Programme into a National IYCF Strategy built on existing efforts

Recommendation #7: Develop an integration plan for IYCF within existing child care Programmes


Recommendations Context Addresses Time

Recommendation #1

Scale up the Programme to an integrated Programme including IYCF, ECD, newborn and child care/ANC/PNC, and mental health. The Programme proved to be useful and needed, filling a gap in IYCF. At the same time, the Programme presents an opportunity to provide a fully integrated set of services that address the same target population and require a set of skills that are relevant to those provided by the Programme team. In order to improve on efficiency and effectiveness of the Programme, the evaluation team recommends that IYCF services continue to be provided in the camp settings, however, using a different integrated modality. Building on existing literature, there is a strong link between early childhood development and early nutrition. This has been further emphasised and confirmed in the new recently published Lancet series, where the message that child health and childhood development services should be integrated is strongly emphasized. The series highlights the importance of multi-sectoral interventions starting with health, which can have wide reach to families and young children through health and nutrition 91 . The series also highlights 'nurturing care', especially of children below three years of age. Based on the above and given the potential of the Programme to be integrated as such, the evaluation recommends that the Programme modality be modified to encompass in addition to IYCF, other early childhood development activities. The evaluation found that a number of the ECD messaging coincide with those provided through the IYCF-P. Furthermore, newborn care messaging and counselling, ANC and PNC, hygiene promotion and mental health are also valid to be incorporated within the Programme.

Camp UNICEF SC-J

Short-term

91 lancet series : http://www.thelancet.com/series/ECD2016 accessed on October 7, 2016

http://www.thelancet.com/series/ECD2016


The evaluation recommends that the Programme be scaled up to an integrated Programme that includes all the above-mentioned activities. The Programme does not necessarily need to be named IYCF-P, however, consideration should be given to show that the Programme targets the first 1000 days in a child’s life with essential services that would ensure a healthy start. This is to continue the advocacy of IYCF. Given the above, SC-J’s role would be to build capacity of existing volunteers and professionals to provide IYCF services. Capacity building would also encompass the array of services that will be provided through the Programme including ECD, Newborn Care, and Early Nutrition. Additional recommendations related to Programme operation are included below. Specifically for Zaatari camp, special consideration needs to be given in order to decrease the unit cost per beneficiary. Engagement of more Syrian refugee community mobilisers, educators and even counsellors might be useful.

Recommendation #2

Consider incorporating additional kinds of IYCF support to influence Programme outcome including 1) providing mother-to-mother support to contribute to sustainability of the Programme, 2) Incorporate a “men’s component” into the Programme where men will be targeted separately, 3) conducting regular capacity building and sensitizing activities targeting camp service providers including NGOs. Findings show that the role of community mobilizers was seen as essential and contributing to Programme outcomes. In order to contribute to the sustainability of the Programme, the evaluation recommends that the mother-to-mother component of the project, previously planned to be implemented, be acted upon. The “mother care group” model implemented by IMC or the “mother action group” model developed by World Vision could serve as examples to follow. The mother-to-mother group will provide the opportunity for ‘lead’ mothers from the targeted community to provide regular support to other

Camp UNICEF SC-J

Short-term


mothers PLWs as previously shown (Kushwaha et al., 2014)92. This could be particularly useful in Zaatari camp, given the saturation of the number of beneficiaries. Given the role that men have in a number of family health decisions and in view of the cultural sensitivities, it is recommended that the Programme looks into different modalities for targeting men. One way could be to hire male community mobilizers or staff who would be trained on health messaging. To counteract the conflicting messages on IYCF between different service providers, the evaluation recommends that continuous dissemination of IYCF messages be conducted targeting camp service providers. Regular workshops and sensitization sessions could be implemented to raise awareness about IYCF guidance including protection, support and promotion. It is also important to look into existing opportunities of integrating IYCF messaging into other sector material such as mental health, ECD, food security and reproductive health.

Recommendation #3

Address gaps in monitoring, data and financial management. Although data including output indicators and financial information has been shared through this evaluation, it is evident that there is a gap in data management, monitoring and financial management of IYCF-P. First, the evaluation recommends that a gap analysis/barrier analysis or baseline assessment be incorporated. Second, the evaluation recommends inclusion of an organized tracking system for reporting that would be integrated and/or shared with MOH. The tracking would match Programme-identified indicators and serve as a monitoring tool. There is a need to adopt a unified results framework developed based on Theory of Change that would be used for regular reporting of indicators. In addition, further financial management capacity development is warranted to ensure consistent and accurate reporting. Third, there is a need to include an evidence base research approach that would guide and feed into the

Camp and Host Community

SC-J and UNICEF

Short-term

92 Kushwaha, K. P., Sankar, J., Sankar, M. J., Gupta, A., Dadhich, J. P., Gupta, Y. P., ... & Sharma, B. (2014). Effect of peer counselling by mother support groups on infant and young child feeding practices: the Lalitpur experience. PloS one, 9(11), e109181.


Programme evolution.

Recommendation #4

Develop an exit strategy that includes steps for phasing out with clear actions such as capacity building, coaching/mentoring of partner staff to be assumed and monitoring of service provision including documentation. These can be implemented within MOH centres or CBO centres and serve as a pilot for integration of the program within MoH. The way the Programme currently runs, there is lack of ownership by partners of Programme activities. There is a need to ensure that primary health centers provide the IYCF services through an integrated approach. Health care providers need to be mandated to provide the IYCF services. They also need to have the capacity to do so. Recommendations #6 and #7 below addresses how to ensure health care providers are mandated to provide the services while this recommendation focuses on improvement of capacity through mentoring and coaching. It is assumed that through this step, which is short-term, lessons learned will emerge which will feed into the longer-term recommendations (steps). Given the lack of integration of activities within centres used as outlets, the evaluation recommends that a clear integration plan be devised that would ensure capacity building and include the following: a. Assessment of capacity of the centre hosting the

services. b. Identification of one or two focal points within

the centre that would serve as key agents for implementation of the Programme.

c. Developing a capacity building plan for focal points that would include transfer of knowledge provided to SC-J staff.

d. Providing regular on-the-job coaching and mentoring for identified focal points that would start with job shadowing and end with a handover. At the start of the process, focal points would first observe SC-J staff provide the services, the former would then gradually start providing counselling and education with mentorship from SC-J staff. At the end of the coaching period, it is expected that focal points

Host Community

UNICEF MOH SC-J

Short-term


within centres will be conducting counselling and education independently with minimal support from SC-J.

e. Monitoring of services through regular visits. SC-J would provide technical support and regularly monitor service provision.

f. Develop an integrated monitoring and reporting system that is linked to MOH health information system.

It is recommended that centres with the capacity of implementing the above plan be selected. SC-J can conduct assessments for different centres and choose centres that are deemed capable of moving forward with this plan. The above plan would be accompanied with the below suggested sustainability measures.

Recommendation #5

Conduct a thorough assessment of bottlenecks in existing structures and health systems within MOH to improve effective IYCF service coverage and accordingly devise a plan to strengthen IYCF within existing Primary Health Care system platforms Although the evaluation shed light on some of the existing gaps in the structure within MOH to support IYCF, there is still a need for an in-depth assessment of health and nutrition systems bottlenecks as a whole. Such actions are crucial in order to build a strong base for a sustainable national IYCF-P. The assessment should include, in addition to a review of existing human resources, capacity and structure, an examination of policies and systems governing IYCF. This would help ensure a constructive and concrete plan is devised to support IYCF at the central level. At the present time, the evaluation showed that there is willingness and openness from MOH to support IYCF, however, capacity is lacking and therefore a gap analysis is necessary. The assessment should also include an overview on existing child health Programmes and thus identify outlets and opportunities for integration of IYCF activities within existing Primary Health Care and Reproductive Maternal, Newborn and Child Health (RMNCH) Programmes, such as newborn care and child development Programmes. Conduct an assessment of existing capacity within the social structure including CBOs, NGOs, and

National / Host community

MOH UNICEF SC-J

Strategic


other professional entities. In addition to existing capacities and structures at the central / MOH level, the evaluation recommends that a mapping of potential technical resources be conducted. For example, entities such as the Syndicate of Midwives or Nurses could be used as potential outlets for administering one-on-one counselling. Similarly, as also shown in the results section, partners such as JHAS, with high potential for reach can also be examined as prospective outlets for administering IYCF services.

Recommendation #6 National UNICEF MOH

Strategic

Integrate the IYCF-P into the developed National Health Strategy and support the implementation of the strategy or development of a revised one. This Programme can form the basis for moving a national strategy forward and would be complemented by other activities. Based on the findings of this evaluation, the following actions can be taken to move forward with the established strategy that supports IYCF at the National level: a. Establish political will and prioritization of IYCF

within MOH through allocating more budget and enforce existing policy implementation (BMS code).

b. Form a Technical Committee to strengthen IYCF practices through Primary Health Care services and ECD services and identify a champion who would influence the moving forward of the plan.

c. Institutionalize the IYCF-P within the PHC and ECD level (following Recommendation #4 above) in order to incorporate IYCF services within the newborn care package.

d. Develop an IYCF emergency preparedness plan that would include contingencies for supporting IYCF during crisis. This would include provisions for enforcing the WHO BMS Code and providing counselling and frontline support.

e. Build the evidence about the need for IYCF in Jordan through documenting best practices and conducting regular assessments and IYCF reporting system. The last assessment was conducted 5 years ago and there is little evidence about the importance of supporting IYCF in Jordan. Ensure next DHS/JPFHS will assess the


rates of IYCF indicators but also evaluate the impact of IYCF practices on key health indictors within Jordan. This would build the base for advocating to further prioritize IYCF within MOH’s strategy. Develop an IYCF reporting system at the MOH level that is included in the health information system. This would serve as a surveillance tool and would also contribute to building the evidence about IYCF.

f. Incorporate Baby Friendly Hospital Initiative (BFHI) within the National Plan with clear implementation and sustainability measures. It has been shown that implementing the BFHI is considered a key contributor to successful breastfeeding initiation and continuation, including in the region93. In addition to capacity building and monitoring, MOH might consider incorporating BFHI into its accreditation requirements for hospitals in order to ensure sustainability.

g. Develop a public awareness IYCF plan to raise awareness about IYCF. This would also serve as an advocacy tool to lobby for IYCF within communities.

h. Integrate IYCF training material within the curriculums of health professionals including doctors, nurses, and midwives. The evaluation recommends that the curriculums of key health professionals include essential and sufficient IYCF material to build capacity of service providers at the pre-service level.

Recommendation #7

Develop an integration plan for IYCF activities within an integrated package of newborn, child health, ECD and nutrition services for 0-2 year olds. As mentioned above under recommendation #1, there is enough evidence to show that interventions targeting children under 3 years of age should emphasise an integration between child development, health and nutrition. For that, and based on results of the assessment recommended to be conducted under recommendation #5 above, the

National UNICEF MOH

Long-term

93 Akik C, Ghattas H, El-Jardali F. K2P Briefing Note: Protecting breastfeeding in Lebanon. Knowledge to Policy (K2P) Center. Beirut, Lebanon; August 2015 Saadé N, Barbour B, Salameh P. Congé maternité et vécu des mères qui travaillent au Liban/Maternity leave and experience of working mothers in Lebanon. Eastern Mediterranean Health Journal. 2010 Sep 1;16(9):994. Nabulsi M. Why are breastfeeding rates low in Lebanon? A qualitative study. BMC pediatrics. 2011 Aug 30;11(1):75.


evaluation recommends that a new plan be devised that would detail a plan for integration of newborn care, children health, ECD and nutrition including IYCF. As stated under recommendation #4, this plan could be piloted in existing primary health centres and tested for feasibility including lessons learned. The plan would ensure that health care providers are mandated to provide such a comprehensive service.


11. Annexes

11.1. Appendix A: IYCF Programme Target Areas and Number of Beneficiaries

11.2. Appendix B: Summary of PCAs extensions and amendments

11.3. Appendix C: List of IYCF Programme Partners

11.4. Appendix D: Evaluation Matrix

11.5. Appendix E: Key Informant Interview Questions

11.6. Appendix F: List of IYCF Programme Key Informants

11.7. Appendix G: Topic Guide for Focus Group Discussion with Service Providers

11.8. Appendix H: Topic Guide for Focus Group Discussion with End-Users

11.9. Appendix I: KAP Assessment Questionnaire

11.10. Appendix J: KAP Assessment Results

11.11. Appendix K: Timeline Events

11.12. Appendix L: Summary of Skills and Capacity of Evaluation Team

11.13. Appendix M: Quotes

11.14. Appendix N: Table Matching National Strategy with IYCF Programme

11.15. Appendix O: Programme Log Frames Across Implementation Period

11.16. Appendix P: Mapping of nutrition services


11.17. Appendix Q: Key Beneficiaries, Staffing and Non-Staffing Costs for Camps and Host Communities

11.18. Appendix R: Unit Costs (JOD visit) for Camps and Host Communities

11.19. Appendix S: Key Beneficiaries and Staff Costs per Year of Implementation and Camp

11.20. Appendix T: Unit Education and Counselling Unit Costs (JOD visit) by Target and Year of Implementation

11.21. Appendix U: Proportions of IYCF and non-IYCF cost by non-staff cost item

11.22. Appendix V: Organizational chart for IYCF Programme and number of staff

11.23. Appendix W: Theory of Change

11.24. Appendix X: Case Study Report

Date post:	11-Jul-2020
Category:	Documents
Upload:	others
View:	1 times
Download:	0 times