END OF PROJECT EVALUATION FOR A WATER SANITATION AND HYGIENE … · 2019-11-22 · v Executive...

END OF PROJECT EVALUATION FOR A WATER SANITATION AND HYGIENE PROJECT IN

SATELLITE SCHOOLS

Final Evaluation Report

19 January 2019

i

Acknowledgements This evaluation was commissioned by UNICEF and the Government of Zimbabwe. The evaluation team would like to express their sincere gratitude to staff of UNICEF in the Water, Sanitation and Hygiene section. Specific mention goes to Moreblessing Munyaka and Terence Chanakira for their support during the evaluation. Our gratitude also go to staff from the National Coordination Unit (NCU), Ministry of Primary and Secondary Education (MoPSE), Ministry of Local Government, Public Works and National Housing (MLGPWNH), and from the Education section of UNICEF who took time from their busy schedules to participate in a two day theory of change and tools review workshop. The team would like to recognise and appreciate all those that participated in the evaluation as key informants and in group discussions. We would to especially thank members of the District Water and Sanitation Sub-Committees (DWSSCs) who organised our field visits in the three districts of Binga, Hurungwe and Mwenezi. Special mention go to all the school staff and students that participated in the evaluation. The views and opinions expressed in this report are of the independent evaluation team and do not represent the position of UNICEF Zimbabwe, UNICEF Australia, or the Government of Zimbabwe.

Evaluation Team Ngonidzashe Marimo (Team Leader)

Nontokozo Sibanda (WASH Consultant) Engineer Remembrance Mashava (WASH Expert and Quality Assurance)

Lillian Siziba (Statistician and Data Manager) Joyline Chikuni (Project Manager)

ii

Table of Contents

ACKNOWLEDGEMENTS ..................................................................................................................................... I

TABLE OF CONTENTS ........................................................................................................................................ II

LIST OF BOXES ........................................................................................................................................................ III LIST OF FIGURES ...................................................................................................................................................... III LIST OF TABLES ....................................................................................................................................................... III

ACRONYMS .................................................................................................................................................... IV

1 EXECUTIVE SUMMARY ............................................................................................................................ V

2 INTRODUCTION ....................................................................................................................................... 1

3 BACKGROUND ......................................................................................................................................... 1

3.1 PROJECT CONTEXT...................................................................................................................................... 1 3.2 PROJECT DESCRIPTION ................................................................................................................................ 2 3.3 PURPOSE AND OBJECTIVES OF THE EVALUATION ................................................................................................ 5 3.4 EVALUATION SCOPE .................................................................................................................................... 5 3.5 AUDIENCE OF THE EVALUATION AND USES ....................................................................................................... 6

4 METHODOLOGY ...................................................................................................................................... 8

4.1 EVALUATION DESIGN ................................................................................................................................... 8 4.2 TOOLS AND TECHNIQUES ........................................................................................................................... 10 4.3 DATA COLLECTION .................................................................................................................................... 15 4.4 DATA QUALITY CONTROL ............................................................................................................................ 15 4.5 DATA ANALYSIS ........................................................................................................................................ 15 4.6 STAKEHOLDER VALIDATION ......................................................................................................................... 15 4.7 ETHICAL CONSIDERATIONS.......................................................................................................................... 16 4.8 LIMITATIONS OF THE EVALUATION ............................................................................................................... 16

5 FINDINGS ............................................................................................................................................... 18

5.1 RELEVANCE ............................................................................................................................................. 18 5.2 VALIDITY OF DESIGN ................................................................................................................................. 21 5.3 EFFICIENCY ............................................................................................................................................. 29 5.4 IMPACT .................................................................................................................................................. 32 5.5 PARTNERSHIPS AND COORDINATION ............................................................................................................ 61 5.6 SUSTAINABILITY ....................................................................................................................................... 63

6 CONCLUSION, LESSONS LEARNED AND RECOMMENDATIONS ............................................................... 68

6.1 CONCLUSION ........................................................................................................................................... 68 6.2 LESSONS LEARNED .................................................................................................................................... 68 6.3 RECOMMENDATIONS ................................................................................................................................ 68

ANNEXES ....................................................................................................................................................... 72

ANNEX 1: TERMS OF REFERENCE .............................................................................................................................. 72 ANNEX 2: LIST OF PERSONS INTERVIEWED .................................................................................................................. 73 ANNEX 3: LIST OF DOCUMENTS CONSULTED............................................................................................................... 74 ANNEX 4: EVALUATORS BIODATA ............................................................................................................................. 75 ANNEX 5: EVALUATION MATRIX............................................................................................................................... 76 ANNEX 6: THEORY OF CHANGE OF THE PROJECT .......................................................................................................... 83 ANNEX 7: RESULTS FRAMEWORK OF THE PROJECT ....................................................................................................... 84 ANNEX 8: TOOLS AND CONSENT FORMS .................................................................................................................... 86

iii

List of Boxes BOX 1: QUALITATIVE DATA COLLECTION METHODS ........................................................................................................... 14 BOX 2: GIRLS AND BOYS TOILETS .................................................................................................................................. 21 BOX 3:DIFFERENCES IN DISABILITY FRIENDLY SQUAT HOLES ................................................................................................ 22

List of Figures FIGURE 1: SUMMARY OF INTERVENTIONS FOR THE WASH IN SATELLITE SCHOOLS PROJECT ....................................................... 3 FIGURE 2: LINK BETWEEN WASH IN SATELLITE SCHOOLS PROJECT RESULTS CHAIN AND QUANTITATIVE TOOLS ........................... 11 FIGURE 3: JMP SERVICE LADDERS FOR MONITORING WASH IN SCHOOLS IN THE SDGS ........................................................... 12 FIGURE 4: PROBLEM FOR THE WASH IN SCHOOLS PROJECT .............................................................................................. 24 FIGURE 5: REVISED THEORY OF CHANGE FOR A WASH IN SCHOOLS PROJECT ........................................................................ 25 FIGURE 6: RETROSPECTIVE RESULTS CHAIN FOR THE WASH IN SATELLITE SCHOOLS PROJECT .................................................... 33 FIGURE 7: IN THE LAST THREE MONTHS, HAVE YOU SEEN, HEARD OR RECEIVED ANY MESSAGES OR MATERIALS ON WATER, SANITATION

AND HYGIENE? ................................................................................................................................................. 47 FIGURE 8: HANDWASHING AFTER TOILET USE .................................................................................................................. 49 FIGURE 9: HANDWASHING AFTER TOILET USE BY DISTRICT .................................................................................................. 50 FIGURE 10: HANDWASHING BEFORE EATING ................................................................................................................... 50 FIGURE 11: NUTRITION GARDEN SET UP AT MUUYU PRIMARY SCHOOL ............................................................................... 55 FIGURE 12: CLASSROOM BLOCKS UNDER CONSTRUCTION AT DAMBANZARA PRIMARY SCHOOL ................................................. 56

List of Tables TABLE 1: TOOLS FOR THE QUANTITATIVE SURVEY ............................................................................................................. 11 TABLE 2: MINIMAL SAMPLE SIZE ................................................................................................................................... 13 TABLE 3: SAMPLED SCHOOLS ....................................................................................................................................... 13 TABLE 4: SCHOOLS SELECTED FOR THE QUALITATIVE SURVEY ............................................................................................... 13 TABLE 5: QUALITATIVE SURVEY TOOLS ........................................................................................................................... 14 TABLE 6: COST OF BUILDING A SCHOOL LATRINE IN BINGA ................................................................................................. 22 TABLE 7: MINIMUM STANDARDS FOR SANITATION FACILITIES ............................................................................................. 36 TABLE 8: RATION OF STUDENT TO SQUAT HOLE FOR CHILDREN IN GRADE 3-7 ......................................................................... 37 TABLE 9: ADEQUACY OF SQUAT HOLES FOR CHILDREN IN INFANT GRADES .............................................................................. 38 TABLE 10: IS THE CLEANING AGENT OR ASH OR SOAP ALWAYS AVAILABLE AT THE SCHOOL HAND WASHING FACILITY? ..................... 39 TABLE 11: AVAILABILITY OF MHM COMMODITIES ........................................................................................................... 40 TABLE 12: SUPPORT BY SCHOOL TO PROMOTE SAFE MENSTRUAL HYGIENE............................................................................. 40 TABLE 13: CRITICAL TIMES TO WASH HANDS ................................................................................................................... 44 TABLE 14: SIGNS OF AN UNHYGIENIC ENVIRONMENT ........................................................................................................ 46 TABLE 15: SOURCE HYGIENE INFORMATION .................................................................................................................... 46 TABLE 16: OUTCOME INDICATORS TO MONITOR ALONG IMPACT INDICATORS ........................................................................ 58 TABLE 17: ALTERNATIVE INDICATORS TO MONITOR IN A SCHOOL WASH PROGRAMME ........................................................... 59

iv

Acronyms BCC Behaviour Change Communication BVIP Blair Ventilated Improved Pit CWD Children with Disabilities CwD Children with Disabilities DDF District Development Fund DEHO District Environmental Health Officer DWSSC District Water and Sanitation Sub-committee DWT ECD Early Childhood Development ECD Early Childhood Education EHT Environmental Health Technician EMIS Education Management Information System ESAP Economic Structural Adjustment Programme FCTZ Farm Community Trust Zimbabwe FGD Focus Group Discussions GEROS UNEG Global Evaluation Reports Oversight System IMF International Monetary Fund INGOs International Non-Governmental Organisations KAP Knowledge Attitude and Practice MDGs Millennium Development Goals MHM Menstrual Hygiene Management MLGPWNH Ministry of Local Government Public Works and National Housing MOHCC Ministry of Health and Child Care MoHCC Ministry of Health and Child Care MoPSE Ministry of Primary and Secondary Education MoPSE Ministry of Primary and Secondary Education NCU National Coordination Unit NGOs Non-Governmental Organisations RDC Rural District Council SDC School Development Committee TOC Theory of Change WASH Water Sanitation and Hygiene ZDHS Zimbabwe Demographic Health Survey ZIMSTAT Zimbabwe Statistics Agency

v

Executive Summary Project description This report presents findings of the end of project evaluation for the WASH in Satellite Schools Project. UNICEF was supporting the National Action Committee1 for WASH in implementing WASH in Satellite Schools Project in 3 districts in Zimbabwe (Binga, Hurungwe and Mwenezi) covering 27 satellite schools2. Funded by the UN Australia National Committee, for a total of US$558,036.35, The WASH in satellite schools project had five outcomes:

• Outcome 1: 8,000 pupils and 300 teachers and their families at 27 satellite schools, 9 (schools) from each of in Zvishavane, Mwenezi and Hurungwe districts have access to safe water supplies, improved sanitation and hand-washing facilities by June 2018.

• Outcome 2: 8,000 pupils, 300 teachers, their families, and 27 School Development Committees (SDCs) at 27 satellite schools are reached with hygiene promotion messages by June 2018

• Outcome 3: 500 ECD pupils at 27 satellite schools have access to appropriate sanitation facilities by June 2018

• Outcome 4: 60 special needs pupils have access to user-friendly sanitation facilities at 27 satellite schools in 3 selected districts by June 2018

• Outcome 5: 300 teachers trained in child friendly teaching methodologies by June 2018

Three Non-Governmental Organisations were recruited as implementing partners for the project: Mvuramanzi Trust, Farm Community Trust, and Plan International covering Binga, Hurungwe and Mwenezi respectively. Evaluation objectives and intended audience This was a summative evaluation based on school level experience of the WASH in Satellite Schools project. The main purpose of the evaluation as per the Terms of Reference was stated as follows:

1. Generating lessons and providing evidence to guide decision making on WASH in schools

2. Improving project design for future WASH in schools’ programs. This evaluation should provide evidence of what works, under which circumstances thereby support in the design of quality programs and innovations.

3. Promoting accountability through communicating to internal and external stakeholders the value of the WASH in schools’ interventions.

Evidence provided through this evaluation were to assure all stakeholders of how investments made in WASH in Schools transform into tangible benefits for the children. The main audience of the evaluation are: 1) Ministry of Primary and Secondary Education (Duty Bearers); 2) beneficiary schools (school authorities and children) and communities (Rights holders); the National Action Committee (Duty Bearers); and Development Partners (UNICEF, NGOs) (Duty Bearers). An elaborate plan for results dissemination was implemented that involved all stakeholders involved in implementation of the project from district to national level and other stakeholders with an interest in supporting WASH in schools at these two levels (district and national level).

1A governmental and non-governmental structure for WASH coordination in Zimbabwe 2 Satellite schools are those that are not properly registered due to various reasons and operate under another school’s name.

vi

Evaluation methodology Evaluation design: Because no baseline was undertaken by the project, the evaluation adopted a One-Shot Retrospective Design premised on recall during structured and qualitative surveys to reconstruct the baseline. The evaluation data collection was undertaken in all districts of the project: Binga, Hurungwe and Mwenezi. UNEG guidance on human rights centred and gender responsive evaluation were adopted in the tools and data collection approaches. Methods: Qualitative and quantitative methods were utilised to collect primary and secondary data to inform evaluation findings. Multiple quantitative data collection tools were designed aligned to the project’s theory of change. This allowed the evaluation to capture results of the project beyond the results framework outcomes mentioned in the project background. To improve evaluability, the theory of change was revised through a participatory tools review workshop and subsequent development of a retrospective results chain that elaborated outcomes and impacts of the project (in addition to the stated output indicators in the results framework). This process culminated in the development of a set of indicators to adequately measure project results. Four tools were developed to measure project performance on various indicators and were a: 1) Student questionnaire; 2) Head Teacher questionnaire; 3) Infrastructure observation checklist; and 4) Sanitation and hygiene practice observation checklist (at critical times). The Infrastructure observation checklist was guided by the UNICEF WASH in Schools Monitoring Package3 and the UNICEF/WHO Core questions and indicators for monitoring WASH in Schools in the Sustainable Development Goals4. The Sanitation and hygiene observation checklist was a special addition to the tools box to verify handwashing behaviour results from the student questionnaire. The qualitative component of the evaluation, included an array of tools targeting all stakeholders of the project: learners; teachers; School Development Committees (SDCs); Water Point Committees (WPCs); Village Pump Mechanics (VPMs); Latrine Builders; Environmental Health Technicians (EHTs); DWSSCs; NCU; UNICEF; MoPSE; MoHCC; and NGO partners. Each data source generated specific information as well as providing opportunities for the evaluation team to triangulate findings between respondents. Focus Groups Discussions were used as the main data collection method at the school and community level, while key informant interviews were held at the national level. Some individual interviews were undertaken with members of DWSSCs in Mwenezi and Binga. Three participatory ethnographic techniques were used in group discussions with girls and boys to elicit in-depth discussions on topics of interest to the evaluation: 1) photo elicitation; 2) project posters; and 3) pathway through life. Sample size: UNICEF engaged the Zimbabwe Statistics Agency (ZimStat) to undertake sampling for the evaluation. Fourteen (14) schools were deemed adequate at 95% confidence interval and 5% precision. A Head Teacher and Infrastructure Observation Checklist was completed for each school. A total of 185 students was found sufficient at 95% confidence interval and 5% precision. This was surpassed as 196 children (Binga (56), Hurungwe (85) and Mwenezi (56) were interviewed. The target for Hygiene Observation, 185 observations, was surpassed as 980 observations were conducted at two points in time: after toilet use and before eating (during break time). FGDs with DWSSCs were conducted in each district. At each school an interview was conducted for each group of respondents.

3 https://www.unicef.org/wash/schools/files/wash_in_schools_monitoringpackage_.pdf 4 https://washdata.org/monitoring/schools

https://www.unicef.org/wash/schools/files/wash_in_schools_monitoringpackage_.pdf

https://washdata.org/monitoring/schools

vii

Data collection and analysis: Quantitative data was collected by research assistants using tablets to allow for real-time data quality review while qualitative data was collected by the evaluation core team and assisted by research assistants. All data from the quantitative survey was transported to SPSS to facilitate data analysis. This analysis mainly comprised descriptive statistics. Qualitative data on the other hand was transcribed in Microsoft Excel® following the evaluation criteria of relevance, validity of design, efficiency, partnerships and coordination, impact and sustainability. Discourse analysis was used to draw key themes and quotes that best illustrates these emerging themes from the evaluation. Validation and dissemination: The evaluation results were validated in meetings with national stakeholders and those from the targeted districts. A child friendly version and an abridged report of the evaluation findings were produced for wider dissemination to beneficiaries and stakeholders. Ethical considerations: an ethics plan and protocol was developed for the evaluation which included ensuring consent was sought from the school and children for participation in the data collection. A set of procedures were developed to guide research assistants in handling various situations with respondents they may come across. Limitations: The evaluation survey was undertaken during grade seven examinations. Therefore study participants excluded this group of students; and a majority of girls at menarche were not captured by the evaluation because of the age group included in the survey, therefore results on Menstrual Hygiene Management (MHM) outcomes are not conclusive. The project had no baseline. The evaluation relied on recall and existing data sets such as the Education Management Information System reconstruct a baseline. Nonetheless for some indicators there were baseline values could not be obtained Findings Relevance: The project was relevant in addressing the evident needs of access to clean and safe water, poor sanitation and hygiene. Children in ECD, girls and those with disabilities were particularly affected. It addressed lower access to water and sanitation for ECD children and the burden of fetching water for girls which improved their positive school experience. Studies in Zimbabwe have shown a correlation between a child’s negative perception of school experience and dropping out5. Therefore these benefits are likely to contribute to this group children staying in school. The project was also relevant in changing community knowledge and attitudes towards the importance of latrines and good hygiene. Between one in 10 and four in 10 households in the targeted districts had toilets with open defecation common place while handwashing was reported to be limited. Demonstrating the effects of good WASH at the schools, was therefore having a positive ripple in community perceptions about WASH. In terms of meeting policy standards, at the school level the project met policy standards for adequate toilets for boys (46.2% of the schools) but did not achieve the same for girls (69.2% of the schools) at the time of latrine construction in 2016. The policy standard for the distance of the water point from the school was met in a majority of schools (71.4%). Children in infant grades were worse off as the shortage of toilets was toilets was three to eight times greater than that for girls and boys in senior grades. These results show equity was not fully captured in implementation. Equity was considered at the level of target schools by selecting the most

5 Surridge, M. Chiroro, C. Marimo, N. (2018) Baseline Study: Longitudinal Study to Gain a Deeper Understanding of the Process of Survival and the Experiences of those who drop out of school. Report prepared for the Ministry of Primary and Secondary Education and UNICEF.

viii

vulnerable or WASH deprived satellite schools in each of the districts. However, the provision of a blanket number of squat holes per school further negated equity objectives as some schools ended up with over supply of toilets (38.5% of schools) while others faced a huge shortage (46.2% of schools) at the time of construction in 2016. There were policy gaps that needed to be addressed to enhance WASH in schools including: 1) the package of support for MHM in schools including nationally approved national designs for girl friendly toilets; 2) hygiene education and related child friendly materials; and 3) ensuring satellite schools are established in places with easy access to water. At the time of the evaluation proposed girl friendly toilet designs were developed an awaiting approval. The Education Management Information (EMIS) was observed to collect relevant information to inform WASH in Schools, however there were gaps in measurement of MHM support in schools. Validity of design: The project design was based on the experience of UNICEF in the Rural WASH programme and data from the Education Management Information (EMIS). Thus no specific needs assessment was undertaken to determine causality. NGO partners did undertake needs assessments but these were adhoc and not collecting all the information required to inform a WASH in schools intervention. The result was that while the project design was largely appropriate to address the needs of targeted schools improvements could have been made to address the specific needs of: 1) girls; 2) children in infant grades or Early Childhood Development (ECD)); and 3) Children with Disabilities (CwD). For example, the project did not fully address issues of menstrual hygiene management (MHM) for girls at menarche and those of children with disabilities. For ECD, toilet and hand washing access were a challenge as handwashing tanks were far from their toilets and the toilets were too few for the enrolment numbers of ECD. The project missed an opportunity to influence policy direction on MHM and disability friendly WASH as these issues were not fully captured in implementation despite a consciousness of the project’s need to address them among implementers. This situation seemed to be influenced by two reasons: 1) the project design not fully capturing these issues and therefore not providing a standard which left partners to implement according to their own experience leading to variances in implementation between districts; and 2) the lack of adequate policy guidance on standards. A participatory review of the project’s theory of change and supported by evaluation findings showed that it provided a strong link to the results chain of the project but with missed elaboration on MHM, policy influence, support for school based WASH planning and climate risk management key issues to ensure WASH in schools is adequately and sustainability addressed. When compared to global standards for WASH in Schools, the design and implementation of the project missed aspects of MHM, and integrating hygiene education in the school curriculum as focus was placed on health clubs. While there was a logical flow in the causality chain, some assumptions of the theory of change did not hold true and transformed to significant risks which disrupted the results logic. Lastly, all activities of the project were necessary but may have been insufficient to bring about the full extent of the desired change for all groups of learners. The activity on child pedagogy seemed an appendage with no direct link to the desired WASH results. Efficiency: While the evaluation did not undertake a cost benefit analysis, the changes brought about by the project such as increased sanitation facilities, water and hygiene transformed school experience for learners. Unresolved issues of equitable access, limited availability of water in handwashing tanks all undermine the value of the investment by the project.

ix

The quality of support was generally acceptable with beneficiaries and stakeholders expressing satisfaction. Latrines received the highest satisfaction for quality while concerns were raised on the quality of water from boreholes, leaking handwashing tanks and poor quality taps on the handwashing tanks. In terms of timely delivery, after a one year delayed start, implementation progressed well in the second year but the project faced teething challenges in Binga with regards availability of materials provided by communities (bricks especially) and the challenges with drilling contractors, which necessitated a one year no cost extension. UNICEF WASH Section has institutionalised reporting templates for value for money. However, this is limited to economy – the unit costs of procurement (cost per unit) and implementation (e.g. cost per beneficiary). Partnerships and coordination: The coordination and management structures for the project were sufficient. Ensuring national leadership through existing WASH coordination structures in implementation was commendable and represented good practice. It allowed the project to access an array of skills and knowledge which enhanced quality of interventions and timeliness of implementation that would have been otherwise too expensive or challenging for the project to sustain or achieve had it relied solely on NGO partners. Monitoring of the project was guided by existing tools used for the Rural WASH programme. While the monitoring tools were viewed as adequate by stakeholders, there may need to be reviewed to make them more relevant for a WASH in schools project including: 1) progress on MHM; 2) equity in availability of water and sanitation facilities; and 3) availability of materials and trained teachers on hygiene. Impact: The project’s impact was measured using the WHO/UNICEF Joint Monitoring Programme (JMP) for WASH in Schools in the Sustainable Development Goals (SDGs) four level ladder grading system of “Advanced Service”, “Basic service”, “Limited Service” and “No Service” for Drinking Water, Hygiene and Sanitation in schools. The project made significant contributions to Millennium Development Goals on access to water and sanitation ensuring all schools (100%) achieved “Basic Service” from “No Service” for drinking water but only managed to move less than half (42.8%) of the schools from “No Service” to “Basic Service” for hygiene. When districts are compared, Binga was the worst affected with all schools not reaching “Basic Service” for hygiene. In Hurungwe, 50% of schools had reached “Basic Service” for hygiene while all schools in Mwenezi had reached this level. Non-availability of cleaning agent (soap/soapy water) was the main drawback for provision of hygiene basic services in these schools as water was available in handwashing tanks at all schools at the time of the survey. While water was available in handwashing tanks at the time of the evaluation, more often than not handwashing tanks are not filled with water due to the burden of manually filling them. So while at the time of the evaluation a majority of schools had “Basic Service”, the reality is that for long periods the schools have “No Service”. Good hygiene behaviour was high for higher age groups than ECD. More boys (89.6%) than girls (86.8%) were washing hands after toilet use. The difference was more visible in handwashing with soap, boys (72.9%) and girls (61.3%). Hygiene behaviour for infant children is poor as 36.2% of ECD do not wash hands after toilet use. Presence of handwashing facilities that were absent prior the project were credited with improved hygiene behaviour while the limited access to such handwashing facilities by infant children was undermining good hygiene behaviour. There were some unintended impacts of the project. Project boreholes facilitated establishment of nutrition gardens in schools enhancing agriculture lessons and providing opportunities for income generation for schools. Availability of water sources at schools sped

x

up the process of building learning blocks thereby enhancing prospects for qualifying the schools for registration with MoPSE6. The school toilets and boreholes were providing relief to households surrounding the school but in some cases it undermined access to water by the intended beneficiaries who are boys and girls from the targeted schools. An interesting finding from Binga was that setting aside one or two girls’ toilets for use by girls during menstruation increased their stigmatisation resulting in their limited use. Sustainability: The strategy of working through existing structures at district level and having these structures lead implementation enhanced ownership of the project which increased prospects for sustainability. Community level structures and cadres that included: Water Point Committees (WPCs), Village Pump Mechanics (VPMs), and latrine builders, all contribute to building community capacity to operate and maintain water, sanitation and hygiene infrastructure. To support hygiene education the project trained two teachers per satellite school. Sustainability of the project benefits will be affected by a number of factors. First, financial constraints faced by government and the low income status of communities will hamper: 1) continued monitoring and support of project benefits; and 2) maintenance of infrastructure. Non-availability of critical supplies for boreholes, especially, will likely increase costs of repair and in turn undermine ability of communities to maintain the facility in the long term. Prioritisation of WASH maintenance by schools and SDCs was lower than other needs at the schools e.g. building of classroom blocks that will facilitate school registration. Registration would enable the schools access government funding, increase enrolment which would in turn increase per capita grants from government and levies from students. The lack of prioritisation was apparent in the long delays in replacement of taps at handwashing tanks and the non-repair of cracks and leaking roofs in some schools. Conclusions While the project performance varied, it is clear it had a positive impact on transforming school experience for children by enhancing access to safe water, sanitation facilities and handwashing. The situation before the project was dire, and it brought a welcome relief to children and communities. However, the inadequacies of the project in fully addressing equity (ECD and disability) and gender had negative effects on access to appropriate sanitation for girls and hygiene behaviour for ECD. Hygiene education also needed strengthening by ensuring it was systematised in the education curriculum at the schools with teachers and students having access to age appropriate education materials. Main recommendations This section presents a summary of the recommendations from the evaluation. More detailed recommendations are presented in section 5.3. Relevance 1. The siting of satellite schools does not consider water availability. This posed challenges

for the project as wet holes were only identified in distances further than the recommended 500m from the school. To address this challenge it is recommended that:

a) piped water schemes would need to be installed. Responsibility: NCU, MoPSE and

UNICEF

b) the MoPSE includes, in its school planning guidelines, the assessment of availability of ground water at viable depths and water yield within the school yard or recommended distance of up to 500m. Responsibility: MoPSE

6 Satellite schools are regarded as informal schools as they do not meet the minimum standards for registration as formal schools. The main reason for establishment of satellite schools is to reduce walking distances to school. These schools are attached to a formal “mother school”.

xi

2. To improve accessibility for children with disability there is need to review the system design of the access point and ensuring water is available within the school yard through piped water schemes. Responsibility: UNICEF, MoPSE

3. There is need to review the data collection form for EMIS (ED46 form) to enable it to

capture issues of support offered by schools for MHM and availability of MHM friendly latrines. Responsibility: MoPSE

Validity of design 4. It is recommended that a future project adopts a theory of change based on a full causal

analysis of the challenge of WASH in schools. It should have a balance between activities at school level and addressing policy gaps that undermine WASH in schools. Responsibility: UNICEF

5. Several assumptions turned into risks. There is need for a future project to design a

monitoring mechanism oversight of these assumptions. This should be underpinned by a flexible project that addresses a shifting context. Responsibility: UNICEF

Efficiency 6. While UNICEF monitors value for money through economy indices, there is need to

ensure, efficiency and effectiveness are also captured to have a fuller understanding of the project’s performance on value for money: Responsibility: UNICEF

7. The project used monitoring tools from the Rural WASH programme. While rationale in

order not to introduce an additional monitoring burden for implementers, there was need to add indicators that enable planning for WASH in schools. Responsibility: NAC

Impact 8. The WASH sector and partners need to work on the development of IEC material suitable

for infants and primary school level to enhance effectiveness of hygiene education for improved hygiene practices. Responsibility: NAC

9. There is need for the establishment of a national standard for hygiene promotion and

education for implementation in schools. If all schools are to adopt hygiene education as part of the core curriculum standardisation has to be ensured for effective implementation. This also ensures sustainability in case of transfer of teaching staff. Responsibility: MoHCC, MoPSE and UNICEF

Sustainability 10. Projects should ensure that there are more contact sessions with water governance

structures e.g. WPCs, to ensure continued clarification of roles and conflict resolution to avoid their collapse or non-functional committees. Ensuring each WPC has a clear and implementable constitution or code of operation will also help in this regard. Responsibility: NAC and DWSSCs

Coordination and Partnerships 11. There was no formal platform created for partners to share information on implementation

to ensure lessons sharing that would contribute to standardisation across the districts. The creation of platforms for partners in similarly structured projects for sharing of notes and innovations to reduce the differences in implementation and increasing benefit to all targeted beneficiaries will therefore be important. Responsibility: UNICEF

12. Child involvement in project planning and implementation is key in ensuring

appropriateness of facilities targeting them. Projects should develop ways of consultation and engagements to gather views from this target group. Responsibility: UNICEF

1

1 Introduction This report presents findings of the WASH Satellite Schools End of Project Evaluation. It presents the context of the project, its description and approach and methodology adopted for data collection, analysis, verification and dissemination. It first provides a background to the evaluation including the evaluation’s purpose and objectives. From this, the report proceeds to detail the methodology used for the evaluation. This is followed by a discussion of the findings. Conclusions, lessons learned and recommendations comprise the last section of the report.

2 Background

2.1 Project Context Why WASH in Schools: Children spend a majority of their time in school. Therefore, improving WASH provision in schools enhances children’s learning potential as disease transmission is limited and dignity, especially for girls is upheld. For girls, the presence of adequate WASH including privacy and comprehensive menstrual hygiene management, improves their school attendance, safety, and performance contributing to gender equity in learning. There is evidence that improved WASH can also have wider effects in the community. Children acting as change agents, in families and communities, from improved knowledge and practice of WASH in their schools, can influence the hygiene and sanitation practices of their parents, siblings and community members7. When the children grow up, their sustenance of this improved WASH behaviour can be passed on to their children facilitating future generational change in WASH behaviour. The importance of WASH in schools is codified in the Sustainable Development Goals under targets 4.a, 6.1, 6.2. Under these targets, WASH in schools is a key component for “safe, non-violent, inclusive and effective learning environment” and “universal WASH access”. The Zimbabwe Education Sector and WASH in Schools: From 1980 Zimbabwe invested significantly in education. This saw Zimbabwe rise to become one of the most literate countries in Africa. This investment has left a legacy of high literacy rates, and at 92% literacy, Zimbabwe is the second most literate nation in Africa. The legacy of this investment has seen the sector relatively withstanding economic shocks, first experienced from 1992 when austerity measures were initiated through the International Monetary Fund (IMF) supervised Economic Structural Adjustment Programme (ESAP) and economic stressors experienced from 1998 to present day due to a variety of factors with the fast track land reform process and its resultant isolation of the country from international financial markets being the chief cause. Despite the reduction in investment in the sector, with government expenditure limited to staff salaries, community support at the school level, through school fees and levies, has largely contributed to continued resilience of the sector. The sector has also managed to retain highly skilled and motivated teachers, school heads and supervisors. This scenario has strengthened the sector at the bottom allowing for beneficial interactions between the school, communities and district level. Nonetheless the heavy reliance on community support has created a bottom up school financing system which unfortunately disadvantages schools in poorer communities. This has resulted in deep rooted inequity in school investments with children in rural and remote areas

7 For more information on the evidence base see the multi-agency advocacy document on WASH in Schools: Raising Clean Hands: Advancing Learning, Health and Participation through WASH in Schools (UNICEF New York, 2010)

2

being most affected learning outcomes and pass rates at different levels8. Satellite schools located in remote areas and resettlement areas. A study by Tarisayi (2016) noted that while there was an apparent need for funds to build toilets, classrooms and teachers’ accommodation, efforts were being hampered by lack of resources at the satellite schools9. Because communities are poor, they are unable to meet these infrastructure demands. All schools sampled in the study had no reliable water supply and toilets for both students and teachers. Another study, SNV (2012)10, demonstrated the inequity in provision of WASH between urban and rural areas with the former more affected. It also noted that there was limited investment in WASH at schools with the major draw-back being the lack of a clear cause and effect in the Zimbabwe context for the lack WASH in Schools and education outcomes. Schools infrastructure development plans do not prioritise investment in WASH infrastructure when compared to education materials and classrooms etc. This draw-back is despite a clear support for WASH in schools in the legislation and policies for the education sector. The Education Act, amended in 2006, requires every school to have two blocks of toilets, one for boys and the other for girls, the ratio of the toilets to school enrolment. The recently launched, March 2018, Zimbabwe School Health Policy, provides broad based support for WASH in schools by providing a framework to equip learners with comprehensive life skills for healthy living and reduce health barriers to learning.

2.2 Project Description In response to the prevailing situation, UNICEF was supporting the National Action Committee for WASH in implementing WASH in Satellite Schools Project in 3 districts in Zimbabwe with a total budget of US$558,036.35. The project aimed to improve access to safe water by pupils and communities in the catchment areas of twenty-seven (27) satellite schools in three districts of Hurungwe, Mwenezi and Binga. It also aims to provide adequate sanitation for pupils in 27 target satellite schools. Initially planned to start in June 2014 and end in December 2017, the project received a one year cost extension to June 2018. Implementation was undertaken was contracted to three NGOs Mvuramanzi Trust, Farm Community Trust, and Plan International working with District Water and Sanitation Sub-Committees in the three districts11. The WASH in Satellite Schools Project aimed at delivering the following major results by the end of the project period:

1. An estimated 8,000 pupils and 300 teachers and their families at 27 satellite schools in Binga, Mwenezi and Hurungwe districts have access to safe water supplies, improved sanitation and hand-washing facilities by June 2018.

2. An estimated 8,000 pupils and 300 teachers and their families and 27 School Development Committees (SDCs) at 27 satellite schools are reached with hygiene promotion messages by June 2018.

3. An estimated 500 ECD pupils at 27 satellite schools have access to appropriate sanitation facilities by June 2018.

4. An estimated 60 special needs pupils have access to user friendly sanitation facilities at 27 satellite schools in 3 selected districts by June 2018.

8 This is confirmed by the Zimbabwe Early Learning Assessment (ZELA) that has been undertaken annually at the end of Grade 2 since 2012, and by pass rates at Grade 7 and ‘O’ level. 9 Tarisayi K. (2016) An Exploration Of The Challenges Encountered By Satellite Schools In Masvingo District, Zimbabwe. https://internationaljournalofresearch.com/2016/11/28/an-exploration-of-the-challenges-encountered-by-satellite-schools-in-masvingo-district-zimbabwe/ 10 SNV (2012) Masvingo Schools WASH Report. http://www.snv.org/public/cms/sites/default/files/explore/download/masvingo_-_wash_in_schools_-_2012_.pdf 11 Each NGO covered a district as follows: Plan International in Mwenezi; Mvuramanzi Trust in Binga; and FCTZ in Hurungwe.

3

5. An estimated 300 teachers trained in child friendly teaching methodologies by June 2018.

The project paid special attention to students with special needs such as those living with disabilities and students in Early Childhood Development (ECD) classes. In this regard, sanitation infrastructure were designed to make them accessible to these target groups as well. Interventions of the project are categorised into four (Water, Sanitation, Hygiene and Sustainability). Figure 1 provides a summary of these interventions. Figure 1: Summary of Interventions for the WASH in Satellite Schools Project

2.2.1 Project theory of change According to the project’s theory change (Annex 3), the project addresses problems of high diarrheal incidence, and low school attendance, especially among girls, caused by the lack of adequate water and sanitation facilities, poor hygiene practices and the lack of menstrual hygiene management. The project addresses these challenges through the project components discussed earlier:

1. Construct boreholes in target schools which are accessible to the disabled; 2. Train community latrine builders; 3. Train School Health Club; and 4. Train and equip water point committees and village pump mechanics.

These investments were expected to lead to increased access to clean water, sanitation and hygiene facilities and increased correct knowledge of water, sanitation, and hygiene. With continued support through health promotional activities of the school health clubs, sustainability of constructed infrastructure, these changes were expected to translate to sustained use of clean water, appropriate hygiene and sanitation practices in the school

4

environment. Positive WASH practices would in turn lead to reduction in diarrheal incidences and increased school attendance by boys and girls.

2.2.2 Project management and implementation structure The project was managed through existing Rural District Council structures, school structures and community management structures. Implementation is supported by Civil Society Organizations (INGO/NGOs). UNICEF was responsible for providing technical support. Implementation of the WASH in Satellite Schools Project was anchored on existing structures in order to promote ownership and sustainability. At district level the project was coordinated through the Rural District Council’s inter-ministerial subcommittee, the District Water and Sanitation Sub-Committee (DWSSC). The Ministry of Primary and Secondary Education (MoPSE) is a member of the DWSSC and provided access to schools for the WASH in Satellite Schools Project. At school level, the project is implemented through school structures such as the School Development Committees (SDCs) and school health clubs. The SDCs provide the linkage between the school and the community. Civil Society Organizations (CSOs) engaged through UNICEF were supporting implementation of the project.

2.2.3 Project activities By the time of the evaluation the project had achieved the following under water provision:

– drilled 26 boreholes covering 26 schools of the 27 targeted – Installed solar powered piped water systems in 3 schools in Binga district

For sanitation, the main achievements recorded include:

– Construction of 333 BVIP latrines in the targeted 27 schools and across the 3 districts (Binga, Hurungwe and Mwenezi)

– 54 of the latrines are for ECD level children so have smaller diameter squat holes

– Another 54 of the latrines are for children with special needs

Achievements in hygiene were the establishment of 27 school health clubs, training of 54 teachers as school health coordinators, and construction of handwashing tanks in each of the 27 schools. These interventions, together with the project’s theory of change and results framework (Annexes 6 and 7) formed the basis of the evaluation.

Boys latrine block at Muuyu Primary School in Mwenezi

School health club members at

Madanwa Primary school in Mwenezi

5

2.3 Purpose and objectives of the evaluation This was a summative evaluation based on school level experience of the WASH in Satellite Schools project. The main purpose of the evaluation as per the Terms of Reference was stated as follows:

1. Generating lessons and providing evidence to guide decision making on WASH in schools

2. Improving project design for future WASH in schools’ programs. This evaluation should provide evidence of what works, under which circumstances thereby support in the design of quality programs and innovations.

3. Promoting accountability through communicating to internal and external stakeholders the value of the WASH in schools’ interventions.

Evidence provided through this evaluation were to provide assurance to all stakeholders of how investments made in WASH in Schools transform into tangible benefits for the children. The specific objectives were stated as follows:

1. To assess the performance and achievements of the project against the planned project objectives, expected results, targets and key indicators as per the logical framework.

2. To assess impact, relevance, effectiveness, efficiency, sustainability, partnership and coordination, and design of the overall intervention. This should give due emphasis on gender, disability, community participation (including children), value for money and environment.

3. To assess the major strengths and limitations of the project and draw lessons for future WASH in Schools Projects improvement and sharing with stakeholders.

4. To assess the validity of the risks and assumptions indicated in the project document in relation of the emerging issues and how the project responded with necessary mitigation measures.

5. To assess the quality and rigour of the monitoring, learning and knowledge managements systems.

The TOR, in Annex 1, provides specific questions for the evaluation.

2.4 Evaluation scope The evaluation covered the period of project implementation (June 2014 to June 2018). All three districts of the project were covered. It was structured according to the OECD DAC evaluation criteria of relevance, efficiency, impact and sustainability. Effectiveness a standard evaluation criterion of the OECD DAC criteria was not considered for this evaluation. However, the measurement of progress on project outputs is detailed under the impact section as part of the analysis of the project’s performance on envisaged outcomes and impacts. Additional criteria of validity of design and partnerships and coordination provided for in the TOR and formed further analysis of the project. An evaluation framework was developed at inception to frame the thematic and issues scope of the evaluation. This is presented in Annex 5. Definitions of the evaluation criteria used for this evaluation are presented below:

Criterion Definition

Relevance This evaluation criterion determined the extent to which the project design, activities are fit for purpose and context. It sought to understand how the project was aligned to policies and standards for WASH in schools. This includes adherence to minimum standards required by policy (MoPSE standards).

6

Criterion Definition

Validity of design Evaluation of design will determine the extent to which the project addressed the underlying challenges of WASH in targeted schools, fidelity to design during implementation and if modifications to design were responsive to change in context or situation. Validity of the TOC and the underlying assumptions were also assessed using retrospective reconstruction of the TOC based on findings of the evaluation.

Efficiency This evaluation criterion focused on how resources were translated into activities and outputs. Quality of infrastructure, timeliness in implementation, and use of value for money indices in planning were also assessed. The evaluation determined monitoring systems put in place for the project with a view to determining: 1) adequacy of the monitoring systems to provide relevant, and timely information for decision making; 2) the type of indicators and their adequacy to meet the project information demands for accountability to the donor (results framework) and project implementation; and 3) extent to which the monitoring systems were by the project partners, UNICEF and the DWSSC.

Impact This criterion assessed: 1) achievement of outputs; 2) planned outcomes; and 3) changes in the lives of the learners and teachers as a result of the intervention.

Partnerships and coordination

Partnership and coordination measured the extent the project was coordinated internally at all levels: national to district level. External coordination with existing WASH projects was also assessed including the partnerships forged to enhance project efficiency and effectiveness. An assessment of the extent of child participation in the project cycle was also undertaken.

Sustainability In evaluating sustainability of infrastructure developed under the project, the evaluation team assessed the Technical, Economic, and Environmental factors that include: the presence of skills to undertake repairs and maintenance, financial resources for repairs and maintenance, extent of mainstreaming effect of natural disasters etc.

2.5 Audience of the evaluation and uses The main audience of the evaluation and their roles during implementation are provided below:

Audience Interest in the evaluation Role in project implementation

UNICEF Australia National Committee (NatCom)

Performance of the project Lessons learned for future similar projects

Funder

UNICEF Global Lessons learned for design and implementation of similar projects

No specific role –guidance framework for WASH in Schools

UNICEF Eastern and Southern Africa Regional Office

Lessons learned for design and implementation of similar projects

No specific role

UNICEF Zimbabwe Performance of the project Recommendations for improving implementation in the Zimbabwe context

Technical support to NGO partners and local implementers

The NAC Project Coordination and oversight

7

Ministry of Primary and Secondary Education

Lessons learned for design and implementation of future similar projects

Project coordination Access to schools during implementation Project monitoring and oversight

Members of DWSSC Project implementation Project coordination Project monitoring

Beneficiary schools (school authorities and children) and communities

Performance of the project

Project implementation

It was envisaged that the evaluation results will be used to:

a) To demonstrate effectiveness of WASH in Schools in improving education outcomes b) Improve institutional framework to support WASH in Schools; c) To guide school budgetary allocations for improved education outcomes d) Generate knowledge among practitioners

8

3 Methodology

3.1 Evaluation design Because no baseline was undertaken by the project, the evaluation adopted a One-Shot Retrospective Design premised on recall during structured and quantitative surveys to reconstruct the baseline. The evaluation used qualitative and quantitative approaches. Multiple data sources were utilised to inform the evaluation that were inclusive of the beneficiaries and other stakeholders of the project. Key principles that guided the evaluation included: Independence, impartiality and credibility: External third party evaluations are premised on impartiality, independence and credibility of findings. As external evaluators, we had no interest in the outcome of the evaluation but more critically, ensure that our whole approach is underpinned by independence from the programme under assessment, its funders and its beneficiaries, which is a defining condition for a quality evaluation output. Therefore the findings represent an independent opinion. The evaluation team had no prior interaction with the project or interests during its implementation. To enhance credibility of findings, data collection and reporting was based on evidence obtained from the field. This evidence was triangulated between sources including a validation process with stakeholders of the project. Any divergence from the evaluation’s view was required to be supported by evidence. Where there remained points of disagreements, project staff were requested to respond through a management response. Honesty and Integrity: All findings and conclusions are evidence based. Were evidence is inconclusive this was highlighted in the relevant sections. The next sections detail the methodological details, key data sources and associated instruments used to inform the evaluation.

3.1.1 Evaluation framework As mentioned under section 2.4, an evaluation framework was developed to guide the evaluation (see Annex 5). This section provides a summary of this evaluation framework particularly the key questions investigated by the evaluation. Relevance: focused on appropriateness of activities to context and meeting of policy standards. While the project was implemented before the Zimbabwe National School Health Policy was approved, it was agreed the evaluation should note how the design responded to current policy provisions for WASH in Schools. The aim was not to criticise the project’s design or implementation, as the policy was approved at the tail end of the project, but to identify design deficiencies to align to the policy as lessons for future similar projects. The evaluation of relevance sought to determine the extent to which the project responded to needs of the beneficiaries (students and teachers) with regards to WASH. Ability of the project to address the specific needs of different target groups that include girls, boys, children with disabilities (CWDs) and different age groups (ECD through to grade seven) was explored. Detailed questions reviewed for this criterion were:

1. What problems did the project help schools and learners overcome in relation to WASH?

2. What challenges did girls and CWDs face with regards WASH and how did the project help overcome them?

9

3. Did the infrastructure follow policy guidelines on WASH infrastructure in school (number of squat holes, menstrual hygiene management (MHM), distance to water source etc)

Design: assessed extent to which the project addressed underlying problems for lack of WASH in schools and validity of the assumptions. The underlying problems were validated through retrospective development of a problem tree for the project. The evaluation used this problem tree to validate the Theory of Change (TOC) of the current project noting its strengths, weaknesses and lessons for future similar project. Through discussions with stakeholders of the project and literature review the evaluation team validated and critiqued the project’s assumptions. The evaluation of design answered the following key questions: 1. Did the project address the underlying challenges for lack of WASH in the schools? 2. How do the underlying problems result in negative effects of lack of WASH? How does the

causality chain compare with the project’s TOC? 3. Were the assumptions in the TOC valid given the context of the project? 4. Were the activities based on good practice/prior experience? Did the partners have

sufficient capacity to implement them? 5. Were the activities including approaches sufficient address the underlying challenges? 6. Was the theory of change followed through during implementation? 7. Was the design based on available literature? Efficiency: determined timeliness, quality and quantity of outputs delivered. Adequacy and effectiveness of monitoring systems for the project were also assessed under this criterion. The specific questions included:

1. Are infrastructure of good quality (no cracks on walls, floors, demonstration of good building practice)?

2. What was the investment by the local community and UNICEF? 3. Were activities implemented according to work plans? 4. Were mechanisms for supervision of works adequate? 5. How was VFM used in project planning, implementation and reporting? Was it well

understood by partners and shared with communities involved? 6. What systems were put in place for monitoring the project by UNICEF, partners and

DWSSCs? 7. Were there specific tools/data collected by the system? 8. Was monitoring data used in planning? 9. What coordination mechanisms were in place (UNICEF, Partner and district) and were

they multi-sectoral? Impact: This criterion assessed changes in the lives of the learners and teachers as a result of the intervention. Key questions were:

1. In line with the project’s theory of change, to what extent has it contributed to the following: Improved hygiene knowledge among students; Improved hygiene practices; Improved menstrual hygiene management in schools; reduced school absence due to diarrhoea or poor MHM at school; Improved MHM in schools

2. What are the specific effects of the WASH project on boys, girls, and CWD? 3. What changes can be observed at the teacher, learning environment and learners as

a result of the training on child friendly pedagogy? 4. What are the specific effects of the project at the school level e.g. enrolment (girls and

boys), teacher retention, improved learning experience etc? 5. What individual, community, organisational and institutional, factors supported

achievement of these results? 6. Is the project package and implementation approaches similar across the project

sites? What have been the influence of any differences? 7. What it the influence of DWSSCs on performance of the project?

10

8. What is the project’s contribution the MDGs? Partnership and Coordination: Partnership and coordination a will measure the extent the project was coordinated internally at all levels: national to district level. External coordination with existing WASH projects will also be assessed. In particular child participation will be a focus of the evaluation. The key questions for this evaluation criterion are:

1. What is the contribution of the DWSSC to project coordination? 2. What platforms exists for lessons sharing between partners and what are the benefits? 3. What is the performance of the different partners? Are there specific differences? 4. At what stage were children consulted? Are the children’s perspectives on being

involved in the project 5. Key drivers and constraints to coherence and coordination?

Sustainability: In evaluating sustainability of infrastructure developed under the project, the evaluation team will assess the Technical, Economic, and Environmental factors that include: the presence of skills to undertake repairs and maintenance, financial resources for repairs and maintenance, extent of mainstreaming effect of natural disasters etc. Key questions for the evaluation will include:

1. What exit plan is in place and the challenges and benefits of it? 2. Has this plan been consistently implemented at all project sites? 3. Will capacity of health clubs, water point committees, village pump mechanics and

latrine builders remain after the project? 4. Have teachers trained in child friendly pedagogy remained in post?

How has the long term context (economic, political and social) of the project been considered in the planning and implementation of the project?

3.2 Tools and techniques

3.2.1 Documentary review An extensive document review was undertaken for the evaluation. Documentary review helped the team understand the project design, its implementation including challenges and recorded successes and failures and project alignment to national and global standards for WASH in Schools. Reviewed documents comprised: 1) project design document; 2) quarterly and annual reports; 3) UNICEF WASH guidance documents and strategies; and 4) national policies. A list of documents reviewed is presented in Annex 2.

3.2.2 Quantitative survey The quantitative survey comprised the use of five tools premised on collecting primary and secondary data. A summary of these tools and target groups is presented in Table 1. The tools are presented in Volume II of the Evaluation Report. The five tools measured outputs, outcomes and impacts of the WASH in Satellite Schools project as depicted in Figure 2. Outputs related to service provision and healthier school environment. This included the functionality of infrastructure provided, their quality, and adequacy. It also measured hygiene promotion and learning in the school environment, Waste management and menstrual hygiene management. (MHM). Equity of service provision was a specific focus, determining whether services are accessible to all genders and age groups. At the outcome level the Handwashing Observation Checklist and the Student Questionnaire were the primary quantitative tools collecting information on knowledge and its translation to hygiene behaviour change and reduction in health risks. Impact level results related to education outcomes and

11

health status of beneficiaries. This included reduced absenteeism, less frequent occurrence of WASH related illnesses for target age groups etc.

Figure 2: Link between WASH in Satellite Schools Project Results Chain and Quantitative Tools

Table 1: Tools for the quantitative survey

Tool Target group Males Females Total

Student questionnaire Female and male students in grade 4 to grade 7

95 90 185

Head Teacher questionnaire School Health Coordinator and Headmaster

14

Infrastructure observation checklist

All WASH infrastructure and School Health Coordinator

14

Sanitation and hygiene practice observation checklist (critical times)

Female and Male learners leaving sanitation facilities

185

Health Facility records checklist

One clinic within the catchment area of the selected school.

14

The infrastructure observation checklist was mainly guided by the UNICEF WASH in Schools Monitoring Package12 and the UNICEF/WHO Core questions and indicators for monitoring WASH in Schools in the Sustainable Development Goals13. These documents provided a global standard for measuring the status of Drinking Water, Hygiene and Sanitation in schools using a four level ladder grading system of “Advanced Service”, “Basic service”, “Limited Service” and “No Service” as part of the WHO/UNICEF Joint Monitoring Programme (JMP) for WASH in Schools in the SDGs. This four level ladder monitoring system measures only the output level results of WASH in schools interventions. Figure 3 provides descriptive details of these four levels. While the global standard descriptions are used to guide categorisation of schools, this evaluation adopted this description but within the context of minimum standards

12 https://www.unicef.org/wash/schools/files/wash_in_schools_monitoringpackage_.pdf 13 https://washdata.org/monitoring/schools


https://washdata.org/monitoring/schools

12

of the Ministry of Primary and Secondary Education (MoPSE) e.g. of number of pupils per squat hole, accessibility for children with disability etc. For areas where there were no specific country level standards e.g. menstrual hygiene management, the evaluation adopted global standards as depicted in the WHO/UNICEF guidance. Figure 3: JMP service ladders for monitoring WASH in schools in the SDGs

DRINKING WATER SANITATION HYGIENE Advanced service: Additional criteria may include quality, quantity, continuity, and accessibility to all users

Advanced service: Additional criteria may include student per toilet ratios, menstrual hygiene facilities, cleanliness, accessibility to all users, and excreta management systems

Advanced service: Additional criteria may include hygiene education, group handwashing, menstrual hygiene materials, and accessibility to all users

Basic service: Drinking water from an improved source and water is available at the school at the time of the survey

Basic service: Improved sanitation facilities at the school that are single-sex and usable (available, functional and private) at the time of the survey

Basic service: Handwashing facilities with water and soap available at the school at the time of the survey

Limited service: Drinking water from an improved source but water is unavailable at the school at the time of the survey

Limited service: Improved sanitation facilities at the school that are either not single-sex or not usable at the time of the survey

Limited service: Handwashing facilities with water but no soap available at the school at the time of the survey

No service: Drinking water from an unimproved source or no water source at the school

No service: Unimproved sanitation facilities or no sanitation facilities at the school

No service: No handwashing facilities available or no water available at the school

Note: Improved sources include piped water, boreholes or tubewells, protected dug wells, protected springs and packaged or delivered water. Unimproved sources include unprotected wells, unprotected springs and surface water

Note: Improved facilities include flush/pour flush toilets, ventilated improved pit latrines, composting toilets and pit latrines with a slab or platform. Unimproved facilities include pit latrines without a slab or platform, hanging latrines and bucket latrines.

Note: Handwashing facilities may be fixed or mobile, and include a sink with tap water, buckets with taps, tippy-taps and jugs or basins designated for handwashing. Soap includes bar soap, liquid soap, powder detergent and soapy water but does not include ash, soil, sand or other handwashing agents.

Source: WHO/UNICEF (2018) Core questions and indicators for monitoring WASH in Schools in the Sustainable Development Goals,

Furthermore the checklist was used to ascertain functionality of water point committees, presence of village pump mechanics and plans for infrastructure maintenance, etc. The sanitation and hygiene practice observation checklist was a special addition to the tools mix of the study specifically to triangulate hygiene practice from the student questionnaire. It mainly focused on measuring the extent of hand washing with soap after latrine use for both girls and boys after toilet use, during break time (before eating). Data collection involved the enumerator observing hygiene practice (or lack thereof) among girls and boys leaving specified latrines and tallying their behaviour (washing with soap, washing with water only and no washing).

Enumerator conduction the sanitation and hygiene observation

13

Sample size UNICEF engaged ZIMSTAT to undertake sampling for this evaluation. Table 2 provides minimum sample sizes identified. Table 2: Minimal sample size

Boys Girls Planned Total Achieved total

Number of schools 14 14 Learners 95 90 185 196 Observation checklist 14 14 Hygiene Observation 95 90 185 980 Head teachers 14 14

Source: Zimstat sampling data sheet

Probability based random selection was used to identify the specific schools to be visited for the evaluation. Table 3 provides the list of schools visited. Table 3: Sampled Schools

Obs District School Name

MOS Number of Hits

Sampling weight

SRN

1 Mwenezi Madamwa 362 0.652840397 1.531767956 1

2 Mwenezi Chemhara 167 0.301172227 1.549143372 2

7 Mwenezi Muuyu 210 0.378719567 2.62037872 3

9 Mwenezi Takunda 298 0.5374211 3.556357078 4

11 Binga Champongo 204 0.367899008 4.57348963 5

14 Binga Mulambe 455 0.820559062 6.07213706 6

16 Binga Mabula 146 0.263300271 6.660054103 7

19 Binga Muunde 267 0.481514878 7.568981064 8

20 Hurungwe Chipapa 526 0.948602344 8.517583408 9

22 Hurungwe Chehamba 684 1.233543733 10.03606853 10

23 Hurungwe Madzimoyo 305 0.550045086 10.58611362 11

25 Hurungwe Dambanzara

454 0.818755636 11.6735798 12

27 Hurungwe Momba 345 0.622182146 12.62759243 13

28 Hurungwe Songalalo 529 0.954012624 13.58160505 14

Source: Zimstat sampling data sheet

3.2.3 Qualitative survey Multiple sources at national, district, community and school level provided qualitative data. Table 4 presents a summary of the data collection tools, target groups and sample sizes. Two schools per district were visited for qualitative data collection. These schools, were drawn from the sample of schools targeted for the quantitative survey (See Table 4). Simple random sampling was used in selecting the schools. Table 4: Schools selected for the qualitative survey

District School

Mwenezi Madamwa

Mwenezi Takunda

Binga Mulambe

Binga Mabula

14

Hurungwe Chehanda

Hurungwe Dambanzara

A summary of respondents for the qualitative survey are presented below. Table 5: Qualitative survey tools

Level Tool Target group Sample size (per school/ districts)

National KII with implementers UNICEF, MoPSE, NGOs/INGOs, MOHCC

5

District KII with implementers stakeholders

Rural District Council (RDC), District Administrator, MoPSE, MoHCC, DDF, District Water Technician

24

District FGD with DWSSC DWSSC Chair 1

School FGD with teachers Teachers trained in child friendly teaching methodologies

2

School KII with School Health Coordinator

School Health Coordinator 1

School Case study with teacher Teacher trained in child friendly teaching methodologies

1

School FGD with SDC Members of SDC 1

School FGD with girls Student girls grades 4-7 1

School FGD with boys Student boys grades 4-7 1

School Case study Students 2

Community FGD with water point committee

Water point management committee at the new water supply infrastructure

1

Community KII with village pump mechanic

Local artisans trained by the project 1

Community KII with environmental health technician

Environmental health technician 1

Community KII with community leader Traditional or elected leaders (councillors)

1

FGDs with learners used three participatory methods: photo elicitation; project posters and pathway through life. Photo elicitation is an ethnographic technique where children took photos of their school and life at school and discuss issues they like or dislike about the photos they have taken. Project posters allow children to narrate changes they observe in their school through drawing posters about the good and the bad aspects brought by the project. Pathway through life technique was used to understand the impact of the project on lives of children.

Box 1: Qualitative data collection methods

15

Children drawing project posters Children during photo elicitation

3.3 Data collection Data collection was undertaken over a two week period in October 2018. Six enumerators were contracted for the quantitative survey with four recruited for the qualitative survey with equal numbers of males and females. They underwent a three day training on the tools. The team was supervised by the main consultants in the field. Quantitative data collection used tablets running on ODK which ensured real-time availability of data. Mobilisation of respondents for the qualitative survey was done by the DWSSCs in all districts. Random selection was used to identify respondents for the student questionnaire.

3.4 Data quality control Data quality measures were put in place and included:

1. thorough training of enumerators and testing of their knowledge; 2. using of mobile data collection the evaluation team was able to access data being

collected by enumerators real-time allowing for daily review of data quality; 3. core team members providing support and supervision to data collection teams; and 4. employing data storage and security measures to limit access to authorised personnel.

3.5 Data analysis Quantitative data: Data collected through questionnaire-based interviews will be captured and analysed using the Statistical Package for Social Scientists (SPSS). Data analysis will seek to (1) describe the data; (2) show the associations and correlations between variables; (3) the effect of variable on a particular outcome; (4) the combination of variables that produces particular outcomes, among others. The programme’s indicators in the log frame and evaluation questions will guide the analysis. Qualitative data: All qualitative interviews will be recorded and transcribed in an excel data entry template organised by theme of the evaluation: validity of design, relevance, effectiveness, impact, sustainability, partnerships and coordination, etc. Sub-themes will also be captured based on the evaluation framework main questions under each theme. Discourse analysis will be used to identify emerging commonalities in the data on sub-themes between data sources. Such analysis will not only conclude on common opinions between data sources but will also identify outlier opinions that may need further clarifications with stakeholders. Triangulation of data: multiple data sources were used for data. This approach allowed the evaluation to triangulate and validate information from these sources before conclusions were reached. Two validation meetings were held – one at national level with national level stakeholders and the other with district level stakeholders involved in the project implementation. These processes and methods ensured the data was properly triangulated and validated.

3.6 Stakeholder validation Findings of the evaluation underwent validation by stakeholders. A first validation meeting was held with UNICEF Zimbabwe’s Technical Research and Evaluation Group (TREG). This was followed by validation with national stakeholders and lastly district level stakeholders.

16

3.7 Ethical considerations Our ethical standards and guidelines were guided by the UNICEF (Innocenti) Guidelines on Ethical Research Involving Children (ERIC14). All members of the research team were trained to meet the highest ethical standards of data collection and analysis throughout the duration of the evaluation. An informed consent process was undertaken. A safe space and an appropriate time was identified for the interview to ensure confidentiality and minimal disruption to the lives of respondents. Referral processes to authorities were also discussed with the fieldwork team in cases of due responsibility (e.g. when research team comes across cases of child abuse). Adherence to ethical standards was particularly important in this evaluation as it involved children as sources of data. The following were issues considered in developing the ethics protocol:

i. Children’s competencies, perceptions and frameworks of reference, which may differ according to factors including – but not only – their age, may differ from those of adults;

ii. Children’s potential vulnerability to exploitation in interaction with adults, and adults’ specific responsibilities towards children;

iii. The differential power relationships between adult researcher and child participant; and iv. The role of adult gatekeepers in mediating access to children, with concomitant ethical

implications in relation to informed consent. Therefore, the following were adhered to:

• Initial consent from gatekeepers – parents and/or teachers or others with a duty of care for the child.

• receiving consent from each individual child – not from children as a group (because individuals in the group could feel pressured to take part against their wishes).

No formal ethical approval was sought for the evaluation as this was deemed not required for the type of research undertaken.

3.8 Limitations of the evaluation The following are limitations of the evaluation:

1. the evaluation survey was undertaken during grade seven examinations. Therefore study participants excluded this group of students. This however did not affect the evaluation as the grade sevens (excluded by the evaluation) and those in grade six (included in the evaluation) had similar exposure to project activities;

2. majority of girls at menarche were not captured by the evaluation because of the age group, therefore results on MHM outcomes are not conclusive;

3. Besides the unavailability of grade seven pupils to respond to our questions, at two schools the evaluation team could not meet EHT as they were occupied in other activities at the time of the evaluation. Village Pump Mechanics (VPMs) at two schools could also not be interviewed as they were unavailable on the day of the visit;

4. The project had no baseline. The evaluation relied on recall and existing data sets such as the Education Management Information System reconstruct a baseline. Nonetheless for some indicators there were baseline values could not be obtained;

5. Bias in the responses was addressed in the sample size used for the evaluation but also employing multiple methods of collecting the same data e.g. handwashing practice was collected in two ways – interviews with students and primary observation of handwashing behaviour. Using multiple methods of data collection also unearthed

14 https://childethics.com/

17

biases such as schools repairing broken down water taps a day or week before the evaluation or embarking on renovations to latrines etc. Despite these methods undertaken to offset bias in data collection, the evaluation does acknowledge some biases in the data but they were not significant enough to influence the findings of the evaluation.

18

4 Findings

4.1 Relevance

4.1.1 How relevant is the project in the addressing the needs of beneficiary communities

with emphasis on the needs of girls and people with disability? Finding: The project was relevant in addressing the evident needs of access to clean and safe water, poor sanitation and hygiene. Children in ECD, girls and those with disabilities were particularly affected. It addressed lower access to water and sanitation for ECD children, the burden of fetching water for girls which resulted in improving positive school experience. Before the project’s intervention, majority of the schools visited for the evaluation had limited service level for water sanitation and hygiene. Available toilets were inadequate for the enrolled children, with the infrastructure dilapidated and in some cases a risk to children. Children had misgivings in using them for fear of falling in the pit, or that they had no privacy as one could be seen by others outside as they were made from sticks and grass. This was particularly worrying for girls. Open defecation was common place across all schools. Even with open defecation, girls complained that during the dry season when there is little vegetation it was difficult to get privacy in the bushes. Access to clean and safe water was a challenge with the main source across all schools being close by rivers.

Latrine and Water Source at Kampandu Primary School Before the Project

Source: Mvuramanzi Trust human interest story, 2016

Children in infant grades, those with disabilities and girls were particularly affected. First because latrines were few and overcrowded it crowded out children in ECD as mentioned by one Teacher in Hurungwe, “The toilets used to be over packed with people such that ECDs would urinate on themselves and others would go in the bush.” ECD children would also avoid use of the existing toilets for fear of falling in the pit as the squathole sizes were too big for ECD use. Because of the challenges to water access, majority of children would bring water from home or fetch at the main source. This posed a particular challenge for children in ECD, as they could not carry water from home as said by an SDC in Mwenezi that, “The water challenges at the school affected children who were disabled and the ECD/infants mostly with the burden of carrying water from home and sometimes from the river”. This challenge was addressed in cases where other children would fetch water for teachers and ECD use in the

PIT ATRINES FOR STUDENTS AND

TEACHERS BEFORE THE PROJECT

THE CURRENT WATER SOURCE

FOR HUMAN CONSUMPTION

19

schools. However, this introduced a burden on girls as they were tasked with fetching water for teachers, ECD and use by boys. Girls at a school in Hurungwe said,

“We would fetch water for our teachers using 20 litre buckets to use for cooking, washing and drinking. Only girls would go fetch water and boys would look for firewood. We still fetch water for our teachers from the borehole but not often as it a bit far.”

One of the reason for this was that fetching water is recognised as a domestic chore and therefore girls are tasked with the responsibility. While these gendered roles are common knowledge an interesting finding highlighted by one teacher in Hurungwe was that because girls have more need for water than boys it should be their responsibility to fetch it.

“Learner girls are the ones who used to fetch water as they were [require] more hygiene than boys. Girls form grade 5-7 would fill up buckets of water for use at the school especially for the ECD and the boys”. Interview with a teacher in Hurungwe. “Boys do not go and fetch water, it is considered a girls chore. boys would only fill the containers and the girls carry them. Students are still asked to go and fetch water for the teachers at the borehole.” FGD with Boys in Hurungwe.

This need for better hygiene is a result of water requirements for menstrual and other personal hygiene. Schools allowed girls to collect water at different times with various effects on the experience of school. In one group of schools, girls would fetch water during class time. By the time they finished fetching water they will be too tired to learn or have very little time to learn, negating their experience of schooling which can contribute to lack of motivation and eventual dropping out of school. In another group, girls would fetch water after school or during lunch. This would delay their time for going home exposing them to risks of violence as one teacher put it, “Usually children would be tasked to fetch water during lunch and would spend hours at the river and would end up coming back home very late”. Girls at menarche faced challenges of missing school as the absence of water and appropriate facilities made menstruation unmanageable while at school as one head teacher from Mwenezi said: “For those students who were on their menstrual period they would request their teachers to go home because there was no water to clean themselves and made life difficult for them”. A group of girls in Binga highlighted how they would use their teachers’ cottages as changing rooms while some teachers would provide relief by providing sanitary pads. This group of girls had this to say:

“It was a difficult time. a) used teachers' cottage as changing rooms. b) Water to wash up was rarely provided. c) used papers which were at their immediate disposal and in some cases teachers provided pads for sanitary purposes. These were disposed by digging holes and covering the dirt up”.

Challenges for those living with disability were not evident as the study could not find children with disabilities or disability teachers within the schools. However, in one group of SDC members in Hurungwe, it was highlighted that ECD together with those children living with disabilities were the most affected by the lack of toilets and water. As with children from infant grades, children living with disabilities (CwD) were facing specific challenges with access to

Girls only carrying water: Dambanzara, Hurungwe

20

appropriate sanitation facilities and water. However, the depth of these challenges could not be ascertained. Finding: The project was relevant in changing community knowledge and mind-set on the importance of latrines and good hygiene. Perhaps one of the biggest benefit of the project was to enlighten communities on the importance of toilets and the need for better hygiene. At the time of the evaluation, majority of households did not have latrines (about 1 out of 10 in Binga and 4 out of ten in Hurungwe households had functional latrines) and open defecation was common place. As with the schools, the main source of water were nearby rivers, and hygiene was limited. Because children grow up under these circumstances, poor sanitation and hygiene become normalised experiences. The project provided an alternative normal helping to increase positive perceptions and needs for better sanitation and hygiene. For adults, while they still faced financial constraints in investing in sanitation facilities, there was recognition of its importance and a desire to change the status quo.

4.1.2 To what extent is the project aligned with the policies and strategies of the GoZ at

district level? Finding: At the school level the project met policy standards for adequate toilets for boys (53.8%) but did not achieve the same for girls (30.8%). The policy standard for the distance of the water point from the school was met in a majority of schools (71.4%). Ten out of 14 schools had the water access point in the school yard or within 500m from the school yard. As shall be discussed in section 4.4, 53.8% of schools met the minimum squat hole to children ratio of 1:20 for boys and only 30.8% did the same for girls at the time of constructing latrines in 2016. Children in infant grades faced a worse shortage of toilets of three to eight times than that for girls and boys in higher grades. Finding: There were policy gaps that need to be addressed to enhance WASH in schools including: clarity of support for MHM in schools, hygiene education and related child friendly materials and ensuring schools are established in places with easy access to water. While the Education Act provides for standards for WASH in schools in particular squat hole to student ratios (for both girls and boys), and the distance to a water point, there are several gaps that undermine provision of WASH in schools. The evaluation noted that because handwashing tanks are manually filled, they remain without water for long periods. There are no policy or legislative provisions for handwashing facilities in schools. Each school implemented hygiene education differently according to training and in cases motivation of the teacher. There was no standard for this aspect of WASH in schools. The recently launched school health policy is limited on this aspect as well. Challenges with regards borehole siting experienced by the project were because the schools were located in places where the water table was too low to provide adequate water throughout the year. Siting of satellite schools does not consider water access. The Education Act does not provide adequate guidance or provisions on this issue. Planning WASH in schools requires data to inform extent of the challenge. The Education Management Information System (EMIS) does collect information on availability of toilets and access to water and provides evidence for planning WASH in schools. However, it does not collect information on appropriateness of sanitation facilities for menstrual hygiene

21

management for girls and availability of support for MHM in the schools. Inclusion of these elements will enhance the evidence base for WASH in schools.

4.2 Validity of Design

4.2.1 Was the design based on a needs assessment and a context analysis? Finding: No specific needs assessment were conducted at selected schools to inform the programme. However, partners undertook pre-implementation adhoc assessments but these were not comprehensive. No specific needs assessment was undertaken. Some partners e.g. Plan and FCTZ did undertake needs assessments or baseline studies at each school before the project but these were not systematic. While the identification process of schools could make for needs assessment, the identification criteria was limited to water access, and sanitation facilities not the broader information needs to facilitate programming: issues of hygiene education, child friendly pedagogy, and status of menstrual hygiene management were missing. The project could also have benefited from prior feasibility assessments especially with regards accessibility of water, as shall be discussed under section 4.4, the siting of boreholes meant the challenge of distances to water source and the burden on girls were not eliminated.

4.2.2 Was the design the most appropriate to meet the needs identified? Finding: The project design was largely appropriate to address the needs of targeted schools but improvements could have been made to address the specific needs of: 1) girls; 2) children in infant grades (ECD); and 3) CwD. The design of the programme as elaborated in its Theory of Change (Annex 7), provided WASH facilities, and behaviour change communication (BCC) to ensure all categories of children have access to appropriate WASH that meets their specific needs. There was a consciousness in the project document of the need to address equity in access and ensure a human rights approach to the delivery of WASH in satellite schools. All these provisions were necessary ingredients for a highly relevant programme to the varying needs of children in school. This said, there were challenges in implementation. The project missed an opportunity to test out models for girls’ friendly latrines and influence policy guidelines using UNICEF global experience and leadership in the WASH in Schools agenda. Partners were left to use their own experience. The differences on the ground was stark. Mvuramanzi Trust, using their experience built girl friendly facilities that met all the provisions of a girl friendly toilet. For Plan International and FCTZ there were no differences between the girls and boys toilets (see picture collage in Box 2 for differences). Box 2: Girls and Boys Toilets

Girls toilets in Mwenezi Boys toilets at same school in Mwenezi

22

According to figures provided by Mvuramanzi Trust (see Table 6), the cost of building a responsive girl friendly latrine with adequate private and facilities for MHM costs 22% more than the boys or conventional latrine. The costing of the project design did not take these differentials into account, thus partners, to meet their grant targets, focused on building the conventional latrines negating the desired impacts on MHM. Therefore while the design of the document was adept to the needs for improving MHM this was not translated to costing and implementation. Table 6: Cost of building a school latrine in Binga

Type of latrine Cost, US$ Girl Friendly latrine 517.26 Disability Friendly latrine 534.26 Conventional latrine 424.26

Source: Mvuramanzi Trust

The trend was the same for disability friendly toilets (see picture collage in Box 3). While all schools had ramps and assistive holding bars inside the latrine, the pits were designed differently by each partner. In Mwenezi, the pits provided a raised seating platform while in Binga and Hurungwe the normal squat hole was provided. It was noted by teachers that the squat hole was not ideal for children with physical disabilities as they found it difficult and uncomfortable to use. Box 3:Differences in disability friendly squat holes

Squat hole in toilet for CwD, Hurungwe Squat hole in toilet for CwD, Mwenezi

Equity and human rights focus of the project was limited to identifying the most vulnerable schools but did not consider: 1) extent of need at each identified school; and 2) addressing fully, accessibility to WASH for all groups of children. The identification process for schools was led by the DWSSC using MoPSE knowledge. Using this local knowledge, the project was able to identify the most WASH deprived schools in the targeted districts enhancing equity in WASH access. Second level concerned ensuring equitable access of WASH facilities across the targeted schools. As shown in Sections 4.2.1 and 4.4.1, providing a standard number of toilets across all schools negated equity objectives as some school had under and over supply of latrines.

“The toilets are few considering the number of students that amount to 544 as they queue a lot on toilets. There are times that teacher would control traffic to the toilets so as to manage queues at the toilets. This applies also to the ECD, 2 more toilets for them would help ease the challenge”. Focus Group Discussion with SDC in Hurungwe.

23

“The toilets are more than enough hence we rate them 10, because we have a surplus”. Teacher in Mwenenzi.

The third level was access by different categories of children. As shall be demonstrated in Section 4.4.1, there were differences in access by different categories of children. The absence of latrine designs that responded to specific hygiene needs for girls at menarche in Hurungwe and Mwenezi, limited their responsiveness to the needs of this group of children. In Binga, the presence of only on girl friendly latrine did not provide the desired result as it inadvertently increased stigma around menstruation (See Section 4.4.4). Across all schools there was a chronic under supply of toilets for children in infant grades (See 4.1.2). Infant grades were also disenfranchised as their latrines in most cases were built close to their classrooms whereas the handwashing was built close to the main latrines. In one school in Hurungwe, which represented the worst case, infant children had to walk 200m to the handwashing facility. This negated hand washing behaviour among this group as discussed in Section 4.4.1. Besides the deprivation of infant children, one handwashing facility per school was inadequate for the population of students in schools. Hygiene education through health clubs was appropriate but targeted only a small proportion of the school (38.4%). While it was important to strengthen child participation in hygiene promotion, they were generally inactive in this regard. A better strategy would have been to strengthen schools to support hygiene promotion in class through capacity building of teachers and development and distribution of standardised child friendly hygiene promotion materials. This is not to say health clubs should have been abandoned but that a combination of the classroom and club based approaches were necessary. Such an approach would also offset the high staff turnover rates at satellite schools which contributed to underwhelming performance of health clubs.

4.2.3 Does the theory of change (TOC) clearly and accurately depict the results chain? Finding: The theory of change provides a strong link to the results chain of the project but with missed elaboration on MHM, policy influence, support for school based WASH planning and climate risk management all important for the targeted schools. The Theory of Change provides clear linkages between the inputs activities and results (see Annex 7). However, it was not based on a problem causality analysis and therefore missed critical elements that address fully the needs of a WASH in schools project. As part of the evaluation, the evaluation undertook a participatory review of the Theory Change with stakeholders from UNICEF, NCU, MoPSE and Ministry of Local Government Public Works and National Housing. A revised Theory of change was developed that best depicts an appropriate WASH in Schools project. This was preceded by a problem tree illustrated in Figure 4. Low water availability due to climate change induced water stress, poverty among community members, limited investment in school infrastructure by government, limited prioritisation of investment in school WASH by communities/schools, limited information on WASH and MHM are all secondary causes for the problems of limited access to safe drinking water, poor hygiene practices and limited MHM.

24

Figure 4: Problem for the WASH in Schools Project

25

Figure 5: Revised Theory of Change for a WASH in Schools Project

26

When the project theory of change (Annex 7) is juxtaposed to the revised theory of change (Figure 5) the following gaps were noted: 1) no provisions for MHM from inputs to outputs and outcomes; 2) absence of climate risk management; 3) support to schools to enhance school led investment in WASH infrastructure; and 4) interventions in support of appropriate policy provisions. When compared to Global the standard for WASH in schools, the ‘WHO/UNICEF Guideline on Water, Sanitation and Hygiene Standards for Schools in Low-cost Settings’, the original theory of change for the project also falls short with limitations on MHM and hygiene education (which was limited to health clubs). The guidelines describe the standard for WASH in Schools as follows:

"A school with adequate WASH has a functional and reliable water system that provides sufficient water for all the school’s needs, especially for handwashing and drinking. The school must also have a sufficient number of toilet facilities for students and teachers that are private, safe, and clean and gender segregated. The school should have several handwashing facilities, including some that are close to toilets to facilitate handwashing after defecation. Facilities should cater to the needs of the entire student body, including small children, girls of menstruation age and children with disabilities. Hygiene education should be included in the school curriculum to instil good hygiene, sanitation and water-handling practices, and students should be encouraged to transmit hygiene knowledge to their families and communities."15

Although there was consciousness among implementers of the need to build sanitation facilities that have appropriate provisions for girls at menarche, they were hamstrung by the design of the project which did not provide an adequate budget for the slightly more expensive facilities as discussed earlier in this section. While training of teachers in participatory hygiene education was undertaken, as shall be discussed in section 4.3.1, this training was not supported with provision of appropriate and adequate materials. Secondly, according to the head teacher survey an average of two teachers were trained per school16. Given the reported high staff teacher turnover in satellite schools and that teachers seldom shared back their training with peers to the same level as they received it, left a gap in capacity among the teachers.

4.2.4 Was there a logical flow in the project theory of change? Finding: There was a logical flow in the Theory of Change however, some assumptions turned to risk and disrupted the results logic. The logic of the Theory of Change cannot be argued. This logic was premised on several assumptions that included:

• Assumption 1: Water would be available within the school yard and at reasonable distance from the school (500m);

• Assumption 2: Schools would be able to continuously fill the handwashing tanks;

• Assumption 3: Communities and schools will have the capacity to maintain the infrastructure;

• Assumption 4: Teachers will remain and continue with health clubs;

• Assumption 5: Water will be of good quality and sustainable yield; Assumption 1: Water would be available within the school yard and at reasonable distance from the school (500m) As discussed earlier (Section 4.1.2) in a majority of schools visited for the evaluation (71.4%) water was available in the school yard or within 500m from the school. This assumption therefore held for a majority of schools. Of the 71.4%, the siting was mostly not in the school

15 http://www.sanitationmonitoringtoolkit.com/images/SMTdocuments/31_WASH_in_schools.pdf 16 Binga and Mwenezi had an average of two teachers trained and Hurungwe an average of three.

27

yard and sometimes in hilly locations making it difficult for children to fetch water. Wet holes could not be located within the schools. Because of this children complained of the distance to the borehole. Assumption 2: Schools would be able to continuously fill the handwashing tanks Schools had the responsibility for filling water tanks. Unlike in some schools in Binga where piped water was available, schools often tasked children with the responsibility of carrying water. Because of the distance and labour involved, the tanks were not often filled with water as one Teacher in Hurungwe said:

“Filling up the handwashing tank is burdensome to the children because in many cases ECD children misuse the water by leaving it open. It’s a challenge to always have soap available because it is expensive so we opt for ash.”

Another key informant from Binga concurred, “Water situation in some of the schools [in schools without piped water] is not sufficient to address the behaviour change because water is still a burden to them especially in filling up the handwashing tanks to ensure handwashing at critical times.” As demonstrated in some schools in Binga availability of piped water in the school increases access to water for hand washing. One teacher in Mwenezi had this to say:

“[The project should] Construct piped water at the school specially to ease the burden of filling up the handwashing tank”.

Therefore the assumption that the availability of the borehole and handwashing facility will lead to consistent availability and accessibility of water for handwashing did not always hold in all schools and piped water may have been more appropriate. Assumption 3: Communities and schools will have the capacity to maintain the infrastructure The Theory of Change for the Wash in Satellite Schools assumes that project benefits will be sustained when communities and schools continue to provide materials, and financial support for maintenance of project infrastructure. Boreholes in all the schools visited had not had mechanical challenges that required the schools’ intervention. Handwashing tanks had mixed experiences. In a majority of schools (approximately 70%) taps at the handwashing tanks had broken down or were stolen prior to the visit. In these schools, for a majority of the taps (57.4% of the schools that replaced the taps), the replacements were made just before the evaluation visit as one group of girls and boys in Hurungwe and Binga had this to say:

“The handwashing tank only started working this year on Monday, 1st of October 201817. It has never worked last year.” “Also the tap at the hand washing facility was stolen a long time ago and was only installed back yesterday.”

Using the Infrastructure Observation Checklist it was observed that in 64.3% of the schools, anal cleansing material was not available. Where it was available students in the student questionnaire reported they used paper (67.9-89.3%), stone (5.4-37.5%), and wood (1.2-50.0%). This further demonstrates the incapacity of schools to meet the operation and maintenance costs for established WASH facilities. Two reasons exist for this situation. One is that schools and the SDC do not prioritise WASH investment even though they do include them in school development plans. For example, 11

17 This was two days before the arrival of the evaluation team.

28

of the 13 schools that had a school development plan at the time of the visit did have some plans for maintaining WASH infrastructure. However, when it comes to competing needs it seems WASH is at the bottom. This is expected given WASH itself is not a priority for the communities in which these schools are found with a majority in FGDs saying they do not wash hands after use of the toilet and that communities estimate over 70% of households use open defecation: “Washing of hands is depended on households’ level of hygiene. But most people do not wash hands”18. “Our toilets are clean especially early morning but get messy later in the day…Handwashing is done mostly without soap [use water only]”. FGD with girls in Binga The second reason is that satellite schools are generally located in the most deprived communities whose financial capacity to pay levies to support the school to maintain the infrastructure is constrained. As a result the school in unable to meet the maintenance requirements:

“It’s a challenge to always have soap available because it is expensive so we opt for ash. Teacher in Hurungwe

Assumption 4: Teachers will remain and continue with health clubs School health clubs depend on the availability of a school health coordinator. About 21.4% (three out of 14) of schools visited for the evaluation still had the trained school health coordinator who was aware of the club and activities. In the rest, high teacher turnover meant continuity of health clubs was a challenge.

“Biggest challenge on the software component: Binga until 2017, 90-95% of teachers were temporary teachers. Even during the time were doing the Natcom project, most teachers were temporary teachers. Big blow to the health clubs as teachers were changed every term.” NGO Partner.

“Retention is better amongst teachers but people are still trying to leave but it is way better. Those teachers trained on school health clubs all left. 4 Teachers were trained and all left.” Interview with a teacher in Hurungwe

Therefore this assumption did not hold. According to the District Supervision Inspectors (DSIs) of Binga, Hurungwe, and Mwenezi satellite schools experience higher staff turnover than other schools because they often lack amenities and infrastructure for teachers’ stay. As a result, majority of teachers posted to satellite schools do not last in post. Assumption 5: Water will be of good quality and sustainable yield In 57.1% of schools visited the water was not palatable. Children and teachers alike complained of the water being rusty (Hurungwe), and the water being too salty for consumption (Mwenezi). In some schools, students and teachers were resorting to old sources of water (home and the river). One case in Binga was recorded of a low yielding borehole. This assumption, therefore, did not hold true in all schools and was reversing project gains.

4.2.5 Are the activities of the project sufficient and/or necessary to bring about desired

change? Finding: All activities of the project were necessary but may have been insufficient to bring about the full extent of the desired change for all groups of learners. The activity

18 FGD with community members from Hurungwe

29

on child pedagogy seemed an appendage with no direct link to the desired WASH results. As noted earlier in this report, all activities of the project were necessary to bring about the desired change. However, inadequacies were observed in ensuring equitable distribution of results (girls, ECD and CwD) which undermined benefits for them as is discussed in detail under section 4.4. What is less linked to the results chain and not represented in the Theory of Change is the training on child friendly pedagogy. The thrust of the training was to enhance child friendly teaching in schools. While teachers who participated in the training mentioned benefits they derived from the training as improved teaching approaches its ink improved WASH outcomes was less clear. There were recognised inadequacies in hygiene education which resulted in the project having minimal impact on hygiene knowledge.

4.3 Efficiency

4.3.1 Is the program worth the resources it costs? Finding: While the evaluation did not undertake a cost benefit analysis, the changes brought about by the project such as increased sanitation facilities, water and hygiene transformed school experience for learners. Unresolved issues of equitable access, limited availability of water in handwashing tanks all undermine the value of the investment by the project. As highlighted in Section 4.2, the project managed to transform a dire WASH situation in targeted schools. Going to school was a burden especially for girls, while ECD and those CwD would face challenges of accessing drinking water and latrines. The investment by the project, has had multiplier effects as shall be discussed in section 4.4. These include: 1) building of new classroom blocks as water was now available moving the schools towards registration19; 2) slight change in teacher retention; and 3) start-up of nutrition gardens which provided opportunities for income generation. However, the evaluation did not undertake a detailed value for money assessment to gather empirical evidence on the cost efficiency and effectiveness of the project. Therefore this question could not be answered in full20. Nonetheless, the evaluation undertook an assessment of the two components of value for money: adequacy and quality of support. Quality of support Finding: The quality of support was generally acceptable with beneficiaries and stakeholders expressing satisfaction. Latrines received the highest satisfaction for quality while concerns were raised on the quality of water from boreholes, leaking handwashing tanks and poor quality taps on the handwashing tanks. Toilets: Discussions with community members, leaders, learners, SDC, and teachers all agreed that the quality of toilets was satisfactory. There were few cases of toilets that had cracking walls or roofs that were leaking and about to fall. In one case children were still afraid to use the toilets because the slab was weak and one would feel like falling inside and another toilet had the slab collapsing:

19 Satellite schools do not meet the standards required for formal registration with MoPSE. 20 Discussions at inception agreed that no value for money assessment will be conducted as there was no budget for this exercise.

30

“We are not so happy about our toilets because there are cracks and some floor toilets are not so strong that sometimes you feel like you want to fall inside. We are not using one of the toilet because the toilet floor collapsed.” Girls Hurungwe

In Mwenezi, SDC and children from one school were worried about the longevity of the latrine structure given large cracks that were already appearing after one year of construction. The main reason relates to the adequacy of supervision of construction by Environmental Health Technicians (EHTs) tasked with this responsibility. Because there is a critical shortage of ward based EHTs, they oversee multiple wards. Although they have motorbikes to facilitate this, fuel was sometimes a challenge as the 20l allocation per month would not be enough to cover the project area. In a discussion with the EHT at the concerned school, his second supervision visit came when the roof was being installed.

“The quality of infrastructure is good and up to standard except the block we didn’t supervise”. EHT Hurungwe

All schools had a schedule for cleaning toilets by students from grades four to seven ensuring they were clean with no faecal matter on the floors. Girls and boys were also highly satisfied with the cleanliness of the toilets they used.

“Toilets are always clean since we clean them every day in the morning and afternoon. There are no flies in the toilet because of the mash wire.” Boys Binga

However there was a challenge of cleanliness in Mwenezi discussed in section 4.4.1. Handwashing: Majority of handwashing tanks were built well. There were isolated concerns raised in three of the 14 schools visited for the evaluation about leaking handwashing facilities.

“When we fill up the tank with water it leaks from the top and bottom hence it does not store water all the time.” Girls Hurungwe

The major challenge with handwashing tanks was the poor quality of taps used across all the schools which could not sustain the volume of usage. Schools were also struggling or delaying their replacement.

“The taps are sub-standard and were breaking down frequently.” FGD with DWSSC. “Handwashing tank was well built and protected. It allows learners to wash their hands after using the toilet. It is a reminder also for students to wash their hands. However, we had challenges that the tap was always broken or stolen by the gold panners.” Teacher Hurungwe

Boreholes: All boreholes had not broken down since installation demonstrating the quality of materials used. However, as highlighted earlier, in a majority of the schools the water was not palatable because it was either too rusty or too salty. In two schools, beneficiaries were resorting to previous unsafe water sources. At one school in Mwenezi, the borehole was reported to dry up during the dry season. Of major concern regarding materials was the rusty water. Adequacy There were mixed findings on adequacy. There was oversupply and under supply of latrines as noted under section 4.1.2. Handwashing tanks with a single tap were inadequate to meet the schools’ enrolment. Almost all boreholes were producing adequate water all year.

31

However, challenges arose with manually filling the water tanks with water as this burden combined with the distance to the borehole resulted in handwashing tanks remaining without water for long periods. The one and a half day training of teachers in PHE was deemed inadequate by EHTs as it did not allow for coverage of all the content. In all three districts, PHE materials were deemed inadequate by EHTs, DWSSCs and teachers with schools reported to either having an incomplete set or nothing at the time of the evaluation. These assertions were at odds with head teachers whose majority (71.4%) agreed or strongly agreed that there were adequate teaching aids for hygiene. The training of WPCs though adequate was not supported by a systematic schedule of follow up visits or supervision to ensure their functionality. As a result a majority of WPCs were reported to be inactive by DWSSCs. There were concerns by DWSSCs and other stakeholders interviewed that the training of VPMs, 14 days, was too short and that the start-up toolkit was inadequate to fully equip them for the responsibility of maintaining the boreholes. Borrowing from experience and lessons from implementation of the rural WASH programme, DWSSCs noted that three days of theory and 11days of practice were not enough to enable the VPMs understand all the parts of the pumping system and was one of the major causes for collapsing boreholes, and higher burden on DDF (as part of the DWSSC) to assist in fishing fallen parts. Thirty days were noted as a more ideal period.

“…Therefore, we cannot say we have equipped a VPM to repair a borehole without the things to use. The load at the district will remain as they will be asked to come and repair the borehole. When we plan toolkit for VPMs lets provide a full tool kit for maintaining a borehole including bicycles for transportation.” DWSSC, Hurungwe

4.3.2 Were the resources and inputs converted to outputs in a timely and cost-effective

manner? On budget against agreed plans? Finding: After a one year delayed start, the project faced teething challenges in Binga which necessitated a one year no cost extension. The project was delayed by one year from June 2014 to July 2015. In 2015 the project only started with one district, Mwenezi, with additional districts of Binga and Hurungwe added in 2016. Once partners were contracted, the speed of implementation improved. In all districts, stakeholders highlighted implementation was timely and according to timelines. When delays were faced there were due to resistance of communities to provide local materials which needed the intervention of the DWSSC and community leaders. This challenge was worse in Binga where, due to sandy soils, it was difficult to find clay soils to mould bricks. Another challenge specific to Binga, was that as a result of water stress, water tables were too low to provide adequate sustainable yield. This brought challenges in finding wet holes. Because of the difficulties to find wet holes coupled with Unicef’s policy of paying only for “wet holes”, the contractor ZINWA experienced spiralling costs of implementation which resulted in a contractual standoff. Two more contractors were brought but the delays had been significant leading to a no cost extension.

4.3.3 How well the project apply VFM principles of effectiveness, economy, and efficiency

in relation to the delivery of its outcome? UNICEF WASH Section has institutionalised reporting templates for value for money. However, this is limited to economy – the unit costs of procurement (cost per unit) and implementation (e.g. cost per beneficiary). This information is used to make decisions on procurement and implementation approaches. Economy, which measures the value of outputs

32

in relation to the costs, is not part of this measurement. Effectiveness, measures the cost incurred vis a vis the outcomes achieved. This aspect is not measured by UNICEF.

4.3.4 Was the programme management, coordination and monitoring efficient and

appropriate for all levels? The project was managed through the Zimbabwe WASH coordination system comprising the National Coordination Unit, the DWSSCs and Ward Water and Sanitation Sub-committee (WWSSCs). Members of these structured were involved in the planning, implementation and monitoring of the project working in collaboration with the NGO partner. During the project quarterly monitoring visits were undertaken by the DWSSC including joint monitoring visits with the NGO. Providing leadership of implementation to existing national WASH coordination structures enhanced efficiency of implementation and complementarities with the existing Rural WASH programme. The multi-sectoral knowledge in the DWSSC enhanced quality of interventions as the project had available all skills and knowledge required for implementation. For example: the DDF was responsible for overseeing borehole drilling and training of VPMs and WPCs; District Environmental Health department was responsible for training Teachers and Latrine Builders on hygiene and latrine BVIP building; and the District Administrator’s office was responsible for mobilising communities through community leaders including negotiating with communities when communities resisted providing local materials. Monitoring: Monitoring of the project was guided by existing tools used for the Rural WASH programme. The DWSSC was responsible for undertaking monitoring. UNICEF and the NCU would undertake quarterly visits to project sites. DWSSCs in all districts highlighted that feedback from UNICEF and the NCU in resolving issues noted during monitoring visits was timely. While the monitoring tools were viewed as adequate by stakeholders, there may need to be reviewed to make them more relevant for a WASH in schools project including: 1) progress on MHM; 2) equity in availability of water and sanitation facilities; and 3) availability of materials and trained teachers on hygiene.

4.4 Impact

4.4.1 What outcomes were achieved by the project? Can the changes in outcomes be

explained by the project, or are they the result of some other factors occurring

simultaneously? This section follows the Theory of Change developed as part of the evaluation design. A typical retrospective results framework (also guided by the project’s objective) was developed (Figure 6). Three key results areas identified in the results framework, to which the project contributed to, are analysed under this section:

• Results Area 1: Increased availability and access to safe drinking water;

• Result Are 2: Increased availability and access to appropriate sanitation and hygiene facilities; and

• Result Are 3: Availability and access to hygiene education.

33

Figure 6: Retrospective results chain for the WASH in Satellite Schools Project

Result Area 1: Increased Availability and Access to Safe Drinking Water Increased availability of safe drinking water sources Finding: The project significantly increased availability of safe drinking water sources in targeted satellite schools from non-existent to consistent availability as downtime of boreholes was limited. The objective of the program was to increase availability of portable water in schools to improve the WASH situation through drilling of boreholes for portable water. In that regard discussions with the three NGO partners and DWSSCs in Binga, Hurungwe and Mwenezi, indications were that 100% of the sampled schools have drilled boreholes in place thereby increasing the availability of safe portable water. This has in turn resulted in smooth progression of school as highlighted by one teacher explaining that “teaching is no longer disrupted as children no longer take time to go and fetch water”. Provision of water through-out the year is key to successful improvement of the WASH situation in satellite schools hence uninterrupted borehole function is paramount. In light of this, at the time of study 85% of the boreholes drilled in schools were functional while 15% were partially-functional. Partially functional boreholes were recorded in Hurungwe and Mwenezi. Provision of water throughout the year, a function of functionality and down time, was also considered by the evaluation. For this program, the longest down time recorded for these school boreholes was five days and the shortest was one day. In one school, Mabula primary school in Binga, it was reported that while the borehole is available, it does not yield adequate water for both the school and community use. Emergency planning requires that there be an alternative safe water source near schools in case the main water source breaks down. The evaluation revealed that 71.4% of the sampled schools have alternative water sources that the schools can use in case of challenges with their boreholes. The most prominent water sources available to schools were community bush pumps contributing 60%, 30% as unprotected dug-wells and 10% being protected dug-wells. Again, water availability is a function of seasonality of the water source and down time. In that regard, 30% of the alternative water sources are seasonal while 70% are perennial.

34

Utilisation of portable water by school children is influenced by water availability and palatability. Palatability is a function of water quality that may be influenced by various factors explored by the evaluation. 42.6% of the interviewed students indicated that water from these boreholes was palatable and were satisfied with it. The observation questionnaire showed that in 57.1% (8 out of 14 schools) of the boreholes observed the water was not palatable. Cases of non-palatability were worse off in Hurungwe (four schools out of six) and Mwenezi (three schools out of four) than Binga (one school out of four). These water palatability cases as revealed by the evaluation were caused by saltiness and rusty water. In light of this students and teachers are thus forced to resort to alternative water sources with a significant proportion using unsafe water sources. Water quality assessments are undertaken at the completion of drilling to ensure the water is safe to drink. However, the schools are seldom shared the results of this assessment. Speaking with DWSSCs, NCU, UNICEF and NGO partners it was highlighted that results were only shared if there was a major issue with the water quality. Furthermore, no regular assessment of water quality are undertaken providing a risk for unnoticed water contamination. In some schools, the school heads were unsure of the quality of their water given it was rusty (Hurungwe) and salty (Mwenezi) at the time of the survey leading to some schools abandoning the project boreholes. In one school in Mwenezi the water was so salty the school had stopped using it for watering the garden as well. Increased access to safe drinking water Access to safe drinking water is estimated by the percentage of a population able to or using improved drinking water sources. Finding: From results of the survey indications are that the project has increased accessibility of safe drinking water in satellite schools despite some challenges with distances to boreholes and palatability of the water. Equity in access was achieved for a majority of water points through the design of child and physical disability friendly water access points. As earlier indicated, the project increased the number of boreholes in the school yards from zero to one per school which addressed the issue of water availability. Later questions arise as to whether these available water access points are accessible to school children. The evaluation revealed that 71.2% of the students have access to drinking water at school i.e. are using the boreholes drilled by the project which they consider to be a safe water source, while 19.8% indicated they bring their water from home because they did not trust the safety of the water or water was not available at the school. Inaccessibility of the borehole is particularly grave in some schools to the extent of causing a reversal of the project’s gains. Boys from a school in Hurungwe, in a focus group discussion, noted that issues of access to water were of a major concern for them. They did not see any real changes because the borehole is as far as the river where they used to fetch water before the project and that the water is rusty. This has forced them to revert to sourcing drinking water from the river they had abandoned when the borehole was drilled. The distance to the borehole was a major concern across a number of schools. The main reason was that the siting of schools had not considered the availability of water. In the school in question in Hurungwe, the drilling company had sunk three dry holes in the school yard with a wet hole only identified about a 800m from the school. Binga seems to have been worse affected as all boreholes visited were far from the school. Therefore piped water schemes would have enhanced access to the available water source. To its credit, the project piloted solar powered piped water schemes at three schools in Binga which have been relatively successful. While there were some schools for which access to safe water is an issue, where access has been significantly increased children have realised profound benefits in their lives and schooling. For boys at Chehamba primary in Hurungwe, the availability of the borehole close to the school has enhanced their schooling experience as they no longer walk the long

35

distances to fetch water for teachers. Exhaustion from such long trips would compromise their participation in class. For boys at Mulame Primary in Binga increased accessibility to clean water means lessons are no longer disturbed ensuring there is more learning time. In Takunda primary school in Mwenezi, girls were happy that the availability of water has also influenced their parents to start building classroom blocks for the school.

“Before we had a borehole to the school, teachers used to ask us to go and fetch water for the at the river before we start school by the time we would start school we would be exhausted, or they would ask us to go and fetch water after school, and since the river was far we would get home very late but now that the borehole is close by, it does not take us long to go and fetch water for the teachers.” FGD with girls in a school in Hurungwe

“Our performance as teachers in terms availability at school has improved children`s attendance.” Key informant inter with a teacher in Hurungwe district

“Our parents were motivated to start building learning blocks when there was availability of water close to the school as well as toilets.” FGD with girls in a school in Mwenezi.

Setting up the infrastructure and having water available at the school throughout the year does not guarantee access to water for all children. In that regard, the evaluation explored if challenges existed with regards to access to water for drinking for children in schools. 96.6% of the school children interviewed in the sampled schools indicated that they had no challenges in accessing water for drinking throughout the year. Children with disabilities, a special needs group, has become of paramount importance in assessing access to basic WASH services. In this evaluation, 64.2% of the drinking water facilities were reported to be accessible to children of all kinds of disabilities while 35.8% could not cater for all disabilities. Reasons for inability to access water points mentioned was failure to use lifting device and inability to get to the source without assistance which is often experienced. To improve accessibility for children with disability there is need to review the system design of the access point and ensuring water is available within the school yard through piped water schemes. The evaluation also sought to find out if children in infant grades (Grade 0-2) were able to access the drinking water sources supported by the project. Results show that 50% of the water points in sampled schools were accessible to children in infant grades with regards to the height of the tap. Hurungwe recorded the highest non access level of 50% for children in infant grades as the taps were higher and could not be reached easily by children in infant grades. However in all schools except those with piped water schemes the ability to operate the bush pump lever is a major challenge for infant grades. This is compounded by the fact that majority of schools do not keep secondary storage facilities specifically for drinking water to improve access by infant grades. While the handwashing tanks can provide a source for drinking water, they are poorly managed. For example the evaluation team observed that the container lids were not closed properly, or not at all and students would throw in litter etc. The water was therefore not safe to drink and students only used this water for handwashing purposes. Access to drinking water also entails how students obtain water from the water source. In this evaluation 78.6% of the students access water using their own reusable drinking vessel, 14.2% share a drinking vessel and 7.1% drink directly from the faucet of the pump sprout. Engagements with students and teachers insinuate an increase in the number of students that bring their own reusable drinking containers for drinking mainly because the schools are

36

unable to provide the drinking vessels. There is potential for use of contaminated vessels as majority of the vessels were empty soft drink or beer plastic bottles. Result Are 2: Increased availability and access to appropriate sanitation and hygiene facilities The evaluation measured the availability and accessibility of project sanitation and hygiene infrastructure in target schools. Increased availability of appropriate sanitation and hygiene facilities Findings: The project provided adequate sanitation facilities which resulted in reduced open defecation in and around the school environment. However, availability of girl friendly latrines remained a challenge across the project with the exception of some schools in Binga where especially dedicated girl toilets with a gender responsive design were constructed. Even these remain in shortly supply as only to two were constructed per school. One of the major challenges highlighted by majority of FGDs in the evaluation was that of open defecation around the schools due to inadequate toilets. The unavailability of appropriate facilities worsened the working conditions of teachers leading to high teacher turnover.

“Before the project children used to use the bush especially if there were not monitored to use the toilets. Now there is no need to monitor them as we are assured they will use the toilets because there are clean and well built.” FGD with community members

According to the DWSSC in Binga there was a significant improvement in the WASH situation including reduced open defecation in targeted satellite schools which could be directly attributed to the provision appropriate sanitation and hygiene facilities by the project. In an interview with DWSSC in Binga and supported by KII with teachers in Hurungwe it was noted open defecation in and around school premises had reduced significantly with the advent of construction of sanitation facilities by the project. Furthermore households around the schools have adopted the setting up of BVIPs moving from no toilet at all or from a pit latrine in Binga while some in Hurungwe were reported to be using the school latrines. School were reaping the benefits of improved working conditions for teachers which had a slight impact on teacher retention. Community members also utilise the school toilets during weekends.

“There is a lot of positive changes. There is no longer a bad odour around the area. We never used to have toilets at our residential places so now we can easily go to the toilet. Many teachers would come and couldn’t wait to leave and get a transfer. Retention is better amongst teachers but people are still trying to leave but it is way better”. KII Teacher

The Education Act recommends a student: toilet ratio of 1:20 for girls and 1:20 for boys hence schools should adhere to set regulatory requirements. Evaluation results show that on average each school has 2 toilets for infant class, 6 toilets for females and 5 toilets for boys for grade 3 to 7. Table 7 shows the proportion of schools that fall within the recommended pupil: squat hole ratio disaggregated by sex. 53.8% of schools fell within the recommended standard for boys while less than half (30.8%) of the schools were within the minimum standard for girls. Table 7: Minimum standards for sanitation facilities

Within recommended ratios

(1:20) Above recommended ratios

(1:25)

Boys 53.8% 46.2% Girls 30.8% 69.2%

37

Source: Survey results, Infrastructure Observation Checklist and Head Teacher Survey

While the project failed to achieve the recommended standard for girls, this represented an improvement from the situation before the project. During FGDs and KIIs it was noted that before the inception of the project toilets in most of the satellite schools were few and inadequate, with an average of 3 toilets each for both girls and boys. There were mostly Blair and pit latrines. Through the WASH project, UNICEF promoted the building of appropriate sanitary facilities that resulted in an increase of BVIP toilets. Nonetheless there are indications that there was an oversupply of toilets in 38.5% of surveyed schools in 2016 (See Table 8). While this over supply may be offset by increased enrolment over time, as discussed earlier, it undermines equitable resource allocation. Students from a school in Binga and Mwenezi illustrate this dichotomy:

“There were times when we failed to go to the toilet due to queueing…” FGD with boys at school in Binga

“Now that toilets are many no one is using the bush anymore…” FGD with girls in a school in Mwenezi

Table 8: Ration of student to squat hole for children in grade 3-7

District School

2016 Girls

Grade 1 -7

2016 Boys

Grade 1 -7

2017 Girls

Grade 1 -7

2017 Boys Grade 1 -

7

2018 Girls Grade 3 -

7

2018 Boys Grade 3 -

7

Champongo binga 19 24 24 29 14 18

Mabula binga 11 10 8 14 5 7

Mulambe binga 71 54 67 50 42 30

Muunde binga 21 22 19 20 13 14

Chehamba hurungwe 73 62 69 63 58 45

Chipapa hurungwe 43 52 42 100 29 36

Madzimoyo hurungwe 37 34 35 29 26 21

Momba hurungwe 24 18 24 19 18 12

Dambanzara hurungwe 35 49 39 51 30 31

Songalalo hurungwe 52 66 51 75 37 50

Chemhara mwenezi 19 29 20 27 12 12

Madamwa mwenezi 22 20 22 19

Muuyu mwenezi 25 25 33 39 25 27

Takunda mwenezi 14 19 17 15 10 12

Source: Survey results, Infrastructure Observation Checklist and Head Teacher Survey

Infant pupils are a special class within schools that need to have their needs met by the design so that they are encouraged to use the constructed toilet facilities. As earlier alluded to, the project successfully constructed toilets for children in infant grades in all the 14 sampled satellite schools in the 3 districts. In terms of appropriateness of the design of the infant latrines 78.5% of the school toilets observed were appropriate. This has resulted in children in infant grades using the toilets and recording low cases of open defection in the sampled schools as highlighted by DWSSC in focus group discussions. One teacher also mentioned an improvement in enrolment due to the availability of user friendly toilets for this age group. Mwenezi was the worst affected with 3 out of four schools (75.0%) observed not meeting the standard for squat hole size appropriate for infant use.

38

While the project met recommended standard on number of students per squat hole for boys and made significant progress on the same for girls, infant children were left behind. The average infant children to squat hole was 1:34 (at the time of construction in 2016) compared to the minimum standard of 1:20. When individual schools are analysed this ranged from 1:12 in Mabula primary school to 71 in Madzimoyo primary school (see Table 9). Table 9: Adequacy of squat holes for children in infant grades

District School 2016 ECD Girls

2016 ECD Boys

2017 ECD Girls

2017 ECD Boys

2018 ECD

binga Champongo 65 61 57 44 106

binga Mabula 12 17 10 15 30

binga Mulambe 20 26 24 23 86

binga Muunde 42 20 11 7 45

hurungwe Chehamba 30 20 18 31 75

hurungwe Chipapa 36 29 32 44 25

hurungwe Dambanzara 37 40 48 47

hurungwe Madzimoyo 59 71 48 36 111

hurungwe Momba 20 20 21 21 48

hurungwe Songalalo 38 40 46 42 130

mwenezi Chemhara 30 31 19 22 48

mwenezi Madamwa 30 52 28 55 75

mwenezi Muuyu 34 39 35 42 11

mwenezi Takunda 26 17 33 36 79

Source: Survey results, Infrastructure Observation Checklist and Head Teacher Tool

Handwashing facilities: Construction of hand washing facilities was one notable impact made by the project in all the satellite schools visited. All schools included in the evaluation had a handwashing tank constructed by the project and 99.4% of students agreed that hand washing facilities were present in all schools. With the availability of handwashing tanks, there is need for continuous supply of water to handwashing facilities at the school to ensure appropriate use. At the time of data collection all the schools were reported to have water flowing from the taps at the hand washing facilities. Interviews with students in the three districts indicated that 87.9% of the facilities have water flowing from the taps throughout the year. 72% of the students that mentioned non availability of water in the handwashing tanks were females, of which 74% of these female students were from schools in Binga district. A group of girls and boys from Dambanzara in Hurungwe had this to say,

“Our lives have greatly improved because we never used to have a handwashing tank to use after using the toilet. Previously we never used to wash our hands, if you didn’t bring a bottle water you would not wash your hands unless your friends are willing to share their water. When the handwashing tank stopped working we resorted to using our bottles to fetch water to wash our hands.” FGD with girls at a school in Hurungwe.

“The hand washing facility was not being used all thus while, so we cannot really say there is really an impact in our lives.” FGD with boys at a school in Hurungwe.

From interactions through FGDs non availability of water in the handwashing tanks may be associated with the distance that students have to walk to fetch water that is poured into the tank. This was noted as a burden by one community leader in Binga district which also is affected by the low yielding borehole earlier mentioned. When students at Dambanzara

39

school in Hurungwe were asked to take pictures of what they liked or disliked about their school, both girls and boys took pictures of the 800m uphill walk from the borehole to the handwashing tank which they are tasked to fill. Apart from the handwashing tanks constructed by the project the evaluation also made an inquiry on the location of handwashing facilities in the school. Handwashing facilities were inside the toilet block or very close to the toilets for 55.8% (24 out of 43) of the toilets observed during the evaluation. For 27.9% (12 out of 43) of the toilets observed the handwashing facility was within the school grounds but too far from the toilets. This was mainly the case for infant grades toilets as they were constructed close to their classrooms and further away from other toilets whereas the handwashing tank was constructed close to the majority of toilets. This affected handwashing practice by this group of children. Only one school in Hurungwe had handwashing facilities in the classroom and at the time of study that facility had no water available. The evaluation also delved into the type of agents available for handwashing at the handwashing facilities. 58.97% of all students in the sample highlighted that soap or ash were always available at the handwashing facility. However, there were stark differences between districts (See Table 10). 83.64% and 35.7% of students in Binga and Hurungwe noted that soap or ash was not always available at the hand washing facility. This was in contrast to only 7% in Mwenezi. Table 10: Is the cleaning agent or ash or soap always available at the school hand washing facility?

District female male Overall

no yes N no yes N no yes N

Binga 74.10 25.90 27 89.70 6.90 28 83.64 16.36 55

Hurungwe 47.60 52.40 42 23.80 76.20 42 35.70 64.30 84

Mwenezi 3.60 96.40 28 10.70 89.30 28 7.10 92.90 56

Overall 42.30 57.70 97 39.40 59.60 98 41.03 58.97 195

Source: Survey results, student questionnaire

Among these agents, student interviews point to water and soap being the mostly used for handwashing as 47% of respondents gave positive feedback. Soap and water is mostly used by students from Hurungwe (76.2%), water and ash in Mwenezi (60.7%) while Binga had the least number of children using soap or ash as washing agents (18.2% and 20.0%). About 61.8% of students in schools in Binga district said they use water only when handwashing. Based on this, it can be concluded that the availability of washing agent is mixed in the districts which has a negative effect on enhancing positive behaviour change on handwashing. Increased availability of menstrual hygiene management (MHM) commodities/materials and facilities Finding: The project had minimal impact on MHM across schools. Binga seemed to perform relatively better than Mwenezi and Hurungwe because of girl friendly toilets were constructed. In terms of responsiveness to gender issues, first, in all schools there were separate toilets for girls and boys and male and female school staff. This contributes to privacy for girls. Furthermore girl friendly latrines should enhance this level of privacy by providing lockable doors, presence of hand washing facilities inside the toilet and discrete facilities for safe disposal/cleaning of menstrual material. Achievement of the project is mixed in this regard (See Table 11). Binga is the only district with facilities fully responding the needs of girls by putting in place facilities for MHM. Hurungwe and Mwenezi schools in general not responsive

40

to girls’ MHM needs. All toilets for girls in Hurungwe and Mwenezi had no lockable doors while 33.3% and 25.0% in Hurungwe and Mwenezi had a private location for girls to dispose or clean menstrual hygiene materials. The same trend was observed for the two districts on privacy of the handwashing facilities for girls to dispose of clean menstrual hygiene materials and facilities for safe and discreet disposal/cleaning of menstrual hygiene materials. In Binga, only one toilet per set of toilet for girls was dedicated for MHM. While good it introduced challenges discussed in Section 4.4.4. Inadequate provision for MHM in the project emanates from two main reasons: 1) the design which did not incorporate this component and hence no guidance for partners; and 2) the unavailability of nationally accepted designs for girl friendly latrines. It is important to note that constructing girl friendly latrines is 22% more expensive than the conventional latrine. Table 11: Availability of MHM commodities

District

Handwashing facility has a private location for girls to dispose of or clean menstrual hygiene materials (n/%)

Are girls' individual toilets compartments lockable from inside? (n/%)

Handwashing facility has a private location for girls to dispose of or clean menstrual hygiene materials (n/%)

Appropriate facilities exist for safe and discreet disposal/cleaning of menstrual hygiene materials (n/%)

Binga 4 (100.0%) 4 (100.0%) 4 (100.0%) 3 (75.0%)

Hurungwe 2 (33.3%) 0 (0.0%) 2 (33.3%) 2 (33.3%)

Mwenezi 1 (25.0%) 0 (0.0%) 1 (25.0%) 2 (50.0%)

Total 7 (50.0%) 4 (28.6%) 7 (50.0%) 7 (50.0%)

Source: Survey results, Infrastructure Observation

In terms of school support for MHM, the head teacher tool showed that schools are not providing a comprehensive package of information and material support for girls to prepare onset of menarche or manage menstruation while at school (See Table 12). Binga seems to be better off than schools in Hurungwe and Mwenezi but the package of information is not standard across all the schools (See Table 12). No schools in Hurungwe and Mwenezi provided information on MHM to girls approaching menarche. In Hurungwe, only one school out of four while only two out of four schools in Binga provided this information. Table 12: Support by school to promote safe menstrual hygiene

Support provided Binga Hurungwe Mwenezi

None 0 3 2

Menstrual hygiene education sessions for girls

4 1 1

Private washing facilities for sanitary ware [e.g. washbasin with tap inside a lockable toilet cubicle]

3 0 0

Private disposal/incinerator facilities for disposable sanitary ware

2 2 0

Provision sanitary ware 3 2 1

Provision of soap for washing self and sanitary ware

0 1 0

41

Support provided Binga Hurungwe Mwenezi

Hygiene promotion materials on menstrual hygiene in appropriate languages for students

2 0 0

N 4 6 4

Source: Survey results, Head Teacher tool

Disability: Designing appropriate sanitation facilities includes considerations for people with disabilities. In that regard, the project, according NGO partner key informants, sought to ensure availability of sanitation facilities that would cater for children with disabilities. From observations made by the evaluation team, disability friendly latrine had ramps for easy access with wheel chair, hand rails inside the toilets and toilet seats in some instances. Of note were the inconsistencies however on the availability of these features in the latrines within the districts and across the consortium. In Mwenezi for instance latrines had toilet seats but no handrails. In Binga also some schools did not have the handrails and in 2 of the schools, these were reported to have been inaccurately installed. The evaluation showed that 85.7% of the schools had toilets set aside for children with disabilities. Access to appropriate sanitation and handwashing Sanitation facilities: From various FGDs conducted during the survey both boys and girls pointed to an increase in access to sanitation facilities and reduced instances of open defecation. This was attributed to the availability of sufficient toilets at the schools as noted by children:

“We no longer defecate everywhere and we are able to wash our hands after using the toilet. Because each and every class has its own toilet which reduces the number of people per toilet and allows people to access the toilet.” FGD girls at a school in Mwenezi “Now that toilets are many no one is using the bush anymore and we do not bring water from home as much although some do because the water is not so tasty.” FGD with boys at a school in Mwenenzi.

The survey solicited for responses as to whether the toilets were conveniently located for use by children and teachers during day and night time. All the 14 schools indicated that the toilets were conveniently located making access easier during any time of the day for both children and teachers. Most of these toilets are accessed by foot paths and so these need to be always clear to encourage access. All the 14 schools indicated that footpaths to the toilets were always clear and encouraged access to the toilets even during the rainy season. Children were asked if they had challenges in accessing toilets in the wet and dry season of the year. 98.4% of the students in the sampled schools indicated that they had no difficulties in using the toilets with 1.6% indicating they do have difficulties. Further the study solicited for the reasons why the few children had difficulties in using the toilets and two main reasons highlighted by girls in Hurungwe and Mwenezi21 were dirty state of the toilets and abdominal cramps that made the students not to go the toilets. Issues of security and privacy were also included in the study. 97.2% of the students had privacy in the toilets while 94.6% felt secure in the toilets. Binga recorded highest number of children highlighting insecurities when in the toilets. Most insecurities in Binga were recorded among male students where 31 % of those insecure said the toilets did not provide adequate

21 See later discussion on cleanliness of toilets in this section.

42

security the girls on the other hand seemed to be content with the level of security of the latrines. This is an interesting observation as security in latrine use is regarded as an issue for girls. However, the low percentage of boys who felt insecure may not warrant a full scale shift in the current design but more information is required on this aspect of the project. For example, sentiments from boys could be a reaction to girls’ toilets that have a lockable door and demonstrate the impact. Allowing students to go to the toilet whenever they felt they needed to, was also investigated. In this survey school children were asked if they went to use the toilets whenever they felt they need to. 97.2% of the school children indicated that they went and used the toilet whenever they felt the need to without being stopped by anyone at the school. 3.8% of the students indicated that they were not allowed to go to the toilet anytime they felt they needed to use the toilet and the reason stated was prohibition by the teacher. Access to latrines with varying seasons seems not to be a challenge in all the schools visited as during both the rainy and dry season majority of students across all three districts pointed out ease of use of the toilets however 13% of female students from Binga and Mwenezi brought to attention difficulty in using the toilets during the rainy season due to smell emanating from the latrine. With regards access by girls at menarche, only five girls in the sample had reached menarche. Of these four strongly agreed or agreed that they were happy to use the school's toilet during my period. However, the small numbers did not give the evaluation sufficient power to make conclusions on this issue. Handwashing facilities: While hand washing facilities may be available in schools, students need to be able to access them at all times. The evaluation revealed that 98.2% of the school children in the sampled schools had access to handwashing facilities. Handwashing facilities for young children especially those in infant grades have to be designed in such a manner as to improve access as issues of height from the ground should be considered. 85.7% of the children interviewed in the sampled schools highlighted that children in infant grades were able to access their handwashing facilities with ease. Observations were made of the students using these handwashing facilities and they seemed to access with ease. With regards access by children with disabilities (CWD), 71.4% of the students interviewed in the schools indicated that CWD could access handwashing facilities with ease while 28.6% indicated that not all disabilities were catered for in the design of these handwashing facilities. While all schools have handwashing tanks constructed, a key informant in Binga highlighted an issue of the burden of collection of water from the borehole and pouring into the tank for drinking water. This points out to instances where tanks go for sometime without water for handwashing. In Dambanzara primary school in Hurungwe one of the teachers highlighted that the school has a very large catchment area and has a huge enrolment as such the tank constructed for handwashing is inadequate as there is a lot of congestion during peak periods. As a result many of the students do not wash their hands at the critical times. Of the five girls at menarche at the time of the survey, three strongly agreed or agreed that they could access water and soap at school during their period. Four girls out of five indicated that they do not miss school or get embarrassed because of their monthly periods. Parents highlighted a reduction in the number of girls that do not attend school because of their monthly

Children queuing for to wash hands in Mwenezi

43

period. This was mainly due to improved information dissemination and provision of an enabling environment through latrine construction in all the schools. Result Area 3: Availability and access to hygiene education 13 out of the 14 schools evaluated confirmed existence of a school health club. Over 90% of student health club members confirmed the club was beneficial to them. In the three districts 34.8% of the student interviewed were members of a school health club. Binga had the highest proportion of students being members of a health club (55.4%) with 63% being female. In Hurungwe only 19% of the respondents were members of a school health club while 30.4% in Mwenezi confirmed their membership to the school health clubs. In all the three districts there were more female respondents who are members of a school health club than males. The mostly covered topics were revealed to be Hygiene, Water and sanitation, general hygiene and personal hygiene there is little or no information shared on reproductive health in all the three districts. While students from Binga and Hurungwe pointed to acquiring knowledge mostly on hygiene and water and sanitation, an alarming 29.4% of the health club members from schools in Mwenezi said they had not received knowledge on WASH topics that should be pivotal to the existence and function of every school health club. The most common health club activity across all three districts was that of cleaning the schools. Though raising awareness to other students was mentioned as one of the main activities of the school health clubs less than half of the interviewees participate in this activity with only 29% of the females and 37% males highlighting that they participate in this activity. In contrast with responses of the school heads who highlighted that students transmit hygiene knowledge to the community and families through educational material and sponsored events only 32% of health club members interviewed participate in such activities. Female respondents in Hurungwe and Mwenezi have never participated in such outreach programs while only 23.5% in Binga confirmed participation. Boys in all three districts were the ones that reported they have participated in these outreach programmes. In all the schools results show that teachers received training on hygiene to facilitate dissemination of hygiene education in schools. Details of the content of training were however not looked into as the survey indicated that trainings were conducted by NGOs and the government. Inconsistencies in the training content may influence the ultimate dissemination of information as shown by the results from students ‘responses and the activities and information shared in the school health clubs. A total of 37 teachers, 23 males and 14 females received hygiene training. From key interviews training of at least 2 teachers was a strategy to ensure sustainability of capacity in instances of transfers. Only three schools highlighted that their staff had received training in 2018 while the rest were trained in 2016 with some last trained in 2013. There is also a clear lack of standardization in the way hygiene education is taught in the three districts. In Hurugwe and Mwenenzi hygiene is said to be a component of the core curriculum as it is integrated as a part of a special module on healthy living. It is only in 2 schools in Binga where hygiene is a standalone special module taught exclusively as well as through sponsored extracurricular activities. While hygiene education is a component of the core curriculum knowledge levels from the students interacted with do not exhibit a satisfactory comprehension of the subject especially students that mentioned that they were members of the schools health club. Superimposing the results of the responses from the students on their knowledge levels one would assume the lack of adequate teaching material on hygiene education, 71% of schools said they have adequate WASH teaching aids. Of note is the teaching of handwashing which results highlight to be considered as a prominent part of hygiene lessons in all the schools. 77% of schools in the study indicated that they stress the importance of handwashing with soap at critical times yet there is a disconnect with the results from the

44

students especially in schools in Binga where students highlight that they mostly use water only for handwashing. There are a few practical demonstrations by the teachers/trainers on how to appropriately wash hands as only 5 schools in all the districts have role models of staff washing hands at critical times. Students in their responses exhibit their understanding of handwashing with soap especially after toilet use in the prevention of disease there is a disconnect in implementation on the failure of most schools in providing soap for handwashing at all times. It is also emerging that there is little involvement of teachers in promoting hygiene behaviour to buttress the information disseminated as only 3 schools in Hurungwe (2) and Mwenenzi where staff are involved in the clean-up campaigns. Hygiene promotion from analysis of responses from health club members may not have been effectively done due to the failure to associate or perceive lack of sanitary facilities and their use thereof, treatment of water and storage and food hygiene with presence of diarrheal diseases. Only 23% of the entire school health club members across the three districts interviewed were able to find association of latrine use, while only 12% associated water treatment with diarrheal disease prevention. Also, in response to identification of critical times for handwashing students mostly identified handwashing after toilet use and before eating as the major critical times paying little or no attention to other times that are critical for handwashing. Open defecation, presence of flies in all three districts were not considered to be a sign of lack of hygiene and therefore no linkage with prevalence of diarrheal disease is made. This becomes a clear indication of gaps in terms of information dissemination by the School health coordinators as well as EHTs that are expected to provide support to schools in hygiene promotion and information dissemination. Knowledge of hygiene Finding: The evaluation found varying levels of hygiene knowledge among students in targeted schools. Binga consistently had higher proportion of students showing hygiene knowledge with Mwenezi being the lowest. There were also gender disparities with boys more knowledgeable than girls. The main sources of knowledge were teachers (binga (89.40), Hurungwe (82.10), and Mwenezi (68.60)) followed by the radio (Binga (23.40), Hurungwe (23.90) and Mwenezi (28.60)). Despite this messaging seemed not consistent as 37.5% in Mwenezi, 20.2 in Hurungwe had not received any hygiene messaging three months prior the survey. A majority of students, 91.1% in Binga, 92.9% in Hurungwe and 83.9% in Mwenezi, noted hygiene was important for good health or to avoid getting sick. The main method of maintaining good hygiene known by students was washing hands with soap (Binga 87.5%, Hurungwe 64.3% and Mwenezi 58.9%). There were no significant differences between boys and girls. The second method was a clean environment. Worrying was that key methods such as treating or boiling water, consistent use of latrines, storing water and food properly, were unknown with less than 15% highlighting them in Hurungwe and Mwenezi. Binga seemed the exception, although the proportion of students was still low, between 21 and 25% of students mentioned these methods. Knowledge of critical times to wash hands was also measured and is presented in Table 13. The two most common critical times were: 1) after using latrines, 2) before eating with over 80% of the respondents in all districts. When further analysis was undertaken to determine the proportion of respondents knowledgeable of “both” these two methods Hurungwe had the highest proportion (83.3%) followed by Binga (82.8%) and Mwenezi (75.0%). Table 13: Critical times to wash hands

45

female male Overall

Binga Hurungw

e Mwenez

i Binga

Hurungwe

Mwenezi

Binga Hurungw

e Mwenez

i

After using latrine

100.0 88.1 82.1 10*0.0 92.9 78.6 100.0 90.5 80.4

Before eating

92.6 88.1 78.6 89.7 92.9 92.9 91.1 90.5 85.7

After cleaning children’s bottom

14.8 2.4 0 10.3 0 0 12.5 1.2 0

Before preparing meal

29.6 26.2 10.7 10.3 4.8 0 19.6 15.5 5.4

After handling children’s faeces

25.9 7.1 0 13.8 0 0 19.6 3.6 0

After touching animals

11.1 2.4 0 17.2 0 0 14.3 1.2 0

After handling animal faeces

11.1 2.4 3.6 13.8 2.4 0 12.5 2.4 1.8

Before feeding others

33.3 4.8 3.6 17.2 2.4 0 25 3.6 1.8

After taking care of sick family members

0 2.4 0 3.4 0 0 1.8 1.2 0

Don’t know

0 14.3 28.6 0 0 3.6 0 7.1 16.1

Other [specify]

0 9.5 7.1 3.4 9.5 7.1 1.8 9.5 7.1

N 27 42 28 29 42 28 56 84 56

Source: Student survey

Students were also asked to identify the signs that show an environment is not hygienic. Table 14 shows that a majority of students view garbage and waste water as the main sign followed by open defecation and bad/ foul smell in the environment/ village. A sizable proportion of students in Hurungwe (15.5%) and Mwenezi (23.2%) did not know the signs of an unhygienic environment. Interestingly the presence of adult and infant faeces was not recognised a sign of an unhygienic environment (Binga 60.7%, Hurungwe 86.9% and Mwenezi 94.6%). This lack of knowledge was more apparent in Hurungwe and Mwenezi. More telling is that only 8.9%, 3.6% and 0% in Binga, Hurungwe and Mwenenzi were knowledgeable of all signs of unhygienic environment.

46

Table 14: Signs of an unhygienic environment

Female Male Overall

Binga

Hurungwe

Mwenezi

Binga

Hurungwe

Mwenezi

Binga

Hurungwe

Mwenezi

Adult and infant faeces

44.4 14.3 0 34.5 11.9 10.7 39.3 13.1 5.4

Animal faeces

25.9 19 21.4 17.2 2.4 7.1 21.4 10.7 14.3

Garbage and wastewater in surroundings

51.9 73.8 71.4 72.4 45.2 42.9 62.5 59.5 57.1

Bad/ foul smell in the environment/ village

63 38.1 42.9 65.5 4.8 7.1 64.3 21.4 25

No/ lack of latrines

22.2 23.8 3.6 20.7 11.9 7.1 21.4 17.9 5.4

Open defecation

51.9 9.5 3.6 44.8 42.9 25 48.2 26.2 14.3

Don’t know

7.4 21.4 32.1 3.4 9.5 14.3 5.4 15.5 23.2

Others, specify

11.1 14.3 10.7 3.4 19 10.7 7.1 16.7 10.7

A lot of flies around

7.4 0 0 6.9 2.4 0 7.1 1.2 0

N 27 42 28 29 42 28 56 84 56


Despite high knowledge levels on methods of diarrhoea transmission, fewer girls than boys were knowledgeable. For example 93.1%, 78.6% and 82.1% of boys in Binga, Hurungwe and Mwenezi knew not washing hands before eating as a method in which diarrhoea can be transmitted, compared to 74.1%, 64.3%, and 35.7% of girls in Binga, Hurungwe and Mwenezi. The main source of information are teachers for Binga (89.4%), Hurungwe (82.1%) and Mwenezi (68.6%). This is followed by the radio in Binga (23.4%), Hurungwe (23.9%) and Mwenezi (28.6%) (see Table 15). While teachers are the most significant sources of knowledge their messaging on hygiene may not be consistent (see Figure 7). For example, 37.5% and 20.2% of students had not received any information on hygiene in the three months prior the evaluation. Therefore while project interventions improved the availability of trained teachers, hygiene may not be part of a learning schedule at the schools giving credence to the earlier findings of the project’s limited impact in ensuring hygiene becomes an integral component of the curriculum at targeted schools. Table 15: Source hygiene information

Response Binga Hurungwe Mwenezi

Class Teacher 89.4 82.1 68.6

Posters or leaflets in the school 0 10.4 8.6

On TV 0 1.5 0

47

On Radio 23.4 23.9 28.6

At community meetings 8.5 3 0

When visiting a health facility 0 3 2.9

Material received at your home 0 6 5.7

Other [specify] 10.6 10.4 2.9

SMS 21.3 3 8.6

N 47 67 35

Source: Student questionnaire

Figure 7: In the last three months, have you seen, heard or received any messages or materials on water, sanitation and hygiene?


Good hygiene behaviour Assessment of hygiene behaviour in the evaluation comprised an observation of students at critical times for handwashing: after using the toilet and before eating. Finding: Consistent with high knowledge of hygiene, more boys (89.6%) than girls (86.8%) were washing hands after toilet use. The difference was more visible in handwashing with soap, boys (72.9%) and girls (61.3%). Hygiene behaviour for infant children is poor as 36.2% of ECD do not wash hands after toilet use. Presence of handwashing facilities that were absent prior the project were credited with improved hygiene behaviour while the limited access to such handwashing facilities by infant children was undermining good hygiene behaviour. Figure 8 and Figure 1

14.320.2

37.5

83.979.8

62.5

BINGA HURUNGWE MWENEZI

Not Received Received

48

Figure 9 show progress in handwashing behaviour after toilet use in selected schools. About 58.1%, 72.9% and 61.3% infants, boys and girls wash their hands with water and washing agent after toilet use (Figure 8). When districts are considered, despite lower knowledge levels of hygiene in Mwenezi than in Hurungwe, handwashing after toilet use was more prevalent in the latter than the former (

49

Figure 9). Infants are particularly disadvantaged in Hurungwe with over half of this group (52.1%) not washing their hands after toilet use. The differences could be a result of consistency in water availability at the handwashing tank which influences handwashing behaviour as noted by a group girls in Hurungwe: “Every time we see the handwashing tank we are reminded that we should wash our hands”. For those children in infant grades the difference could be a result of the handwashing tank being further away from their toilets as was noted in previous sections. Non-equitable distribution of facilities and inconsistent filling of handwashing tanks was compromising handwashing behaviour. Figure 8: Handwashing after toilet use

Source: Hygiene observation checklist

5.7

16.7

25.5

58.1

72.9

61.3

36.2

10.413.1

INFANTS BOYS GIRLS

Washing with water only Washing with water & detergent No Washing of hands

50

Figure 9: Handwashing after toilet use by district


In contrast to handwashing after toilet use, for the schools in Hurungwe where this was measured, a larger proportion of students were washing hands before eating including those from infant grades. For example, all infant grades observed were washing hands with soap before eating, with a slightly lower percentage of girls (92.0%) and boys (86.7%) doing the same. Figure 10: Handwashing before eating

8.2

24

36.4

0 0 0

39.7

61.0

47.00

100 100

95.2

52.1

15.0 16.7

0 0 4.8

INFANTS BOYS GIRLS INFANTS BOYS GIRLS

HURUNGWE MWENEZI


51


Cleanliness of toilets was observed to measure hygiene behaviour at the time of the survey. Finding: 71.4% of toilets were clean with no presence of faecal matter on the floors. Mwenezi had the most unhygienic toilets as all toilets were somewhat clean or not clean and three of the four schools had toilets with faecal matter on floors. About 57.1% (9 out of 14 schools) of schools had toilets that were clean and 42.9% were somewhat clean or not clean. The challenge was worse in Mwenezi. All four schools had somewhat clean or not clean toilets. Worrying is that three of the schools had toilets with faecal matter on floors with a significant fly problem.

4.4.2 To what extent did the project support achievement toward the MDGs? To measure contribution of the project to Millennium Development Goals (MDGs) the evaluation measured the extent to which targeted schools had reached “Basic Service” level. Finding: The project made significant contributions to MDGs on access to water and sanitation ensuring all schools achieved “Basic Service” from “No Service” but only managed to move less than half of the schools from “No Service” to “Basic Service” for hygiene. All schools had achieved “Basic Service” for drinking water and sanitation. Only 42.8% of schools had achieved “Basic Service” for hygiene. When districts are compared, Binga was the worst affected with all schools not reaching “Basic Service” for hygiene. In Hurungwe, 50% of schools had reached “Basic Service” for hygiene while all schools in Mwenezi had reached this level. Non-availability of cleaning agent (soap/soapy water) was the main drawback for provision of hygiene basic services in these schools as water was available in handwashing tanks at all schools at the time of the survey. As discussed in earlier sections, while water was available in handwashing tanks at the time of the evaluation, more often than not handwashing tanks are not filled with water due to the burden of manually filling them. So while at the time

0 0 0.0

100

86.7

92.00

013.3

8.0

INFANTS BOYS GIRLS


52

of the evaluation a majority of schools had “Basic Service”, the reality is that for long periods the schools have “No Service”.

4.4.3 What helped or hindered the project to achieve these impacts? Positive factors for project achievements Finding: Establishment of structures and capacity building at community level greatly helped to enhance the project`s efficiency The WASH in satellite schools project established WPCs, and capacitated VPMs and latrine builders in the community. These institutions contributed to project`s success through monitoring and overseeing of technical activities. The Water Point Committees set up by the project played an essential part in its success by ensuring that the communities and schools felt responsible for the constructed infrastructure (borehole). Training of WPCs strengthened the capacity of communities to operate and maintain their boreholes, thereby ensuring their long term sustainability. FGDs with WPC and SDC, revealed that WPCs in the project districts were well coordinated and ensured boreholes were always functional and hygiene was maintained. Observations made suggest that all constructed boreholes were functional and handwashing tanks were working with challenges in quality of water and availability of water respectively. Enabling this were efforts by WPCs to keep the infrastructure working despite being faced with challenges in mobilising funds to maintain borehole and replace broken tapes. The project trained Village Pump Minders at each established borehole. At least two VPMs were trained at each school and tasked with attending to the borehole. Eighteen latrine builders were supported with builders’ start-up kits and builder’s manuals which provided guidance on minimum standards. Using local builders was a strong point for sustainability, the communities and schools did not have to rely on IPs for latrine maintenance, for example. While this was the case the ability of the schools to pay for their services varies and is hampering renovation works on some toilets. At Dambanzara some toilets were destroyed by heavy rains and the roof was leaking, however, no renovations had been undertaken at the time of the evaluation. At Takunda Primary School in Mwenezi, the SDC managed to raise money to plaster the walls and resurface the floor that were damaged due to high rainfall the previous season. There were also concerns of the quality of work done on some toilets which was attributed to inadequate training of the latrine builders and in some cases supervision by EHT tasked with oversight of construction. At Chehamba primary school in Hurungwe, builders constructing a new school under contract from the MoPSE offered their help to correct mistakes made by the latrine builders regards structural strength and wind pipe. The renovations were underway at the time of the evaluation. Finding: Utilizing existing government structures and building on existing programmes such as the rural WASH program enhanced project`s efficiency The project’s efficiency was enhanced by engaging the various government departments already operating in the Rural WASH project through DWSSC (national, district and ward level structures). DWSSCs and WSSCs were trained on the components of the project to enable them to supervise building of latrines, borehole drilling, surveying and siting of boreholes and monitor water quality testing. Members of the DWSSC were also involved in the training of VPMs, latrine builders and WPCs and took leadership in targeting and district level planning working with the local NGO partner. All this enhanced ownership of the project as it cultivated greater interest from the district and hence strengthened government institutions’ capacity to monitor the project. Because of this involvement, government was actively involved in supporting the implementing partners to mobilize communities to provide locally available

53

resources for construction (pit sand, river sand, and bricks). In Mwenezi, where the partners were dealing with community resistance, DWSSC members would help by engaging the communities through EHTs. In Hurungwe the support of community leaders through the DWSSC helped convince parents to contribute towards the school construction. Without this involvement mobilising community contribution would have been a huge challenge for the project thus undermining achievement of objectives. In some instances, e.g. in Hurungwe, the project enrolled VPMs trained under the Rural WASH programme. This reduced the cost of training but may have inadvertently affected their capacity to provide adequate support to the school level boreholes because of the large geographic areas they covered. The project also piggybacked on existing interventions in WASH by the chosen implementing partners. Through the Rural WASH programme and other activities in the districts, partners had established long standing relationships at district and community levels. IPs were therefore trusted by communities, and positively perceived to be best by the various government officials. For instance, an interview with WASH NCU highlighted that Mvuramanzi Trust has been in operating in Binga since 2011 implementing the Zimbabwe Communities Approach to Total Sanitation. It was easy to coordinate and work with district officials they were already familiar with. The full involvement of EHTs ensured sustainability and success of the project. EHTs would visit sites and provide technical support to latrine builders and supervise drilling of boreholes. EHTs would combine ward visits for both the rural WASH program and the WASH in satellite schools project which was efficient in dealing with fuel shortages. Despite these efforts, EHTs continued to face transport challenges such as shortage of fuel and breakdown of motorbikes which hindered them from covering all wards during monitoring. Also, the inadequate number of EHTs available in each district presented a set back to the project. This meant that there were times EHTs were not able to carry out their tasks as planned.

“The technical support is not sufficient because the Ministry has inadequate EHTs. The District has 18 Wards i.e. 12 in Communal Wards and 6 in the Resettlement areas. There are 14 EHTs in the district and this implies that some wards do not have EHTs. Wards 13 and 17 are the biggest wards. Ward 17 has no EHT. Coverage and monitoring is a challenge. EHTs do not have enough motor cycles and some are broken down. Fuel for motor cycles is another challenge even if the EHTs repair the motor cycles at their own”. DEHO Mwenezi

Finding: Community participation in latrine building and drilling of boreholes promoted ownership which contributed to project success. Community participation in the project was strong which enhanced project effectiveness. Their contributions of labour, river and pit sand and bricks helped the project achieve the targeted number of latrines. Further, their active participation through the SDC, local leadership and WPC facilitated support for the project interventions including commitments for Operation and Maintenance costs22. For example, local leaderships were central in helping thaw impasse between communities and the project with regards provision of local materials and labour. However there some challenges (mentioned under efficiency) with distance to water source for moulding bricks in Hurungwe and Mwenezi and the unavailability of clay soil for brick moulding in within walkable distances from the school in Binga. Factors negating project performance

22 This is despite challenges in mobilising money for spare parts and payment for VPMs and latrine builders.

54

There were two sets of factors that undermine project performance: 1) those within the project’s control related to project design, implementation approaches; and 2) those outside the control of the project such as environmental factors. Finding: Challenges with dry boreholes sub-sequent challenges with the sub-contractor for borehole drilling affected the project`s success in Binga The project initially sub-contracted ZINWA to drill boreholes in all the three districts. The contract UNICEF had with ZINWA was that they would not be paid for dry boreholes. However in Binga because of water stresses and low ground water levels, drilling sub-contractor failed to yield water from the appropriate borehole sites and the contract was abandoned. A further two contractors were contracted and were able to identify high yielding holes but some distance from the schools (above the standard 500m radius). The lengthy contract cancellation and sub-sequent contracting of two additional sub-contractors protracted project implementation in Binga and necessitated a no cost extension of the project. The unavailability of water had a knock on effect on the construction of latrines and handwashing facilities. Finding: Unavailability of teaching materials/manuals on health issues for teachers to be utilized during school health clubs hindered achievement of effective dissemination of hygiene promotion information The project lacked training materials on hygiene aspects that were specific for children Government officials and Implementing partners facilitated training utilizing the PHHE manuals and teachers would adopt the manuals or schools. Teachers used the PHHE materials during the school health club sessions. The generic IEC manuals had relevant hygiene issues but the IEC materials’ pictorials were not child friendly. Whilst some schools in Binga used the generic PHHE materials schools in Mwenezi utilized IEC materials that were developed by other NGOs operating in the district. For instance, KII interview with the School Health Coordinator for Takunda Primary School in Mwenezi, revealed that they were using hygiene training materials provided by the Red Cross. This hindered project`s efficiency and affected effectiveness of dissemination of hygiene information as the trainings were not uniform and standard to all schools. Finding: A project design that did not fully integrate equity and gender undermined equitable access to WASH among the target groups. Results analysed earlier (Section 4.4.1) show differences in availability of sanitation facilities for grades 3-7 and 0-2 with more schools reaching the minimum standard for the former than the latter. In fact on average the student to squat hole was 1:22 and 1:68 for grades 3-7 and grades 0-2 respectively. As discussed under relevance and design, the determination of number of toilets per school did not consider the school population. At best the decision was based on the number of functional toilets present at the time of implementation. further the decision for inclusion of infant grades only toilets were left to the discretion of the school hence the challenge with adequacy. “We were supposed to build 13 squat holes. Out of the 13, two were for disabled and the other female and male. Out of the 11, six for females and the other five for the boys. Out of the six for the females three were for the girl friendly latrines. For ECD – these were based on the school’s request. There was no particular strategy.” Key informant interview with Implementing Partner MHM was not consistently implemented across the project as this was absent in the theory of change. This resulted in inadequate implementation of MHM across all districts. In Binga, the building of girl friendly toilets was commendable but the inadequacy led some unintended consequences in Binga of increased stigmatisation of girls at menarche. Finding: Unavailability of water and challenges in provision of washing agent by schools was undermining handwashing.

55

As highlighted under section 4.2.2, because schools’ siting did not take into consideration availability of water, a majority of boreholes were sited more than 500m from the school resulting in schools failing to ensure consistent availability of water in the handwashing tank. In some schools the quality of taps used at the handwashing were of poor quality and unable to handle the huge traffic of students (on average 381 students). Some schools would go for lengthy periods without replacing the tap. This was harming handwashing behaviour. “There are times when the tank is not working because the tap would have broken down those who have bottles of water use it to wash their hands and those who don’t they go back to class without washing their hands.” FGD with girls in a school in Hurungwe district Capacity of schools to provide the washing agent (soap or ash) at all times was a challenge across all districts but more pronounced in Binga and Hurungwe.

4.4.4 To what extent are differences in impact amongst implementation sites, groups

explained by variations in implementation? Within each district implementation from results obtained points to a level of consistency in approach. However, variations in Binga notably impacts vary with schools that benefited from the piped water schemes and those that received boreholes. In Mulambe one of the schools that got a piped water scheme for instance, uptake of handwashing was observed to be highest compared to the rest of the schools in Binga district under study. This, as highlighted by one of the councillors during an interview, may be influenced by the labour that comes with filing up of water tanks to ensure availability of water for handwashing. In the same light the same school, there is no burden on children to fetch water for teachers compared to sentiments shared by students from the other two districts in Mwenezi and Hurungwe as water in this school is just a turn away. Project sought to improve menstrual hygiene management in all schools results of the evaluation show that only two districts; Mwenezi and Hurungwe have 80% respondents highlighting that they have started their menstruation. Results however show that these two districts have sanitation facilities providing an enabling environment for menstrual hygiene management. It is only in Binga where Mvuramanzi trust was IP that has private handwashing facilities inside the latrines, where schools provide sessions for MHM and hygiene promotion material on MHM for girls. While all the three districts ensure that of the new latrines constructed were facilities to cater for children with disabilities, there are inconsistencies in the accessories installed to make the latrines disability friendly. In Binga, all schools had hand rails installed although these were not present in all the schools across the three districts. From observations also their installations were not uniform.

4.4.5 What unintended impacts (positive and negative) did the project produce? Finding: Project boreholes facilitated establishment of nutrition gardens in schools enhancing agriculture lessons and providing opportunities for income generation for schools.

Of the six schools covered by the qualitative evaluation, four had established nutritional gardens. Such gardens tended to be located proximal to boreholes constructed through project assistance, except Mulambe Primary School in Binga, where the nutrition garden was established after the setting up of piped water scheme. Of the two schools that did not have a nutrition garden, one of them, Takunda

Figure 11: Nutrition Garden set up at Muuyu Primary School

56

Primary School in Mwenezi, reported having failed at establishing a functional garden due to the water from the borehole being too salty to sustain crop growth. Respondents in targeted schools reported that the established school gardens were being considered as viable platforms for students to learn crop agriculture in a practical way. An emerging narrative from schools such as Chehamba Primary School and Dambanzara School, is that nutrition gardens are in fact “learning gardens.” Nutrition gardens were also reported to play a significant financial role in the school through income generation. In Binga District, for example, Mulambe Primary School was selling the produce from the garden to raise funds for the school’s development e.g. construction of classroom blocks and revamping toilets. As one respondent put it: “Some schools now have gardens which appear to be helping them with raising funds for various development initiatives and projects”. FGD DWSSC Binga Finding: Availability of water sources at schools was speeding up the process of building learning blocks thereby enhancing prospects for qualifying for school registration with MoPSE. The WASH in satellite schools project facilitated the construction of boreholes and sanitation facilities targeting 27 satellite schools that were significantly needy and lacking of proper WASH infrastructure which would enable the schools qualify for school registration. The project saw not only an improvement of WASH profiles but also other infrastructural development at the school such as building of learning blocks. Improved availability of water in schools made it easier and less labour intensive for parents moulding bricks for school learning block construction. In the pre-project period, it was reported that distance to water source was stalling cooperation by parents around brick making. Of the six schools visited by the qualitative survey, 3 schools located in Hurungwe, Mwenezi and Binga Districts were constructing learning blocks with others having finished building one or more blocks. These schools include: Mulambe Primary School in Binga, Takunda Primary in Mwenezi and Dambanzara School in Hurungwe. A group of SDC members reiterated that it was now easy to organise parents to provide bricks for building classrooms due to availability of water closer to the school. One SDC member commented,

“Changes as a result have largely been in the availability of water in schools, parents are now willing to participate in activities like building because water is now available in close to the school” FGD SDC Mwenezi

Finding: Establishing exclusive toilets for girls to use during menstruation increased their stigmatisation resulting in girls reverting to the old practise of missing school during their menstruation. Girl friendly toilets were built in Binga mostly. Among the latrines allocated to girls, one latrine was designed to ensure girls at menarche are able to privately, securely, and safely clean, and dispose of sanitary ware. While according to the student survey four of the five girls that were at menarche at the time of the survey strongly agreed or agreed that they happy to use the school toilets during menstruation there some interesting findings from the qualitative assessment that can have an impact on the future design of WASH in satellite schools. First having one latrine dedicated for girls at menarche during their menstruation cycle reinforced

Figure 12: Classroom Blocks under construction at Dambanzara Primary School

57

stigma associated with menstruation. Because only one latrine was available it was obvious to all what the girl who visited that latrine was doing increasing stigmatisation by boys. As a result, all school in Binga faced this challenge. As one EHT put it:

“All girl toilets should be girl friendly to reduce the stigma attached to girls using the girl friendly latrines as it will be known that they are menstruating.” EHT Binga

The challenge of stigma emanates from negative cultural beliefs. Popular local belief in Binga was that children should not come to school during menstruation because they would have to dispose of their pads leading to their blood having to come to contact with that of others, something considered taboo. Reinforcing these stereotypes was the culture of silence and secrecy around menstrual and reproductive health within the affected communities in Binga. As a result, girls seen using the girl- friendly latrines that had locked screens were laughed at, shunned by both boys and some girls who were not yet of menstruation age. Instead of using girl friendly toilets, affected girls have been forced to miss school during their periods in avoidance of stigma by school mates. While all five girls at menarche in the student survey reported they do not miss school, findings from the head teacher revealed in three out of four schools in Binga girls do miss school during their menstruation cycle. So missing school due to menstruation is still an issue in the targeted schools. Finding: While the boreholes have indeed provided needed relief to the wider community, in some cases it undermined access to water by the intended beneficiaries who are boys and girls from the targeted school. While there is an improvement in accessibility to safe and clean water by students there are some issues that were brought to the fore by the a FGD with girls in Hurungwe noting that the school borehole shared with the community has congestion during peak hours that forces that students to resort to the river.

“We are still fetching water from the river because sometimes the borehole has long queues from parents from around the community who sometimes do not allow us to get water first, rather they would want us to wait for them to finish filling their buckets with water.”

Finding: The school toilets are providing relief to households surrounding the school. FGDs with community members and community leaders revealed that a majority of households within communities (ranging from six out of ten to nine out of ten households in the community) have no toilets and practice open defecation. Toilets at the school provide relief to households living in close proximity to the school as they use the toilets during weekends when schools are closed. Community members surrounding the

“We also noted that toilets are being used during weekends by two groups of churches (SDA and Lutheran) this might cause them to block quickly.” FGD with Community members in Mwenezi

4.4.6 Which impact indicators are relevant to monitor alongside outcome indicators? Finding: Basing on the findings of the evaluation and the Theory of Change Developed for the Project, impact, output and outcome indicators were identified to be monitored by a similar project in the future in line with three result areas of: increased availability and access to safe drinking water; increased availability and access to appropriate sanitation and hygiene facilities; and increased availability and access to hygiene education.

58

As discussed before to improve the evaluation undertook a retrospective development of a results framework to assist in the measurement of outcomes and impact. The results framework was presented in Figure 6. Three key result areas were identified and already discussed in section 4.4.1:

• Results Area 1: Increased availability and access to safe drinking water;

• Result Are 2: Increased availability and access to appropriate sanitation and hygiene facilities; and

• Result Are 3: Availability and access to hygiene education. Drawing from the Results Framework, Table 16 presents indicators at outcome and impact level that could be tracked by a similar project. These indicators can be added to the current monitoring tools for WASH in Schools and further into the Education Management Information System (EMIS) (especially those concerned with equity and MHM). All the indicators will be disaggregated by gender, disability and age group (infant and non-infant grades) Table 16: Outcome indicators to monitor along impact indicators

Result Indicator Source

Impact 1: Fewer missed days of school

School attendance Head Teacher questionnaire

Impact 2: Less incidences of WASH related illnesses

Proportion of students that had diarrhoea in the last 2 weeks prior the survey

Student questionnaire

Outcome 1: Good WASH behaviour

Proportion of children washing hands with water and soap/ash at critical times (after using the toilet and before eating)

Hygiene behaviour observation checklist

Outcome 2: Increased knowledge of WASH

% of students with correct knowledge of handwashing (able to mention at least three critical times for handwashing)

Student Questionnaire

Number of schools with adequate age appropriate hygiene learning materials

Head Teacher Questionnaire

Number of schools with hygiene as an integral part of learning (i.e as a component of the core curriculum (e.g. in Science classes); as an integral part of a special module on healthy living/life skills; and as a stand-alone special module on hygiene exclusively)


Outcome 3: Increased availability of appropriate MHM materials and commodities

Number of schools providing support to girls during menstruation


Type of support provided by schools Head Teacher Questionnaire

Outcome 4: All children have access to sanitation facilities that suit their needs

Proportion of students that face difficulties in using toilets


Proportion of girls’ toilets with lockable doors Infrastructure Observation checklist

Proportion of girls’ toilets with private space for cleaning and disposal of menstrual hygiene materials

Infrastructure Observation Checklist

Number of schools with toilets allocated for children with disabilities

Infrastructure Observation checklist

Number of schools with handwashing facilities with water and soap at the time of the survey

Observation checklist

Outcome 5: All Children have access to safe water

Number of schools with water facilities accessible to children with physical disabilities


59

Number of schools with palatable water (not rusty, not salty, no odour, no colour, no particles)


Number schools with drinking water sources that can be accessed by the youngest children at school


Number of schools with water points conveniently located (within 500m from the school)


Proportion of students that face difficulties in accessing water by sex and disability


Source: Evaluation data collection tools

There are other important indicators that are important to monitor which support the results mentioned in Table 16 above. These are presented in Table 17. Table 17: Alternative indicators to monitor in a school WASH programme

Issue Indicator Source

Prioritisation of WASH by communities, schools and responsible authorities

Number of schools with annual plans including WASH


Number of schools with Parents contributing towards WASH


Number of schools with functional WPCs


Investment in WASH infrastructure by responsible authorities, schools and communities

Amount paid by school towards O&M of WASH infrastructure


Availability of safe drinking water sources

Distance of water point from school Infrastructure Observation Checklist

Availability of sanitation facilities

Number of students per functional squat hole


4.4.7 For whom (which groups), in what ways and in what circumstances did the

interventions work? Spanning across all three districts in the study children alluded to the relief of availability and access to water sanitation and hygiene facilities in their schools which creates a more conducive and healthier environment for learning for all students in all the schools.

“Before the borehole we would go and fetch water for the teachers at the river. Even now that the borehole is closer to the school, we are still sent to fetch water for them during lunch time.” FGD Boys Hurungwe

While this benefit was felt by the whole students body an FGD with boys in Mwenezi brought an eye opening dimension to the benefits of the projects being felt more by the girls.

“Boys do not go and fetch water, it is considered a girls chore. boys only fill the containers and the girls carry them.” With reduced distances from the water sources girls have a lesser burden of carrying the heavy containers for longer distances. “. Usually girls between grade 4- 7 were the ones tasked to fetch water for teachers and school use. Some girls still fetch water for teachers but it is not as often as before we had the borehole.”

Girls in Hurungwe feel such a relief as a result of the borehole drilling as they shared sentiments on the burden that they had of fetching water. They pointed out the long distance

60

they had to travel uphill fetching water for the teachers which seemingly was a mandatory task for them.

“Tinonzwa moyo wedu kurwadza nekuti pane kamufambo kuenda kunochera mvura uye kune makata anorwadza kukwira wakatakura mvura inorema” We are forced to fetch water when we are sick and tired. “Tinotsiga ma20 litre buckets zvatisingaite tiri kumba” When we are home we use scotch carts to carry water to our homes but when we are at school we have to carry water on our heads.

With the advent of improved WASH services and education for girls in schools has also been of significance in schools in the project schools and to some this has also improved the attitude they have towards coming to school translating to improved attendance. MHM as mentioned in a community FGD in Binga has been made easy for the girls because of the increase in the number of latrines which means they have enhanced privacy. Not only do they have more enabling facilities but also availability of water in close proximity for the girls to clean themselves up insinuating fewer cases of absenteeism by girls as there are now facilities for cleaning up and disposing of used sanitary ware.

“Girls also enjoying more days in schools and are not disadvantaged in any way enjoy the same privileges as the boys.” EHT Mulambe, Binga.

In Mwenezi there was a report of reduced cases of absenteeism was made, the teacher also highlighted that more education and information on MHM for the staff has also helped in assisting the girls and creating more awareness on MHM. Training of male teachers also helps in fighting the stigma associated with menstruation and is quite a huge benefit to the girls. Considerations that designs of WASH facilities have made for children with disabilities and infants (ECDs) have also largely contributed to the universality of access to WASH by all. The study also brought to the fore the lessening of the burden of carrying water by these students with special needs who usually would not be able to carry the sufficient quantities of water. An FGD in Hurungwe made mention of the sad reality of the disabled children and infants who in the past would not be able to bring sufficient water quantities to school from home As enlightened by one teacher the enrolment of ECDs has also improved with increase in access and availability of WASH facilities in schools giving the young ones a chance at enjoying access to education, a right they are entitled to. As highlighted by KII from Mvuramanzi Trust, most satellite schools are characterized by a higher proportion of temporal teachers in transit. Reasons for such high staff turnover were pointed out to be highly associated with poor facilities and services including WASH. In Hurungwe even students mentioned that some of their teachers would not live in the school premises because of long distances travelled to fetch water and poor sanitation facilities. In affirmation to the students’ observations a teacher in Dambanzara, Hurungwe said they were now staying with their family at the school. Teachers also mentioned that in the past classes would resume late because of the burden of fetching water but with the advent of boreholes in close proximity that is now a thing of the past making their work easier and increasing contact time with student.

“Children now enjoy their allocated learning period with little disruptions.”

4.4.8 What were the particular features of programme that contributed the most, and the

least to the results? As discussed earlier in the impact the project made, the setting up of key safe and accessible WASH infrastructure in all the schools i.e. the drilling of boreholes and setting up piped water schemes for provision of safe drinking water as well as the construction of safe sanitation

61

facilities in all the three districts were the features that contributed most to the results. In sentiments across the three districts from district right down to community levels are hailing the provision of water which benefited both the schools and their catchment areas. While the project earmarked schools in ensuring availability of water it also managed to meet the dire needs of communities that are battling with water shortages. As indicated by one of the EHTs most of these schools are in resettlement areas that have very little or no infrastructure at all. The availability of infrastructure enables and complements the hygiene education that is being conducted in the schools for instance the availability of handwashing is buttressed by the presence of handwashing facilities and implementation of hygiene practices.. Involvement of the community in the setting up and construction of the infrastructure was also a huge success as most of them are still functional and have contributed to the functionality of infrastructure post project implementation. The software component of the project through hygiene promotion contributed the least results because of little understanding exhibited by the students when it comes to implementation of hygiene practices. There seems to be a gap and inconsistencies in the implementation of hygiene education across the three districts and observations point to very poor adoption of good hygiene practices by children. While there is emphasis made on the importance of handwashing with soap there are very few provisions made by majority of the schools in ensuring availability of soap for handwashing. Also, results point to lack of provision of anal cleansing material for children with most using papers and other materials like stones and wood.

4.5 Partnerships and Coordination

4.5.1 What coordination arrangements for the project exist? The study revealed that IPs facilitating the Rural WASH Project in the 3 districts were selected to implement the project in the 3 districts. The project rode on the existing Rural WASH project that was due to IP experience in WASH in schools programming as well s leveraging on already existing implementation mechanisms for a smooth transition. This was also a cheaper option Value for money for UNICEF as there were no additional overhead costs that were to be incurred as highlighted by Unicef WASH officer. Existing WASH structures were used to coordinate the project at both National and Provincial levels. Rural WASH meetings were used for reports and updates on the progress of the progress. There was no specific platform created for IPs to meet and share lessons, challenges and innovations. As pointed out by all three IP project coordinators, the project had no structured platform for interaction for the IPs, informal ways of sharing information and ideas on project implementation and progress among partners were adopted. As highlighted in all 3 districts project coordination at district level was spearheaded by the DWSSCs, a multisectoral body that governs all WASH activities in the districts with technical departments such as MoHCC and DDF being focal in quality assurance. The DWSSC has a lower tier stricture at ward level that is also multisectoral composed of all extension workers operating in each ward. The survey through the DWSSC FGDs revealed that implementation at project sites was monitored and supervised mainly by the EHTs together with the respective IPs, local leadership i.e. the councillors and headmen, SDCs and teachers as well as WPCs. Reports generated at ward level are shared through the departmental heads. Leveraging on the Rural WASH project being implemented in the three districts ,From the FGDs it is evident that while DWSSCs are responsible for coordination ,monitoring and supervision there seem to be inconsistencies and glaring gaps. “ It is very difficult to get monitoring reports from the districts (DWSSC) , the district heavily relies on our officer to produce reports” Mvuramanzi Trust, Key Ïnformant.

62

4.5.2 How were the beneficiaries (including children) included in decision making and

implementation of the project? Finding: There was very little or no inclusion of beneficiaries in decision making Despite active engagement and participation during implementation, responses from all IP key informants and DWSSC FGDs indicate minimal or no consultations of beneficiaries prior to project implementation. As noted by a councillor in Binga ward, there was very little consultation of the community leadership prior to implementation. In his sentiments consultation of such critical players enhances not only implementation and coordination but also ensures that projects are contextualised to suit prevailing conditions in any community. Data triangulation concludes that no platform was created in the project for consultation of children for decision making, instead Key informants at both National and district level pointed out that the closest to involvement of children in the project was their participation in the school health clubs with survey results indicating that only 32.6% of students included in the study were members of a health club in all the three districts.

4.5.3 To what extent have partnerships’ responsibilities been fully and effectively

discharged? Findings: Partnerships’ responsibilities were fully and effectively discharged to a larger extent. Results from district engagements pointed to the efficient availing of resources for timely implementation as funds aligned to activities were reported sufficient for project implementation. From survey results it is indicative that the IPs facilitative role in all three districts was effectively executed as no district raised inefficiency or effectiveness concerns. notably in Binga interactions with DWSSC applauded timely availing of resources and provision of technical staff that assisted with infrastructure quality assurance and training during latrine construction. According to Mvuramanzi Trust KII, the project assumption was that communities would mould their own bricks for latrine construction, however, the district is characterized by sandy soils that are not ideal for production of durable bricks. Communities were to have a burden of importing from other communities with seemingly better soils was going to delay implementation. The IP thus facilitated the purchase of bricks for construction in all the beneficiary schools. This was highlighted by the DWSSC as a very good approach to implementation and significantly contributed to the timely completion of the project. All IP key informants consented to efficiency in procurement by Unicef with regard to borehole drilling. Contracting ZINWA seemed to have lessened the hurdles associated with the tendering procedures. However, Mvuramanzi Trust noted challenges with the contractor for the borehole drilling in Binga district as the project had to consider 2 or 3 other contractors resulting in boreholes being drilled post project implementation. This also resulted in untrained water point committees for those areas. Quarterly visits in all three districts were conducted as scheduled with little or no challenges noted. EHTs in all districts continuously monitored the construction of the latrines certifying standards and providing timely feedback to the district and IPs. At ward level SDCs, WPCs, VPMs were available to conduct and assist with smooth project implementation and monitoring. In all districts data indicates that majority of SDCs played a pivotal in the construction of school toilets ensuring timely mobilization of resources. In most schools success of the project was indicative of a highly effective and committed SDC. “if it was not for this SDC that worked tirelessly against all odds the project completion would have taken longer.” EHT Mabula, Binga. However, it is worth noting that there were instances of poor community participation as made mention in all three districts. In Hurungwe communities went on to reiterate that at times the locally available resources mobilisation was too labour intensive and competed with some other demands and household activities that were

63

prioritized over project activities. Also in Mwenezi DWSSC FGD highlighted some encounters of community resistance.

4.6 Sustainability

4.6.1 To what extent has capacity been successfully developed at the different levels for

sustaining the results of the project? Finding: to a large extent sufficient capacity has been developed at district and community level to sustain results of the project. Capacity is the ability of a human system to perform, sustain itself and self-renew. This working definition makes clear that capacity is not a static state or quality. It is about creating some form of added value for the members and the outside world (perform), it is about staying alive and active (sustain), it is about adjusting and developing over time (self-renew) on the basis of external pressures and internal drivers. Through various trainings conducted at community level the project sought to build and enhance the capacity of communities. Trainings targeted the VPMs, latrine builders and WPCs to enable at community level ability and knowhow of governance, operation and maintenance of infrastructure set up by the project. For every new water point in all three districts water point committees were selected and trained to ensure that there is a governing body for each water point. These were trained and as a measure of sufficient capacity built is the number of water sources that were fully functional at the time of the study is indicative of the capacity built. In this light from data collected, the longest down town recorded for these school boreholes was five days and the shortest was one day with 85% of the boreholes being fully functional at the time of study. The confidence exhibited by communities as reported through community FGDs may also be used as a measure to ascertain the sufficient capacity that structures set to oversee water points have. Analysis of data also points to a level of confidence by the WPCs and SDCs that emanates from a clear understanding of their roles and responsibilities which could also be an indication of effective training of these structures. In Binga district due to drilling of boreholes post project implementation have 3 water points committees that were not established, these however, are being run by the SDCs. 64% of the schools included in the survey reported that they had renovations to latrines done in the past year which may also point to a degree of ability of SDCs to ensure continuity with little external help. In every school it was the projects objective to train 2 school health coordinators that would ensure the functionality and running of school health clubs. With regular support mentioned by EHTs there is sufficient capacity to keep the school health clubs functional. On the other hand low retention of staff/ rampant transfer of teachers in schools may affect the capacity that schools have in retaining capacity to keep school health clubs functional.

4.6.2 To what extent are the project results (impact and outcomes) likely to continue

after the project? Finding: Capacity at district level has a bearing on the sustainability of project outputs and outcomes. Existence and functional DWSSCs in the districts is a major contribution to the sustainability of the project outcomes and impacts. The structure of coordination of WASH projects in the districts highlighted was that of the integration of the project with other existing project allows for the DWSSC s to have a surveillance system that will keep an eye and ensure the sustainability of the benefits.

64

The representatives of the RDCs highlighted that it is the mandate of the RDC to provide WASH services in the district, partners are just there to complement and support the RDC. Though where possible, the RDC will provide the necessary support to the communities. Mwenezi and Binga pointed out that in the Rural Districts’ budgeting processes there is an allocation for WASH activities and which may help to sustain the infrastructure and impacts of the project. However, mention was made of the financial hurdles faced by the RDCs of low revenue streams that may affect the allocation to the WASH sector. “Competing needs in the districts may affect the availability of funds to attend to all issues and that may have a bearing on the sustainability of the project gains” It is worth noting that DWSSC points out to budgetary constraints in the post project phase that is not funded. Line ministries DDF and MoHCC as technical departments understand their pivotal role in ensuring the sustainability of the outcomes and impacts in the districts and mentioned that their departments are fully committed to ensuring that they provide the required support to communities and schools. In Hurungwe, the DEHO highlighted that despite their commitment there are some schools in some wards that may not get the desired support because the ministry is understaffed. “The district has 18 wards 12 in communal and 6 in resettlement areas, and only 14 EHTs. There is a gap when it comes to providing technical support to schools in the wards without EHTs.” The Ministry as highlighted by Binga DEHO is mandated to provide support to the schools and communities; the presence of EHTs on the ground ensures constant and sustainable support. Participation by all member departments in the meetings was also noted to be an issue at district level. Form interactions during an FGD in Binga, participation seems to be largely influenced by the availability of incentives and allowances. In instances where there are no allowances most members of these committees at district level will not attend which then defeats the multisectoral approach design. In some instances there is also a lack of consistency in the individuals sent as representatives to meetings and this also affects the consistency in contributions to planning, monitoring and follow ups by line ministries. In most cases there is internal poor feedback and communication by the various individuals representing the ministries this approach was thus pointed out as a reason to silo operations with technical departments seemingly working without other departments. Finding: Poor community ownership of project may have a negative impact on the sustainability of the project. In identifying sustainability measures stakeholders at district level are of the notion that schools incorporate O&M costs in the development levies. The Community Based Management concept introduced has also been one way raised as a tool towards achieving sustainability of benefits not only from this project but many others implemented before. Communities according to the DWSSC have been capacitated on the essence of CBM in sustaining the impacts of community development projects. Though efforts have been made by the project implementation through sensitisation meetings as alluded to by the implementing partners, there are glaring gaps and very few communities have embraced the concept. A number of communities exhibit a donor dependence syndrome that stifles sustainability of development projects. Sentiments from one of the community leaders pointed to a poor or little ownership by the communities where feelings pointed out to the need for payment of communities by the project for the construction of school latrines in Mabula primary schools in Binga. “If builders are being paid, then the rest of the community that assisted in the construction should also be given thank you incentives for the contribution made to setting up infrastructure.” In his displeasure the same key informant also pointed out that lack of these incentives for the community disheartened the community and participation was very poor. Though it is not magnified in all three districts poor community participation as results show Hurungwe and Mwenezi slowed down the rate of completion of most activities. As such, unattended or without counteractive mechanisms to address community participation isses sustainability of project outputs and outcomes may be questionable. DSI for Binga suggested

65

that in tackling such dependency syndrome related issues projects may need to pay more attention to empowerment of local leadership, community sensitisation to enhance buy in and ownership prior to project implementation. With low levels of ownership of community development by beneficiary communities may have a huge bearing on the implementation of strategies put in place for sustainability. Finding: Presence of community level governance structures will enhance sustainability At community level, WPCs were trained on the operation and maintenance of water points. All schools in the 3 districts have functional SDCs who are working hand in glove with the water point committees in ensuring functionality and governance of the water sources. Local artisans i.e. the builders and village pump mechanics received training as highlighted by member of the DWSSCs in the FGDs, which are perceived at all levels to have been sufficient in enhancing capacity for sustainability. “we are very confident that we are going to enjoy the benefits that came with the project because there were committees set up, that have been monitoring use of infrastructure. In all schools across all 3 districts water sources service both the school and communities. Water point committees set up regulate and monitor usage of these water sources. Nonetheless, there are challenges especially during peak hours where community members get preference to students, resulting in the latter fetching water from unsafe sources. The induction of water point committees, awareness of community leaders need to ensure that children always have first preference in accessing water from the project boreholes. Finding: Schools and communities setting up policies and regulations will enhance the likelihood of sustainability of Infrastructure Together with the communities Water point committees have taken measures such as fencing and protecting the water sources to reduce vandalism and contamination. Communities and schools have contributed to the resources required for the protection of the water sources. In Hurungwe Chehamba primary school for instance, the SDC availed funds for the purchase of a fence for the water point. The presence of builders has also seen SDCs working on the repair of non-functional toilets and construction of new ones to ensure increase in availability of safe latrines. One out of the fourteen schools managed to construct new latrines while nine out of the fourteen (64%) schools in the study renovated their latrines. Communities and schools have set up regulations on the usage of water sources to ensure sustainability. This includes rationing of water usage in most low yielding boreholes. In Mwenezi, households have been forbidden to do their laundry at the borehole, and in Hurungwe the WPC ensures that borehole water is not used for brick moulding especially during the dry season when the water table is low. As highlighted in majority of responses across the three districts, the key to sustainability of project outcomes is driven by a high level of ownership by the beneficiary schools and communities and the most highlighted was that of beneficiaries having a fund set aside for Operation and maintenance costs. Results show that some communities have already set up a fund where monthly contributions are made by the members of the community for purchase of spares and payment of VPMs for maintenance. A case to note is one in Binga, Mulambe Primary school where every household benefiting from the Piped water scheme is contributing $1 towards an operation and maintenance fund. According to the EHT in the area, the community also adopted the notion that every child pays an additional $2 in levies per term for operation and maintenance costs. Another school highlighted that they have a constitution drafted for WASH operation and maintenance for the sustainability of infrastructure. SDCs through levies paid also have seen to the functionality of infrastructure in the schools notably the toilets and handwashing facilities conducting repairs and building of new

66

infrastructure where need be. In all the schools respondents highlighted that roasters are drafted where classes take turns to clean the toilets. Student respondents also reiterated that they clean their own toilets girls cleaning girls’ toilets and boys cleaning their own. Cleaning toilets has enhanced usage of the toilets and desisting from open defecation in most schools especially in Binga. All these efforts made by beneficiary schools and communities increase the likelihood of sustainability of project outcomes and impacts. Finding: Presence of local artisans has a huge contribution to sustainability of Infrastructure and project outcomes. According majority of the Water point committees in all 3 districts VPMs have also played a significant role which has contributed significantly to the sustainability of the project. Ensuring the functionality of water sources and a lesser down time in case of down time. These have been regularly carrying out regular maintenance such as greasing of the boreholes. “we do not wait for Plan International to come and apply grease for us we do it ourselves.” Responses from the schools also indicate of an acceptable down time period in the three districts. Respondents were asked if the boreholes from installation have ever broken down 9/14 schools reported that boreholes in their schools have broken down from the time they were installed. An inquiry into the down time shows the longest down time to be 5 days and the least being 2 days, with a school in Binga having the longest down time of 5 days. VPMs timeously attend to breakdowns especially those that do not require material externally. Finding: Lack of financial capacity and unavailability of spares may be a hindrance to sustainability On the flip side unavailability of spare parts is a big challenge faced by some schools and communities and stifle the sustainability of water sources. Results from key informants at schools highlighted that 2/14 schools Hurungwe (1) and Mwenezi (1) have partially functional water sources and these have been in this partially functional state for over a month. Further inquiry into the causes of this functionality state was attributed to poor initial design of the system and lack of availability of spares. A VPM in Binga pointed out the challenge of spare parts as a problem in ensuring reduced down time and functionality of boreholes, “spares are not locally available in our wards and sometimes at the District centre and have to be purchased in Bulawayo,” to add to the challenge of procuring the spares is the challenge of additional transport costs for the transportation and purchase of the spares. One of the DWSSC members form DDF alluded that sometimes the district does not have the spares required by the communities hence it becomes a huge challenge and increases down time.

“Increased costs of spare parts becomes an affordability nightmare to the already poverty stricken communities.” Binga; Ward 19 councillor.

This has a high impact on the sustainability of the water points. The issues of sustainability in the districts come with a lot of mixed feelings from various stakeholders in the districts. A case in point was made by the DSI in Binga who made mention of a school that benefited from a piped water scheme , upon damage of a few pipes by animals the scheme has since become obsolete because the school and community does not have capacity to purchase the pipes required for the restoration to functionality. The hardships faced before they benefited from this development have even become more desperate. Lack of funds/ financial capacity was reported to be the biggest hurdle for sustainability of infrastructure across all 3 districts. Key informants from MoPSE in all three districts reiterated that most schools face challenges of non-payment of school fees and levies by parents and this has a direct impact on the availability of funds to the school for operation and maintenance purposes. While one of the assumptions made by the project was that schools will be self-sustaining and prioritize WASH infrastructure operation and maintenance, schools and the SDC do not prioritise WASH investment even though they do include them in school

67

development plans. For example as earlier stated, 11 of the 13 schools that had a school development plan at the time of the visit did have some plans for maintaining WASH infrastructure. As most satellite schools are battling to obtain registered status prioritization of other activities like erection of classroom blocks takes precedence over WASH related developments. In highlighting some of the challenges faced by the MoHCC in addressing WASH issues in satellite schools, the DEHO raised an issue of failure by schools and communities to pay builders’ fees that are hefty in many instances. While some may take payment in kind usually in the form of grain some demand cash payment which is out of reach for many communities the compounding effect thus leaves WASH operation and maintenance as a secondary issue with little priority affecting sustainability. Finding: High teacher turnover in satellite schools undermines sustainability of capacity for hygiene education. As highlighted under section 4.4, because of poor working conditions in most satellite schools, teacher turnover is significantly high. Of the two to four teachers trained on hygiene education, an average of one remained in post at the time of the evaluation. Without a system of regenerating the capacity - e.g. without a standing arrangement for continuous capacity development from the district education office staff turnover in satellite schools will remain a significant risk to sustainability of hygiene education including the continuation of health clubs.

4.6.3 How effective has been the sustainability/exit strategy employed by the project? The exit strategy was to have the WASH governance structures take over the project as highlighted by DWSSCs and district key informants. Like all other WASH projects, WASH in Satellite schools in the 3 districts was coordinated and spearheaded by the DWSSCs which has still continued post project implementation through monitoring and supervision largely done by the Lower level structure WSSCs EHTs being more pivotal in the regular monitoring visits and support to the schools which is also embedded in their key result areas. Due to budgetary constraints post project implementation DWSSCs have largely depended on the extension workers for feedback without visiting beneficiary schools. Despite budgetary constraints districts have taken leverage of other existing projects like the Rural WASH project where possible to provide support to beneficiary schools.

68

5 Conclusion, Lessons Learned and Recommendations

5.1 Conclusion While project performance varied, it is clear it had a positive impact on transforming school experience for children by enhancing access to safe water, sanitation facilities and handwashing. The situation before the project was dire, and it provided a welcome relief to children and communities. The project was buoyed by a sound implementation framework that was fully driven by local governmental and non-governmental partners and communities. This framework supported efficient delivery of activities and enhanced prospects of the project being sustainable in the medium to long term. Working through multi-stakeholder structures at the district level allowed the project to access an array of skills and knowledge at low cost and ensuring the multi-dimensional WASH project had access to relevant skills and knowledge at all times. However, the inadequacies of the project in fully addressing equity (ECD and disability) and gender had negative effects on access to appropriate sanitation for girls and hygiene behaviour for ECD. Hygiene education also needed strengthening by ensuring it was systematised in the education curriculum at the schools with teachers and students having access to age appropriate education materials. Deficiencies in the project design and facilitating similarities in project implementation across the three districts had a negative effect on project wide performance. The evaluation identified variances in implementation among the three NGO partners as major contributors to subsequent variances in project performance.

5.2 Lessons learned Some key lessons distilled from the evaluation findings are as follows: 1. Effective engagement of local leadership has a significant impact on the achievement of

project results and intended objectives. 2. Investment in piped water schemes in Satellite schools increases impact of WASH in

schools project by significantly reducing the distance and labour to obtain water for handwashing, drinking and other hygiene needs but this needs to take into account the community level WASH needs when designing and implementing WASH in schools.

3. A focus on privacy for girls’ latrines only can result in boys feeling insecure in the use of their own latrines.

4. Separating a few toilets for exclusive use by girls at menarche increases stigma of these girls and resultantly limited or non-use of the especially designated toilets.

5. Establishment of good relations between school and EHTs and their involvement in the school health promotion and hygiene education fosters the effectiveness of session delivery and continuity in case of transfer of teachers.

6. Use of existing governance structures like DWSSC and WSSCC (EHTs) ensures effective coordination, implementing and monitoring of project progress.

5.3 Recommendations Recommendations based on the findings are presented according to the evaluation criteria below. The recommendations were discussed and agreed with stakeholders during validation meetings.

69

5.3.1 Relevance 1. The siting of satellite schools does not consider water availability. This posed challenges

for the project as wet holes were only identified in distances further than the recommended 500m from the school. To address this challenge it is recommended that:

c) piped water schemes would need to be installed. Responsibility: NCU, MoPSE and

UNICEF

d) the MoPSE includes, in its school planning guidelines, the assessment of availability of ground water at viable depths and water yield with the school yard or recommended distance of up to 500m. Responsibility: MoPSE

2. 35.8% of the boreholes were inaccessible for children with disabilities for various reasons

that include: failure to use lifting device and inability to get to the source without assistance which is often experienced. To improve accessibility for children with disability there is need to review the system design of the access point and ensuring water is available within the school yard through piped water schemes. Responsibility: UNICEF, MoPSE

3. Insufficient information is captured on the EMIS to facilitate planning of this important

aspect at school level. There is need to review the data collection form for EMI (ED46 form) to enable it to capture issues of support offered by schools for MHM and availability of MHM friendly latrines. Responsibility: MoPSE

4. Creation of a national standard for girl latrine and disability friendly latrines to ensure

consistency in construction of facilities. Responsibility: NAC/MoPSE

5.3.2 Validity of design 5. There were several deficiencies in the design identified by the project that include:

addressing MHM; school prioritisation of WASH; having hygiene education as an integral part of the curriculum; and climate risk management. It is recommended that a future project adopts a theory of change based on a full causal analysis of the challenge of WASH in schools. It should have a balance between actual implementation and addressing policy gaps that undermine WASH in schools. Responsibility: UNICEF

6. Several assumptions turned into risks. There is need for a future project to design a

monitoring mechanism to support monitoring of these assumptions. This should be underpinned by a flexible project that addresses a shifting context. Responsibility: UNICEF

5.3.3 Efficiency 7. While UNICEF monitors value for money through economy indices, there is need to

ensure, efficiency and effectiveness are also captured to have a fuller of the project’s performance on value for money: Responsibility: UNICEF

8. Concerns were raised by stakeholders on the adequacy of training of Village Pump

Mechanics. There is need to undertake an in-depth review of the training of these critical cadres for borehole sustainability to ascertain changes required to the training content and period and associated material and tools. Responsibility: NCU

9. The project used monitoring tools from the Rural WASH programme. While rationale in

order not to reduce an additional monitoring burden for implementers, there was need to add indicators that enable planning for WASH in schools. Responsibility: NCU

70

5.3.4 Impact 10. Though training was conducted for school health coordinators, there is a gap in the

involvement of teachers in health and implementation of good hygiene practices in schools. School health teachers should get refresher trainings and not a once off training to enhance their capacity in running effective school health clubs and improving health and hygiene practices in their schools. Responsibility: MOHCC

11. The lack of age appropriate hygiene promotion materials was apparent. The WASH sector

and partners needs to work on the development of IEC material suitable for infants and primary school level to enhance effectiveness of hygiene education for improved hygiene practices. Responsibility: MoHCC, MoPSE and UNICEF

12. It would be ideal for such projects to develop performance standards for school health

clubs to ensure functionality. These will enable assessment of performance of health club and ultimately the effectiveness of activities carried out by these organisations. Responsibility: MoHCC, MoPSE and UNICEF

13. There is need for the establishment of a national standard for hygiene promotion and

education for the implementation in schools. If all schools are to adopt hygiene education as part of the core curriculum standardisation has to be ensured for effective implementation. This also ensures sustainability in case of transfer of teaching staff. Responsibility: MoHCC, MoPSE and UNICEF

5.3.5 Sustainability 14. Governance structures at community level i.e. WPCs only received a once off training on

the running of their institutions. The lack of follow up monitoring was leading to disintegration of WPCs. Therefore projects should ensure that there are more contact sessions for these governance structures to ensure continued clarification of roles and conflict resolution that will enable the structure to continue post project implementation to avoid collapse and non-functional committees. Ensuring each WPC has a clear and implementable constitution or code of operation will also help in this regard. Responsibility: NCU

15. Communities are financially incapacitated to ensure sustainability of infrastructure.

Operation and maintenance issues should be discussed with communities during inception meetings in order for projects and communities to devise the most appropriate mechanisms for sustainability of interventions. Responsibility: NCU

5.3.6 Coordination and Partnerships 16. There was limited involvement of local leadership i.e. councillors and traditional leaders at

the onset of the project. There is need for inception meetings to ensure the presence of all relevant community leadership for coordination and community mobilisation, which enhances the ownership of the project. Responsibility: UNICEF

17. There was no formal platform created for Partners to share information on implementation

to ensure lessons sharing and contribute to standardisation across the districts. Creation of platforms for partners in similarly structured projects for sharing of notes and innovations to reduce the differences in implementation and increasing benefit to all targeted beneficiaries. Responsibility: UNICEF

18. Equity needs to be a central part of implementation of the project. A similar project should

also take into consideration the already existing developments going on in their target schools, for instance school already constructing latrines may be assisted to complete

71

without necessarily constructing new ones this may help in channelling resources to other areas that may require attention. Responsibility: UNICEF

19. Child involvement in project planning and implementation is key in ensuring

appropriateness of facilities targeting them; projects should develop ways of consultation and engagements to gather views from this target group. Responsibility: UNICEF

72

Annexes

Annex 1: Terms of Reference TORs are attached at the End of the Report.

73

Annex 2: List of Persons Interviewed Name Position Organisation District

Barnabas Mahoma WASH Manager Farm Community Trust Binga

Blood-well Rusike Program Manager Mvuramanzi Trust Hurungwe

Tapiwa Chiro Program Manager Plan International Mwenezi

Terrence Chanakira WASH Officer Focal Person

UNICEF Harare

Mr Nesbert Shirihuri National Coordination Unit

Harare

Mrs Muzenda District Administrator DWSSC Hurungwe DWSSC

Mr Muchenje MOHCC DWSSC Hurungwe DWSSC

Mr Musena MOPSE DWSSC Hurungwe DWSSC

Mr Kagoro Ministry of Youth DWSSC Hurungwe DWSSC

Mr Mlazi Chairperson DWSSC Hurungwe DWSSC

Mr Makuverere RDC DWSSC Hurungwe DWSSC

Mr Nyamasika DDF DWSSC Hurungwe DWSSC

Jesca Jeche DEHO MOHCC Mwenezi

Ms Dhliwayo District Lifelong Coordinator

MOPSE Mwenezi

Enock Mugande Social Services RDC RDC/DWSSC Binga

Pollina Ngwenya MoPSE DWSSC Binga

Sukoluhe Ngwenya DDF DWSSC Binga

Tobias Toringa Officer Mvuramanzi Trust/DWSSC

Binga

Bekezela Moyo WASH Officer Mvuramanzi Trust/DWSSC

Binga

Marinyame Farai Local Government DWSSC Binga

Magwabani Noel MoHCC DWSSC Binga

Promise S. Muleya WASH Officer CARITAS/ DWSSC Binga

Ashwin Maseko EHO MoHCC Binga

Mutale Moffat EHT MoHCC Binga

Japera Chamukwanda Councillor RDC Ward 19 Binga

Julius Mugande VPM Ward 19 Binga

Bongani Sibanda EHT MoHCC Binga

Mudimba S. J Schools Inspector MoPSE Binga

74

Annex 3: List of Documents Consulted 1. UNICEF 2014. WASH in Satellite Schools Project Document 2. Zimbabwe (2018) National School Health Policy 3. The Education Act, amended in 2006 4. Partner Quarterly Monitoring Reports 2015, 2016, 2017 and 2018 (48 reports in total) 5. UNICEF Annual Reports for the WASH in Satellite Schools Project (2015, 2016, and

2017) – three in total 6. WHO/UNICEF, Guidelines for WASH in Schools Monitoring,

https://www.unicef.org/wash/schools/files/wash_in_schools_monitoringpackage_.pdf 7. Evaluation reports in WASH in Schools for UNICEF


75

Annex 4: Evaluators Biodata Ngonidzashe Marimo has 14 years’ experience as an evaluator. He is an expert evaluator with some of his evaluation reports recommended as good practice globally. In 2015, he conducted an evaluation of the UN Joint Programme for Gender Equality and Women’s Empowerment in Uganda (involving seven agencies). The evaluation was recognised top three best evaluations by UN Women globally as is referred among good practice evaluations conducted for the agency. In the same year, Ngoni led an evaluation another UN Joint Programme on Green Jobs in Zambia led by ILO. The evaluation is also referred to as good practice within the ILO global evaluation office. Ngoni has several published evaluation reports including Evaluation of Austria Development Agency’s Uganda Country Strategy, Evaluation of the Swiss Agency for Development Cooperation Regional Programme for Southern Africa and the Impact Assessment evaluation of the HORIZONT3000 Technical Assistance Programme in Eastern Africa to mention a few. Contact: [email protected]; [email protected] Nontokozo Sibanada has 7 years experience in WASH implementation including participatory hygiene promotion. She has worked for several NGOs as project manager on WASH assignments. Rememberance Mushava was the Water and Sanitation Infrastructure Specialist. She has 15 years of experience working in water and sanitation in Zimbabwe and Africa. Nkosilathi Mpala, the proposed WASH and Public Health expert, has 15 years’ experience in WASH in schools programming. His experience spans Zimbabwe, West Africa and Asia and the pacific. The proposed Statistician, Lillian Siziba, has 20 years’ experience in evaluation design and statistical analysis. T

76

Annex 5: Evaluation Matrix Criterion Questions in TOR Detailed questions for the evaluation Evidence Data Source Methods

Relevance How relevant is the project in the addressing the needs of beneficiary communities with emphasis on the needs of girls and people with disability?

What problems did the project help schools and learners overcome in relation to WASH? What challenges did girls and CWDs face with regards WASH and how did the project help overcome them?

Project meets needs identified by project stakeholders Problems that are no longer present in selected schools Perceptions of girls and CWDs on the project’s ability to meet their needs

Learners institutional stakeholders (partners, UNICEF, Government, School administration and communities)

FGDs Case Studies Key informant interviews

To what extent is the project aligned with the policies and strategies of the GoZ at district level?

Did the infrastructure follow policy guidelines on WASH infrastructure in school (number of squat holes, menstrual hygiene management (MHM), distance to water source etc)

Number of squat holes to student population Distance to toilets Distance to water point

Head teacher Direct observation UNICEF

Structured head teacher questionnaire Key Informant Interviews Observation Checklist

Design Was the design based on a needs assessment and a context analysis?

What information was used to guide project design? Did the information provide sufficient knowledge on the situation and an appropriate TOC?

Literature used for project design Adequacy and relevance of information available

UNICEF NGO Partners Government Partners Learners Head teacher SDC DWSCC members

Key informant interviews FGDs with students

Was the design the most appropriate to meet the needs identified?

How do the underlying problems result in negative effects of lack of WASH? How does the causality chain compare with the project’s TOC? Did the project address the underlying challenges for lack of WASH in the schools?

linkage between identified problems and negative effects Comparison of causality chain identified by the evaluation and that at design Underlying challenges for WASH problems Project activities directly respond to the underlying problems

UNICEF NGO Partners Learners Head teacher SDC Community members DWSCC members Project design document Project reports

Key informant interviews FGDs Case Studies Structured questionnaire for students Observation checklist Literature review

77

Criterion Questions in TOR Detailed questions for the evaluation Evidence Data Source Methods

Does the theory of change (TOC) clearly and accurately depict the results chain? Was there a logical flow in the project theory of change?

Does the TOC provide an accurate linkage with the links between the primary and core problem and its effects? Were the assumptions in the TOC valid given the context of the project? Did the need to be changed as the project progressed?

Link between primary and core problem are clear Assumptions are comprehensive Changes in context

UNICEF NGO Partners Government Partners Head teacher SDC Community members DWSCC members Project design document Project reports

Key informant interviews FGDs Case Studies Observation checklist Literature review

Are the activities of the project sufficient and/or necessary to bring about desired change?

Were the activities based on good practice/prior experience? Did the partners have sufficient capacity to implement them? Were the activities including approaches sufficient address the underlying challenges?

Activities are based on good/global practice Partners have individuals with skills, knowledge and experience in the work, have organisation history in the work The number and type of activities addressed all the necessary challenges for the core problem

UNICEF NGO Partners Government Partners Head teacher SDC Community members DWSCC members Project design document Project reports


Efficiency Is the program worth the resources it costs?

Are infrastructure of good quality (no cracks on walls, floors, demonstration of good building practice)? What was the investment by the local community and UNICEF? Have boreholes continued work after support from UNICEF?

Infrastructure with cracks Boreholes functioning Toilets and hand washing facilities functioning Number of students and teachers benefiting Local investment by communities in the project

UNICEF NGO Partners Government Partners Head teacher SDC Observation Community members DWSCC members Project design document Project reports


Were the resources and inputs converted to outputs in a timely and cost-effective manner? On budget against agreed plans?

Were activities implemented according to work plans and budget? If there were differences, what caused the differences?

Budget compared to expenditure Project completion rates Reasons for any differences

UNICEF NGO Partners Government Partners Head teacher SDC Community members DWSCC members Project reports and work plans of partners

Key informant interviews FGDs Case Studies Literature review

How well the project apply VFM principles of effectiveness,

How was VFM used in project planning, implementation and reporting?

VFM indices used in planning and reporting

UNICEF NGO Partners

Key informant interviews

78


economy, and efficiency in relation to the delivery of its outcome?

Was it well understood by partners and shared with communities involved?

Perceptions of partners and stakeholders on VfM in the project

Government Partners DWSCC members Project reports and plans

Literature review

Was the programme management, coordination and monitoring efficient and appropriate for all levels?

What systems were put in place for monitoring the project by UNICEF, partners and DWSSCs? Were there specific tools/data collected by the system? Was monitoring data used in planning? What coordination mechanisms were in place (UNICEF, Partner and district) and were they multi-sectoral? Were mechanisms for supervision of works adequate?

Monitoring systems in place Tools used for data collection Reports reflecting monitoring data Adequacy of coordination mechanisms at all levels Mechanisms for supervision of works

UNICEF NGO Partners Government Partners Head teacher Community members DWSCC members Project reports and plans

Key informant interviews FGDs Literature review

Impact Did the WASH in Satellite Schools produce the intended (primary and secondary) results?

In line with the project’s theory of change, to what extent has it contributed to the following: Improved hygiene knowledge among students; Improved hygiene practices; Improved menstrual hygiene management in schools; reduced school absence due to diarrhoea or poor MHM at school; What changes can be observed at the teacher, learning environment and learners as a result of the training on child friendly pedagogy? What are the specific effects of the project at the school level e.g. enrolment (girls and boys), teacher retention, improved learning experience etc?

% students with hygiene knowledge # of schools with menstrual hygiene management infrastructure # of schools with menstrual hygiene management plans school attendance for girls, boys and CWD % students washing hands at critical times (after toilet use and before eating) # of schools with signs of open defecation # schools with safe water access at the time of the survey # of schools with adequate improved sanitation

Head Teacher Observation checklist Learners Teachers

Structured head teacher questionnaire Structured observation checklist Structured learner questionnaire FGD

79


facilities at the school that are single-sex and usable (available, functional and private) at the time of the survey # of schools with handwashing facilities with water and soap available at the school at the time of the survey

What helped or hindered the project to achieve these impacts? To what extent are differences in impact amongst implementation sites, groups explained by variations in implementation?

What individual, community, organisational and institutional, factors supported achievement of these results? Is the project package and implementation approaches similar across the project sites? What have been the influence of any differences?

Community factors Individual learner factors Organisation factors Institutional factors Implementation approaches and activities at each project site

UNICEF NGO Partners Government Partners Head teacher SDC Learners Community members DWSCC members Project design document Project reports


What unintended impacts (positive and negative) did the project produce? Which impact indicators are relevant to monitor alongside outcome indicators?

What were the positive and negative impacts of the project on learners?

Effects on learners Head teacher SDC Learners Teachers

Key informant interviews FGDs Case Studies

To what extent did these impacts reach all intended beneficiaries?

How are ECD, girls and boys benefiting from the project? Are there groups that are excluded and reasons for exclusion?

Benefits accruing to different categories of learners: ECD, boys, girls, and CWD

Learners Teachers Community members

Structured learner questionnaire FGDs Case Studies Key Informant Interviews

Can the changes in outcomes be explained by the project, or are they the result of some other factors occurring simultaneously?

What are the factors influencing change? Are these factors consistent throughout the sampled schools?

Reasons for outcomes by school

UNICEF NGO Partners Government Partners Head teacher SDC Learners Community members


80


DWSCC members Project design document Project reports

How do program impacts vary across implementation sites and amongst different groups (children with disabilities, women, and children)? For whom (which groups), in what ways and in what circumstances did the interventions work?

What are the specific effects of the WASH project on ECD, boys, girls, and CWD? Which districts have performed well and which performed less well?

Effects of the project on ECD, boys, girls and CWD

Head teacher Teachers SDC Learners Community members

Key informant interviews FGDs Case Studies Structured learner questionnaire

What were the particular features of programme that contributed the most, and the least to the results?

How did different features of the project contribute to results? Which contributed most or least?

Contribution of different project components

UNICEF NGO Partners Government Partners Head teacher SDC Learners Community members DWSCC members Project design document Project reports


To what extent did the project support achievement toward the MDGs?

To what extent did the project enable schools to meet basic service delivery under the JMP scale?

# of schools with Basic Service Delivery under the JMP scale

Observation Head Teacher UNICEF NGO Partners Government Partners

Key informant interviews Observation checklist

Partnership and Coordination

What coordination arrangements for the project exist?

What is the contribution of the DWSSC to project coordination? What platforms exists for lessons sharing between partners and what are the benefits?

Contributions of DWSSC Platforms for lessons sharing Benefits gleaned from platforms

UNICEF NGO Partners Government Partners Head teacher SDC DWSCC members Project design document Project reports

Key informant interviews Literature review

How were the beneficiaries (including children) included in decision making and implementation of the project?

What structured existed to facilitate beneficiaries input in project implementation? At what stage were children consulted? Are the children’s perspectives on being involved in the project?

Structured for beneficiary input into the project Structures to facilitate participation of children

UNICEF NGO Partners Government Partners Head teacher SDC Learners


81


DWSCC members Project design document Project reports

To what extent have partnerships’ responsibilities been fully and effectively discharged?

What is the performance of the different partners? Are there specific differences? What were the key drivers and constraints to coherence and coordination and how were they dealt with? Key drivers and constraints to coherence and coordination?

Partners performance with regards: implementation quality, timeliness, and monitoring and support Constraints and drivers for coordination

UNICEF NGO Partners Government Partners Head teacher SDC Learners DWSCC members Project design document Project reports

Key informant interviews Literature review

Sustainability To what extent has capacity been successfully developed at the different levels for sustaining the results of the project?

Will capacity of health clubs, water point committees (WPCs), village pump mechanics (VPMs) and latrine builders remain after the project?

structures (WPCs, VPMs, latrine builders) functional at the time of the survey Plans in place for O&M and financing plan

Head teacher SDC DWSCC members WPC VPMs Latrine builders


To what extent are the project results (impact and outcomes) likely to continue after the project?

Have teachers trained in child friendly pedagogy remained in post? Are the infrastructure likely to continue functioning? What plans are in place for O&M are these effective?

Perceptions on teacher turnover Plans for O&M and financing plan O&M plans operational and effective

Head teacher Teachers SDC DWSCC members WPC VPMs Latrine builders


How effective has been the sustainability/exit strategy employed by the project?

What exit plan is in place and the challenges and benefits of it? Has this plan been consistently implemented at all project sites?

Exit plan in place Challenges and benefits of the exit plan Project reports demonstrate progress in implementing exit plan

UNICEF NGO Partners Government Partners Head teacher SDC DWSCC members WPC Project design document Project reports ne builders

Key informant interviews FGDs

Literature review

To what extent has the long-term context of the project been taken into consideration?

How has the long term context (economic, political and social) of the project been considered in the planning and implementation of the project?

Changes in context Effects of context on sustainability

UNICEF NGO Partners Government Partners Head teacher SDC DWSCC members


82


Have changes in the context affected sustainability of project interventions?

WPC Project design document Project reports ne builders

83

Annex 6: Theory of Change of the Project

84

Annex 7: Results Framework of the Project

Expected Result

Indicators

Baseline

Target

Means of

Verification

Assumptions

Outcome 1:

8,000 pupils and 125

teachers and their

families at 27 schools

have access to safe

water supplies,

improved sanitation by

June 2017.

• Number of

pupils (girls and boys)

and teachers with

access to safe water

• Number of


and teachers with

access to improved

sanitation facilities

•

0

0

0

8,000

((4,000

girls,

4,000

boys)

300

teachers

• Drilli

ng Log

sheets

• Pum

p test

reports

• VPM

s & Latrine

builders’

training

reports

• Moni

toring/progr

ess reports

• Site

visits

Funds released

on time

Prices remain

stable

Socio-economic

and political

situation remains

stable

Maximum

community

participation

especially in

supplying locally

available

materials such

as sand and

stone

aggregates,

water and bricks

for construction

works

Output 1.1:

27 schools and

surrounding

communities have

adequate water

supplies that are within

a radius of 500 metres

of the school

• Number of

schools with

adequate water

supplies that are

within a radius of 500

metres of water

source.

• Number of

boreholes drilled

• Number of

Village Pump Minders

refreshed.

0

0

0

27

27

27

Output 1.2:

27 schools have

adequate& appropriate

sanitation facilities for

girls and boys and

children living with

disabilities

• Number of

schools with

appropriate sanitation

facilities

• Number of

builders trained

• Number of

latrines constructed

0

0

0

27 schools

27

builders

trained

324 BVIP

squat

holes

Output 1.3:

27 schools have

adequate hand

washing facilities for

girls and boys and

children living with

disabilities

• Number of

schools with hand

washing facilities

• Number of

hand washing

facilities constructed

0

0

27 schools

27 hand

washing

facilities

Outcome 2:

8,000 pupils and 300

teachers and their

families at 27 schools

• Number of


and teachers able to

0 8,000

pupils

(4,000

girls,

Reports on

hygiene

promotion

sessions

85

Expected Result

Indicators

Baseline

Target

Means of

Verification

Assumptions

are benefiting from

with hygiene promotion

by June 2017.

practice hygiene at

school and home.

4,000

boys)

300

teachers

Hygiene

Club

registers

SHC

Activity

Plans

86

Annex 8: Tools and Consent Forms

Revised

quantitative tools.docx

Revised Qualitative

Tools.docx

Consent Forms Student consent form

Hello, my name is _____________________ and I work with the MUTHENGO DEVELOPMENT SOLUTIONS. We are conducting a survey to determine experiences of students at this school and others in Zimbabwe supported by UNICEF to have toilets, boreholes and hand washing facilities. We would like to ask you questions about your schooling, water, sanitation and hygiene at school and at home. The results of this study will inform the performance of this project and preparations of similar projects. We would like to speak with you in private about these issues. All the responses you provide are confidential and will be used for the purposes of this study only. This interview is not expected to cause you any harm or discomfort. However, if you feel uncomfortable with certain questions you can choose not to answer them. We, however, hope you will participate in this survey since your views are very important. This interview will take about 1 hour of your time.

Consent 1=Yes 2=No

(FW:CHECK THAT REFUSAL IS NOT RELATED TO TIME BEING INCONVENIENT FOR RESPONDENT, PLEASE MAKE APPOINTMENT TO COME BACK AND DO THE INTERVIEW). OTHERWISE THANK RESPONDENT FOR HIS/HER TIME AND END THE INTERVIEW AND RECORD THE FINAL. STATUS OF THE INTERVIEW ON THE COVER SHEET

DATE OF INTERVIEW DD MM YYYY

TIME START HH MM

TIME END HH MM

87

TERMS OF REFERENCE FOR EVALUATION OF THE SMALL TOWNS WASH

PROGRAMME (STWP) IN ZIMBABWE

Title Evaluation of the Small Towns WASH Programme (STWP) in Zimbabwe

Purpose To determine the extent to which anticipated outputs were achieved

Location Bindura, Chipinge, Chiredzi, Chivhu, Gokwe, Gwanda, Hwange, Karoi,

Mutoko, Mvurwi, Plumtree, Rusape, Shurugwi and Zvishavane

Duration 6 months

Start Date January 2018

Reporting To Chief of WASH

Budget Code/Grant FR 9000016972

Project and Activity Codes 6260/A0/06/103/001/005

CONTEXT

Zimbabwe historically boasted one of the region’s most advanced levels of water, sanitation and

hygiene (WASH) service provision. At the start of the MDG period, almost all urban areas had piped

water supply but this has declined in recent years; access to piped water in urban areas suffered a

decrease from 98.1 to 71% over the period 2000 to 2015. Similarly, access to sewerage dropped from

47.5 to 42.8% over the same period23.

This decline has stemmed from a host of factors; 1) ageing water and sewerage networks, plants and

equipment 2) poor maintenance practices 3) lack of capital investment in water and sewerage

infrastructure 4) population growth exacerbated by rural to urban migration resulting in development

of unplanned peri urban settlement areas that are not serviced by the local authorities 5) Low billing

and collection efficiencies

Provision of water and sanitation services in urban areas of Zimbabwe is decentralized responsibility

of urban local authorities, which mandates are delegated in terms of the Urban Council’s Act [Chapter

29:15] and the National Water Policy of 2013 and the Public Health Act 15.09 Government has shifted

the mandate24 for water and sanitation service provision between ZINWA and ULA and back. The

standards for water and sanitation infrastructure for urban areas in Zimbabwe as enshrined in the

Building Models by law are high and expensive to maintain. In the face of significant economic decline,

Urban Local Authorities (ULAs) have not been able develop new municipal infrastructure in line with

the increase in urban population or undertake adequate maintenance regimes for the old

infrastructure. Local authorities struggle to maintain the high cost WASH infrastructures and yet are

23 Joint Monitoring Programme 2017- https://washdata.org/data#!/zwe 24 The GoZ shifted the mandate to ZINWA in 2006 but subsequently again restored the mandate to ULAs in 2009/10

88

unwilling to adopt more affordable lower cost option in line with the latest development in urban

WASH.

The poor status of urban water and sanitation infrastructure was one of the key reasons for the

unprecedented cholera outbreak of 2008/9 in which 4389 lives were lost25 . Poor WASH infrastructure

continues to be the underlying cause of the current cholera outbreak which has affected of 9,218

people of which 8,990 were suspected and 228 confirmed. This includes 54 deaths by 14 October

2018.

Ensuring WASH outcomes are sustained needs to go beyond hardware provision alone; this requires

multi-faceted interventions including strengthening urban local authorities’ accountabilities,

hardware provision, maintenance and upgrades; engagement with consumers for trust building and

responsiveness to WASH services; hygiene promotion and institutional support to local authorities for

effective management of WASH facilities.

SMALL TOWNS WASH PROGRAME

In 2013, the Government of Australia provided 28.96 million Australian Dollars (AUD) to UNICEF for

the implementation of an Urban WASH programme in 14 small towns (12 urban and 2 rural towns-

Bindura, Chipinge, Chiredzi, Chivhu, Gokwe, Gwanda, Hwange, Karoi, Mutoko, Mvurwi, Plumtree,

Rusape, Shurugwi and Zvishavane) to facilitate the transition from the recovery phase of the

Emergency Rehabilitation and Risk Reduction Programme (ER&RR) to development in the urban

WASH sector in Zimbabwe. The STWP is a development-focused programme that emerged as UNICEF

transitioned from the ER&RR, initiated as a humanitarian response to the unprecedented cholera

outbreak of 2008/9 . Implementation started in January 2013 and ends in December 2018.

The STWP consists of three thematic areas; 1) Rehabilitation of water and sewerage work 2) Hygiene

promotion and customer care 3) Institutional strengthening. The multi thematic integrated approach

aimed to improve and strengthen the sustainability of the water and sanitation provision by urban

local authorities and Zimbabwe National Water Authority (ZINWA) through rehabilitation of water

supply and sewer systems; improved cost recovery; hygiene promotion for WASH related disease

prevention and improved engagement between the service providers (Urban Local Authorities and

ZINWA) and town residents through improved customer care.

Key STWP activities have included:

• Rehabilitation of water intakes/abstractions systems; water treatment plans upgrade;

distribution network and water storage improvements; rehabilitation and upgrade of sewer

conveyance pipes and pumps; upgrade of sewage treatment plants

• Hygiene promotion activities including establishment of community health clubs and school

health clubs, and support to town councils for improved customer care for improved service

delivery. This involved establishing a real-time complaints and feedback mechanism.

• Provision of ICT equipment and capacity building on Promun municipal billing software, for

effective management, particularly on billing; Capacity building on WASH Operation and

Maintenance

25 Brocklehurst C, Murtaza M, Sebunya K, Salama P 2013 Engineering in the time of cholera; overcoming institutional and political challenges

to rebuild Zimbabwe’s water and sanitation infrastructure in the aftermath of the 2008 cholera epidemic Journal of Water, Sanitation and

Hygiene for Development pp 222-229 - https://www.researchgate.net/publication/273980791_Engineering_in_the_time_of_cholera_Overcoming_institutional_and_political_ch

allenges_to_rebuild_Zimbabwe%27s_water_and_sanitation_infrastructure_in_the_aftermath_of_the_2008_cholera_epidemic

89

The intended STWP outputs are as follows:

a. Improved operational capacity and efficiency of water supply systems in the target towns

b. Improved operational capacity and efficiency of sewerage systems and sewage treatment plants

in target small towns

c. Improved knowledge of safe hygiene practices among the target communities through

participatory hygiene education and with special focus on gender, vulnerability and disability

d. Enhanced human resources and technical capacity of the target towns for sustainable operation

and maintenance of the water and sanitation services

e. Enhanced cost recovery in the target towns

f. Enhanced community participation, improved accountability and responsiveness in target towns

g. Strengthened urban WASH sector coordination, planning, and management and monitoring.

The Small Towns WASH Programme has a logical framework that articulates the results the

programme intends to achieve as well as a theory of change on which shows how the change is

expected to happen (Annex 1- please double click link on ToR document or see attachment). The

logical framework and theory of change provide a description of the programmes intended results.

Implementation of the STWP was steered through existing GoZ WASH National Action

Committee(NAC) structures. At national level the programme is steered through the Programme

Management Team (PMT), a subcommittee of the Urban WASH Sub Sector of the NAC. The

Programme Management Team is chaired by the Ministry of Local Government, Public Works and

National Housing. At town level, the project was steered by the Project Steering Committee (PSC).

This committee replicates the Programme Management Team in terms of its membership

representation. The Project Steering Committee is chaired by the Town Secretary.

Programme implementation was done by civil engineering contractors and engineering consultants

for the hardware and Non- Governmental Organizations for the software aspects of the project. Civil

engineering contractors and engineering consultants conducted the rehabilitation of water supply and

sewer systems under the supervision of the resident Town Engineers. Seven Non-Governmental

Organizations (NGOs) facilitated the hygiene and customer care component of the programme.

Consultants were also engaged to support monitoring of the programme. The rights holders- the town

residents were engaged in various aspects of the project including having representation through the

Residents Association in the Project Steering Committee.

RATIONALE

Zimbabwe recognizes the human right to safe drinking water and safe sanitation. It has identified SDG

6 on availability and sustainable management of water and sanitation for all as one of its 10 priority

SDGs26. The SDG aims for higher targets beyond the MDG targets including safely managed water and

sanitation27 respectively as well as universal access for all. A direct consequence of this is that the

26 Sibanda DM, Zimbabwe Position Paper presentation on SDG, May 2016 (https://www.parlzim.gov.zw/component/k2/zimbabwe-

position-paper-on-sdgs-presented-by-dr-d-m-sibanda) 27 Safely Managed water – Improved facility located on premises, available when needed and free from contamination. Safely managed

sanitation – Private improved facility where faecal wastes are safely disposed on site or transported and treated off-site; plus a handwashing facility with soap and water

90

rapidly emerging urban and peri-urban areas which are un-served as well as the majority which also

have inadequate and limited service will need strengthened focus. This is critical as Zimbabwe did not

meet its target on water and sanitation for the MDGs28 which are lower than that of the SDG.

UNICEF has identified Urban WASH programming29 as one of the key new areas the organisation needs

to work in to contribute to meeting the needs of the most vulnerable everywhere30. Although UNICEF

has worked extensively on Rural WASH programming, the urban WASH work done has been on a far

lower scale. This requires the organization to have a good understanding of urban WASH (including to

capture, examine and document urban WASH experiences and identify knowledge gaps to inform

urban WASH programming) at national and global level which will be useful for influencing

government and donors’ WASH agenda.

Justification

Urban WASH programming in Zimbabwe is gaining increasing importance and becoming quite critical

due to the poor state, or in some cases non -existent municipal WASH infrastructure in many urban

and peri-urban areas. This is reinforced by the continued typhoid outbreak in urban areas of 2017 to

2018, and the series of cholera outbreaks experienced in 2018, with urban and peri urban areas being

the epi -center. The poor status of urban water and sanitation infrastructure was one of the key

reasons for the unprecedented cholera outbreak of 2008/9 . The joint UNICEF – GoZ STWP is one of

UNICEF’s major urban WASH Programme and has been a significant proportion of the UNICEF

Zimbabwe Country Office urban WASH portfolio, implemented in 14 towns over a five-year project

span. As the project comes to an end there is need to conduct a summative evaluation to determine

if the expected results were achieved and to draw lessons to inform future urban WASH programmes

in Zimbabwe.

Purpose of the Evaluation

This is a summative evaluation of the Small Towns WASH Project which is commissioned by UNICEF Zimbabwe Country Office (ZCO) and Government of Zimbabwe (GOZ) with the main purposes as follows:

▪ Provide evidence and lessons on UNICEF’s potential added in value, strengths and weaknesses in Urban WASH programming to guide decision making and further interventions in the sector in Zimbabwe and provide a reference point for future programme

▪ Contribute to evidence and data for improved urban WASH programme design, implementation and monitoring including best practices, challenges, barrier and success factors

▪ Provide advocacy and promote accountability through communicating to internal and external stakeholders the value of Urban WASH interventions.

▪ Enable stakeholders understand about quality programming and innovations that will emerge, in order that they may improve the programmes they support in light of the evidence. Evidence provided through this evaluation should provide assurance to all stakeholders of how investments made transform into tangible benefits for the children

Specific Objectives of the Evaluation

28 Zimbabwe MDG Final Progress Report 2015 (http://www.zw.undp.org/content/zimbabwe/en/home/library/mdg/mdg-final-progress-

report-2000---2015.html) 29 UNICEF Strategy for WASH 2016-2030, UNICEF New York, 2016

(https://www.unicef.org/wash/files/UNICEF_Strategy_for_WASH_2016_2030.PDF) 30 UN Resolutions 64/292 and 70/169 affirms the right to safe and clean drinking water and sanitation as human right

http://www.cawater-info.net/library/resolutions_e.htm

91

1. Undertake an End Line Survey comprising both quantitative and qualitative components to

assess the performance and achievements of the project against the planned project

objectives, expected results, targets and key indicators as per the logical framework and

against the STWP Baseline Survey

2. To assess impact, relevance, effectiveness, efficiency, sustainability, partnership and

coordination, and design of the overall intervention.

3. To provide evidence on the equity, gender, disability, community participation, value for

money and environment

4. To assess the major strengths and limitations of the project and draw lessons for future Urban

WASH Programme improvement (Including best practices, challenges, barrier and success

factors)

5. Determine if and how UNICEF and GoZ can scale up developments and innovations from the

STWP

6. To assess how the risks and assumptions in the project design affected the project delivery in

relation to emerging issues in the Zimbabwe context and how these were addressed

7. To assess the quality and rigor of the monitoring, learning and knowledge management

systems

TARGET AUDIENCE OF THE EVALUATION

Table 18: Target Audience and Use of the Evaluation

Target Audience Use of the Evaluation

National Action Committee of

WASH – Government of Zimbabwe

Use evaluation evidence to improve the quality of urban WASH programing and prioritization of the various components in Zimbabwe Use evaluation evidence to support improvements in institutional framework to

support Urban WASH

Contribute to the body of knowledge, lessons learnt and best practices in urban WASH for Zimbabwe

Urban Councils of Zimbabwe

(UCAZ)

Use evaluation evidence to improve the quality of urban WASH programing and prioritization of the various components in Zimbabwe

Civil Society, Practitioners

Bilateral and Multilateral agencies

active in WASH in Zimbabwe

Use evaluation evidence to improve the quality of urban WASH programing and

prioritization of the various components in Zimbabwe

Generating knowledge among practitioners

UNICEF To help guide UNICEF position itself in the urban WASH sector based on its relative strength and experience in Zimbabwe

Transparency and accountability

92

Residents of the Town Council Share lessons

Generate knowledge in the community on what works for them and what does

not

,

Uses of Evaluation Results

Table 18: Target Audience and Use of the Evaluation shows that there 4 main uses of the evaluation

as follows

i. To help guide UNICEF position itself in the urban WASH sector based on its relative strength

and experience in Zimbabwe

ii. To provide evidence to inform government and other key stakeholders including donor on

quality urban WASH programing and prioritization in Zimbabwe for the achievement of the

SDGs.

iii. To contribute to the body of knowledge, lessons learnt and best practices in urban WASH for

UNICEF global WASH programming

iv. To account for donor funding and provide transparency on what was achieved

SCOPE

The scope of the evaluation is defined as follows:

Geographical: Bindura, Chipinge, Chiredzi, Chivhu, Gokwe, Gwanda, Hwange, Karoi, Mutoko, Mvurwi,

Plumtree, Rusape, Shurugwi and Zvishavane. (all 14 towns)

Time Boundaries: January 2013 to December 2018. (entire implementation period)

Programme Components: Entire Theory of Change.

Level of evaluative analysis: Town level. Town level analysis will provide evidence to show how each

town performed and provide lessons on how the different programme components work under

different contexts.

Evaluation Criteria and Key Evaluation Questions

The consultant(s) is expected to undertake the evaluation within the framework of the OECD-DAC

evaluation criteria that respond to impact, relevance, effectiveness, efficiency and sustainability. This

evaluation will also consider partnership and coordination and design as well as use an equity lens

(specifically considering disability and gender) in the evaluation process. It should be conducted from

a child sensitive, UN Human Rights Based Approach and a Gender Responsive Approach

(http://www.unevaluation.org/document/detail/1616).

Detailed evaluation questions are listed in Annex 2. They represent UNICEF’s thinking on the essential questions that need to be investigated in order to answer the key evaluation questions. Proposers are

http://www.unevaluation.org/document/detail/1616

93

invited to comment on them and propose adjustments, deletions or additional questions as they feel appropriate. While the comments made on the annex will be considered within the technical review, the actual final decisions on the detailed questions will be taken in the inception phase, based on the following principles:

▪ Importance and priority: the information should be of a high level of importance for the various intended audiences of the evaluation,

▪ Usefulness and timeliness: the answer to the questions should not be already well known or obvious, additional evidence is needed for decision,

▪ Answerability and realism: all the questions can be answered using available resources (budget, personnel) and within the appropriate timeframe; data and key informants are available and accessible, and performance standards or benchmarks exist to answer the questions,

▪ Actionability: the questions will provide information which can lead to recommendations that be acted upon to make improvements.

METHODOLOGY

This is a summative evaluation that will follow a utilisation focused approach and a human rights and gender equality approach (http://www.unevaluation.org/document/detail/1616). Thus, the consulting evaluators should facilitate the evaluation process with careful consideration of how everything that will be done, from beginning to end, will affect use and demonstrate HR and GE mainstreaming. A baseline survey was conducted at the beginning of implementation. An end line is expected as part of this evaluation giving a pretest/post-test evaluation design. The Small Town WASH Programme has a monitoring and evaluation framework. Information sources to support the evaluation include

Table 19: Information Sources

Data Quality Reliability Produced by

Monthly progress reports satisfactory satisfactory Town Councils/NGOs

Quarterly progress reports satisfactory satisfactory NGOs, Consulting Engineer

Engineering Contracts reports satisfactory satisfactory Consulting Engineer

Field Visit Reports satisfactory satisfactory PMT

Group Maturity Index Reports satisfactory satisfactory Town Councils

Service Level Bench Marking Reports 2014, 2015, 2016

satisfactory satisfactory MoLGNHPW, World Bank

It is expected that the evaluation will occur in three phases, which are further described below: 1)

inception phase, 2) data collection and analysis phase, and 3) reporting and communication phase.

The following offers guidance on the evaluation process. It should be commented on, further

developed and improved by the bidders in their respective proposals. Alternative approaches can also

be proposed. The methodology will be further specified and finalized by the selected evaluation

consulting team in collaboration with UNICEF during the inception phase. Note however that the

evaluation should be participatory and the involvement of women and children is of utmost

importance.

http://www.unevaluation.org/document/detail/1616

94

Phase 1: Inception

▪ Convene an inception meeting between the evaluation consulting team, UNICEF Technical

Research and Evaluation Group (TREG) and Government Partners (Evaluation Taskforce)

in Harare, Zimbabwe. The meeting will review the contractual provisions and the technical

and budget requirements, familiarize the evaluation consulting team with UNICEF’s

standards, processes, and tools related to global and WASH evaluations, clarify

expectations, provide the evaluation consulting team with the available resource and

information, and refine the work plan for the next steps.

▪ Gather all necessary information and carry out a desk review of the relevant academic

and grey literature to develop an in-depth understanding of Urban WASH in Zimbabwe

and of UNICEF’s programming in this area. Relevant stakeholders will help the evaluation

consulting team access the documentation available inside and outside UNICEF. Bidders

should demonstrate their familiarity with the available literature as part of their proposal.

▪ Finalise the evaluation questions; identify the data to be collected, identifying data

collection methods and appropriate analysis for data to be collected. This includes

designing the end line survey ensuring comparability to the baseline survey data.

▪ Specify the evaluation work plan

▪ Draft and submit the inception report,

▪ Meet with the Evaluation Taskforce and the UNICEF REG to finalize and receive approval

for the inception report before commencing data collection.

Phase 2: Data collection and analysis

▪ The consultants are invited to propose various and innovative participatory methods and

tools for data collection, analysis and visualisation.

▪ Literature Review: There are a number of sets of documentation that already exist and

can be directly exploited. UNICEF will guide the evaluation consulting team to any relevant

resources at all levels. This documentation includes operational data; donor reports;

programme results and financial monitoring data.

▪ Key Informant Interviews and focus groups: These are to be conducted at national, town

and community levels. Gender, Equity and Social Inclusion should be highly regarded in

the selection of participants. UNICEF will provide a list of key informants, institutions, and

groups including government, NGOS and private sector based and the evaluation

consulting team will be invited to comment, suggest and improve. Use of voice recorders

for key informant interviews and focus group discussion a must – where consent is given.

This is in addition to other methods of recording information.

▪ Surveys: The evaluation team is expected to carry out systematic primary data collection

in the field. It is envisioned that the evaluation consulting team will conduct an in-depth

end line survey to provide understanding of the situation at the end of the project. The

endline survey should seek to answer the key evaluation questions. It should utilise both

qualitative and quantitative data collection methods. It should provide end line data that

can be compared to the base line data. The endlike survey should mirror the baseline

survey in methodology. This will provide pretest- post-test evidence to the evaluation

process.

95

▪ The sample size for the household survey in the baseline was 6 725 households drawn

from across the 14 towns.

▪ Qualitative methods included

Method Total Done

during baseline

1. FGD 85

2. KII 145

3. Transect Drive 14

4. Photos 14

5. Facility Observations 13

▪ Field Visits: The proposal should detail the proposed duration of the field data collection,

the number of evaluation consulting team members participating, and the

training/supervision/support/quality assurance components of the work considering the

geographic spread and typology of the various towns. Field observations and interviews

by the evaluation consulting team are required in order to get direct exposure to the

project.

▪ Robust and objective methods for measuring, scoring and visualising performance need to be used. The proposal should clearly show the linkages between the evaluation questions, data sources, data collection methods and analysis methods.

▪ A field report should be produced at the end of the data collection detailing the data collection experience, challenges and limitations experienced that have a bearing on the quality of data collected

▪ A detailed data analysis plan and suggested tables should be prepared and reviewed by the Evaluation Taskforce.

▪ The consultants are expected to utilise robust analysis software and methods

Phase 3: Reporting and communication of results

▪ Draft the evaluation report as per UNICEF Evaluation Report Standards.

https://www.unicef.org/evaldatabase/files/UNICEF_Eval_Report_Standards.pdf

▪ Draft the PowerPoint presentation which summarizes the content of the reports;

▪ Draft an evaluation summary (or evaluation brief);

▪ Present the first draft report to UNICEF/GoZ

▪ Draft report should have clear SMART recommendations based on evidence from the

evaluation guided by the evaluation criteria. The evaluation recommendations should

address UNICEF MORES – 10 determinants framework

(https://www.unicef.org/about/employ/files/MoRES_Briefing_Note.pdf) including a

clear HR and GE issues.

▪ Lessons learned should be documented and based on the evaluation

▪ The draft reports will be evaluated at town level by the Project Steering Committee and

at national level by the Evaluation Taskforce and the Task Force, UNICEF Research and

Evaluation Group. TREG and local authorities. The evaluation consulting team should

expect to receive and integrate about 2 rounds of comments on the draft evaluation

reports before the final report is approved by UNICEF;

▪ All final reports should be print ready.

https://www.unicef.org/evaldatabase/files/UNICEF_Eval_Report_Standards.pdf

https://www.unicef.org/about/employ/files/MoRES_Briefing_Note.pdf

96

▪ Once approved, the final evaluation report will be submitted to the UNICEF’s global

evaluation reports oversight system (GEROS) for an independent quality review. The

report and the review will be made available on the UNICEF Internet website and the GoZ-

NCU website in compliance with the commitment for transparency of evaluation findings.

▪ The Evaluation Taskforce will endeavor to provide a management response to

recommendations and an action plan with clear timelines and lines of accountability

within 2 weeks of receiving the final approved evaluation report.

▪ The evaluation consulting team is expected to participate in the dissemination of the

results. Refer to Annex 3 for a suggested dissemination plan. Bidders are invited to

comment, suggest and improve on the dissemination plan.

These terms of reference do not intend to define the methodology in detail. Rather the consultants

will be expected, as part of the bidding process, to set out his/her proposed methodology. However,

UNICEF expects the methodology to be participatory and incorporate Gender and Human Rights Based

Approaches and to adhere to certain principles and processes, set out here.

The evaluation should:

1. Ensure that all key stakeholder groups affected by the project (both directly and indirectly) are

consulted and their views considered in the evaluation. The level and form of active participation

by Government Partners should if possible go beyond consultation in order to maximise learning

opportunities. However, this will be determined by the final agreed methodology, but it is critical

that it meets this requirement.

2. Evaluation should be in line with the UNEG principles of:

a. Consultation and Inclusion: The involvement of all stakeholders is important in the

evaluation process. The perspectives of the marginalised groups (incl women, children

and children with disabilities) should be included in the evidence, and a clear picture is

provided of how the project addressed equity issues.

b. Quality of Evaluation: the evidence is generated through methods that are justifiable

given the nature of the enquiry;

c. Triangulation: the evidence has been generated using a mix of methods, data sources and

perspectives;

d. Contribution: the evidence explores how change happens, the contribution of the

intervention and factors outside the intervention in explaining change;

e. Transparency: the evidence discloses the details of the data sources and methods used,

the results achieved, and any limitations in the data or the conclusions.

5.4 Ethical Consideration

This evaluation is commissioned by UNICEF Zimbabwe and the Government of Zimbabwe. The evaluation will not require an independent ethical review. The evaluation will follow United Nations Evaluation Group (UNEG) norms and standards for evaluations, as well as adherence to ethical standard as prescribed in the UNEG – Code of Conduct for Evaluation in the UN System. www.uneval.org/document/download/548; www.uneval.org/document/download/2535

http://www.uneval.org/document/download/548

97

However, bidders are invited to identify anticipated ethical issues throughout the evaluation as well as the measures and methods adopted to mitigate them. Limitations to the evaluation One limitation to the evaluation is that

• Data disaggregated by disability may not be available in all towns

ROLES AND RESPONSIBILITIES

The Evaluation Taskforce will oversee and guide the Institution/Contractor selected to do the evaluation. All evaluation deliverables will have a two-step approval process (i) approval by the Evaluation Taskforce and (ii) approval by the UNICEF TREG which controls quality. The evaluation will be directly managed by the Head of WASH and/or his delegates. The Head of WASH and/or his delegates will be responsible for convening, coordinating and supporting the evaluation in close collaboration with all stakeholders.

Role of the Evaluation Consulting Team

The evaluation consulting team will be responsible for conducting the evaluation as described in these

terms of reference and in their proposal. The evaluation consulting team leader, more specifically,

will be in charge of the following:

▪ Oversight and management of team members,

▪ Managing communications with UNICEF,

▪ Ensuring adequate workspace, work equipment, accommodation, in-country travel, food, security

and insurance arrangement to all evaluation consulting team members and other relevant service

providers,

▪ Orienting and training of team members,

▪ Meeting deadlines and ensuring the quality of all evaluation products and deliverables,

▪ Leading the authorship of the final report,

▪ Designing and facilitating all presentation workshops.

Role of the Evaluation Taskforce

▪ Support the drafting of the ToR

▪ Support the selection process of the evaluation consulting team

▪ Provide technical input into the evaluation process (inception report, data collection tools, final

evaluation questions)

▪ Provide the evaluation consulting team with all information, resource documents and contacts

necessary for the evaluation,

▪ Facilitate the communication and coordination between the evaluation consulting team and sub-

national structures that will be contacted or visited by the evaluation consulting team,

▪ Provide comments on the key deliverables

▪ Develop a management response to the evaluation.

▪ Support the implementation of the evaluation dissemination plan

▪ Participate in all evaluation capacity building exercises

98

Role of the UNICEF TREG

▪ Participate in the key meetings with the evaluation consulting team during the inception phase

and as the evaluation nears completion,

▪ Review all key deliverables (TORs, Inception report including data collection tools, Draft Report

and Final report) and provide feedback that can be formally acted upon by the evaluation

consulting team,

▪ Advise on the management response to the evaluation

▪ UNICEF TREG will provide quality assurance on all evaluation tools and documents based on the

UNEG’s norms, standards, processes and tools and as well as on other best practices related to

UNICEF evaluations. The evaluation consulting team will be familiarized with these and is expected

to observe them during the entire evaluation process.

Role of UNICEF WASH

▪ Manage all contractual aspects,

▪ Organise and facilitate key meetings between the evaluation consulting team and the Evaluation

Taskforce and TREG,

▪ Organise and facilitate debriefing meetings as required,

▪ Facilitate communications between the evaluation consulting team and the various stakeholders

▪ Provide technical support to the evaluation consulting team,

▪ Provide a first quality review of all deliverables presented by the evaluation consulting team before

they are submitted to the Evaluation Taskforce and TREG

▪ Approving submitted work for payment upon verification that the work satisfies UNICEF

evaluation standards and guidelines, which are closely linked to those of the UNEG

EXPECTED DELIVERABLES AND PAYMENT TERMS

Table 20: Expected Deliverables and Payment Terms

Activity Deliverable Proposed Activity

Schedule

Payment Schedule

Phase 1: Inception

Activity 1: Entry meeting in UNICEF

Harare Meeting minutes Weeks 1 (1 day)

Activity 2: Drafting of inception report

including draft data collections tools. Draft inception report

Weeks 1-3 (10

days)

Activity 3: Evaluation Taskforce/ TREG

Meeting- Presentation of the Draft

Report

Meeting minutes Weeks 3 (1 day)

Activity 4: Review of Inception Report; in

cooperation of comments from

Evaluation Taskforce/ TREG

Week 4 (3 days)

99

Activity 5: Evaluation Taskforce/ TREG

Meeting- Presentation of the Final

Inception Report

Final inception report Week 5 (1 day)

25% of total

payment

Phase 2: Data collection and analysis

Activity 6: Training on Tools, Piloting and

Field Data Collection Work Field Work Report

Weeks 6-11 (30

days)

25% of total

payment

Activity 7: Data Entry, Transcription,

Coding, Analysis; drafting of the interim

report

Weeks 12-15 (20

days)

Phase 3: Reporting and communication of results

Activity 8: Presentation of the draft

report to the Taskforce, Local Authorities

and the UNICEF REG

Draft Comprehensive

Evaluation Report

comprising End Line

Survey Report and

Evaluation Findings

Meeting minutes

Weeks 16 (1 day)

20% of total

payment

Activity 9: Incorporation of comments

and circulation to local authorities and

PMT for validation and commenting

Weeks 17 (3 days)

Activity 10: Incorporation of comments

and presentation to National

Stakeholders

Workshop report

/Presentation Weeks 19 (3 days)

Activity 11: Finalisation of Report

1.Final

Comprehensive

Evaluation Report

(print ready)

2. Final End line

survey report (print

ready)

3. Abridged Evaluation

Report (max 8 pages)

(print ready)

4. Two-page summary

of key findings and

recommendations

(print ready)

5. Child Friendly

summary of key

findings (max 5 pages

print ready)

Weeks 23 (20

days)

30% of total

payment

100

6. Field Note as per

UNICEF template

highlighting the key

findings from the

evaluation (print

ready)

7. Power Point

Presentation of the

Evaluation Findings

8. Cleaned data files in

SPSS

All qualitative data

transcripts in word,

code books and

reflexivity notes and

intercoder reliability

test results

All minutes and reports will be in Microsoft Office Word format while all presentations will be in Microsoft Office PowerPoint. No PDF or hard copy will be submitted by the evaluation team.

The evaluation report should be not more than 60 pages

A comprehensive end line survey report should not be more than 60 pages

The use of bullet points, tables, graphs and other visualization methods is encouraged.

The use of annexes is required for the evaluation tools, for all secondary information that is not directly related to the evaluation findings, as well as for any long technical documentation intended to a specific audience.

PowerPoint presentations must include notes below each slide to make them easy to understand for

people who could not attend the meeting

All documentation must be in professional level standard English and in compliance with UNICEF Style Book 2015 and UNICEF Brand Toolkit 2012.

6 Estimated Cost

To be proposed by the bidder using Annex 7. The Consultant will be selected based on the “best value

for money” approach. UNICEF will effect payments for the services of the Contractor upon acceptance

of the deliverables as per the payment schedule depicted in the section “Expected Deliverables and

Payment Terms”.

Reporting

In addition to the reports described in the deliverables section. The bidder shall have two weekly

meetings with UNICEF to brief on progress and monthly meetings with the Evaluation Taskforce for

the same purpose.

https://icon.unicef.org/iconhome/ICON%20Document%20Library/UNICEF%20Style%20Book%202015.pdf

https://icon.unicef.org/iconhome/ICON%20Document%20Library/UNICEF%20Style%20Book%202015.pdf

http://www.unicef.org/ethiopia/BrandToolKit_EN_September2012.pdf

101

EXPECTED BACKGROUND AND EXPERIENCE OF CONSULTANT

The evaluation will be carried out by a gender balanced, culturally diverse evaluation team from an

institution. The institution should be a consulting firm or research institute in the business of

conducting similar work. It should have the legal authority to conduct business in Zimbabwe.

International institutions are encouraged to partner with registered local firms. UNICEF does not

provide advance payments; thus, the institution should demonstrate ability to deliver within the

payment framework shown in Table 3 on Expected Deliverables and Payment Terms.

The number and profile of the evaluation consulting team members should be such that the team size,

experience, qualifications, mix and complementarity of expertise, availability and level of effort is

convincing that the proposed work plan can be met. The following is suggested as a guidance:

1. The evaluation consulting team—which must have a designated evaluation leader--should

include the following competencies:

Required:

▪ Be an evaluation specialist with a minimum of 10 years’ experience in programme

evaluation in a development context;

▪ Have technical expertise in high-level and high-quality evaluations and studies;

▪ Have an ability to design and plan the evaluation approaches and research

methodologies, including quantitative and qualitative research methods;

▪ Have relevant subject matter knowledge and experience;

▪ Have appropriate country knowledge and/or experience.

The lead evaluator must ensure that the team is able to cover all ToR areas of interest within the

proposed timeframe and to an adequate level of coverage and depth and rigor.

▪ There should be adequate expertise within the team of the principle technical areas of

the Projects’ thematic areas (Municipal WASH Infrastructure, Institutional Strengthening

for Urban WASH, Hygiene Promotion and Customer Engagement);

▪ Knowledge of the country context;

▪ Expertise in human rights and gender responsive approaches (with evidence of use of

various analytical frameworks).

▪ Strong qualifications in statistics and data analysis;

▪ Strong expertise and experience in designing and implementing Urban WASH

programmes with a distinct advantage for team members that have acquired this

experience in Africa

▪ Fluency in English

Considered an advantage:

▪ Advanced Qualifications in Monitoring and Evaluation, development studies, public

health, engineering, social sciences, environmental health, Education

▪ A Public Health/WASH specialist, Engineer, Social Scientist, Statistician, gender

specialist/ Gender Equity Social Inclusion (GESI) specialist or other relevant disciplines.

▪ Prior experience in similar work

In addition to details in the technical proposals summaries of available expertise and relevant

experience should be highlighted in Annexes 3-6.

102

EVALUATION CRITERIA

TECHNICAL PROPOSAL FROM

POINTS

AWARDED

80

1. OVERALL RESPONSE

- Demonstrated understanding of requirements, objectives and

deliverables.

- Demonstrated understanding of scope, including ability to perform the

work assignments articulated in this RFP.

- Adequacy of the technical plan demonstrated through the overall concord

between RFP requirements and the Proposal submitted.

- Demonstrated understanding of the situation of Urban WASH in

Zimbabwe

- Work plan, Methodology and Timeline.

- Overall clarity and completeness of the Proposal.

5

2. PRESENTATION OF TECHNICAL PROPOSAL TO TECHNICAL EVALUATION

PANEL

- Quality, delivery and effectiveness of presentation of the Technical

Proposal with emphasis on the study methodology and other relevant

details to the Technical Evaluation Panel.

15

3. STRATEGY/METHODOLOGY

- Quality of proposed approach/methodology.

- Quality of proposed Implementation Plan, i.e. how the institution/

consultants will undertake and execute each stage, with proposed project

schedules

15

4. PROPOSED TEAM AND PROFESSIONAL ORIENTATION

- Structure, qualifications, experience and skills of proposed Team to

undertake this assignment.

- Team leader: relevant experience leading similar projects and

qualifications

15

- 5. Evidence-supported TECHNICAL CAPACITY

- Range and depth of relevant experience in conducting similar evaluations

including experience in research in development sector.

- Sample of previous evaluation/study conducted in similar context

NOTE: Institutions/Contractors that fail to submit relevant documentation that

will enable a thorough review of all relevant areas, do so at their own risk and

will be subject to disqualification or penalty of reduced points. The assessment

of the Institution will be made on the basis of documents submitted for review in

the Proposal.

20

103

TECHNICAL PROPOSAL FROM

POINTS

AWARDED

80

6. Capacity of Institution

• Financial status (please provide financial statements to enable judgements

on the capacity to prefinance)

• Registration of company (for international institutions- it includes

registration status of the local partner)

• Company/ Institutional profile which clearly demonstrates capacity to

conduct evaluations of complex projects and experience with similar work

10

POSSIBLE MARKS - TECHNICAL PROPOSAL FROM

Pass mark (80%) 80

FINANCIAL PROPOSAL 20

Weightage is allocated to the lowest priced proposal. The financial scores of the

other proposals will be in inverse proportion to the lowest price

Considerations:

Cost benefit comparison related to number and quality of personnel in the

proposal who will conduct the evaluation.

Completeness of the financial proposal (ensure that all costs, including

professional fees, costs of travel, salaries, insurance etc. are included in the price

offered)

Timeline proposed

NOTE: As the most financially attractive offers will be at an advantage, if the

institution/consultancy is open to discounts and price negotiations, this should

clearly be marked in the financial proposal to facilitate the comparison

The overall minimum pass mark is 70%

General Conditions: Procedures and Logistics

Official Travel Involved

The consultants will be required to make a presentation of their proposal, either in person or through

use of skype or similar technology options to the technical committee as part of the selection process.

104

Negotiations/Technical discussion and other pre‐award activities and expenses are to be borne by the

Institution/Contractor and should not be included in the Financial Proposal submitted.

For the entire period of this project/contract all travel related to any contract emerging from this RFP,

including travel for consultations and discussions, meetings for planning and execution of potential

works etc. will have to be borne by the Institution/Contractor and must be included in the Financial

Proposal submitted. The Institution/Contractor is responsible to make all relevant travel

arrangements, including hotel, transportation and other relevant costs.

UNICEF may, as a courtesy, facilitate through the provision of a letter of certification that travel is

related to this project/contract, provide/indicate names of hotels/guest houses used by UN, but all

arrangements are the responsibility of the Institution/Contractor, both financially and action‐wise.

Institution/Contractor’s Work Place:

The institution is expected to work from its own premises, with official travel to target towns when

necessary.

Policy both parties should be aware of:

• Members of the contracting company are not entitled to payment of overtime. All

remuneration must be within the contract agreement.

• No contract may commence unless the contract is signed by both UNICEF and the

Contractor.

• No member of the contracting company may travel under the consultancy contract prior to

contract signature.

• Members of the contracting company will not have supervisory responsibilities or authority

on UNICEF budget.

• Contracting company will be required to sign the Health statement for

consultants/Individual contractor prior to taking up the assignment, and to document that

they have appropriate health insurance, including Medical Evacuation.

105

Annex 1: SMALL TOWNS WASH PROGRAMME LOGICAL FRAMEWORK

ANNEX 2: LIST OF DETAILED EVALUATION QUESTIONS

Criteria Definition of criteria

(OECD)

Key evaluation questions

Impact Positive and negative changes produced by a development intervention, directly or indirectly, intended or unintended.

• Did the Urban WASH programme produce the intended (primary and secondary) results? What helped or

hindered the project to achieve the results?

• What unintended impacts (positive and negative) did the project produce? What helped the project to achieve

these impacts?

• To what extent did these impacts reach all intended beneficiaries? For whom (which groups), in what ways and in

what circumstances did the interventions work?

• How do program impacts vary across implementation sites and amongst different groups (children with

disabilities, women, and children)?

• To what extent are differences in impact amongst implementation sites, groups explained by variations in

implementation?

• To what extent did the project support achievement toward the SDGs?

Relevance The extent to which project is suited to the priorities of the target beneficiaries.

How relevant is the project in the addressing the needs of beneficiary communities with emphasis on the needs of girls and people with disability?

• To what extent is the project aligned with the policies and strategies of the GoZ at local authority level?

Effectiveness A measure of the extent to which an aid activity attains its objectives.

• Were the planned objectives and outcomes in the project document achieved?

• How and to what extent has gender, disability and environment been applied in the implementation of the project?

• How and to what extent can observed results and changes in outcomes be attributed to interventions.

106

• What percentage of the needs was covered by the project?

• Quality of hardware?

• What are the key drivers and barriers affecting the delivery of results for the project?

Efficiency Signifies that the aid uses the least costly resources possible in order to achieve the desired results.

• Is the program worth the resources it costs?

• Were the resources and inputs converted to outputs in a timely and cost-effective manner? On budget against

agreed plans?

• How well the project apply VFM principles of effectiveness, economy, efficiency in relation to the delivery of its

outcome;

• Was the programme management, coordination and monitoring efficient and appropriate for all levels

Sustainability

Measuring whether the benefits of an activity are likely to continue after donor funding has been withdrawn

• To what extent has capacity been successfully developed at the different levels for sustaining the results of the

project?

• To what extent are the project results (impact and outcomes) likely to continue after the project?

• How effective has been the sustainability/exit strategy employed by the project? To what extent has the long-

term context of the project been taken into consideration?

Partnerships and Cooperation

The extent to which partnerships have been sought and established, and synergies been created in the delivery of assistance.

• What coordination arrangements for the project exist?

• How were the beneficiaries (including children) included in decision making and implementation of the project?

• To what extent have partnerships’ responsibilities been fully and effectively discharged?

• What were the key drivers and constraints to coherence and coordination and how were they dealt with?

Design The extent to which the project design is appropriate and cost effective in bringing about the desired change

• Was the design based on a needs assessment and a context analysis?

• Was the design the most appropriate to meet the needs identified?

• Does the Log Frame clearly and accurately depict the results chain?

• Are the activities of the project sufficient and/or necessary to bring about desired change?

107

The questions posed in this ToR represents the key questions stakeholders need investigated in order to provide essential information for their needs. Bidders are invited

to comment on them, propose adjustments or additional questions as they feel appropriate. Do not delete the questions provided. While proposed adjustments and

additions will be considered during the technical review, the actual final decisions on the questions will be taken in the inception phase, based on the following principles:

• Importance and priority: the information should be of a high level of importance for the various intended audiences of the evaluation,

• Usefulness and timeliness: the answer to the questions should not be already well known or obvious

• Answerability and realism: all the questions can be answered using available resources (budget, personnel) and within the appropriate timeframe; data and key

informants are available and accessible, and performance standards or benchmarks exist to answer the questions,

• Action ability: the questions will provide information which can lead to recommendations that be acted upon to make improvements.

108

ANNEX 3: Draft Dissemination Plan

Stakeholder Use of Evaluation How disseminated Format Responsible for

Dissemination

By When

National Action

Committee of

WASH –

Government of

Zimbabwe

Use evaluation evidence to improve the quality of urban WASH programing and prioritization of the various components in Zimbabwe Use evaluation evidence

to support

improvements in

institutional framework

to support Urban WASH

Contribute to the body of

knowledge, lessons

learnt and best practices

in urban WASH for

Zimbabwe

Review/Validation

workshops

Final evaluation work shop

NCU Website

Abridged Evaluation Report

(max 8 pages) (print ready)

Two-page summary of key

findings and

recommendations (print

ready)

Power Point Presentation

Evaluation Taskforce Within 2 weeks of

final report

Urban Councils of

Zimbabwe (UCAZ)

Use evaluation evidence

to improve the quality of

urban WASH programing

and prioritization of the

various components in

Zimbabwe

Review/Validation

workshops

Presentations

Pamphlets




findings and


ready)

Evaluation Taskforce 2 weeks of national

level dissemination

109


Civil Society,

Practitioners

Bilateral and

Multilateral

agencies active in

WASH in

Zimbabwe

Use evaluation evidence

to improve the quality of

urban WASH programing

and prioritization of the

various components in

Zimbabwe

Generating knowledge

among practitioners

Review workshops

Final evaluation

conference

Website

1. Final Comprehensive

Evaluation Report (print

ready)

2. Final End line survey

report (print ready)

3. 2- pager pdf- Key results and recommendations

Presentation ppt


final report

UNICEF To help guide UNICEF position itself in the urban WASH sector based on its relative strength and experience in Zimbabwe

Transparency and

accountability

Review workshops

Final evaluation

conference

Website

Article in local Sector

Newsletter

Field Note

Summary Emailed to

Brown-Bag list serv

Full Final Report

Presentation ppt

Articles

Evaluation Taskforce 4 weeks after

presentation to

project stakeholders

110

Urban Local

Authorities

Use evaluation evidence to improve the quality of urban WASH programing and prioritization of the various components in Zimbabwe Use evaluation evidence

to support

improvements in

institutional framework

to support Urban WASH

Contribute to the body of knowledge, lessons learnt and best practices in urban WASH for Zimbabwe

Review/Validation

workshops

Final evaluation work shop

NCU Website




findings and


ready)



final report

Residents of the

Town Council

Share lessons

Generate knowledge in

the community on what

works for them and what

does not

Distribution of pamphlets

Distribution of child

friendly reports

Child friendly report

Pamphlets

U- Report

111

ANNEX 4: TEMPLATE FOR THE PRESENTATION OF BIDDER’S REFERENCES

➢ If the Bidding Institution/Contractor already has its own template, it may simply use that template in the submission as long as all the requested

information is present.

➢ This is the minimum required information for each reference. Additional information beyond the minimum may be provided at Bidders’ discretion.

➢ Bidders to submit a sample report of similar work undertaken (just 1 report)

Brief description of the assignment (Title, objective and scope):

Country(ies) of assignment:

Location (region, district, commune):

Client’s name, address, and valid email and phone number:

Start date (month/year):

End date (month/year):

Duration of the assignment (in months):

Approximate contract value (in USD):

Key experts mobilised for the assignment (number, names):

112

ANNEX 5 – TEMPLATE FOR THE SUMMARY PRESENTATION OF PROPOSED KEY TECHNICAL EXPERTS

➢ This a compulsory template and must be completed and submitted as part of the proposal in addition to the detailed CVs.

Function in the

evaluation Name Age

Level of seniority

Number of year of relevant professional

experience Specific role

e.g. Team leader

e.g. Water supply

specialist

e.g. Gender Specialist

Level of seniority:

− Senior = more than 10 years of relevant professional experience

− Intermediate = between 6 and 10 years

− Junior = 5 years or less

113

ANNEX 6 – TEMPLATE FOR THE LEVEL OF EFFORT OF ALL PERSONNEL BY ACTIVITY

➢ This is a compulsory template and must be completed and submitted as part of the proposal.

➢ Bidders are asked to list in this table all personnel that will be mobilised for the evaluation,

not only the key technical experts. This includes evaluation/research experts, assistants, etc.

For research assistants please indicate their name if known, their function and level of effort

by activity.

Number of Days

Experts Function in the

evaluation

Phase 1 Phase 2 Phase 3 Total person/

days

Name 1 Team leader

Name 2 WASH Specialist

Name 3 WASH

Specialist/Local

Name 4

Name 5

Name 6

Name 7

Etc. … …

Total number of person/days

114

ANNEX 7 – TEMPLATE FOR THE STATEMENT OF AVAILABILITY OF SENIOR AND INTERMEDIATE

EXPERTS

➢ This is a compulsory template and must be completed and submitted as part of the proposal.

Understanding UNICEF’s concern that the senior and intermediate level experts listed in the proposal

actually be available for the assignment and not be already committed to other responsibilities and

activities incompatible with this assignment, the Bidder commits that if awarded the contract, the

named senior and intermediate experts will be available for the period and time necessary to fulfil

their responsibilities. UNICEF reserves the right to terminate the contract should the key experts listed

be unavailable for the assignment after contract award.

Name of expert 1:

Title / Function:

Signature:

Date:

Name of expert 2:

Title / Function:

Signature:

Date:

Name of expert 3:

Title / Function:

Signature:

Date:

Name of expert 4:

Title / Function:

Signature:

Date:

115

Annex 8- Budget Template(bidders may export the template to excel)

BUDGET TEMPLATE

Activity Unit

Number

of Units

Unit

cost Total Cost

Budget Notes

(explain the

costs in detail)

1. Consultancy Fees (Specify)

e.g. Team Leader

e.g. Gender Specialist

Sub- Total Consultancy Fees

2.Contracted Staff (specify)

e.g.

Sub- Total Contracted Staff

3.Planning and Inception

Sub- Total Planning and

Inception

4. Training

Per diem and Allowances (specify

break down per budget line if

different for each staff category

e.g. per diem consulting staff;

per diem contracted staff, per

diem driver etc.)

Stationery

Transport (specify costs)

Other costs (specify)

Sub- Total Training

5. Data Collection

Per diem and Allowances (specify

break down per budget line if

different for each staff category

e.g. per diem consulting staff;

per diem contracted staff, per

diem driver etc.)

116

Stationery

Transport (specify costs)

Other costs (specify)

Sub- Total Data Collection

7. Data Entry and Analysis

Sub- Total Data Entry and

Analysis

8. Report writing and

presentation

Sub- Total Report Writing

and Presentation

9. Other Costs (specify)

Sub- Total Other Costs

(specify)

GRAND TOTAL

Date post:	11-Mar-2020
Category:	Documents
Upload:	others
View:	0 times
Download:	0 times