External evaluation of ACF’s project,
“Improve Access to Water, Sanitation
and Hygiene in the Ger areas of
Ulaanbaatar, by providing proven and
innovative solutions”
May 2009-April 2012 (first phase)
May 2012-June 2015 (second phase)
June 2015
Julie Patinet Anne Delmaire
2
Final external evaluation of the ACF project Improve access to WASH in the Ger areas of Ulaanbaatar, Mongolia
Groupe URD - TDM | June 2015
Groupe URD is a support structure for the humanitarian sector. For over 20 years it has endeavoured to improve the aid practices
for the populations affected by crises, with several types of activity: operational research projects, programme
evaluations, methodological tool design, institutional support and training processes, both in France and abroad.
Toilettes du Monde has worked since 15 years in France and abroad with public and private sanitation players to improve the hygiene
and living conditions of underprivileged populations by promoting the development of ecological sanitation (dry
toilets, grey water) through professional training actions, action research, bibliographic studies, techniques,
regulations, socio-cultural actions, lobbying the authorities.
Note on the authors Julie Patinet is in charge of "Environment" research in Groupe URD. She studied Geography and has a Master’s degree
in Geomatics and Development. She is a qualified Risk Prevention Territorial Engineer and worked in French local
development associations before turning to the international aid sector, specializing in the field of ecological
sanitation.
Anne Delmaire is a benchmark in social project management concerning ecological sanitation and solidarity. She
joined the TDM team in 2010 (and is now the general manager) after working for over 7 years in Mexico as a
coordinator of water and ecological sanitation projects. She trains trainers in participatory methodologies such as
MARP, PHAST and SARAR for the social and technical implementation of WASH projects and has created tools for
workers on the ground. She has likewise carried out methodological support and distance techniques.
Acknowledgements I would like to express my sincere thanks to all those who took part in this evaluation and took the time to answer my
questions, as well as for the logistical support that I received, and the confidence that I was shown. I would
particularly like to thank Pier-Francesco Donati and Federica Voltolini for their extremely warm welcome, their
availability and the quality and nature of our discussions.
Groupe URD
La Fontaine des Marins
26170 PLAISIANS
Tel : + 33 (0)4.75.28.29.35
Fax : + 33 (0)4.75.28.65.44
E-mail : [email protected] | www.urd.org
Toilettes du Monde
28, place des Arcades
26110 NYONS
Tél : +33 (0)4 75 26 29 98
Fax : +33 (0)4 75 26 19 02
E-mail : [email protected]
http://www.toilettesdumonde.org
This evaluation was financed by ACF.
The opinions expressed in this report are those of the authors alone.
© Groupe URD | July 2015
3
Final external evaluation of the ACF project Improve access to WASH in the Ger areas of Ulaanbaatar, Mongolia
Groupe URD - TDM | June 2015
Table of contents
Abbreviations/Acronyms _____________________________________________________ 5
Executive summary __________________________________________________________ 6
1. The project and its context ________________________________________________ 8
Characteristics of the area of intervention _______________________________________ 8
Brief outline of the project ___________________________________________________ 10
2. Evaluation objectives and methodology ____________________________________ 12
3. Between the interim and final evaluations… _________________________________ 13
4. Results based on the OECD DAC criteria _____________________________________ 16
Coherence /Coordination: the project is coordinated well with its institutional environment
_________________________________________________________________________ 16
A project which is coherent with the new urban development plans ___________________________ 16
ACF an important player in WASH coordination ____________________________________________ 17
Internal coherence of the project ______________________________________________ 18
Relevance: decentralised solutions adapted to the capital’s Ger areas________________ 19
Effectiveness: an ambitious development project ________________________________ 20
Impact ___________________________________________________________________ 21
Impact on behaviour change at the household/community level ______________________________ 21
Impact on a larger scale and at the institutional level _______________________________________ 23
Impact of the advocacy strategy ________________________________________________________ 23
Development of the advocacy strategy ____________________________________________________ 24
Analysis of the content of the advocacy strategy____________________________________________ 24
Analysis of the advocacy strategy process _________________________________________________ 26
Degree of autonomy of local structures to take over kiosks, emptying services, and
composting sites ___________________________________________________________ 27
Lack of anticipation in ACF’s exit strategy _________________________________________________ 27
Multi-service water kiosk managed by Tolgoit for almost three years __________________________ 28
Major concern about the sustainability of MonESIC taking over the ecological sanitation service ____ 29
WaSH Action, a young NGO that is still fragile, but has a lot of potential ________________________ 37
4
Final external evaluation of the ACF project Improve access to WASH in the Ger areas of Ulaanbaatar, Mongolia
Groupe URD - TDM | June 2015
Conclusion and recommendations _____________________________________________ 41
Conclusion _________________________________________________________________________ 41
Recommendations for ACF: ____________________________________________________________ 41
Recommendations for WaSH Action: ____________________________________________________ 42
Recommendations for MonESIC: ________________________________________________________ 43
Recommendations for the Water Agencies: _______________________________________________ 43
Recommendations for all actors involved in advocacy: ______________________________________ 44
Annexes __________________________________________________________________ 45
Annex 1: Best practice: Hygiene promotion ______________________________________ 46
Annex 2: Classification according to the DAC criteria _______________________________ 48
Annex 3: People met ________________________________________________________ 50
Annex 4: timetable _________________________________________________________ 52
Annex 5: Bibliography _______________________________________________________ 53
Annex 6: Workshop with the ACF / WaSH Action national staff ______________________ 55
Annex 7: online survey _______________________________________________________ 59
5
Final external evaluation of the ACF project Improve access to WASH in the Ger areas of Ulaanbaatar, Mongolia
Groupe URD - TDM | June 2015
Abbreviations/Acronyms DAC Development Aid Committee
EPA U.S. Environmental Protection Agency (who regulates the maximum Ecoli quantity accepted)
GADA Ger Area development Agency
Ger Mongolian term for "yurt"
IEC Information, Education, Communication
Khooro neighbourhood
MIRIM Mongolian branch of IRIM: Independant Research Institute (marketing consultants)
M&E Monitoring & Evaluation
MNT Tugrik (Mongolian currency) 1€ = approx 2 000 MNT
NGO Non-governmental organisation
OECD Organisation for Economic Cooperation and Development
SDC Swiss Agency for Development and Cooperation
SWOT strengths, weaknesses, opportunities and threats
TDM Toilettes du Monde
Tolgoit Grassroots community organisation with the name of the neighbourhood
UB Ulaanbaatar
USTB University of Science and Technology Beijing
USUG Public company responsible for supplying water and sanitation services in the city of Ulaanbaatar
WASH Water, Sanitation and Hygiene
WC Water Closet (flush toilet)
WECF Women in Europe for a Common Future
6
Final external evaluation of the ACF project Improve access to WASH in the Ger areas of Ulaanbaatar, Mongolia
Groupe URD - TDM | June 2015
Executive summary
Main Observations / Conclusions Main Recommendations
Su
sta
ina
bil
ity
ACF’s exit strategy was not sufficiently anticipated and
will have a different impact on each local partner. ACF’s
withdrawal will have an obvious negative effect on
MonESIC, who will not meet all the challenges facing it
without external support. The pilot solutions have been
tested technically, but have not been tested socially for
long enough.
Tolgoit is likely to continue to manage the multi-service
kiosk in the long term.
For ACF: the next time ACF implements a project on
the margins of its mandate, it would be a good idea
to anticipate the exit by working with local partners
sufficiently in advance.
For ACF in connection with this project: maintain
strong links with WaSH Action, in the form of a
partnership.
ACF should invest one last time in the EcoSan sector
and support social project management activities,
without which the service is in danger of collapsing.
WaSH Action should also support the EcoSan sector
in the short term, as it will help to make it better
known.
For WaSH Action: in the short term : survive ! and
for the longer term: pick up the torch of WaSH
advocacy
WaSH Action has a lot of potential, but is still very
fragile and still has too little financial visibility. One of
the challenges for this young national NGO will be to
take over the advocacy strategy and to adapt it to their
means.
- the pursuit of the implementation of the advocacy
action plan should not be WaSH Action’s main
priority in the months ahead; but it should
nevertheless remain in the background (with
activities which do not require any budget) because
it is in keeping with Wash Action’s mission (vision)
and allows it to remain visible and involved vis-à-
vis key decision-makers.
For the Water Agencies and the other potential
financial partners: in order for advocacy to be able
to continue, it is NECESSARY to have financial
support for WaSH Action.
Imp
act
Several improvements to national regulations directly
attributable to ACF and its advocacy activities.
Sanitation: the project led to changes in behaviour
(acceptance) vis-à-vis the ecological sanitation
approach, but this remains at the pilot level, which
could well be abandoned if there is no investment in
social project management (households’ enthusiasm
for the EcoSan toilets proved to be reversible as soon
as they had to pay for the service).
Hygiene: excellent long-term impact, which has led to a
drop in the number of illnesses in the targeted schools.
7
Final external evaluation of the ACF project Improve access to WASH in the Ger areas of Ulaanbaatar, Mongolia
Groupe URD - TDM | June 2015
Co
he
ren
ce
ACF’s decentralised solutions remain relevant with
regard to the new urban development plans: the two
types of solution (large-scale decentralised public
infrastructure vs decentralised solutions at the
household level) are, in fact, complementary.
It is not likely that all the Ger areas will be transformed
into apartments and connected to the network.
The models that have been tested can meet the WaSH
needs of people in the Ger areas in the short/medium
term.
Recommendation for all actors involved in advocacy
activities: include the need for public showers in
advocacy vis-à-vis the institutions (to be included in
the urban development and facilities plans for the
Ger areas).
Long term recommendation for WaSH Action:
resume action research into improving the
management of grey water (look for partnerships
with researchers + volunteers +funding).
It would be useful, in terms of advocacy, to compare
the different WaSH solutions (economically,
environmentally, and in terms of job creation) and
how they meet the short/ medium/ long-term
needs of the inhabitants of the Ger areas.
Re
lev
an
ce/
ap
pro
pri
ate
ne
ss
The project meets people’s essential WaSH needs, even
though it does not cover needs in terms of public
showers or grey water management. ACF’s innovations
are promising and adapted to an extremely cold
climate, they have inspired large organisations like
UNICEF, and have been integrated into the urban
development preliminary studies for the2nd biggest
city in the country.
The relevance of the solutions is apparent also in the
amount of public demand which continues to grow.
This success needs to be qualified: the technical option
of urine infiltration into the ground is a source of
pollution, and is not coherent with the EcoSan
philosophy which campaigns for complete reuse of all
nutrients.
Eff
ect
ive
ne
ss The project was unable to meet its objectives in
terms of the dissemination/accompaniment of the
tested solutions for sanitation and water, which was
predictable with regard to the duration of the project.
This objective was too ambitious.
8
Final external evaluation of the ACF project Improve access to WASH in the Ger areas of Ulaanbaatar, Mongolia
Groupe URD - TDM | June 2015
1. The project and its context
Characteristics of the area of intervention
A capital city that is facing mass countryside-city migration
The country is mountainous and has an arid, almost desert climate. There is thus little farmed
agricultural land and, at present, the population of nomadic or semi-nomadic herders is
estimated as 30% of the Mongolian population. The recent evolution of the climate, with a
reduction in the rainfall, combined with the development of industrial activity (significant
exploitation of mineral resources and development of tanneries) has entailed deterioration in
the quantity and quality of the country’s water resources.
In 2013, Mongolia was divided into 29 river basins, each with a basin Agency, a model that is
very similar to the water agency management in France.
In the 90s, Mongolia swung from an economy based on the Soviet model to a free-market
economy. In addition to this major economic transformation and the weak environmental and
industrial policies, there was a quick succession of particularly harsh winters (dzuud), which
decimated the livestock of numerous nomadic families. Mongolia now has to confront a crisis
caused by a combination of many climatic, environmental and structural factors, which have led
to the rapid mass exodus of the populations to the capital. It seems that this trend is beginning to
slow down now.
Administrative complexity and political crisis
It is not easy to understand the coordination between responsibilities and regulations (even
for the Mongolian institutions themselves). The evaluation team recommends the institutional
analysis of the WaSH sector carried out in connection with the project (January 2014) and the
study, “Sanitation: experiences, challenges and recommendations” (January 2015), which are
both of very high quality.
At the political level, the government experienced a major crisis at the end of 2014 with the
forced resignation of the Prime Minister followed by a ministerial reshuffle. These changes and
the high turnover in key positions which exists in Mongolia, have resulted in certain decisions
and laws being held up.
The specific characteristics of the climate
Four seasons, but which can be summarized as two: a very cold winter period (temperatures
going down to minus 50°C at night and often minus 35°C during the day), and hot in the summer.
This is a central consideration in the Water, sanitation and hygiene sector, and is a major
technical constraint (the need to bury the pipes deep or to insulate them considerably above a
depth of 2.5 metres, all liquids freezing on the surface and at a shallow depth, making the ground
slippery and dangerous, etc.).
As a result, the majority of sanitation options which are known and well-understood in
temperate climates are not applicable in Mongolia. This is all the more true for low-cost local
9
Final external evaluation of the ACF project Improve access to WASH in the Ger areas of Ulaanbaatar, Mongolia
Groupe URD - TDM | June 2015
sanitation: there are too few tried and tested solutions for autonomous sanitation in areas where
there is an extremely cold winter. Difficulties to use onsite technologies suitable for the Mongolian
harsh climate conditions and the lack of consistency from WaSH actors in piloting options from
internationally acknowledged models have slowed down the introduction of sound onsite
sanitation technologies for Mongolia (SDC report, June 2015)
The study by the World Bank which is due to be published in September 20151 on local
sanitation in cold climates is a very interesting experience sharing initiative. We should also
point out here that WECF produced a very good technical guide in 2007 on EcoSan toilets, re-
using nutrients, sanitisation and regulatory aspects.
http://www.wecf.eu/cms/download/2007/WP-26_web-07.pdf
This specific climatic characteristic should be seen as an opportunity (for example, negative
temperatures make bacteria inactive).
In the Ger areas
There is a clear disparity between the city centre and the Ger areas in terms of access to
basic services (water, sanitation, communication channels, electricity, health, education).
Access to water is mainly through around 500 water kiosks (the overwhelming majority
public, managed by USUG), of which about half are connected to the mains, the other half being
supplied by water tankers. The management of the water kiosks located in the yurt area is loss-
making for the USUG, because the operating costs are significantly higher than the income from
the sale of water. In the winter, the absence of paved roads makes the journeys complicated and
dangerous, especially those to transport water.
In addition to the problem of quantity (some families live with less than 10L/person/day), there
is the bad quality of the water consumed, notably due to contamination from the transport or in
the home. The water containers used for transport/storage are often unsuitable, as they
previously contained harmful substances (reuse of containers from the petrochemical
industries).
Sanitation essentially consists of basic latrines which, once full, are “closed”. They then dig
another pit. On the one hand, the current system contaminates the land, and on the other hand it
is not sustainable, due to the limited space in individual homes, preventing them from changing
the pits regularly.
The inadequacy or bad quality of the WASH infrastructures, and the lack of knowledge about
how diseases are transmitted, combined with poor hygiene practices, contribute greatly to the
development of diseases, in particular hepatitis A virus.
1 The recommendations of the study were presented for the first time at a conference in June 2015.
10
Final external evaluation of the ACF project Improve access to WASH in the Ger areas of Ulaanbaatar, Mongolia
Groupe URD - TDM | June 2015
Brief outline of the project
The project evaluated is formally the one financed by the Seine Normandy and Artois-Picardie
Water Agencies2, which began in May 2012 for a period of 3 years (which ended on 30 June 2015
following a two-month no cost extension). It would, however, make little sense to ignore the
project financed by Swiss Cooperation– SDC (with co financing by ACF) with a 2-year duration
(May 2012-April 2014), the “research” aspect, as well as the “hygiene promotion” project funded
by UNICEF (dates?). Indeed, these three projects were conceived as complementary.
In simple terms (in order to give the reader some points of reference), at the time of ACF’s
withdrawal, the main (and frequently mentioned) achievements of the project are as follows:
Main outputs Comments
Organisation of high quality WASH events,
seminars, exhibitions, media campaigns, etc.
This mobilisation is based on WASH technical
expertise and increases ACF’s recognition in
Mongolia. In effect, these are the main
advocacy activities.
sanitation
Around 280 eco-toilets3 and testing of a peri-
urban emptying service
Robust, tried and tested solution4. The only
solution to date for local sanitation in the Ger
areas (solution recommended by the
engineers who carried out the pre-feasibility
study of WASH facilities for the 2nd largest city
in Darkhan region).
Cost recovery difficult for the emptying
service (only around 10% of users pay)
Need for significant support and
accompaniment (notably advocacy for grants)
before it can genuinely be scaled up.
Ability to produce compost in winter An impressive scientific exploit, but without
any operational use in the present context.
Production of compost in the summer Accumulated experience -> today there is
genuine expertise. Excellent visual and
olfactory quality of the finished product.
Laboratory analyses from June 2015 show
that there is a good level of sanitization, even
though there is still room for improvement
with regard to the summer composting
process (which is still very new) and the
analysis protocol.
2 Including co-funding from Cap Calaisis community agglomeration, ACF.
3 Of the 370 EcoSan toilets built by the project, this figure does not include all the initial prototypes and initial tests or
failures which have been abandoned. 4 Even though it is still possible to improve the designs and optimise the service (with a view to continuous improvement) in
order to optimise costs.
11
Final external evaluation of the ACF project Improve access to WASH in the Ger areas of Ulaanbaatar, Mongolia
Groupe URD - TDM | June 2015
Standard approved at the end of May 2015 for
the compost
A significant success which was needed in
order to be able to sell the compost
water
Model of autonomous multi-service water
kiosks for three years
Very promising model, which is economically
viable if it is run by a non-profit association
and we exclude the initial investment cost
(kiosk construction) which remains high.
The model could be refined further to achieve
a perfectly replicable solution.
No dissemination for the time being.
20 000 25L water containers made in UB, then
sold at subsidised prices
Major success in the sense that this activity
was mentioned by all those who were
interviewed for the evaluation. A Mongolian
company is now capable of producing this
type of container. The goal now should be to
analyse demand (a consumer or social
marketing study) in order to determine
whether the « business » should be developed
at cost price or whether to campaign for the
price of the containers to be subsidised.
Hygiene promotion
A lot of WASH awareness-raising materials in
a variety of formats, disseminated in an
innovative way5 (television, radio, social
media…)
One of the most successful outputs both in
terms of quantity and quality (see annex 1
best practice). Materials can easily be re-used
by organizations other than ACF/Wash Action
High quality toolbox for WASH training
courses validated by the official institutions
and integrated into health courses throughout
the country, widely disseminated and used
Long-term impact
Health clubs in 22 schools + networking and
creating Facebook pages for the clubs
Clubs being replicated in certain schools.
Other clubs will perhaps not continue due to
insufficient dynamism and support.
Contribution to major changes in regulations
at the national level
Long-term impact
As was predicted in the intermediate evaluation, it was not possible to achieve the specific
objective of “Accompanying the dissemination of tried and tested solutions which allow
sustainable access to water and sanitation for the inhabitants of Ulaanbaatar’s Ger areas”
(except for the hygiene promotion component): dissemination had hardly begun at the time of
ACF’s withdrawal, which is completely “normal” considering the overall duration of the project
and the real challenge of developing a peri-urban EcoSan sector from A to Z, in a country where
there is strong cultural resistance.
5 This was the first time that WASH related messages were disseminated in this way.
12
Final external evaluation of the ACF project Improve access to WASH in the Ger areas of Ulaanbaatar, Mongolia
Groupe URD - TDM | June 2015
Location
The two intervention districts already targeted by the previous research programme carried out
between March 2009 and August 2012 are Bayanzurkh and Songino Khairkhan. Relatively
quickly, activities were focused on Songino Khairkhan district alone, as this was the only district
involved in the second phase (2012-2015).
Map 1: location of ACF’s operational areas in Ulaanbaatar
2. Evaluation objectives and methodology
For us, the evaluation, as well as allowing accountability, must be useful for the project
stakeholders and their partners, enabling them to learn from their experiences and to improve
their actions. It is thus essential to start from their concerns that the evaluation should be
constructive, the players should take ownership of it and it must allow real progress to be made.
It was thus important for future users of the evaluation to be able to discuss the terms of
reference, perceive them as their own, explain the issues involved in the evaluation questions
and express the priority objectives of the evaluation for them. The main expectations which
were expressed by all ACF staff, both at headquarters and in the field, concerned the criterion of
sustainability (particularly for WaSH Action of Mongolia: expectation of recommendations in
order to be able to evolve in the best possible way after ACF’s withdrawal), and the criterion of
impact (for the Water Agencies: the issue of achieving results after a long-term engagement, and
for ACF: reviewing their involvement at the time of withdrawal). As the criteria of coherence and
relevance were covered a year and a half ago, and as there has not been any change since, they
are not developed at length in this report (see interim evaluation).
13
Final external evaluation of the ACF project Improve access to WASH in the Ger areas of Ulaanbaatar, Mongolia
Groupe URD - TDM | June 2015
The evaluation was carried out in the unusual context of the last month, when staff were clearing
up and leaving. Despite this, they did everything they could to be relatively available.
Classic evaluation methodology was applied (described in detail in the inception report): the
evaluation is based on a qualitative methodology and is based on the triangulation of
information (written and oral sources, and field observations) in order to draw conclusions.
- Interviews with a wide range of stakeholders: (see annex 3 for more details about the
people met)
- Gathering and analysis of documents (see annex 5) :
- Direct observation of project outputs (see annex 3)
As is often the case for a final evaluation, an online survey was sent to around 200 Mongol and
international representatives of institutions and partners working in Mongolia (see annex 7).
3. Between the interim and final evaluations…
The following table reviews the recommendations made in the interim evaluation: Have they
been implemented? Why? Are they still relevant in relation to the findings of the final
evaluation? Etc.
Crit
eri
on
Main observations Main Observations Reviewed in
June 2015
Ex
tern
al
co
he
ren
ce
The first official development
plan for the Ger areas was
established after ACF’s project
began
ACF’s local solutions remain
relevant considering the new
urban development plans
• All the Ger areas will
probably not be transformed
into apartments and
connected to the network
• A model which will allow the
WASH needs of Ger area
inhabitants to be met in the
short/medium term
• A relatively light and flexible
(adaptable) model in the case
of toilets and the emptying
service
ACF: Coordinate regularly with the municipal
authority, and the competent institutions,
particularly the Ger Areas Development
Agency, and establish contact between ACF’s
partners who are taking over the activities and
these institutions
Collaboration
with the GADA
proved to be
difficult
(turnover,
distrust from the
Agency)
Nevertheless, ACF
continued to
engage in
coordination
activities
ACF is a major WASH player in
Ulaanbaatar
Continue to integrate the tools (in particular
hygiene promotion) and skills into the
different Mongol institutions, and other WASH
This was done
remarkably well
14
Final external evaluation of the ACF project Improve access to WASH in the Ger areas of Ulaanbaatar, Mongolia
Groupe URD - TDM | June 2015
organisations In
tern
al
co
he
re
nc
e
Four complementary
components (Water Agency +
research + SDC + UNICEF)
Some progress made possible
by the research (development
of composting technique in a
cold climate, toilet survey,
assurance of scientific
reliability…)
But the combination of research
and operational activities was
very ambitious and
insufficiently prepared with all
the actors, which led to
misunderstanding and
frustration: it could have been
conducted more effectively
For the next time: prepare, clarify and
communicate the objectives with all the
stakeholders from the beginning: anticipate
and clarify how the research and field
activities are going to complement each other.
Ensure that information is regularly shared
between the field and research activities.
Continue to disseminate in Mongolia and
research results at a global level, and use the
results during the advocacy phase.
Recommendation
remains valid6
Pe
rtin
en
ce
The project meets the essential
WASH needs of the population,
even though it does not address
needs in terms of public
showers and managing grey
water
Pursue research into simple, low-cost solutions
for the management of grey water
Recommendation
remains valid, but
for the long term
(is not a priority
for WASH Action)
The innovations developed by
ACF are promising and adapted
to an extremely cold climate
Use multi-service kiosks to:
- provide an example to the population,
USUG and the other WASH stakeholders
(social marketing, promotion of the
multiservice aspect, EcoSan cycle, making
EcoSan toilets available, etc.)
- experiment (re-use grey water in summer,
recycle compost made from faeces…)
As part of the advocacy strategy: collect past
and future lessons from running multi-service
water kiosks, organize visits for WASH
organizations and conduct advocacy in favour
of a new legal standard for multi-service water
kiosks.
Recommendation
remains valid, but
depends on
obtaining funding
There is a good level of
acceptance amongst users and
this is gradually becoming more
widespread
Continue improving the comfort of EcoSan
toilet users to increase the level of acceptance
Recommendation
remains valid
Test the level of re-use of urine and grey water
per plot
Still valid, but not
a priority
Develop a strategy for promoting/selling
compost
Recommendation
remains valid
E F More fine-tuned approaches For a future project: Recommendation
6 The main initial misalignment was due to the different needs in terms of schedules and timeframe between the research
and the program. The research became operative on the ground only during the second year when most of the project
activities were already defined and no more open to technical inputs from the research.
15
Final external evaluation of the ACF project Improve access to WASH in the Ger areas of Ulaanbaatar, Mongolia
Groupe URD - TDM | June 2015
(more development oriented)
would probably have increased
the level of acceptance7 and
effectiveness:
- Creation of the supply
without working in parallel
on the demand
- The expats feel that they did
not properly engage with
the Mongolian culture, and
that this was a mistake
Include sociological expertise from the
beginning of the project
Co-create supply in a participatory manner so
that it corresponds to demand
remains valid
The project will probably have
difficulty reaching its objectives
in terms of disseminating tested
solutions by June 2015
Make the accompaniment and consolidation of
capacity building of local organisations a
priority for the last year of the project, and
extend the duration of the project
This was one of
the main
recommendations
. The
accompaniment
of local partners
was made a
priority during
the last year of
the project, but
ACF’s
(premature)
withdrawal may
weaken their
sustainability.
DU
RA
BIL
ITE
The handover to private sector
EcoSan partners is coherent,
but very new and fragile and
there is a danger that it will be
rushed if the accompaniment is
not extended beyond the
remaining year
IMP
AC
T
The project made a decisive
contribution towards changing
behaviour vis-à-vis the EcoSan
approach
Extend the project beyond the 6th year (3 + 3
years is not enough for a development project
of this kind)
As was acknowledged from the
beginning, the pilot project
could not be aimed at the most
vulnerable sections of the
population
Future strategies should look at solutions for
vulnerable households8
This is relevant
for all
humanitarian
organizations
(not a priority for
WaSH Action)
7 This concerns mainly acceptance at the level of decision-makers and institutions concerning the use of compost. 8 This concerns economic vulnerability (the poorest households): WaSH vulnerability is very high even amongst households
with an average income.
16
Final external evaluation of the ACF project Improve access to WASH in the Ger areas of Ulaanbaatar, Mongolia
Groupe URD - TDM | June 2015
4. Results based on the OECD DAC criteria
Coherence /Coordination: the project is coordinated well with its institutional environment
A project which is coherent with the new urban development plans
In general, ACF’s intervention in Ger area based on a decentralized logic (construction of a multi-
service water kiosk, renovation of two kiosks, establishment of an EcoSan branch…) is still
relevant when considering the urban development plans implemented by the Mongolian
institutions (6th Master Plan) and consisting broadly of progressively transforming the Ger areas
into blocks of apartments (whose appearance recalls the post-WWII low-income social housing
that was built in Europe) and connecting them to the centralized water and sewer mains.
It is indeed unlikely that it will be possible to transform all the Ger areas into apartments
connected to the mains, especially as new families arrive regularly, and it will take several
decades even for those that are transformed.
New apartment blocs & Gers
This point has been confirmed again and again over the past year and a half.
(SDC report) Ecosan Scale up in Darkhan
As mentioned in the context analysis CDIA proposed in the prefeasibility study for “Water Supply and
Sanitation Infrastructure Improvement Project (WSSIIP) for the City of Darkhan” the combination of onsite
sanitation solutions and infrastructure expansion. After a consistent assessment of sanitation options and the
comparison of MoMo project ecosan cycle with ACF one, the CDIA team selected ACF ecosan toilet and waste
management ACF ecosan cycle for the onsite sanitation project component in Darkhan. The prefeasibility
study foresee the construction of around 9300 ecosan toilet that initially will constitute the first access to
improved sanitation and that later will be slowly replaced, where orographic conditions allows, by sewerage
lines. The adoption of the ecosan technology in a big scale prefeasibility study leads to two considerations
about consistent piloting of actions and coordination of actors
17
Final external evaluation of the ACF project Improve access to WASH in the Ger areas of Ulaanbaatar, Mongolia
Groupe URD - TDM | June 2015
The two types of solutions and levels of intervention (large-scale decentralised public
infrastructure on the one hand, and decentralised household-level solutions on the other)
should no longer be viewed in opposition to each other, because they are complementary9.
However, issues of coordination and the development of a strategic vision remain a neglected
priority. The study, “Sanitation in Mongolia: Experiences, issues and recommendations”, which
was completed in January 2015, came to the same conclusions:
The work carried out by actors working on improving the housing and sanitation conditions in the ger area of
Ulaanbaatar is sometimes not welcomed by the authorities because these interventions might potentially increase
the investment done on the Khasha, its market value might also slow down the process for land acquisition from
construction companies redeveloping the Ger areas. At the same time actors working on the improvement of onsite
sanitation facilities in the Ger area are facing difficulties in engaging with households in improving their latrine
because households expect in a very short time to have their living environment connected to the water and
sewerage network. This is due to the lack of a vision strategically integrating the sector of service provision with
onsite sanitation solutions.
� Recommendation for all actors involved in advocacy, including WaSH Action: it
would be useful to compare different WaSH solutions (economically, environmentally
and in terms of job creation) with regard to meeting the short/ medium/ and long-term
needs of those who live in the Ger areas.
In addition, the fact that certain regulations have evolved in the last year (validation of a
standard for compost10, minimum WaSH dictates in schools, the establishment of a budgetary
line by the Ministry of Finance for all schools to have soap11) is directly linked to the work of
ACF. This also shows the relevance of the strategies that ACF has defended with regard to the
institutional context, despite the difficulties mentioned above.
ACF an important player in WASH coordination
Over and above participation in existing coordination bodies and working groups (e.g. WASH,
Education and Health clusters), ACF was clearly a driving force in the creation of ad-hoc working
groups and the organisation of experience sharing events.
Several indicators show that ACF’s WASH expertise is highly regarded:
-ACF was invited by the WHO to run a training course on water quality in 10 Khoroos involved in
the development of health security plans.
-ACF/WaSH Action were invited to take part in the municipal authority’s Waste Management
working group.
-ACF’s participation in a restricted meeting with the World Bank’s WaSH experts on 18 June to
discuss the initial recommendations of the study on sanitation (before its official presentation in
September), two weeks before the end of the mission, shows not only that ACF is recognized, but
also the quality of its networking which allowed it to be invited into this kind of network.
9 This is what the WASH pre-feasibility study for the city of Darkhan shows: decentralised solutions that are planned to be
temporary. 10 Standard which determines the characteristics of the product 11 These two last changes had not been fully officialised at the time of the evaluation.
18
Final external evaluation of the ACF project Improve access to WASH in the Ger areas of Ulaanbaatar, Mongolia
Groupe URD - TDM | June 2015
Internal coherence of the project
The first three years (2009-2012) were conceived without necessarily imagining a “continuation”, and
then at the end of 2011 discussions began with the Water Agencies on continuing the activities. A call
by the SDC (beginning of 2012) and a visit by the Water Agencies (March 2012) endorsed the
continuation and development of the project for a further 3 years (only 2 years with the SDC).
Between the end of the first Water Agency project (end 2011-beginning 2012), and the decision to
continue with the operations in Ulaanbaatar, there was a moment of hesitation between the exit
strategy and the new operational strategy, which could be called a “crisis”. This was at the beginning
of 2012 and was marked, among others, by the fact that the expatriate team was completely
renewed, and by several months without a coordinator (accompanied by a loss of institutional
memory).
The three components of the project evaluated are currently as follows:
1. Project financed by the Water Agencies, the regional authorities and ACF, April 2012-
June 2015 “Improve access to water, hygiene and sanitation in the Ger areas in
Ulaanbaatar, Mongolia, through the dissemination of innovative and proven solutions.”
2. Project financed by Swiss Cooperation and ACF, July 2012-February 2015: “Ger EAH
project: Sustainable Water, Sanitation and Hygiene services - “Improving Access to Safe
Water and Improved Sanitation, and Knowledge on Proper Hygiene Practices” for local
population in Songino Khairkhan district – Ulaanbaatar / Mongolia”
3. UNICEF: three consecutive projects between February 2013 and June 2015.
+Research aspect: “Sustainable Sanitation for vulnerable peri-urban population - Operational
Research Study in Mongolia”
The SDC project clearly allowed an extension of the activities which had been tested and
validated in the previous research phase (without being able to talk of ‘scaling up’), with the
construction of 250 eco-toilets from 2013 in the SDC project, which led to the recruitment of
around 2/3 of the employees.
19
Final external evaluation of the ACF project Improve access to WASH in the Ger areas of Ulaanbaatar, Mongolia
Groupe URD - TDM | June 2015
Since May 2012, the project has focused primarily on consolidation and handing over the
decentralized WASH solutions to local partners (full EcoSan cycle with supply of eco-toilets +
emptying service + composting site, construction and running of a multi-service water kiosk, a
vast hygiene-promotion campaign mainly aimed at schools, and the design, production and
distribution of containers for transporting water). The handing over of the EcoSan cycle to the
local partner also involved looking for ways to establish financial viability and transform the
beneficiaries into clients.
This second phase also involved developing an advocacy strategy which was implemented from
December 2014.
Only the experiment which consisted of building two treatment units for grey water in two
different households during the first phase was more or less abandoned because the technique
was not appropriate (too complex, not appealing) for the household level.
Relevance: decentralised solutions adapted to the capital’s Ger areas
Access to potable water and improved sanitation, and hygiene promotion are priority needs for
the population. The ACF programme is therefore relevant: there are no other solutions for a
large proportion of the Ger neighbourhoods other than to develop and reinforce appropriate
decentralized solutions. Even though one of the main needs remains unresolved: the absence of
solutions for grey water management (insalubrity and mosquito breeding pools in the summer,
and dangerous patches of ice in the streets in winter), and the lack of public shower facilities12;
the ACF programme provides alternative solutions which are adapted to the context and meet
the needs of the inhabitants of the Ger areas, whether temporary (in relation to the major urban
development plans for the capital) or long-term13. What is more, the solutions that have been
developed are capable of meeting the needs created by the mass arrival of new migrants
(estimated at 30 000 families each year) in the peripheral areas.
Ecological sanitation is also relevant due to the fact that there is a water shortage looming in
Ulaanbaatar, as well as low soil fertility and the need for organic enrichment14.
The relevance of the solutions can also be measured in terms of public demand which is growing
both for subsidized water containers and EcoSan toilets. In addition, those who use the emptying
service have expressed concern about it stopping due to ACF’s departure.
12 Activities for provision of bath houses had to be investigated at the beginning of the definition of pilot options and maybe
integrated to the concept of MSWK, nevertheless it has to be said that even nowadays the legal framework is not allowing
this kind of combination due to hygiene rules. 13 Other priority needs are solid waste management, the lack of charcoal for heating, the lack of schools and kindergartens,
unemployment, the poor state of the roads, and street lighting. 14 A large number of studies have shown that there is demand and a market for compost. For example, ACF took part in an
exhibition on ecology organized in April 2014 by the Ministry for the Environment and the municipal authority. The focus of
the event was primarily the production and use of compost. Apart from the public interest in this type of compost,
numerous gardening companies (20 in number) were also interested.
20
Final external evaluation of the ACF project Improve access to WASH in the Ger areas of Ulaanbaatar, Mongolia
Groupe URD - TDM | June 2015
Some solutions have inspired large organizations and have already been replicated: for example,
UNICEF outdoor emptyable pit toilets for rural areas schools.
This success nevertheless needs to be qualified in terms of households’ acceptance of EcoSan
and the emptying service: of the 370 toilets that have been built in connection with the project,
the real number of toilets used today in the spirit of the EcoSan cycle is quite disappointing. As
soon as the service becomes fee-paying, public support tends to be withdrawn.
Another qualification is technical: the project’s eco-toilets are based on the principle of urine
infiltration into the soil, which seems to be the most realistic option (based on previous
experience), and which was retained in the WASH pre-feasibility study for Darkhan.
Nevertheless, it is important to remember that this infiltration represents a source of pollution
depending on the type of soil and the level of the water table, and does not correspond to the
EcoSan approach which campaigns for the complete recycling of nutrients.
� Recommendation for all actors involved in advocacy15 : include public showers in the
advocacy work vis-à-vis institutions (to be included in urban development and facilities
plans for Ger areas)
� Long-term recommendation for WaSH Action: resume action research to improve the
management of grey water (look for partnerships with researchers + volunteers +
funding)16
Effectiveness: an ambitious development project
The specific objective of the “Agencies” project is as follows:
Accompany the provision of proven solutions permitting sustainable access to water and
sanitation for the inhabitants of the Ger areas of Ulaanbaatar
The expected results are detailed below with comments about how well they have been
achieved:
Result 1: Accompany the public
and private players to develop
decentralized water and sanitation
branches with the aim of meeting
the needs of the population of the
Ger areas of Ulaanbaatar.
As was predicted in the interim evaluation, concerning the
dissemination of multi-service water kiosks and EcoSan
(“the development of sectors of activity”), considering the
ambitious nature of the project and the “common”
difficulties encountered (cultural resistance, political
crises with numerous changes in terms of organisation,
responsibilities and personnel), several additional years
would be necessary to achieve sustainable results.
15 not applicable to WaSH Action in the short term, as it has other priorities 16 same remark: not a priority for WaSH Action (except, of course, if funding opportunities open up on this subject)
21
Final external evaluation of the ACF project Improve access to WASH in the Ger areas of Ulaanbaatar, Mongolia
Groupe URD - TDM | June 2015
Result 3: The transfer of technical
and economic skills in water,
sanitation and hygiene, to public,
private and association players
contributes to attaining the
Millennium Development Goals
Result broadly achieved: very good level of skills transfer
to a wide range of institutional actors, to USUGs, to
Tolgoit, and to the national ACF staff who are taking over
as a national NGO specialised in WaSH.
Result 2: The institutional
framework of the water, sanitation
and hygiene sector is analyzed and
reinforced on the national level.
It seems reasonable to say that the changes in national
policies and regulations directly related to the ACF project
go beyond the initial objectives which aimed to “analyse”
and “reinforce” the institutional framework.
Impact
Impact on behaviour change at the household/community level
Sanitation:
The interim evaluation stated that beneficiaries of pilot infrastructures were satisfied, proud and
talked about them to their immediate entourage (neighbours, family, and friends). Word of
mouth appeared to be working well. ACF had received requests from several districts of the
capital and from other Provinces.
At the time of the final evaluation, this trend remains true for the (rare) beneficiary households
who effectively use the emptying service. These same users say that they are worried about ACF
leaving as they are afraid that the service will stop one day. What is more, the municipal services
at the khooros level receive requests from households who want to install eco-toilets17.
However, considering the alarming drop in the number of toilets that are emptied (see
paragraph on MonESIC in the chapter “degree of autonomy of local structures”), it should be
noted that though households are happy with / interested in EcoSan toilets, they obviously have
not understood the importance of the emptying service. They are not prepared to pay for this
service which is nevertheless essential for the proper functioning of the system as a whole.
Though pilot solutions were tested technically and were accepted well, there are levers of a
more sociological nature that are missing to establish acceptance of the following idea: someone
has to pay so that the service can function. And, to some extent, this someone has to be the
beneficiary household.
Hygiene:
Cases of water-borne diseases have fallen considerably in the schools targeted by the project,
which is a very positive indicator of impact.
Water:
According to USUG, average water consumption in apartments has fallen. USUG attributes this
phenomenon to the numerous campaigns for saving water (of course, in this area ACF is just one
of several contributors).
17 The social worker from the 7th khoro says that he has recieved around 250 requests since January 2015 and thinks that
requests are going to rise considerably in summer due to the problems with pit-latrines (unpleasant smell, flies).
22
Final external evaluation of the ACF project Improve access to WASH in the Ger areas of Ulaanbaatar, Mongolia
Groupe URD - TDM | June 2015
Regarding the two resource centres set up in connection with the project, it is very difficult to
evaluate their real impact (no data about the number of visits to the centres). It is clear that the
library, the museum and the technical training room in the USUG training centre were clearly
improved as part of the ACF project. USUG is very proud and satisfied with this, and this can only
have a positive impact. Those who use it are USUG staff from the whole country, students, school
children and the general public. The small room in school n°106 is more like a meeting room
than a resource centre, but again, even though the real impact of the facilities provided during
the project is impossible to evaluate, despite its very small size, it is already a good thing that it
exists. The publications in the cupboard are not only about hygiene promotion, and are very few
in number. Even though communication through social media about materials, toolkits and key
publications, and the networking of groups via internet will probably have a much greater
impact than this room in an isolated school on dissemination and the reinforcement of health
clubs in schools: the two strategies are complementary.
Resource center at school number 106
Museum, library and technical training room at USUG training center
23
Final external evaluation of the ACF project Improve access to WASH in the Ger areas of Ulaanbaatar, Mongolia
Groupe URD - TDM | June 2015
Impact on a larger scale and at the institutional level
It should be underlined that the impact of ACF’s activities at the national level, which were a real
challenge, was remarkable. Here are some of the main changes attributed to ACF and mentioned
by partners directly (non-exhaustive list).
Contribution to changes in national regulations:
- Minimum requirements related to WASH in schools: ACF took part in the working group
organized by the Ministry of Education to define WASH requirements within schools, and
influenced the results significantly (in the process of being officially validated in June
2015). ACF provided training about these new requirements to heads of department
working for the Ministry of Education and Culture from the country’s 21 Provinces
(aimags) at the beginning of 2015 (150 people trained). These people will have begun
implementing concrete changes on returning to their Provinces (see list of changes with
Ariuna)18
- Integration of WaSH tools and training courses developed by ACF and validated by the
official institutions into health courses. These tools are widely disseminated and used.
- Setting up of health clubs in 22 schools (potential for autonomous replication)
- It is probable that (on the basis of ACF’s advocacy work) the Ministry of Finance (or the
schools themselves) will allocate an annual budget to buy soap for every school.
- The standard for compost as a finished product was successfully established, which will
make its commercialization much easier.
- In connection with the working groups created by the Ministry of Health in order to
prepare specific standards for non-collective sanitation19, ACF introduced the EcoSan
toilet model, which was accepted as a technical solution by the members of the working
group.
ACF’s influence will therefore continue after it has left.
Impact of the advocacy strategy
Definition of advocacy, from the ACF International Advocacy Toolkit, 201320
There are many definitions of advocacy but fundamentally, advocacy is a set of organised
activities designed to influence policies and actions of those in power, with the aim of achieving
lasting and positive change. Hence, advocacy requires a good understanding of power
relationships (who makes decisions and at what level, and who has the power to influence
them?) and sets out to effectively engage with and influence the multiple expressions of power
to achieve a desired change.
18 This happened thanks to the UNICEF project – There was nevertheless synergy with the projects implemented by ACF. 19 These standards should allow the standards established in 2008 for latrines and septic tanks to be updated. 20 Excellent guide (clear, simple, comprehensive and operational); it was a very good initiative to have translated it
into Mongolian.
24
Final external evaluation of the ACF project Improve access to WASH in the Ger areas of Ulaanbaatar, Mongolia
Groupe URD - TDM | June 2015
Development of the advocacy strategy
1. Draft Advocacy Paper drawn up by ACF headquarters and the WaSH coordinator
2. Support from Bolor LEGIEM, a very good local consultant, from July 2014 on the
basis of the Advocacy Paper and institutional analysis.
3. Exchanges between Bolor and ACF expat staff
4. 2014-2017 advocacy strategy finalised in September 2014
5. Elaboration of an action plan by the heads of project with support/coaching from
the consultant. This step was preceded by substantial and progressive capacity
building (training) in advocacy by the national team.
6. Finalisation of the action plan in the autumn of 2014 (even though it is
continually changing, in order to be able to seize any opportunities that arise).
7. Implementation of the action plan since the end of 2014, but which has been
perturbed by the decision to close the ACF mission and the creation of WaSH
Action in December 2014;
8. In recent months, a connection has begun to be established between the ACF
action plan and WaSH Action. This was not easy, and remains a challenge.
Analysis of the content of the advocacy strategy
Even though the strategy was formalised at the end of 2014, ACF had conducted advocacy
activities21 aimed at the general public and institutions since the beginning of the project which
contributed to ACF’s reputation in Mongolia and led to the remarkable progress made on
regulations already mentioned above.
21 See activity reports for descriptions of all these activities which allowed advocacy messages to be transmitted: world
water days, handwashing, toiltes, conferences, exhibitions, presence in the national media (television, radio), on the
internet and on social media.
25
Final external evaluation of the ACF project Improve access to WASH in the Ger areas of Ulaanbaatar, Mongolia
Groupe URD - TDM | June 2015
The objectives of the formalized advocacy strategy were as follows:
-progress on the ‘polluter pays’ law (strategic focus on water)
-standard for compost as a finished product (strategic focus on sanitation)
-HP curriculum is included in the National curriculum of primary schools and kindergartens
(strategic focus on hygiene promotion)
-Budgets for Schools and Kindergartens cover the hygiene management expenses (strategic
focus on hygiene promotion)
Globally, the 2014-2017 advocacy strategy document is relevant and of good quality. There are,
nevertheless, some comments to be made:
1. The objectives of the water section seem very ambitious (not coherent with the
capacities of the team) and are not always very relevant.
2. There is no advocacy in favour of subsidies to support both the emptying service and
the purchase of eco-toilets by residents of the yurt areas. And yet nothing else is likely
to be done on this important issue without ACF.
ACF’s mandate (as a humanitarian organisation whose mission it is to help the most
vulnerable people) raises questions about the fact that the advocacy strategy does not
include a “social” objective concerning access to services (particularly sanitation) for the
poorest sections of society, with subsidy mechanisms for the construction of eco-toilets
and the emptying service. Something similar was done for the cost of water in the SDC
project (see English/ Mongolian brochure)
3. The challenge is for WaSH Action, which has limited funds, to take over the action
plan which was developed from the perspective of an international NGO (ACF).
4. Several crucial issues affecting all three components of WaSH emerged from the
study, “Sanitation in Mongolia: experiences, challenges and recommendations,
January 2015”.
As this study was conducted after the advocacy strategy was finalized, the
recommendations of the study are not included in ACF Mongolia’s advocacy strategy.
The only activity that was carried out was the awareness-raising campaign aimed at the
general public and local authorities on decentralised sanitation (cf. Facebook).
The study (which is in the process of being published) was presented during the last
WaSH Forum organized by ACF (90 participants) and is going to be sent in the coming
months to the 60 members of parliament.
� Recommendation for all actors involved in advocacy (and, if possible, for WaSH
Action eventually, if they obtain funding): optimize the advocacy potential of this study
(dissemination, organisation of workshops, etc.).
� Recommendation for WaSH Action and all actors involved in advocacy: it would be
a good idea to include a specific section on developing the EcoSan sector in the advocacy
strategy: even though, ideally, the emptying service could eventually be financially
viable, the scaling up process and the social marketing campaigns that are required need
to be subsidised.
26
Final external evaluation of the ACF project Improve access to WASH in the Ger areas of Ulaanbaatar, Mongolia
Groupe URD - TDM | June 2015
Analysis of the advocacy strategy process
- Action plan developed exclusively by the Mongolian heads of project (which, in
retrospect, appears even more important due to ACF’s decision to withdraw and the
creation of WaSH Action). This process was chosen (rather than the development of an
action plan steered by the expat staff) with the aim of building the capacity of national
staff and no doubt also because ACF’s withdrawal was a possibility (even though the
decision had not been made officially).
- Long development process, but the time taken was essential to develop a strategy that
was adapted to the context.
- The implementation of the advocacy action plan only began at the beginning of 2015: it
would be a great shame (and this would raise questions about effectiveness) if all this
solid work, built on ACF’s years of experience in Mongolia, was not pursued.
- But, as stated above, the advocacy action plan was developed by staff working for ACF
(not with a view to becoming WaSH Action) : in its current state, it is too ambitious, and
as such it is not adapted to the capacities of a new local NGO with limited resources
(indeed, no resources for the time being). In any case, it is unlikely that it could be
implemented at the planned rhythm, but it may be possible gradually, little by little,
using WaSH Action’s solid network and by beginning with the “easiest” advocacy issues.
� Recommendation for WaSH Action: pursuing the implementation of the advocacy
action plan should not be an absolute priority for WaSH Action in the months ahead; this
should remain in the “background” (with activities which do not require any budget)
because it constitutes the mission (vision) of WaSH Action and allows it to remain visible
and involved vis-à-vis key decision-makers. For this, WaSH Action will need to adapt the
strategy and the action plan based on their capacity (having several scenarios at the
ready depending on their resources).
� For the Water Agencies and other potential financial partners: for advocacy to
continue, it is NECESSARY for WaSH Action to receive financial support.
Looking ahead, we unfortunately feel that the collaboration between Artois-Picardie Water
Agency and the Mongolian river basins authorities (notably on integrated watershed
management and environmental licence fees) appears to be compromised, considering the
reduced resources of the Agencies for international cooperation programmes.
� Adopt a clearer position about the possibility of pursuing the collaboration with the
Mongolian watershed authorities.
27
Final external evaluation of the ACF project Improve access to WASH in the Ger areas of Ulaanbaatar, Mongolia
Groupe URD - TDM | June 2015
Degree of autonomy of local structures to take over kiosks, emptying services, and composting sites
Lack of anticipation in ACF’s exit strategy
Even though ACF’s strategic decision to withdraw from Mongolia is perfectly justified22 :
1. The withdrawal is too early considering the nature and state of progress of the project:
the majority of WaSH solutions that have been developed are still at the pilot stage, and
have not yet reached their full potential in terms of dissemination and replication. It
would be a shame to abandon them in their current state.
2. The decision to leave was made late (in November 2014), which is a pity, as it was not a
surprise (questions were clearly already being asked a year and a half ago) and an earlier
decision would have made it possible to envisage local capacity building with the main
partner that was taking over (WaSH Action) more serenely.
ACF knew that the chances of receiving funding for new projects were slim (withdrawal of
international donors)… and even though they have intensified their efforts to set up projects
with WaSH Action in the last six months, in the end, what the WaSH Action team lacks is “the
ability to put together projects that can be accepted”: ACF should pursue its support of WaSH
Action in looking for and putting together projects so that they are equipped in relation to
international donors23.
As for the EcoSan sector, in order for scaling up to be able to happen, there needs to be progress
in the following areas:
-advocacy for subsidies to help individuals buy toilets
-advocacy for subsidies for the emptying service (which, like the sale of water at multi-service
kiosks, is similar to a public service)
-promotion / sale of the compost produced
-continuous improvement of toilet design
-using “local champions” to re-launch the process (construction, restoring the functioning of the
service)
Today, ACF still has a central role in activities which aim for replication: the khoroos send the
hundreds of people interested in EcoSan toilets to ACF rather than to construction companies.
The advertising for eco-toilets is done by ACF rather than by MonESIC (who, in any case, do not
have the time or the resources to take care of this kind of promotion and to deal with the people
who are interested), and ACF is supporting the emptying service financially (until the end of
June 2015).
22 Extract from ACF activity report for year 5: “This decision is based on in-depth analysis of the humanitarian situation in
Mongolia, ACF’s international operational context and the weak possibility of funding for new projects. ACF France must
also face multiple humanitarian crises at the global level which will require significant amounts of human and financial
resources. Mongolia has managed to recovery from the dzuds which affected the country in the 2000s, and although WASH
conditions remain worrying, the general situation has improved immensely”. 23 When the evaluation report was being finalised, WaSH Action were on the point of signing a MoU with ACF and the USTB
so that they would receive technical support (and perhaps financial support).
28
Final external evaluation of the ACF project Improve access to WASH in the Ger areas of Ulaanbaatar, Mongolia
Groupe URD - TDM | June 2015
What is more, the unsubsidised cost of the toilets seems prohibitive for households in the yurt
areas. This point needs to be qualified though, as households are capable of paying almost 4
million Tugrik (around 1 700 Euros) per family for new telephones to the detriment of their
toilets which are left in the same state, that is to say, without any improvements for a decade24. It
is clear that spending on “sanitation and health” is not among households’ priorities. This non-
demand for sanitation is a barrier for the development of ecological and sustainable solutions.
Demand needs to be generated by working more on Information, Education and Communication
(IEC) and increasing communication campaigns. If households do not see what they can get out
of it, demand will be limited to those who are “naturally” conscious of ecological issues.
Despite this, significant investment has been made in training and building the capacity of
partners since the decision was made, which has compensated somewhat for the lack of
anticipation.
� Recommendation pour ACF: the next time ACF implements a project which is on the
margins of its mandate, it would be a good idea to anticipate its exit by working with
local partners sufficiently early, in order to be able to accompany them (+ capacity
building) at a gradually decreasing level until the exit.
Multi-service water kiosk managed by Tolgoit for almost three years
Regarding the multi-service kiosk, the quality of Tolgoit’s commitment in running the service
should be underlined, and the fact that it continued despite problems (the economic crisis which
affected household budgets and made it difficult to rent spaces to artisans, and compromised its
financial stability).
It is therefore very probable that Tolgoit will continue to manage the kiosk in the long term. In
addition to the engagement of MonESIC, several factors will help to establish this long-term
stability:
- The training-coaching (development of the business-plan) carried out by MIRIM was
very useful to the extent that Tolgoit already was involved in the financial management
of the kiosk, and that the business plan was developed on the basis of everyday
experience. Another key factor to take into consideration is that the President25 of the
organisation (head of Tolgoit) already has solid grounding in financial management
(previous experience and personal experience).
- ACF was successful in campaigning the Water Service Regulatory Commission to
exonerate Tolgoit from paying tax on water sales. This non-profit NGO provides a public
rather than commercial service.
24 It is estimated that 4 million have been invested on mobile phones in the last 10 years (source: ACF analysis).
25 She has been the President for one and a half years.
29
Final external evaluation of the ACF project Improve access to WASH in the Ger areas of Ulaanbaatar, Mongolia
Groupe URD - TDM | June 2015
TOLGOIT headquarter multiservice waterkiosk
The evaluation team notes that success and long-term stability depends on choosing a key
partner, which was the case with Tolgoit. It should also be underlined that the time and
commitment of the members of the association, and particularly the President (since 1,5 year), is
voluntary, and is essential to make the service work. This needs to be taken into account in
terms of the replicability of the model.
It should also be underlined that the EcoSan toile at the kiosk site (which is used by the
caretaker’s family) was changed to a pit latrine about a year ago (a pit was dug in the space for
the bucket) and is neither cleaned nor maintained, which is a very negative advert for EcoSan.
Tolgoit has no doubt not embraced the idea of EcoSan enough, or it was not a priority for them.
Considering Tolgoit’s recent enthusiasm for planting plants and bushes (+ project to
commercialise plants, which is a very profitable activity in Mongolia), as well as the association’s
dynamism (and its audience in the community), it would have been/would be interesting for the
association to adopt the EcoSan approach. We feel that the recommendations from the interim
evaluation are still worth exploring: selling small quantities of compost and promoting the
EcoSan approach at the kiosk.
In the medium to long term it would also be interesting to test how the grey water produced by
the artisans is re-used (this currently only concerns the hairdresser because the washing
machine service has been replaced by a shoemaker).
Major concern about the sustainability of MonESIC taking over the ecological
sanitation service
The real investment and motivation of the people from MonESIC who were met during the
evaluation should be underlined. The composting technique and the final result of the mature
compost are excellent. One week after the compost pile was established in June 2015, the
temperature that was recorded in the pile on 8 June 2015 was 55°C. Congratulations for these
results.
Compost processing, June 2015
30
Final external evaluation of the ACF project Improve access to WASH in the Ger areas of Ulaanbaatar, Mongolia
Groupe URD - TDM | June 2015
Comments about the analysis of the compost
The analyses in September-October 2014 (4 weeks after the compost pile was put in place)
showed that there was a very high quality of product during the composting : very little EColi
(below the EPA threshold), Giardia lamblia in one sample and grysporidium in another. There
was no Strongyloides stercoralis (from the helminth egg family).
The analyses in June 2015 using samples taken from the same mature compost showed that all
the pathogens had disappeared (Giardia, Grysporidium), but a helminth egg was detected,
despite the fact that it had not appeared in the analysis at the beginning of the composting
process.
Helminth eggs can
sometimes end up in
the ground. They are
the most resistant
parasite -> an excellent
composting process is
needed: 50-60°C° for 7
consecutive days
evenly spread
throughout the pile
(without the
temperature falling
back down).
Several hypotheses are possible: either 1) the composting process did not maintain a sufficiently
high temperature for 7 consecutive days in the whole pile, which would explain why a helminth
egg was found, or 2) the presence of this helminth egg comes from the way of taking samples
(the contamination took place when the sample was being taken), or 3) the helminth egg was
present in the soil (which again raises questions about the sample taking protocol).
The finished product includes 87% of organic matter, which is excellent. The pH was already
basic (8.2): mineralisation was beginning to take place. The compost is still very good as a soil
enricher, but it will be important to check that the pH stabilizes and does not increase too much.
During the composting process, the compost goes through an acidic phase, then neutral, then
basic. When the pH result is acidic, the composting process is not finished. When the pH result is
too basic, this is like going past the optimal “use-by date”26.
26 Comment: in the sanitation sector there is often a focus on composting as a sanitising process, but not on optimising the
compost for its agronomic value with a view to its reuse for agriculture.
31
Final external evaluation of the ACF project Improve access to WASH in the Ger areas of Ulaanbaatar, Mongolia
Groupe URD - TDM | June 2015
� Recommendation for WaSH Action / MonESIC: the protocol for taking samples to
analyse the summer compost could probably be improved (how and when to take
samples).27
MonESIC still does not have a licence (in terms of being an emptying company), but this does not
appear to pose any problems.
Since the handover to MonESIC, ACF has mainly provided the NGO with technical support on the
compost production and management. However, there does not appear to have been enough
support in terms of social marketing or optimising the emptying service. The question of long-
term stability after ACF left was raised at each meeting between MonESIC and ACF (ACF
continues to fund the NGO’s running costs).
ACF leaving will have an obvious negative effect on MonESIC, who clearly will not be able to
meet all the challenges involved without external support.
Today, around 280 EcoSan toilets are effectively usable.
Ecotoilet from summer 2014, SKHD
Ecotoilet from 2011, SKHD
27 What is the protocol for taking samples? For example, was the spade clean (was the sample of mature compost taken
using a tool that was used in making the compost pile, and therefore had been in contact with fresh faeces)?
Do you have an initial analysis of each substance that goes into the compost: sawdust + a little earth (soil) due to the fact
that the sawdust is stored on the ground + faeces + other?? If these analyses are not carried out for each sample, then it is
impossible to know where these helminth eggs came from. It would be good to do this for the next analyses.
32
Final external evaluation of the ACF project Improve access to WASH in the Ger areas of Ulaanbaatar, Mongolia
Groupe URD - TDM | June 2015
When ACF was present, two people were responsible for the emptying round, but they were not
employed full time on this activity to the extent that the round was only carried out once a
quarter and lasted about twenty days.
The handover to MonESIC, when the service was ready to operate at full capacity, actually led to
a reduction of the service, and some hesitation. The emptying service and the composting are
still done manually, still on the same scale as for the pilot project.
Several worrying issues:
-Reduction of the number of toilets emptied: since people have had to pay for the emptying
service, most have returned to their old latrines (or have transformed their EcoSan toilet into a
traditional latrine);
-The service is managed in a somewhat passive way, whereas it should absolutely be managed
proactively;
-The eco toilet on the composting site is not looked after (broken ventilation, dirty) and is not
used much;
-A theft in the staff premises at the composting site.
Raw data:
Quantity of compost produced in August 2014: around 1 tonne (very little!), which corresponds
to about 150 toilets effectively used and emptied.
Since September 2014, around 40 buckets have been collected (as could be seen during the visit
to the composting site in June 2015, on the day when the compost pile was put in place), which
corresponds to roughly 2 to 2.5 tonnes of matter at the most28.
According to MonESIC, since January 2015 around 20 toilets have been emptied (3 rounds); ¾ of
the households paid for the service.
Since September 2014, around 40 buckets have been collected
28 By considering an average family = 4 people
33
Final external evaluation of the ACF project Improve access to WASH in the Ger areas of Ulaanbaatar, Mongolia
Groupe URD - TDM | June 2015
Fragilities:
-MonESIC does not have any human resources of its own (to use an image, the NGO is like an
empty shell);
-The financial support for running the service from ACF ends in June 2015;
-Only one person is fully competent in managing the composting process (a former ACF member
of staff), who is employed as a day labourer;
-The urine separation toilets are not used properly: the faeces that are collected are extremely
liquid.
Faeces that are collected are extremely liquid
Analysis:
The transition from being “beneficiaries” as part of a humanitarian project to “clients” of an
emptying service led to a very large number of households abandoning the service: indeed, it is
common that the shift from a free sanitation service to a fee-paying one does not work (unless
there are extremely solid support measures in terms of social marketing and communication).
Ideally, the households should never have been considered as “beneficiaries” of a free service.
They should have paid a fee for the service from the start.
There was no joint construction of demand or solutions with the households and the other
partners (which would have increased ownership and helped to establish long-term stability).
Even though the idea of developing the business is important in itself, the training course
provided by MIRIM to MonESIC was not appropriate.
The business plan which resulted from this coaching proposed to change the strategy again: the
new idea was to consider eco-toilet owners as suppliers of a resource (faeces) and therefore the
service should remain free. This change was very sudden and created confusion within the
MonESIC team (though this strategy is not absurd in itself29, it should have been worked on
much more in advance). Today this business plan is very attractive on paper, but remains
29 EcoSan systems like this exist in the world (in Kabul for example).
34
Final external evaluation of the ACF project Improve access to WASH in the Ger areas of Ulaanbaatar, Mongolia
Groupe URD - TDM | June 2015
virtual: there is a huge gap between the current situation with the emptying service run by
MonESIC and the business plan. The accompaniment from MIRIM did not make it possible to
plan in concrete terms the steps that are needed to establish a model of this kind.
MonESIC had more urgent needs:
- support in operational management to optimize the emptying service (this is what ACF
tried to provide in the last year and a half);
- help in consolidating (stimulating and maintaining) social demand, that is the will to pay
for the emptying service: also done by ACF, but insufficiently.
- Scaling up of the EcoSan cycle in the form of an increased number of users, which
requires the engagement of decision-makers at the highest level: ACF worked towards
this with the study “Sanitation in Mongolia: experiences, challenges and
recommendations”
There are different levels of targets to create or maintain demand (households / community
relays / institutions, decision-makers / the general public). All these different levels are
complementary. Obviously the government has a crucial role to play in order to achieve
sustainable change on a large scale.
The objective of social engineering (which usually takes the form of social marketing) is not that
households want subsidised (free) toilets, but rather that they understand why it is important to
have a toilet and the need to pay for the service.
35
Final external evaluation of the ACF project Improve access to WASH in the Ger areas of Ulaanbaatar, Mongolia
Groupe URD - TDM | June 2015
Focus on the EcoSan projects:
Sanitation is never a priority for households (a fundamental difference compared to access to
water). Transport is a recurring problem in this kind of project: experiences in other projects
show how important it is in terms of efficiency, and consequently in terms of viability/ long-
term stability to make sure that collection points are close to each other, to the treatment sites30,
and to the compost sales sites. The EcoSan service in Ulaanbaatar will need to eventually look
into this issue.
Even though the EcoSan approach is perfectly relevant, it is important to say that making a
success of an EcoSan project is always a major challenge; as such, it was completely relevant to
invest in the research aspect even though the linking up of the research and the programmes
was disappointing and did not lead to the expected results. If this EcoSan cycle experiment in
Mongolia becomes firmly established (dissemination, scaling-up and consolidation), it will gain
international recognition! Taking the time to fill in a SuSana project file, and disseminating it
globally, could boost alliances and partnerships with the EcoSan service project in Ulaanbaatar.
EXPERIENCES OF URBAN/PERI-URBAN ECOSAN SERVICE PROJECTS
WHAT LESSONS FOR THE MONGOLIAN CAPITAL?
SOURCES:
HTTP://WWW.SUSANA.ORG/EN/RESOURCES/CASE-STUDIES/DETAILS/1049 (FOR ERDOS PROJECT)
HTTP://WWW.SUSANA.ORG/EN/RESOURCES/CASE-STUDIES/DETAILS/791 (FOR SOUTH AFRICA PROJECT)
HTTP://WWW.SUSANA.ORG/EN/RESOURCES/CASE-STUDIES/DETAILS/84 (FOR OUAGADOUGOU PROJECT)
Globally, there have been very few experiments in urban and peri-urban “ecosan” services. We
can refer to three of these, and the lessons that have been drawn from them: (i) the ERDOS
project in Inner Mongolia in China; (ii) the e-Thekwini project, Durban municipality, South
Africa, and (iii) the EU ecosan project in Ouagadougou in Burkina Faso. All three have provided
lessons that are relevant to the Mongolian experience.
In each of these cases, urine separation toilets were installed and an external service was set up
to manage the faeces and/or the urine. In two cases (China and Burkina Faso), the service was
paying from the start; in the South African case, the initial premise was that the families would
empty the toilets themselves, but in practice they were not very willing to do so. As a result, local
micro-entrepreneurs were trained by the municipal authority to do the emptying, for which the
families paid a fee. The experience in Burkina Faso showed that the households were not very
willing to pay for the service (almost 80% of costs not recovered) and that there was not
sufficient demand from farmers for ecosan fertilizer, which meant that the municipal authority
had to subsidise the service at the end of the project, taking over from the international donor. In
China, technical problems in terms of toilet use (large-scale implementation from the beginning),
a lack of openings for ecosan fertilizers in agriculture and a low level of involvement of users at
the beginning led to households becoming demotivated and making the radical change from dry
toilets to WCs.
These different experiences, which are far from being negative, show that the question of an
ecosan service and even the use of dry toilets on a large scale is a process which needs to be
thought through in advance with interlocutors who are all equally concerned that the project
will be sustainable: users, local authorities, farmers, educators, technicians and companies. If
30 This can lead to the introduction of a collection point or a relay treatment point (buffer site).
36
Final external evaluation of the ACF project Improve access to WASH in the Ger areas of Ulaanbaatar, Mongolia
Groupe URD - TDM | June 2015
these connections have not been made from the beginning, the experiences in South Africa and
Burkina Faso show that it is possible to rectify the situation, but this implies spending more time
than initially planned for the ecosan service solution to be stable in the long term!
The positive points which have already been established by the Mongolian experience are as
follows: (i) the interface with users is accepted and (ii) the pilot phase and the test phase before
wider dissemination were done very well. However, to prevent the EcoSan service in
Ulaanbaatar from coming to an end, based on analysis of these 3 EcoSan services, we
recommend the following actions be taken to influence different factors affecting the long-
term stability of the service:
� Subsidise the functioning of the service immediately (fully or in part), to ensure that he
dry toilets are emptied and avoid them being abandoned. It is important to understand that if
the service is paying for households, it is wise to estimate the rate of non-recovery in the
service’s business plan at least 40%.
� Involving the local authorities is essential for the long-term stability of the service. The
government needs to support these projects and help to develop regulations to achieve a wider
coverage, which will then allow the significant scaling-up which goes hand in hand with a more
stable service in the long-term.
� Reinforce links with agriculture so that this outlet is accepted and easy to set up (within a
radius of 20km at most for reuse), notably through the sale of fertilisers from the dry toilets.
� To revitalise the process and improve the image of using ecosan toilets: (i) use local
champions, in the form of model users and influential people in the neighbourhood, and even
people who are well-known nationally to boost the campaign; (ii) in connection with the well-
known people nationally, carry out IEC activities (media, street theatre, and events in schools,
for example).
Recommendations:
� For ACF: Having invested so much (including 400 000€ on the PhD), it would really be
a shame to let the EcoSan experience die as is happening now: ACF should invest one last
time in the EcoSan area of activity and support social project management activities.
� For ACF, WaSH Action/MonESIC and all other actors involved in advocacy: it is
urgent to find subsidies for the construction of EcoSan toilets and social marketing
campaigns, without which scaling up is very unlikely to happen31.
� For MonESIC: Have an immediate survival strategy: be more proactive and develop
niche markets by optimising use of the equipment.
� For WaSH Action and MonESIC: Work in an integrated manner, as this will be to
everyone’s benefit: EcoSan is an innovative approach which could help to make WASH
Action known, but without the expertise and support of WASH Action, there is a danger
that the EcoSan adventure will come to an end.
31 If all goes well, the emptying service could eventually be financially viable, but it will probably need financial support
immediately to replace ACF’s subsidies which will stop and the end of June 2015, and until MonESIC has developed niche
markets to balance the service financially.
37
Final external evaluation of the ACF project Improve access to WASH in the Ger areas of Ulaanbaatar, Mongolia
Groupe URD - TDM | June 2015
WaSH Action, a young NGO that is still fragile, but has a lot of potential
The next six months will be decisive. This assertion is based on:
-informal discussions with staff (“6 months maximum”)32
-rental of a small office from CARITAS for WaSH Action (cost covered by ACF until the end of
2015)
-the Mongolian social protection system: 50% of the salary is covered for 3 months + an end of
contract bonus of one month’s salary which is given by the employer.
ACF Mongolia’s staff benefited from numerous training courses and capacity building activities
(in Ulaanbaatar, Paris, Madrid…) over several years.
What is more, when ACF made the decision to leave, capacity building activities were intensified,
notably with two national consultants:
- Bolor LEGIEM (who has already been mentioned in the paragraph in which the advocacy
strategy was analysed): her 2nd contract focused on capacity building (writing propels,
developing a Wash Action strategy, fundraising, etc.). Wash Action will receive support
from Bolor after ACF has left based on a tacit contract between ACF, Bolor and WaSH
Action (10 days for advocacy and 3 training courses on fundraising, external
communication and M&E).
- Development Solutions: ten workshops (around 120 hours in total) since February 2015.
A very positive point is that the company offered to pursue its coaching on a voluntary
basis until November 2015, which indeed seems essential, because it is when WaSH
Action will have to face reality on its own that it will have the greatest need for coaching.
SWOT analysis of WaSH Action in June 2015
Strengths
Motivation
Competencies, qualifications
Conscious of their real capacity and weaknesses
Team spirit and commitment
Excellent network (links and partnerships with
numerous organisations)
Expertise and knowledge of sellable
decentralised WaSH solutions
The only national WaSH NGO
Weaknesses
Lack of financial visibility
Key competencies still need to be acquired and
put into practice (see map of competencies
below), notably in organizational, administrative
and financial management
Opportunities
The existence of obvious and enormous WaSH
needs
The existence of a market (around 150
consultancy companies in Mongolia in the
environment sector, but integrated
competencies in WaSH are extremely rare)
Threats
Withdrawal of international donors and closure
of certain large WaSH projects
The sector is still not sufficiently prioritized at
the national level, and at the community level
The attraction of other organisations: risk of
losing the most competent members of WaSH
Action
32 confirmed by the final report of Development Solutions
38
Final external evaluation of the ACF project Improve access to WASH in the Ger areas of Ulaanbaatar, Mongolia
Groupe URD - TDM | June 2015
ACF national team, WaSH Action of Mongolia members
WaSH Action map of competencies
The years of experience with ACF combined with the intensive training courses received in
recent months have allowed staff to be trained with a view to acquiring essential competencies.
The table below summarises the competencies needed. Grey boxes signify gaps (the greyer the
box, the less they have of this competence).
Essential
competencies for
WaSH Action as an
autonomous local
NGO
ACF experience and
internal training
Support from Bolor
Legem
Support from
Development
Solutions
WaSH technical
competencies
Excellent level33
English 2 people speak fluent
English34
Networking as WASH
Action
Very good network at the national level, but
new to the international level, improving little
by little
Advocacy Beginner level
Project (cycle) management
Strategic planning The aim is to organise their ideas within the
academic frameworks of project management
and the reality of financial opportunities :
more or less beginner level (lack of
experience)
Communication
Proposal writing
Fundraising Beginner level
33 Though some expressed the need to increase their technical competencies to achieve the level of experts. 34 But this figure depends on the commitment of the people who are effectively going to work for WaSH Action in the
months ahead.
39
Final external evaluation of the ACF project Improve access to WASH in the Ger areas of Ulaanbaatar, Mongolia
Groupe URD - TDM | June 2015
Budget management
Establishment of a
budget plan
The Programme
Managers currently
have theoretical
competencies (no
practical experience)
but the former ACF
Financial Manager
has very good
capacity
Financial procedures
Logistical procedures
Management /
organisation / Senior
management35
Funding and development prospects at the time of ACF’s departure
-Several studies have already been carried out by WaSH Action (water monitoring tool,
menstrual and personal hygiene tools36)
-Ideas for project type funding:
- with Asia Foundation who propose to include WaSH Action in their advocacy activities
(UNICEF funding), which would be useful in terms of visibility even though it is not a
large contract (around 2 weeks, 800 USD in total).
- New Zealand small grant in July 2015
- The balance from the ACF PhD?
- UNICEF in September/October: pushed by the Ministry of Education who wanted WaSH
Action to pursue their work on the minimum WaSH requirements in schools, UNICEF
finally decided (at the end of June 2015) to evaluate WaSH Action’s capacity in August, to
continue working together.
- UNICEF Mongolia are planning to make WaSH a major part of their strategy for the next
6 years, and at the same time they seem to have realized the importance of reinforcing
the capacity of local actors in the current context of diminishing international funds.
-A few ideas of projects for consultants, such as facilitating the training module for USUG (WHO
funding) in the months ahead, or for a number of Ministries.
From ACF to WaSH Action
One of the main challenges and projects to be carried out by WaSH Action will be to adapt
(reduce) ACF’s strategies (for action and advocacy) so that they correspond to their real capacity
and priorities.
� Recommendations for WaSH Action :
In the SHORT TERM:
1. Survive!
This means investing in what are perhaps not the most relevant activities for the WaSH sector,
but in activities which will be the most lucrative, and will give WaSH Action some financial
visibility. From a tactical point of view, this will mean using the competencies and materials
which can already be mobilised (produce that is ready to sell), and focusing on areas in which
WaSH Action already has recognition and visibility.
35 Who is responsible for what? Human resource management, etc. 36 These are the first tools of this kind in Mongolia.
40
Final external evaluation of the ACF project Improve access to WASH in the Ger areas of Ulaanbaatar, Mongolia
Groupe URD - TDM | June 2015
- Have an action plan: who does what in the coming weeks, depending on the strategic
priorities and tactics which have been established.
- Adapt the advocacy strategy to continue at a minimum level (see detailed
recommendation in the advocacy paragraph)
- Be proactive to increase visibility within the international community (INGO, UN, World
Bank, Asian Development Bank, etc.)
- Give ACF and other partners news regularly. In the medium term (after the survival
phase), it would be useful to regularly analyse the WaSH sector and to send this analysis
to partners (and make it available on the website). The aim here would be to remain
visible and become an important local actor in terms of analysis.
2. Support the EcoSan sector
- Contribute to advocacy in favour of subsidies for EcoSan toilets and the emptying
service;
- Draw up a SuSanA file to share lessons learned, which could help to improve visibility at
the international level, and fund-raising.
In the LONG TERM:
3. Promote the WaSH sector in Mongolia, in line with what was done with ACF
- Optimise the 2014-2017 WaSH advocacy strategy, as well as the advocacy potential of
the study “sanitation in Mongolia: experiences, challenges and recommendations”
(dissemination, organisation of workshops, etc.)
- Improve the versatility of the WaSH Action staff (water, sanitation and hygiene);
- Develop a virtual Resource Centre (website) for the Mongolian WaSH sector;
- Resume action research to improve grey water management (look for partnerships with
researchers + volunteers + funding)37.
� Recommendation for ACF: Maintain strong links with WaSH Action, in the form of a
partnership, rather than a simple handover of the facilities38, as this would be to
everyone’s advantage.
Advantages for WaSH Action: 1/ getting advice, and even re-reading of the next proposals (in
English) which are going to be crucially important for the long-term stability of the structure; 2/
being able to show that they are in a partnership with a globally recognised international NGO
would be helpful;
Advantages for ACF: 1/ having invested a great deal in this project, it would be useful to monitor
its impact and future in terms of lessons learned and knowledge management; 2/ continuing to
support the project, even at a distance, is a way of optimising investments; 3/ keeping a local
partner in a country where there is a risk of earthquakes and Dzuud is good strategy for a
humanitarian organisation.
� Recommendation for ACF: Help WaSH Action to write a SuSanA file, underlining all the
research work that has been carried out (PhD and Masters).
37 same remark : this is not a priority for WaSH Action (except, of course, if funding opportunities open up for this subject) 38 This seems to be in the process of being signed (including USTB) at the time of writing this report.
41
Final external evaluation of the ACF project Improve access to WASH in the Ger areas of Ulaanbaatar, Mongolia
Groupe URD - TDM | June 2015
Conclusion and recommendations
Conclusion
To conclude, the evaluators would like to highlight both the relevance of the overall project and
its completely innovative nature in several respects, which logically leads us to encourage the
consolidation and dissemination of the WASH models that have been developed.
This has been a very ambitious project, which has achieved excellent results, notably due to the
complementary nature of the projects by the 3 main financial partners (Water Agencies, Swiss
Cooperation and UNICEF).
At the time of ACF’s withdrawal, despite the fact that solutions that have been developed are
very relevant, are tried and tested and can be replicated, there has not yet been any scaling-up of
the decentralized sanitation solutions (EcoSan cycle) or for the multi-service water kiosk.
The difficulties encountered on the EcoSan section are not surprising considering the challenge
that this kind of innovative project represents and the time that they require. What is more, it is
not realistic to think that this type of sector can become financially balanced within the space of
a few years (even though it is possible in theory).
The results in terms of the dissemination and sustainability of hygiene promotion messages
have been remarkable.
The work of the ACF team in building its reputation and visibility should also be underlined. This
constitutes a very favourable environment for the creation of WaSH Action. Even though there is
a risk that the initial months will be difficult and decisive, the new (and only) Mongolian NGO
specialised in WaSH has all the potential and competencies needed to succeed in this bold
gamble.
From a strategic point of view as regards Mongolia, the development of projects aiming to
improve health conditions in other provinces and secondary urban centres should be
encouraged, as should the advocacy for an improvement in access to WASH in rural areas, in so
far as this type of action could help to curb the mass rural exodus, which could lead to an urban
crisis, at the same time as the disappearance of the traditional nomadic culture.
Recap of detailed recommendations in the previous chapters:
Recommendations for ACF:
FOR A FUTURE PROJECT…
The next time ACF implements a project which is on the margins of its mandate, it would be a
good idea to anticipate its exit by working with local partners sufficiently early, in order to be
able to accompany them (+ capacity building) at a gradually decreasing level up to the exit.
IN CONNECTION WITH THIS PROJECT:
- Having invested so much (including 400 000€ on the PhD), it would really be a shame to
let the EcoSan experience die as is happening now : ACF should invest one last time in
42
Final external evaluation of the ACF project Improve access to WASH in the Ger areas of Ulaanbaatar, Mongolia
Groupe URD - TDM | June 2015
the EcoSan area of activity and support social project management activities (IEC) +
capacity building for WaSH Action in setting up projects for international donors.
- Maintain strong links with WaSH Action, in the form of a partnership, rather than a
simple handover of the facilities39, as this would be to everyone’s advantage.
Advantages for WaSH Action: 1/ getting advice, and even re-reading of the next proposals (in
English) which are going to be crucially important for the long-term stability of the structure; 2/
being able to show that they are in a partnership with a globally recognised international NGO
would be helpful;
Advantages for ACF: 1/ having invested a great deal in this project, it would be useful to monitor
its impact and future in terms of lessons learned and knowledge management; 2/ continuing to
support the project, even at a distance, is a way of optimising investments; 3/ keeping a local
partner in a country where there is a risk of earthquakes and Dzuud is good strategy for a
humanitarian organisation.
- Help WaSH Action to write a SuSanA file, underlining all the research work that has been
carried out (PhD and Masters).
- For ACF, WaSH Action and all other actors involved in advocacy: it is urgent to find
subsidies for the construction of EcoSan toilets and the emptying service, without which
scaling up is very unlikely to happen.
Recommendations for WaSH Action:
� Recommendations for WaSH Action:
FOR THE SHORT TERM:
1. Survive!
This means investing in what are perhaps not the most relevant activities for the WaSH sector,
but in activities which will be the most lucrative, and will give WaSH Action some financial
visibility. From a tactical point of view, this will mean using the competencies and materials
which can already be mobilised (produce that is ready to sell), and focusing on areas in which
WaSH Action already has recognition and visibility.
- Have an action plan: who does what in the coming weeks, depending on the strategic
priorities and tactics which have been established.
- pursuing the implementation of the advocacy action plan should not be an absolute
priority for WaSH Action in the months ahead; this should remain in the “background”
(with activities which do not require any budget) because it constitutes the mission
(vision) of WaSH Action and allows it to remain visible and involved vis-à-vis key
decision-makers. For this, WaSH Action will need to adapt the strategy and the action
plan based on their capacity (having several scenarios at the ready depending on their
resources).
39 This seems to be in the process of being signed (including USTB) at the time of writing this report.
43
Final external evaluation of the ACF project Improve access to WASH in the Ger areas of Ulaanbaatar, Mongolia
Groupe URD - TDM | June 2015
- Be proactive to increase visibility within the international community (INGO, UN, World
Bank, Asian Development Bank, etc.).
- Keep regular contact with ACF and other partners. In the medium term (after the
survival phase), it would be useful to regularly analyse the WaSH sector and to send this
analysis to partners (and make it available on the website). The aim here would be to
remain visible and become an important local actor in terms of analysis.
2. Support the EcoSan activity
- Contribute to advocacy in favour of subsidies for EcoSan toilets and the emptying
service;
- Work closely with MonESIC, as this will be to everyone’s benefit: EcoSan is an innovative
approach which could help to make WASH Action known, but without the expertise and
support of WASH Action, there is a danger that the EcoSan adventure will come to an
end.
- - Draw up a SuSanA file to share lessons learned, which could help to improve visibility at
the international level, and fund-raising.
LONG TERM:
3. Promote the WaSH sector in Mongolia, in line with what was done with ACF
- Optimise the 2014-2017 WaSH advocacy strategy, as well as the advocacy potential of
the study “sanitation in Mongolia…” (dissemination, organisation of workshops, etc.)
- Improve the versatility of the WaSH Action staff (water, sanitation and hygiene);
- Develop a virtual Resource Centre (website) for the Mongolian WaSH sector;
- Resume action research to improve grey water management (look for partnerships with
researchers + volunteers + funding)40.
Recommendations for MonESIC:
- Have an immediate survival strategy: be more proactive and develop niche markets by
optimising use of the equipment.
- Work closely with WaSH Action, as this will be to everyone’s benefit: EcoSan is an
innovative approach which could help to make WASH Action known, but without the
expertise and support of WASH Action, there is a danger that the EcoSan adventure will
come to an end.
- Co-develop an advocacy and fundraising action plan to help develop the EcoSan sector:
indeed even though the emptying service could ideally become financially viable in the
long term, the scaling up process and social marketing campaigns require subsidies.
Recommendations for the Water Agencies:
- Finance a small-scale project by WaSH Action to test an initial partnership (for around
50 000€)
- Support the implementation of the advocacy plan
- Adopt a clearer position about the possibility of pursuing the collaboration with the
Mongolian watershed authorities.
40 same remark : this is not a priority for WaSH Action (except, of course, if funding opportunities open up for this subject)
44
Final external evaluation of the ACF project Improve access to WASH in the Ger areas of Ulaanbaatar, Mongolia
Groupe URD - TDM | June 2015
Recommendations for all actors involved in advocacy41:
- Include public showers in the advocacy work vis-à-vis institutions (to be included in
urban development and facilities plans for Ger areas)
- It would be useful to compare different WaSH solutions (economically, environmentally
and in terms of job creation) with regard to meeting the short/ medium/ and long-term
needs of those who live in the Ger areas. This would not require a great deal of time to be
invested by WaSH Action in the short term to increase its reputation amongst WaSH
actors in Mongolia.
- Optimise the advocacy potential of the study, “Sanitation in Mongolia: experiences,
challenges and recommendations”: dissemination, organisation of workshops, etc.
41 Not applicable to WaSH Action in the short term, as they have other priorities
45
Final external evaluation of the ACF project Improve access to WASH in the Ger areas of Ulaanbaatar, Mongolia
Groupe URD - TDM | June 2015
Annexes
1/ Best practice: Hygiene promotion
2/ Classification based on the DAC criteria
3/ People met
4/ Field timetable
5/ Bibliography
6/ Contents of the workshop with the ACF / WaSH Action national staff
7/ Online survey
46
Final external evaluation of the ACF project Improve access to WASH in the Ger areas of Ulaanbaatar, Mongolia
Groupe URD - TDM | June 2015
Annex 1: Best practice: Hygiene promotion
TITLE Hygiene promotion in 22 schools in Ulaanbaatar, Mongolia: introduction of
sustainable changes at both the individual and institutional levels 2009-2015
HISTORY
- 2009: targeting of poor families
Result: Unsuccessful because the people felt stigmatised, and did not come to the
training courses.
- 2010: change of strategy: work at the school level
Introduction of child to child methods and health clubs
4 schools in SKHD, 1 school in BZD
Interesting impacts observed in that the children transmit good practice to their
families
Distribution of soap in schools to create a context that is favourable to good
hygiene practices
Result: Limited success as there was no high level institutional support (the
teachers were reluctant to have external interventions if they were not officially
recognised).
- Official authorisation signed with Municipal Education Department
- Since 2011 : scaling-up process (+5 new schools)
Development and continuous improvement of hygiene training toolkit (which
gradually becomes a WaSH training toolkit, rather than just about hygiene). The
improvement of the hygiene promotion kit was carried out in collaboration with
UNICEF.
Validation by the Education Research Institute of the changes to health curricula
proposed by ACF
� Integration into health curricula at the national level
- 2012: scaling up (made possible by funding from the Swiss aid agency): +
14 new schools in SKHD (22 in total42)
Key factor of the sustainability of the health clubs: support from the school boards
and the teachers
Main outcomes and impacts:
-a lot of awareness-raising materials (including high quality video spots)
-high quality WaSH toolkit and training courses validated by the official institutions
and widely disseminated and used
-the replication of numerous health clubs
-a resource centre in a model school
-health club Facebook page
-contribution to major changes to national regulations:
-WaSH Minimum requirements at school
-It is probable (based on ACF’s advocacy work) that the Ministry of Finance
(or the schools themselves) will allocate an annual budget to buy soap for
every school.
42 and not 24, because projects have continued in 2 of the schools targetted by previous projects
47
Final external evaluation of the ACF project Improve access to WASH in the Ger areas of Ulaanbaatar, Mongolia
Groupe URD - TDM | June 2015
INNOVATIVE
FEATURES
-Strictly speaking, the peer training method (child to child) and the health clubs are
not innovative, because they have been tried and tested in other countries, but they
were new to Mongolia.
-Similarly, the use of popular communication techniques for hygiene promotion
messages was new in Mongolia: the production of videos shown on the internet,
social marketing, campaigns on social media like Facebook and Twitter (hash tag
campaign), and flash mobs.
SPECIFIC
RECOMMEN
DATIONS
FOR
DISSEMINAT
ION
in Mongolia:
-Continue to invest in social media which appear to be very popular in Mongolia,
and not only amongst the young.
-Look into ways of raising awareness amongst young children, as early as
kindergarten, because habits are learned early, and it is easier to change behaviour
early on, rather than when bad habits have been adopted.
48
Final external evaluation of the ACF project Improve access to WASH in the Ger areas of Ulaanbaatar, Mongolia
Groupe URD - TDM | June 2015
Annex 2: Classification according to the DAC criteria
c note
(1 mini, 5 maxi) Explanation
1 2 3 4 5
Sustainability x
ACF’s exit strategy was not sufficiently anticipated. ACF’s
withdrawal will have an obvious negative effect on MonESIC, who
will not meet all the challenges facing it without external support.
The pilot solutions have been tested technically, but have not been
tested socially for long enough.
Tolgoit is likely to continue to manage the multi-service kiosk in
the long term
WaSH Action has a lot of potential, but is still very fragile and still
has too little financial visibility. One of the challenges for this
young national NGO will be to take over the advocacy strategy and
to adapt it to their means
Impact x Several improvements to national regulations directly attributable
to ACF and its advocacy activities.
Sanitation: the project led to changes in behaviour (acceptance)
vis-à-vis the ecological sanitation approach, but this remains at the
pilot level, which could well be abandoned if there is no
investment in social project management + subsidising of the
service
Hygiene: excellent long-term impact, which has led to a drop in the
number of illnesses in the targeted schools.
Plusieurs améliorations de la réglementation au niveau national
directement attribuables à ACF.
Coherence x ACF’s decentralised solutions remain relevant with regard to the
new urban development plans: the two types of solution (large-
scale decentralised public infrastructure vs decentralised
solutions at the household level) are, in fact, complementary.
It is not likely that all the Ger areas will be transformed into
apartments and connected to the network.
The models that have been tested can meet the WaSH needs of
people in the Ger areas in the short/medium term.
Coverage x Coverage is good in terms of the dissemination of innovations
(even though it is difficult for the economic model of the emptying
service to have toilets to empty which are scattered throughout
the district).
Relevance/app
ropriateness
x The project meets people’s essential WaSH needs, even though it
does not cover needs in terms of public showers or grey water
management. ACF’s innovations are promising and adapted to an
extremely cold climate, they have inspired large organisations like
UNICEF, and have been integrated into the urban development
preliminary studies for the2nd biggest city in the country.
Effectiveness x The project was unable to meet its objectives in terms of the
49
Final external evaluation of the ACF project Improve access to WASH in the Ger areas of Ulaanbaatar, Mongolia
Groupe URD - TDM | June 2015
dissemination/accompaniment of the tested solutions for
sanitation and water, which was predictable with regard to the
duration of the project. This objective was too ambitious.
Efficiency x The training courses for local partners were useful for USUG and
Tolgoit, but not appropriate for MonESIC.
The donors’ high level of flexibility and confidence allowed the
field staff to adapt the project strategy and activities in response to
opportunities and changes in the context, and thereby increase
efficiency as the main decisions made by the project team were
completely relevant. On the other hand, it is regrettable that
Artois-Picardie Water Agency had to repeatedly cancel its
participation in forums organised with the other institutional
actors.
50
Final external evaluation of the ACF project Improve access to WASH in the Ger areas of Ulaanbaatar, Mongolia
Groupe URD - TDM | June 2015
Annex 3: People met
Partenaires de hand-over (MonESIC,
Tolgoit)
-MonESIC: Myagmarsuren Shagdarjav, executive Director
(4/06)
-MonESIC: Zaya Munkhchimeg, financial management
(4/06)
-Tolgoit : Ms Suuri, head of CBO (11/06)
-Gardienne vendeuse d’eau du kiosque multiservice
(11/06)
Consultants mongols clefs ayant
accompagné ACF : MIRIM (Marketing
Innovation Research Institute of
Mongolia), Bolor Legjem, Development
Solution
-BAT-ORSHIKH Erdenebat, executive Director of
Development Solutions (2/06)
-Bolor LEGIEM (4/06)
-David CHACE, MIRIM manager (8/06)
-Yanjinpagma Nyamsuren, Project Manager MIRIM
Consultant (8/06)
Equipes ACF / WaSH Action of Mongolia -Pier Francesco Donati: EAH Head of Department
-Clément Philit: Directeur pays
-Bulganchimeg.B, Ecosan Officer
-Ariunaa, Hygiene Promotion Manager / WaSH Action
-Shijirtuya, Environmental Sanitation Manager / WaSH
Action
-Zolzaya, Finance manager/ WaSH Action
-Baatar, Driver logistics
-Bodigerel, Human Resource Officer/ WaSH Action
-Amarkhuu, Field officer
-Battseren, head of WaSH Action
-Khaliunaa, Communication officer/ WaSH Action
-Javzansuren, Water supply manager / WaSH Action
Bayrmaa.E, Field officer
Différentes catégorie de
« bénéficiaires »/partenaires :
propriétaires d’Eco-toilettes usagers du
service de vidange, écoles, usagers des
kiosques à eau, différentes personnes
formées par ACF (USUG, River basin,
Schools, etc…), institutions
-ecotoilets users 3° & 26° khoroo (5/06)
-schools 106: Tuvshinjargal, teacher; Erkhtuya, doctor;
Bayarmaa & Ariunzul, Health Club members (5/06)
-Director of Batsurkh school (5/06)
-USUG: Dagvasuren, manager + 2 teachers
Acteurs institutionnels43. Ces entretiens
viseront à comprendre l'implication de
ces partenaires et leur point de vue sur le
projet et ses réalisations (Ministères,
-Ms MYAGMAR Jadamba & DULAMKHAND, Ministry of
education and science - department of coordination of
policy implementation – division preschool and primary
education (4/06)
43 The categorisation of the actors met is more or less arbitrary to the extent that the majority of institutional actors at
different levels are also partners or beneficiaris of ACF either as targets of advocacy or capacity building. The same is true
for UNICEF for example, which has several roles.
51
Final external evaluation of the ACF project Improve access to WASH in the Ger areas of Ulaanbaatar, Mongolia
Groupe URD - TDM | June 2015
Municipalité d’UB, autorités du district et
sub-districts…)
-Ms MUNKHJARGAL, consultant (former health
department of UB Municipality) (4/06)
- 7th khoroo governor: Davaanyam, social worker(5/06)
+ online survey
Acteurs humanitaires et de
développement travaillant sur des
projets similaires dans les zones de
yourtes (JICA, Banque Mondiale, GIZ,
Croix Rouge, World Vision…)
� online survey
Expert -D. Myagmar Dovchin, (10/06)
Bailleurs44 -UNICEF: BATNASAN Nyamsuren
& Robin WARD, WaSH officer (9/06)
Universités et étudiants partenaires -Nazim Uddin, pHD reseracher, Université de Sciences
et Technologies de Pékin (by skype 2/06)
Observations
- site et mise en place du compost d’été (3/06)
- Zip-Zap toilet atEmelt compost site (3/06)
- resource center + museum + technical training room at USUG training center (10/06)
- resource center at school (5/06)
- school toilets rehabilitation (5/06)
- Eco-toilettes installées de 2011 à 2014 (5/06)
- MSWK (11/06)
- coaching meeting on business plan provided by MIRIM for MonESIC & TOLGOIT (8/06)
44 L’équipe d’évaluation recommande à ACF d’inviter les Agences de l’eau lors de la restitution finale à Paris (au moins via
skype ou téléconférence)
Final external evaluation of the ACF project Improve access to WASH in the Ger areas of Ulaanbaatar, Mongolia Groupe URD - TDM | June 2015
Annex 4: timetable
1st
we
ek
Monday 1er /06
Day OFF in Mongolia
Tuesday
02/06
Wednesday
3/06
Thursday
4/06
Friday
5/06
Sat
6/06
Sun
7/06
mo
rnin
g
Biblio + Inception
report
-Briefing HoM
-Meeting with
Development Solution
Compost site visit
Compost processing
-Bolor (consultant)
-education Minister
members
-shools (including health club,
resource center, toilets rehab)
Data analysis
+ biblio
aft
ern
oo
n
Briefing with ACF
mission
Planning of evaluation
ACF MonESIC -EcoSan toilets users in SKHD
-7th khoroo social worker
e v Skype with Nazim Data analysis + biblio Skype meeting with TDM
2d
we
ek
Monday 8/06 Tuesday 9/06 Wednesday
10/06
Thursday
11/06
Friday
12/06
Sat
13/06
Sun
14/06
mo
rnin
g
MIRIM workshop with
USUG, Tolgoit &
MonESIC
MIRIM meeting
Data analysis
UNICEF
-Myagmar Dovchin (expert
& consultant)
-Thibault (CARITAS)
TOLGOIT & Multiservice
Water Kiosk
Ministry of Environment, Green
Development & Turism (3 times
postponed …)
Preparation of debriefing seminar
off
off
aft
ern
oo
n
WaSH Action of
Mongolia workshop
(1/2)
WaSH Action of
Mongolia workshop
(2/2)
USUG training center Tuul River basin
authority
Debriefing (=evaluation
workshop) with ACF staff +
partners
ev
e
nin Skype meeting with TDM Data analysis + biblio
AP Water Agency A. Courtecuisse
(Skype) Skype meeting with TDM Skype meeting with TDM
53
Final external evaluation of the ACF project Improve access to WASH in the Ger areas of Ulaanbaatar, Mongolia
Groupe URD - TDM | June 2015
Annex 5: Bibliography
General reports from ACF
- ACF Mongolia, 2012/2014 Knowledge, Practice and Attitude KAP survey on Water,
Sanitation and Hygiene promotion in ACF targeted Ger area in Songino Khairkahn
District, Ulaanbaatar, Final findings report, 2014
Ecosan toilet monitoring report 2014 -> inclus dans cette KAP survey
- ACF Mongolia & WaSH Action of Mongolia, Sanitation in Mongolia: experiences,
challenges and recommendations “We can’t wait”, January 2015
- ACF Mongolia, APPENDIX L. GER AREA INVESTMENT PLAN AND PROGRAM Water
Supply and Sanitation Infrastructure Improvement Project for Darkhan Final Report,
September 2014 (=CDIA prefeasibility study 2014 (mentionné dans les TDR)
- ACF Mongolia, “Improve access to water, hygiene and sanitation in the Ger areas in
Ulaanbaatar, Mongolia, through the dissemination of innovative and proven solutions”
MAY 2013 – DECEMBER 2014, Water Agencies activities report January 2015
- ACF Mongolia, Franziska Bock, Institutional analysis of WATER SUPPLY, SANITATION
AND HYGIENE IN MONGOLIA, 2014
- ACF, Sustainable Water, Sanitation and Hygiene services, GER WASH PROJECT, SDC
FINAL REPORT, april 2015
- coordonnées GPS des éco-toilettes
- Mongolian University of Life Sciences, compost analysis, June 2015
- Enkhtungalag B (Central Laboratory of Agency for Specialized Inspection), Battseren
(ACF, Purevdorj (MNUMS, School of Public Health), Microbiological study of compost in
the open area, 2014
Hygiene Promotion
- ACF Mongolia “Ger WASH project: Sustainable Water, Sanitation, and Hygiene services”
2012-2014/ Hygienic training and public advocacy work and monitoring at the schools
- ACF Mongolia, Survey report on hand washing conducted among secondary school
students in Songinokhairkhan district, Ulaanbaatar, dec 2014
- matériel de formation et plaidoyer développé par ACF de 2009 à 2015 (supports de
cours, boîte à outils, posters, clips vidéo)
Advocacy
- ACF, Eric Reinshtein, Coordinator of WaSH project, advocacy document FOR AN
EQUITABLE AND SUSTAINABLE WATER TARIFF IN ULAANBAATAR, 2014
- ACF MONGOLIA MISSION, ADVOCACY STRATEGY, (2014-2016), sept 2014?
- ACF, ADVOCACY ACTION PLAN, Water, Change goal: National policies and plans are to be
implemented for water conservation, protection, water supply to face water pressure
due to human activities, population movements and poor resource management, déc
2014
- ADVOCACY ACTION PLAN, Sanitation (Change goal: Legal frame and standards for
sustainable sanitation are improved, déc 2014
- ACF, ADVOCACY ACTION PLAN, Hygiene, Change goal: National curriculum on HP
continuously disseminated in primary schools and kindergartens, déc 2014
- ACF International advocacy toolkit, 2013
WASH Action of Mongolia
54
Final external evaluation of the ACF project Improve access to WASH in the Ger areas of Ulaanbaatar, Mongolia
Groupe URD - TDM | June 2015
- WaSH Action of Mongolia, strategy Plan 2015-2017, January 2015
- SWOT analysis of WaSH action of Mongolia, early January
- site internet de WaSH Action
- facebook page of WaSH Action
MonESIC
- statuts
- organigramme
- business plan
- comptabilité (cash book) janvier, février, mars, avril 2015
Consultants
- BAT-ORSHIKH Erdenebat Development Solutions, final report of establishing and
strengthening the NGO through improving operational and financial management, 5 juin
2015
- http://www.irim.mn/index.php/en/mirim
- MIRIM, Business Plan Training Program for Sanitation Enterprises Final progress report
June 18th, 2015
Other:
- http://www.publicfinanceforwash.com/
- World Bank : report on onsite sanitation in cold countries (à paraître en septembre
2015)
55
Final external evaluation of the ACF project Improve access to WASH in the Ger areas of Ulaanbaatar, Mongolia
Groupe URD - TDM | June 2015
Annex 6: Workshop with the ACF / WaSH Action national staff
Monday the 8th June afternoon
& Tuesday the 9th June
I/ Review of past activities
Session 1 :
- Propose following categories (steps): testing solution-> solution selection -> scaling up -
> Advocacy -> adoption at large scale, explaining that these steps can sometimes carried
out in parallel. Give examples.
- Divide the group into three teams: water / sanitation / hygiene
- Ask each group to write on pieces of paper the main activities of the past 6 years,
indicating if possible the year or period of achievements of the activity and the contract
(WA, SDC, UNICEF, ACF )
- The groups will stick on the wall the different activities proposed in the 3 following
categories (1-solution testing / pilot project; 2 / scaling up; 3 / advocacy)
- in Plenary: all groups validate (or move) paper bonded by other groups. Then:
Discussion of the 3 different WaSH sectors in the progressivity of the test to the
widespread adoption through the scaling up. What are the most significant
achievements?
56
Final external evaluation of the ACF project Improve access to WASH in the Ger areas of Ulaanbaatar, Mongolia
Groupe URD - TDM | June 2015
Session 2 :
Plenary Intro: What was the initial strategy? What were the main adaptations (or shift) of the
strategy?
- Division in subgroup water / sanitation / hygiene
- Leave the product materials (papers with activities), and for each activity, consider the
following questions in groups:
1. What has still to be done in relation to what was planned (following the rationale of
the project)? (Optional question depending on available time)
2. In your opinion, what activities should be conducted (new business) to optimize the
work already done, in order to achieve a good impact -> write the new activities on
white paper
3. Do WaSH Action wants to carry out these activities that would maximize the impact?
4. What are the most difficult activities to finance?
5. Imagining that funding is available, does WaSH Action has the capabilities to carry
out these activities?
Each group is asked to prepare a synthetic restitution on paperboard
While the groups are working, and depending on the degree of work-progress, I bring new
questions in each subgroup: among all these activities, did you face failures? Disappointments?
Have you learned lessons? Would have it been possible to implement activities in a different
manner?
57
Final external evaluation of the ACF project Improve access to WASH in the Ger areas of Ulaanbaatar, Mongolia
Groupe URD - TDM | June 2015
II/ Advocacy45
� Working in small groups:
What does advocacy mean for you?
What are the advocacy actions that have succeeded and why? What advocacy actions have failed
and why?
How all ACF advocacy activities planned in the strategy fit in in with WaSH Action strategy?
Who are the others stakeholders doing advocacy in UB?
� Questions for plenary debrief:
-Do you monitor advocacy activities (with performance indicators)? (See Manual ACF)
-How will you promote / disseminate the study "we can not wait"? what are the links with your
advocacy strategy?
III/ Next steps: looking in the future
1. From ACF-> to WaSH Action
� 3 subgroups work on the following questions: What are the main changes
between WaSH Action nd ACF (+ and -)? What will be the similar activities? What
activities will you stop? What new activities will you implement?
cross fertilization for restitution
+ contracts, consultancies and proposals review
2. Competencies map
� Introduction in plenary intro: What are the skills needed to WaSH Action to be
viable?
� Mutual interview with neighbor on 3 questions (the neighbor will present the
answers):
1. In the last 6 years, what are the formations that were the most useful to you? (List)
why?
2. What are the skills that I will bring to WaSH Action
3. What could WaSH Action bring to me? (=Why I want to involve myself in WaSH Action?)
45 this workshop on advocacy was not done in group (lack of time)
58
Final external evaluation of the ACF project Improve access to WASH in the Ger areas of Ulaanbaatar, Mongolia
Groupe URD - TDM | June 2015
* What do you do in 1/3/6 months if Wash Action doesn’t succeed in getting substantial
funding?
How long can you stay without any salary?
From July 2015, what will be your level of investment? full-time? part-time?
59
Final external evaluation of the ACF project Improve access to WASH in the Ger areas of Ulaanbaatar, Mongolia
Groupe URD - TDM | June 2015
Annex 7: online survey
The low number of respondents meant that this could not be used to provide analysis for the
evaluation. The raw results are mentioned here for information purposes only.
Q2: What is your overall feeling about ACF WaSH programs ? / ACF-ийн WaSH
төслүүдийн талаарх Таны бодол ?
3 comments:
-Thank you for helping schoolchildren to create a clean environment for school children
-To keep activities continuesly. The project is only one project, which did improvement based
on the survey and pilot result. Also involved researchers.
-салбарын байгууллагуудтай хамтарч ажиллахад түлхүү анхаарах (Should focus more to
cooperate with relevant stakeholders)
Q3: What are the main changes in the WaSH sector in Mongolia in the last 10 years /
Сүүлийн 10 жилд Монголын WaSH салбарт ямар чиглэлд гол өөрчлөлтүүд гарсан бэ?
(1/2)
60
Final external evaluation of the ACF project Improve access to WASH in the Ger areas of Ulaanbaatar, Mongolia
Groupe URD - TDM | June 2015
3 comments :
Сургуулийн бие засах газрыг үлгэр жишээ болгон өөрчилж, удирдлагуудад хийх
боломжийг мэдрүүлсэн. (School toilet room was rehabilitated as a good practice for others
and school principal had good understanding of importance of the improvement.)
Established Water supply and sewerage regulation committee, improved legal environment.
used different types of technologies on waste water treatment and pit latrine. Missed
coordination and evaluation.
Монголын нөхцөлд тохирох ариун цэврийн байгууламж бий болгох нь бидний хамтын
зорилго, зорилт байх болно (Our main objective, challenge is develop toilet design that can
be adjusted Mongolian weather condition.)
Q4: What are the main changes in the WaSH sector in Mongolia in the last 10 years /
Сүүлийн 10 жилд Монголын WaSH салбарт ямар чиглэлд гол өөрчлөлтүүд гарсан бэ?
(2/2)
3 comments:
Pit latrine new standard approved.
Гар угаах сургалтын материалыг мэргэжлийн түвшинд хийж, сургалтыг явуулсан.
Сурагчдын клубыг бие даан ажиллахад нь дэмжлэг үзүүлсэн. (We have developed hand
washing training material and conducted the training)
In efficient use of budget not based on long term strategy in the WaSH
Q5: What are the changes due to ACF ? (among the previous ones) / Дээрх чиглэлүүдэд
гарсан өөрчлөлтүүдэд ACF –ийн оролцоотой өөрлөлт гарсан нь аль вэ? (1/2)
61
Final external evaluation of the ACF project Improve access to WASH in the Ger areas of Ulaanbaatar, Mongolia
Groupe URD - TDM | June 2015
2 comments:
Эрүүл мэндийн болон Боловсролын сайдын хамтарсан шаардлага гарсан. (Joint order
was approved by Ministery of Health and Ministry of Education.)
улам сайн технологи нэвтрүүлэх (More efficient technologies have to be implemented)
Q6: What are the changes due to ACF ? (among the previous ones) / Дээрх чиглэлүүдэд
гарсан өөрчлөлтүүдэд ACF –ийн оролцоотой өөрлөлт гарсан нь аль вэ? (2/2)
1 comment:
Боловсролын салбарт ялангуяа багш нарын "Гар угаах" сэдвийг заахад өөрчлөлт гарсан.
(Education of teachers, especially in "hand-washing" to teach the subject changes)
Q7: According to you, what will remain from ACF work here? / Таны бодлоор ACF –ийн
ажлуудаас үлдэцтэй ямар ажлууд байна вэ ?
62
Final external evaluation of the ACF project Improve access to WASH in the Ger areas of Ulaanbaatar, Mongolia
Groupe URD - TDM | June 2015
1 coment:
Бодлогын болон нэгжийн түвшинд ажиллаж чадсан. (Policy and worked with the unit level)
Q8: According to you, what will remain from ACF work here? / Таны бодлоор ACF –ийн
ажлуудаас үлдэцтэй ямар ажлууд байна вэ ? (2/2)
Q9: What are the main WaSH coordination events and bodies in Mongolia? / Монголд
WaSH чиглэлээр үйл ажиллагаа, ажил зохион байгуулдаг гол байгууллагууд нь юу вэ?
?
ACF
The Ministry of Health, MECS
MCUD, MOE, MOHealth,
олон байгууллага байгаа (many!)
Q10: Do you know WaSH Action of Mongolia ? / Та WaSH Action of Mongolia-г мэдэх үү?
100% : YES
2 comments:
But I need to learn more about WaSH Action of Mongolia.
Маш санаачлагатай, хүртээмжтэй ажилладаг байгууллага. (Very proactive and inclusive
organizations.)
63
Final external evaluation of the ACF project Improve access to WASH in the Ger areas of Ulaanbaatar, Mongolia
Groupe URD - TDM | June 2015
Groupe URD
La Fontaine des Marins
26170 PLAISIANS
Tel : + 33 (0)4 75 28 29 35
www.urd.org