Monitoring & Evaluation of BORDA DEWATS
Global M&E Report 2017 / 2018: Sustainable performance assessment of BORDA DEWATS
1P R E FA C E
Preface
Developing a tailor-made DEWATS M&E
methodology (Global Monitoring Form GMF2;
2017) was an achievement of cross-regional
collaboration and the much needed next step
to generate knowledge on the performance of
DEWATS in a holistic way. Having a methodol-
ogy in place which is focusing on all relevant
sustainability criteria (defined by so-called
Statements of Change) rather than on tech-
nical aspects only, is a major advancement
and asset for our work & credibility - and that
of our partners – in the future.
With the ratification of the Sustainable Devel-
opment Goals (SDGs), BORDA is finally provid-
ed the mandate and framework to continue its
work and act in line with a globally pursued
agenda. The SDGs highlight the importance
and value of quantitative (i.e. measurable)
data for wastewater treatment on country
and global level and, also donor organisations
worldwide further align their funding strate-
gies alongside the SDGs. Hence, being able to
monitor our work on both programmatic and
on system level provides international donor
organisations the much-needed evidence for
successful investments in/scaling-up of de-
centralized sanitation solutions as part of the
(peri-) urban development. Being able to gen-
erate knowledge within the BORDA network
to advance such – so far – limited evidence
is key for the transformational process the
sanitation sector is about to experience in
the decades to come. Internally, this report
offers us the great chance for cross-regional
learning and measuring the impact of our
work: by acknowledging the challenges in
establishing sustainable sanitation services,
this report is an invitation for constructive and
self-critical reflection of our work as part of a
necessary organisational learning and quality
management process. For that reflection to
take place, we need to take specific findings
forward to engage in a few distinct discussions
within the Global Directorate where needed.
On global level, this work also reminds us of
the required and not yet finalized positioning
of BORDA towards:
i DEWATS implementation within the
new programmatic approaches (Livea-
bles cities, integrated Water Resource
Management),
i DEWATS technology upgrades to meet
new legisations and the requirements of
city-wide cluster concepts and
i BORDAs global responsibility for the
correct application and implementation
of DEWATS in the sector
Equally important, such evidence-based con-
clusions and recommendations, if taken up
and disseminated regionally, will also open
the door for increased cooperation with a
variety of stakeholders and sectors beyond
the water and sanitation sector. Therefore,
we would like to see our regions taking up the
courage to disseminate selected findings and
pro-actively showing our work to the world,
with all of its potential and obstacles, in order
to pin down hot spots of further commitment
and to consolidate our positioning as honest
broker in the sanitation sector.
Enjoy the read
BORDA and its BMZ partner network have committed
themselves to professionalism and quality throughout its
programs and fields of expertise, not only since the
formulation of the BNS Quality Policy in 2012.
‘Monitoring and Evaluation’ has long been emphasized
as a crucial business aspect for accountability, credibility
and quality assurance, thus paving the ground for
our future success. Therefore, ‘M&E’ is also reflected in
BORDA’s current ’Mission & Vision’ statement.
Regarding BORDA’s decentralized sanitation solutions,
the DEWATS inventory 2017 was the first step to
structurally extract the knowledge and assets BORDA
has created over the last 30 years across the globe,
and to make it available to our network – and beyond.
Stefan Reuter Andreas Schmidt Alexander Wolf
2 3N AV I G A T E T H E R E P O R T
The report starts off with the very consensed
Excecutive Summary. It highlights positive
findings & challenges, formulated as con-
densed conclusions and recommendations for
either the Global directorate (GD) or country
offices. This is done to best direct the reader
to the most relevant chapters and findings.
Please always refer to the specific thematic
sections (Statements of Change – ‘SoC’s) to
understand where results, conclusions and
recommendations derive from (Chapter 3).
Thematic sections (Statements of Change) are
always structured as follows:
! Where suitable/available, results and/or parameters are further explained by so-called Background/’Soft’ information. This is information/experience from the regions/
senior staff that puts potentially surprising results in a different light by adding some not-obvious/well-known reasons. Such information is highlighted in boxes and indicated by this icon.
Navigate the report
First, the global evaluation result (Pie chart)
for the SoC is presented, combining all related
parameters and objectives, to an overall score
for the SoC.
Then, a ‘per country’ evaluation for that SoC
is provided (Bar chart).
Secondly, to highlight specific findings for
a SoC, the most often occurring responses
for parameters (i.e. the ones which had the
biggest impact) are shown.
Thirdly, specific (managerial/technical) rec-
ommendations for the SoC are provided.
Picture: Tanzanian Building Agency (TBA) Housing Development Project, Bunju, Dar es Salaam. On-site wastewater management system (designed by BORDA Tanzania) with simplified sewers, biodigester, ABR, PGF, vortex and polishing pond.
4 5G L O S S A R Y
Few definitions, existing in parts already but
useful to be repeated, help to put the M&E
activities into context and understand the
evaluation:
9 Statements of Change (SoC) have been
identified as being crucial for the
comprehensive evaluation of BORDA
DEWATS. Their formulation is based on
the sustainability criteria for small-scale
sanitation systems as defined by ISF through
previous work (Functioning Technology,
Effective Management, Sustaining Demand
and Sustainable Financing). All SoCs were
jointly elaborated by BORDA’s M&E team,
BORDA’s Global Directorate and country
directors in 2016.
Each SoC is defined by 1 to 3 Objectives.
Objectives consist of Prioriy Indicators (PI)
which in turn are devided into Parameters.
Parameters are in most cases the exact
questions as asked during field surveys.
COD - Chemical oxygen demand – is
commonly used at BORDA to express the
level of organic wastewater pollution.
For M&E activities, COD as parameter is
considered the minimum required indicator
for analytical effluent qualification. Additional
indicators (turbidity, nutrient concentrations)
are desirable and suggested, but are not
enforced by the M&E program to keep the
costs for M&E efforts at a minimum.
Management entity (m.e.) The management
entity is a person or a group of people having
the responsibility for all required managerial
tasks relating to DEWATS O&M after handover
(financial administration, employing an
operator, organizing repairs etc). It typically
consists of members of the served community,
institution or SME, the household owner or an
external or governmental agency.
Operator The system operator is defined as
the ‘person/caretaker/operator assigned and
responsible for O&M activities’ and can be a
community member, an employee (of SME or
institution), the house-owner or an external
system provider.
The evaluation method differentiates between
an operator and ‘one or more people without
O&M responsibility taking care of facility’
to cover the situation often observed in the
field where a formal operator does not exist
but some O&M is done by others.
Project types have been re-defined for the
Global DEWATS Inventory in 2017 to have
a standardized description for all BORDA
projects. Here a few selected definitions:
‘CSC - Community sanitation centres’ are
public or semi-public (e.g. in schools or
hospitals) toilet blocks, generally including
showers and laundry areas.
‘RES - Residential DEWATS’ treat domestic
wastewater, e.g. from residential apartments,
single houses or settlements. Community
Sanitation Centres, including public toilets,
connected to DEWATS are also considered as
residential projects.
‘SME DEWATS’ treat the grey- and/ or
blackwater as well as organic industrial
effluent of SME, such as public and private
hospitals (no medical wastewater streams),
slaughterhouses, animal husbandries, food
production, markets, offices, hotels, hostels
and resorts as well as other industries.
‘Institution DEWATS’ treat the grey- and/ or
blackwater of public and private institutions
such as schools and universities (including
vocational schools, orphanages, child centres
and student dormitories), religious centres,
prisons, and all institutions which are not
considered as small and medium enterprises
(SME).
BORDA’s mission and vision according to
BORDA’s mission & vision statement (2017),
all DEWATS are assumed to have the project
objectives of ‘environmental health’ and
‘improvement of living conditions of the
disadvantaged’.
‘Disadvantaged’ in this evaluation method
refers to people with ‘low income according to
project planning’ in order to avoid the difficulty
of formulating a general valid definition for
poverty.
Specific O&M practices There are no clearly
defined standards on DEWATS O&M practices
across all BORDA regions against which to
evaluate ME data. The following definitions
were elaborated by reviewing existing BORDA
SOPs and handbooks as well as outside
publications. These definitions represent a
first draft for global benchmarks on acceptable
DEWATS O&M practices.
The following activities were considered
important for SoC 5 ‘Functioning maintenance
and operation’ and were defined as follows:
Sludge disposal i Safely buried
In specific situations sludge may be safely
buried1 close to the DEWATS. Safely buried
sludge should be covered by at least 30
cm of uncontaminated topsoil. This leads
to anaerobic conditions inside the sludge
helping to further stabilise it, removing smells
and creating a safe pathogen barrier to the
top. Buried sludge should not be disturbed for
a minimum of 3 years to avoid contact with
buried pathogens.2
i Reused
About safe reuse of wastewater sludge (Still
et al, 2012):’Agricultural use at agronomic
rates is permitted if stability class 1 or 2 is
achieved. Vegetables shall not be grown which
are consumed raw, that touch or are below the
soil/sludge mixture or have harvested parts
below the soil surface. Crops whose edible
parts do not touch the soil/sludge mixture,
shall not be harvested until 90 days after
the last sludge application. Animals shall
not be grazed on land until 90 days after the
last sludge application. Notably, there are no
restrictions on the production of industrial
crops (e.g. timber).’
Since raw DEWATS sludge never reaches
class 2 requirements, it should not be used
for agriculture, even when following the rules
for ‘safe reuse’ above.
1 Based on information in Still et al (2014) and ‘DEWATS Operation and Maintenance Manual’.
2 For more details on correct burial technique see Still et al (2014) p. 120 – 122.
i On land, not safely reused or buried
Unsafe on-site burial:
] Burial holes in very sandy or
gravelly soils because of the potential of
groundwater contamination.
] Burial holes on eroded banks or
cutaways where activity could erode
the vertical face into the sludge itself,
exposing pathogens.
] Burial holes located within 15m of a
stream or well.
Scum disposal i Sealed into domestic solid waste
collection
Sealed means at least three layers of knotted
water-tight plastic bags
i Safely stored
Safely stored in a water- and air-tight
container, placed on shaded, stable ground
and inaccessible to animals and children
i Safely buried
Dispose the scum safely into a pit, which is at
least 10 meters away from dug or bore wells.
Cover with at least 30 cm of uncontaminated
topsoil.
Solid waste disposal i Sealed into domestic solid waste
collection
Sealed at least by three layers of knotted
water tight plastics bags
i Safely stored
Safely stored in a water- and air-tight
container, placed on shaded, stable ground
and inaccessible to animals and children
i Safely buried
Dispose the waste safely into a pit, which is at
least 10 meters away from dug or bore wells.
Cover with at least 30 cm of uncontaminated
topsoil.
Glossary
6 7T A B L E O F C O N T E N T S6
Executive Summary — 8
1 Introduction — 14
1.1 Objectives of the BORDA M&E program — 15
1.2 Objectives of this report — 15
1.3 Project selection process — 16
1.4 Visited projects — 17
1.5 Team members involved — 19
2 BORDA’s M&E methodology — 20
2.1 Evaluation matrix — 21
2.2 Scoring — 22
2.3 Assumptions made during evaluation method development — 24
2.4 Data quality-assurance (QA) — 25
2.4.1 Data plausibility testing — 25
2.4.2 Confirmation of draft results between HQ and regions — 25
2.4.3 QA for wastewater concentration measurements — 25
3.4.5 Project specific objective ‘Pre-treat ww before discharge to solid-free sewer’ — 41
3.4.6 Project specific objective ‘Reduction of O&M costs’ — 42
3.5 SoC 4: Functioning technology – systems are operating as intended — 44
3.5.1 Statement of Change structure — 44
3.5.2 Result overview on Statement of Change level — 44
3.5.3 Parameters with strong impact — 45
3.5.4 Recommendations regarding strong impact Parameters — 47
3.6 SoC 5: Functioning maintenance - systems are maintained as intended — 49
3.6.1 Statement of Change structure — 49
3.6.2 Result overview on Statement of Change level — 50
3.6.3 Parameters with strong impact — 50
3.6.4 Recommendations regarding strong impact Parameters — 55
3.7 SoC 6: Sustaining demand - system is available, used to capacity and acceptable — 57
3.7.1 Statement of Change structure — 57
3.7.2 Result overview on Statement of Change level — 57
3.7.3 Parameters with strong impact — 58
3.7.4 Recommendations regarding strong impact Parameters — 58
3.8 SoC 7: Effective management - active and accountable management entity and operator — 60
3.8.1 Statement of Change structure — 60
3.8.2 Result overview on Statement of Change level — 60
3.8.3 Parameters with strong impact — 61
3.8.4 Recommendations regarding strong impact Parameters — 66
3.9 SoC 8: Sustainable financing – sufficient income to cover all short and long term costs — 68
3.9.1 Statement of Change structure — 68
3.9.2 Result overview on Statement of Change level — 68
3.9.3 Parameters with strong impact — 69
3.9.4 Recommendations regarding strong impact Parameters — 73
3.10 SoC 9: Quality planning, design and construction — 74
3.10.1 Statement of Change structure — 74
3.10.2 Result overview on Statement of Change level — 74
3.10.3 Parameters with strong impact — 75
3.10.4 Recommendations regarding strong impact Parameters — 77
4a Appendix A: M&E Matrix structure and Parameter responses
4b Appendix B: Per project average SoC scores
3 Evaluation results — 26
3.1 Global overview across all SoCs — 27
3.2 SoC 1: The sanitation service maintains or improves environmental health — 29
3.2.1 Statement of Change structure — 29
3.2.2 Result overview on Statement of Change level — 29
3.2.3 Parameters with strong impact — 30
3.2.4 Recommendations regarding strong impact Parameters — 32
3.3 SoC 2: The sanitation service improves the living conditions of communities — 33
3.3.1 Statement of Change structure — 33
3.3.2 Result overview on Statement of Change level — 33
3.3.3 Parameters with strong impact — 34
3.3.4 Recommendations regarding strong impact Parameters — 36
3.4 SoC 3: The service achieves project specific impact — 37
3.4.1 Statement of Change structure — 37
3.4.2 Result overview on Statement of Change level — 38
3.4.3 Project specific objective ‘Produce and use biogas’ — 39
3.4.4 Project specific objective ‘Reuse of wastewater’ — 40
Table of Contents
8 9E X E C U T I V E S U M M A R Y
This internal report is intended for BORDA
management on global, regional and country
level.
The 2017/2018 edition is BORDAs first global
M&E report of its kind; its dissemination the
second last step required to complete the M&E
program’s learning cycle by finally translating
the findings into evidence-based corrective
action. All data and observations were
collected by trained staff and conclusions and
recommendations were supported by valuable
knowledge from our network to understand
‘the context behind’ whenever possible. The
global M&E report is intended to be published
in one-year intervals.
The report’s objectives are to highlight general
areas of strength and concern of existing DEWATS
projects, allow for evidence-based global
(programmatic) and local corrective action,
motivate cross-regional learning and identify
topics of possible future in-depth analysis.
Specifically, this report evaluates project
outcomes and impacts in these areas:
Environmental health, Improved living
conditions of served communities, Project
specific impact, Functioning technology,
Functioning maintenance, Sustaining demand,
Effective management, Sustainable financing,
Quality of planning, design and construction.
The underlying M&E methodology was
developed in cooperation with the Institute
for Sustainable Futures (ISF). It is based on
the most relevant sustainability criteria for
small-scale sanitation systems and BORDA’s
M&E requirements as specified by BORDA’s
Global Directorate and country directors in
2016.
In total, 140 systems have been monitored
between April 2017 and March 2018 in 11
countries (Afghanistan, India, Nepal, Lesotho,
Tanzania, Zambia, Cambodia, Indonesia, Laos,
Philippines and Vietnam). 123 systems were
conventional DEWATS systems, 14 consisted
of prefab modules and three designs included
both module types.
Recommendations for the Global Directorate
(GD) are expected to motivate HQ initiated
discussions regarding i) required definitions
or standards and ii) programmatic decisions
and the positioning of BORDA where required.
Recommendations for country directors highlight
areas of DEWATS performance which require
corrective actions depending on countries.
Environmental health (SoC1)
i C1. Globally, effluent COD concentrations
of many systems (67%) don’t fulfill respective
national discharge standards, as they are valid
at present.
R1. Promote the concept of progressive
implementation and engage in discussions
on appropriate national discharge standards.
(All countries, GD)
i C2. In 45% of visited systems, effluent
COD concentrations are above BORDA
internal discharge standards (200 mg COD/l
for anaerobic effluent, 80 mg COD/l for
aerobic effluent).
R2a. Perform in-depth investigations
(country engineers, available R&D staff) to
identify probable reasons for high effluent
concentrations where those are difficult to
comprehend. (All countries)
R2b. Validate the correctness of the
currently widely used definition of aerobic
treatment/effluent when referring to PGF
installations (GD).
i C3. Field observations showed that
practices of handling and disposal of the non-
liquid DEWATS waste streams scum, sludge
and solid waste are unsafe in 21%, 35% and
22% of the investigated cases, respectively.
So far, there is no global BORDA standard
regarding those practices.
R3. Discuss and decide on the here suggested
definitions on safe handling of waste streams.
Update training materials and emphasize the
importance during workshops. (GD)
i C4. For a large number of visited systems
(44%), existing effluent concentration values
were either not considered reliable because
they were measured by external laboratories
of questionable quality (11%) or no effluent
samples were taken in the first place because
of budget constraints (34%).
R4a. Internal laboratories are still
considered the most reliable option. If
external laboratories are considered, their
quality assessment must be completed before
relying on the laboratories. A guideline how
to proceed is available. (All countries)
R4b. Discuss and decide whether COD
measurements should be mandatory for each
M&E visit (see p.27/28 ). (GD)
i C5. So far, neither nutrient nor pathogen
measurements are part of the methodology,
yet both are crucial to appropriately describe
DEWATS’ impact on environmental health.
Also, the topic of GHG contributions by
DEWATS is not touched in any way.
R5. Discuss and decide whether nutrient
& pathogen concentrations or the topic of
GHG emissions should be included in future
evaluations (see p.32). (GD)
Improving livingconditions (SoC2)
i C1. 18% of all visited residential systems do
not cater for a majority of poor people, although
BORDAs mission statement is: “Aim of all BORDA
activities is to support poor people to establish a
life in a healthy and liveable environment”. Also,
in 10% of cases, the predominant sanitation
option before project implementation was
already classified as ‘improved’.
R1: Discuss and potentially adapt
BORDAs mission statement and internal
communication glossary to ensure that all
future projects align with it. (GD)
i C2. Difficult access for elderly and
disabled was observed at numerous projects
which include sanitation facilities, especially
in Tanzania and Zambia.
R2. Improve/highlight the importance of
access to the sanitation facilities in future
projects. Cross regional learning opportunity
exists with Cambodia, where all such systems
are designed with good access for elderly and
disabled. (Tanzania, Zambia)
C3a. The specific project objectives to
1) improve the previous sanitation options
and 2) to minimize the exposure to fecal
pathogens for the surrounding communities,
were largely met throughout projects, globally.
C3b. Exposure to pathogens was found
to be not well or only moderately managed at
residential systems (apartments and multiple
houses) in 7 and 18 cases, respectively.
R3b. Discuss within country teams the
actual pathogen exposure risks for surrounding
communities in the identified projects and – if
necessary – find solutions to minimize the
exposure risks for system users and their
neighbours. (Afghanistan, Indonesia)
Executive summary
10 11E X E C U T I V E S U M M A R Y
Project specific impact (SoC3)
i C1. The evaluation of project specific
impacts showed very different results,
depending on the objective.
i C1a. ‘Produce and use of biogas’ has been
evaluated with ‘cautious’ or ‘bad’ in (83%; n=29)
of the visited systems, either due to biogas not
reaching the appliances or not being used in
the first place. However, all visits in Zambia
indicated regular use of biogas.
R1a. Investigate why biogas is not used
and/or does not reach appliances. (India, Nepal,
Lesotho, Tanzania, Zambia, Vietnam)
Discuss how relevant the appropriate
and rigorous use and promotion of biogas is
for BORDA, also concerning its positioning
regarding climate friendly technologies.
Decide next steps (see p.39). (GD)
i C1b. ‘Re-use of wastewater’. The objective
is met for most (88%; n=51) of the visited
systems.
R1b. Emphasize the importance of
accurate project objective definitions in
Nepal, where, in many cases in which reuse
is stated as one of the project objectives,
wastewater is actually not being reused at all.
i C1c. ‘Pre-treat ww before discharge to
solid-free sewer’ has been achieved in 31%
of the 13 projects with this objective for
which all required data was available. The
main reasons for poor performance scores are
minor building structure damage and under- or
overload. (see p. 41 for the used indicators and
discuss if considered not suitable)
R1c. Repair systems with minor building
structure damages in Zambia.
C1d. ‘Reduction of maintenance costs’ has
been achieved in all projects where the
objective has been initially set.
Functioning technology (SoC4)
i C1. Lack of COD effluent data (see SoC1;
C4 and R4)
i C2. 7 visited systems (1 in Zambia, 6 in
India) were found to not treat any wastewater
at all, 2 of which are repaired and fully
functional by the time this report is written.
R2a. Investigate reasons why 5 systems
don’t treat any wastewater at all and, if
possible, initiate corrective action (India).
R2b. Discuss and define what ‘corrective
action’ means for BORDA. Discuss and
develop a protocol for future financing and
implementation of required corrective actions
and strategies how to deal with factors that
are beyond BORDAs control. (GD)
i C3. Where COD measurements were
taken from reactor effluents, BORDA’s
internal standards for anaerobic and
aerobic reactor effluent were met in
70% and 49% of systems, respectively.
Note: Compliance to national discharge
standards is not considered in this SoC in
order to evaluate ‘functioning technology’ of
all systems against a common and appropriate
benchmark.
R3. (see SoC1; R2)
i C4. The utilisation rates in almost half of
the systems (42%; n=103) are either too low or
too high (i.e. user numbers being either below
66% or above 133% design value). This does
not seem to depend on system age or type
(with the exception of CSCs, where 10 out of
11 systems are underloaded).
R4a. Identify reasons for low utilisation
rates (especially in India).
R4b. Consider the development of
technical solutions to make systems more
adaptable to varying loads (e.g. designs with
several treatment trains which can easily be
by-passed or include space for additional
modules in future). (GD)
i C5. Biogas appliances & usage (see SoC3;
R1a)
i C6. 14% of 134 systems experience severe
flow surges and 19% experience some form
of related issues.
R6. Continue the development of
functional storm-water diversion options for
large scale roll-out. (GD)
The wastewater type discharged to (by far)
most systems matches design
assumptions (96%).
Functioning maintenance (SoC5)
i C1. Overall, system maintenance is
a challenge in all investigated countries
(although less pronounced in Afghanistan,
Cambodia and Indonesia).
i C2a. Almost no or only minor physical
damages to the building structures (not
affecting operation or safety) are observed.
i C2b. 4% of visited systems however are
severly damaged.
R2b. Initiate corrective action for 11
systems with severe structural damage. (India,
Nepal, Lesotho, Tanzania, Vietnam)
i C3a. Desludging of systems (not
underloaded and operating more than 3 years)
took place in 43% of the 56 visited systems.
(All countries)
R3a. Communicate with m.e.’s to
understand and share across the network
why systems are often not desludged within
3 years of operation and to initiate short-term
desludging of systems where needed. (Most
countries)
C3b. Good access to desludging service
providers was observed for almost all
systems (94%).
i C4. Filter blockages were observed at 11
HGFs (14% of all visited HGF) (India)
R4. Investigate and share across the
network observed reasons for filter blockages.
Remediate blocked filters. (India, Nepal,
Lesotho, Tanzania and Zambia)
R5. Investigate and share across the
network observed reasons for insufficient
sewer system maintenance. Initiate repairs
where needed. (India, Nepal, Indonesia)
i C5. Sewer system maintenance was
insufficient in 23% of the 42 systems at which
maintenance was needed in the past. Major
issues were insufficiently repaired in 28%
of the 38 systems. Broken pipes/manholes,
clogging, blockages and bad smell were the
most common issues. (India, Nepal, Indonesia)
i C6. Biogas system maintenance in general
is not good. Major points relate to missing
pressure gauge installation (71%), biogas not
reaching the stove (51%) or, when unused, not
being flared off (92%).
R6a. Investigate and share across the
network observed reasons for poor biogas
system maintenance. Initiate repairs where
needed (India, Nepal, Lesotho, Tanzania, Zambia,
Vietnam)
R6b. Emphasize the relevance and
benefits of biogas usage during m.e. and
operator training materials. (GD)
C7. Pump maintenance seemed to work
very well in most cases (86%).
i C8. Incorrect grease trap dimensioning
and maintenance at one system (SME-
restaurant) lead to complete blockage of the
reactors.
R8. Detailed design and maintenance
guidelines for grease traps should be
disseminated amongst regions and their
importance stressed amongst engineers
(especially for systems treating effluents from
food industry or restaurants). (GD)
12 13
Sustaining demand (SoC6)
i C1. Too low utilization rates (i.e. <66% of
designed rate) are an issue at a significant
number of systems. (34%; n=103) (see SoC 4; R4)
i C2. At 43% of the 21 visited sanitation
centres more (at times far more!) than 20 users
are using one toilet.
R2. Discuss adequate users per toilet
ratios depending on project type and (if nec-
essary) review design recommendations. (GD)
C3. Cultural acceptability appears high
across all systems: in 136 out of 139 cases
(98%), utilisation rates are either high
enough to indicate ‘good acceptance’ or too low
utilisation rates are due to other reasons than
(cultural) ‘in-acceptance’.
Effective management (SoC7)
i C1. Almost all systems (99%) have a
management entity (m.e.) (for the definition
of m.e., see glossary).
R1. Investigate why in two cases no
management entity exists and initiate
corrective action. (India)
i C2. 10% of visited systems do not have an
operator or person that is taking up the tasks
of the operator.
R2. Investigate reasons for non-existence
of designated operators, initiate corrective
action and share lessons learned. (Afghanistan,
India, Nepal, Tanzania, Indonesia)
i C3. There is no global BORDA standard
for the type and content of m.e. trainings.
Data indicates that: (i) the types of trainings
received by m.e. vary a lot across (and
sometimes within) countries and (ii) m.e.s are
often not aware of important responsibilities
such as financial administration and O&M /
operator management.
R3. Discuss and define minimum
requirements for m.e. training modules,
conduct workshops, emphasize the relevance
of trainings for project sustainability and
provide templates for facilitation. (GD)
i C4. Field observations showed that the
onsite documentation of O&M activities
(financial administration, O&M, responsibility
definitions) is missing in 40%, 72% and 21%
of investigated cases, respectively.
R4. Discuss and decide whether onsite
documentation should be emphasized during
training modules for system managers. (GD)
i C5. The operators' awareness regarding
responsibilities is limited.
R5. Discuss and define minimum require-
ments for operator training modules. (GD)
i C6. 17% of operators receive no payment
(excluding household systems).
R6. Investigate and share across the
network observed reasons why operators
receive no payment. Emphasize operator
costs as part of general O&M costs (see SoC8).
(India, Nepal, Lesotho, Zambia, Laos, Philippines,
Vietnam)
Sustainable financing (SoC8)
i C1. 6% of visited systems have no income
source which could contribute to the O&M
budget (for this evaluation, household
systems are assumed to always have an
income source).
R1. Investigate reasons why some projects
have no source of income to cover operational
costs. (India, Lesotho, Zambia, Vietnam)
i C2. In roughly half of the projects (44%;
n=133), an O&M budget was not defined. This
was observed for most project types.
R2. Discuss and decide whether an O&M
budget definition by the local authority,
management entity, members of community
or owner should be compulsory part of the
hand-over procedure of DEWATS (GD)
i C3. Globally, O&M expenses can be
covered in most cases. In some countries
however, solutions for irregular/major O&M
expenses often do not exist and irregular/
major as well as regular O&M expenses have
often not been covered in the past.
R3. Investigate and share lessons
learned across the network why irregular/
major and regular O&M expenses are not
covered. Identify and implement solutions
where needed (cases exist in all countries, but
especially in India, Nepal, Tanzania, Lesotho,
Vietnam).
C4. Where user fees are part of the income
sources (Afghanistan, Nepal, Tanzania,
Cambodia, Indonesia, Laos, Philippines),
the associated processes (agreement on fees,
responsibilities, collection) work well to very well
(exception Afghanistan, where often fees are not
collected).
Planning, design, documentation (SoC9)
i C1. Estimation of O&M costs is not
included in the project documentation of
62% of visited projects. Countries in which
O&M cost estimation is never contained in
project documentation are Lesotho, Zambia,
Indonesia, Laos, Philippines and Vietnam.
In India, for most projects such information
is not available through available project
documentation.
R1. Discuss and decide whether an O&M
cost estimation should become compulsory
part of project planning and documentation
(GD)
i C2. For 50% of projects, no or only 1 out
of the 3 documents considered important for
this evaluation (Design drawing, Design spread
sheet, Feasibility study) was available in the
respective BORDA offices.
R2a. Discuss and decide which documents
and project information should become
compulsory part of project documentation
(see p.73) (GD)
R2b. Emphasize the importance of
complete sets of project documentation,
(Afghanistan, India, Nepal, Lesotho, Tanzania,
Zambia, Vietnam)
C3. For 98% of 118 systems, designs
observed in the field matched the original
design.
i C4. Per capita investment costs of 38
residential systems were found to vary
considerably, ranging from approx. 50 to 300
EUR per user (calculated with design number
of users).
R4a. Discuss and decide whether infor-
mation on per capita investment costs would
support BORDAs project management. (GD)
R4b. Discuss and decide whether bench-
marking and evaluating such costs would
support BORDAs project management. (GD)
E X E C U T I V E S U M M A R Y
15I N T R O D U C T I O N
1 Introduction
1.1 Objectives of the BORDA M&E programBORDA’s M&E program was initiated by BORDA in order to:
i Monitor the functionality of installed systems and the quality of sanitation
service delivery
i Improve availability of project data in order to quantify and qualify:
] The impact over the service on the beneficiaries’ living conditions and
the environment
] BORDA’s collective outputs and outcomes across the installed systems
i Learn from M&E to improve functionality of installed systems and sanitation
service (quality management)
i Learn from M&E to improve the impact of future system implementation and
programs (quality management)
1.2 Objectives of this reportThis internal report is intended for BORDA management on global, region
and country level.
It is intended to be published in one-year intervals.
Its objectives are to:
i highlight general areas of strength and concern of existing DEWATS
projects by providing insights into
] global Statement of Change (SoC) performance
] country specific Statement of Change (SoC) performance
] project type specific differences
i allow for evidence-based global (programmatic) and local corrective action
i motivate cross-regional learning
i identify topics of possible future in-depth analysis
Specifically, this report will focus on the evaluation of project outcomes and
impacts in the areas of 9 SoCs:
i Environmental health
i Improved living conditions of served communities
i Project specific impact
i Functioning technology
i Functioning maintenance
i Sustaining demand
i Effective management
i Sustainable financing
i Quality of planning, design and construction
The spread or publication of this report or parts of it outside of BORDA,
requires BORDA management approval.
16 17I N T R O D U C T I O N
1.3 Project selection processAll investigated projects are part of the DEWATS inventory
(https://bordahq.sharepoint.com/me/SitePages/DEWATS.aspx).
Projects are selected by regional ME coordinators and directors based on
available ressources and the following ‘must have’ and ‘should have’ criteria:
Must have criteria: i Period of DEWATS operation: > 12 months operation
i Minimum of 6 months between two field visits
i No FSTPs. The current version of the Global Monitoring Form (GMF2.3)
does not allow for monitoring of FSTPs.
i For field investigations to represent normal plant activity, projects are not
to be monitored on public holidays or on the 3 days after public holidays.
Same applies to school holidays, especially for SBS projects.
Should have criteria: i Period of DEWATS operation: > 24 months operation
i A representative sub-group of projects should be monitored considering
the project type, size, geographic location, climate zone, income
groups. For example, if CSC projects represent 20% of 100 existing
projects in one country, but resources only allow for the monitoring
of 10 projects, approx. 2 CSC projects should be visited that year.
Other aspects that are considered during the project selection process: i Distance of projects to one another to minimize travel costs. Monitoring
of regional project clusters is prioritized.
i Possible combination of ME field visits with other activities scheduled
in the same area to minimize travel costs.
1.4 Visited projectsOnly systems which were constructed or implemented with BORDA support
or partners were monitored.
This report evaluates data from a total of 140 systems visited between April
2017 and March 2018 in 11 countries (Afghanistan, India, Nepal, Lesotho,
Tanzania, Zambia, Cambodia, Indonesia, Laos, Philippines and Vietnam).
123 systems are conventional DEWATS systems, 14 consist only of prefab
modules and three designs include both module types.
The following 5 figures present information on visited systems per country,
project types, treatment modules, design treatment capacities and years of
operation at time of field visit.
0%
5%
10%
15%
20%
25%
30%
35%
40%
45%
50%
0
5
10
15
20
25
30
35
40
45
50
Per
cent
age
of e
xist
ing
syst
ems
Num
ber o
f vis
ited
syst
ems
Visited systems
Percentage of existing systems
Please note: In Figure 1, the 2nd y-axis is – for scaling purposes - showing
the relation of visited vs. overall existing systems as percentage [%] of
existing systems. The percentage gives the reader an idea about the general
representativeness of the dataset.
Figures 2, 4 and 5 provide similar information by showing the number of visited
systems (1st x-Axis) and the number (2nd x-Axis) of overall existing systems.
0
200
400
600
800
1000
1200
0
5
10
15
20
25
30
35
Num
ber o
f exi
stin
g sy
stem
s
Num
ber o
f vis
ited
syst
ems Residential systems - visited
SME - visitedInstitutional systems - visitedExisting
Figure 1: Number of visited systems per country and the percentage of existing systems visited in 2017
Figure 2: Number of visited systems per project type
18 19
0
20
40
60
80
100
120N
umbe
r of t
reat
men
t mod
ules
0
100
200
300
400
500
600
700
05
101520253035404550
0-5
5-1
010
-15
15 -
2020
-25
25-3
030
-35
35-4
040
-45
45-5
050
-55
55-6
060
-65
65-7
070
-75
75-8
080
-85
85-9
090
-95
95-1
00>
100
Num
ber o
f exi
stin
g sy
stem
s
Num
ber o
f vis
ited
syst
ems
Design treatment capacity (m³/d)
Visited
Existing
0
100
200
300
400
500
600
05
101520253035404550
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 >20
Num
ber o
f exi
stin
g sy
stem
s
Num
ber o
f vis
ited
syst
ems
Years of operation
VisitedExisting
At the time of field-visit more than 65% of visited systems operated between
2–6 years.
Figure 3: Total number of treatment modules
Figure 4: Design treatment capacities of the 140 visited systems (values for systems above 100 m³/d are: 103, 130, 150, 188, 225, 349, 418 and 600 m³/d) and of all existing
Figure 5: Years of operation (rounded up) of the 140 visited systems at time of field visit and years of operation of existing DEWATS (n = 2709) in 2017
1.5 Team members involvedThe following BORDA staff was involved in M&E activities 2017/20183:
I N T R O D U C T I O N
Table 1: Staff involvement
1
3 Please note that this list is not exhaustive. Other staff members might have also participated in M&E activities.
Activity Region/country: Staff
Global coordination of M&E activities, questionnaire- and methodology development, including SOPs and training material
HQ: Elli Rodriguez, Nicolas Reynaud, Alex Wolf
Country-wide coordination of M&E activities, including logisitics, staff training, quality assurance and feedback on questionnaires
Africa: Leonidas Bernado Deogratius, Aubrey Simwambi, Angela Kapembwa,
South Asia: Anusha Narayan, Rohini Pradeep, Shreyas Kumar
South East Asia: Dao Thi Hoa Anh, Jenni Kesip, Mirko Dietrich, Monivanh Boulom, Nguyen Quang Vinh, Pham Thi Hai Yen, Ry Borin
West- and Central Asia: Emal Waziri
Field investigations, laboratory analyses, team support
Africa: Diana Tursarinow, Evelyn Herrera Lopera, Eva Schoell, Lekhotla Lebona, Sophia Hibler, Tobias Pomplun, Tsietsi Leteane
South Asia: Sarani Sasidharan, Daniela Schmitz, Sujatha Gaddipatti, Varshini J. Reddy
South East Asia: Bot Khorn, Chhun Sophear, Dwi Maryanto, Ikatri Wulandari, Louise Stephen, Mang Opasith, Marlon Oiga, Mary Evelina Jose, Miftah Huda, Nguyen Thi Ha Chau, Nguyen Thu Van, Nungki Paradita Kartikasari, Rudi Rudiyanto, Somock Keopashath, Trias Windi Hapsari, Truong Ngoc Hien, Vansoukkhy Keosengsay, Vorn Thary,
Data evaluation and reportHQ: Nicolas Reynaud, Andreas Schmidt, Pascal Siemsen, Alex Wolf, Elli Rodriguez, Wilm Mäteling
21B O R D A’ S M & E M E T H O D O L O G Y
2.1 Evaluation matrixThe underlying methodology has been developed by BORDA in cooperation
with ISF (Institute for Sustainable Futures). It is based on sustainability
criteria for small-scale sanitation systems (Functioning Technology, Effective
Management, Sustaining Demand and Sustainable Financing) and BORDA’s
M&E requirements as specified by BORDA’s Global Directorate and country
directors.
Nine Statements of Change (SoC) have been identified as crucial for the
comprehensive evaluation of BORDA DEWATS. Each SoC is defined by 1 to 3
Objectives, shown in Table 2.
Statement of Change Objective
1. The sanitation service maintains or improves environmental health
1.1 Effluent meets discharge standards
1.2 Removed waste is safely disposed or reused
2. The sanitation service improves the living conditions of communities
2.1 Underserved4 people are connected to the sanitation service
2.2 Potential exposure to faecal pathogens for surrounding commu-nities is managed
3. The service achieves project specific impact
3 Sanitation service achieves intended Objectives
4. Functioning Technology - systems are operating as intended
4.1 System operating as designed - acceptable loading5 and system hydraulics
4.2 Systems operating as designed - treatment meets BORDA requirements
5. Functioning Maintenance - systems are maintained as intended
5.1 Systems maintained - no major damage
5.2 Maintenance activities occurring as intended
6. Sustaining Demand - system is available, used to capacity and acceptable
6.1 Service is adequately available to users
6.2 Utilisation rate: Service is used to capacity
6.3 Acceptability: Culturally acceptable, users satisfied with system
7. Effective management: Existing, active and accountable management entity and operator
7.1 Active and accountable management entity
7.2 Trained and equipped operator
8. Sustainable Financing: Sufficient ongoing income to cover all short and long term costs
8.1 Regular income
8.2 Sufficient income to cover all short and long term costs
9. Planning, design, construction
9.1 Project design appropriate to context
9.2 Systems built to design
9.3 Acceptable investment cost per user
Table 2: Statement of Change structure
1 2
4 ‘Underserved’ refers to low income households without access to basic sanitation
5 ‘Acceptable loading’ is defined as actual user numbers being between 66 – 133% design value, treatment of intended wastewater type only, signs of feed and effluent flow and no evidence of severe flow surges
2 BORDA’s M&E methodology
22 23
Objectives consist of Prioriy Indicators (PI) which in turn are devided into Parameters. Parameters are in most cases the exact questions as asked during field surveys.
Statement of Change Objective Priority Indicators Parameters
1
1.1 1.1A1.1A.1
1.1A.2
1.1B 1.1B.1
1.2 1.2A 1.2A.1
In the results section, the structure of each SoC is discussed in detail.
Data collection is conducted through surveys on technical design details,
field observations, interviews with the management entity and the operator
and field and laboratory measurements.
2.2 ScoringParameter responses are rated as: ‘good’ (numerical score: 3), ‘caution’
(numerical score: 2), ‘bad’ (numerical score: 1), ‘data not available’ or ‘not
relevant’.
In this report, all scores for Statements of Change, Objectives and Priority
Indicators are calculated by averaging all relevant Parameter response scores.
Averaged scores smaller than 1.5 are rated as ‘Bad’ (red), scores larger than
1.5 to 2.5 as ‘Caution’ (yellow) and scores larger than 2.5 are rated as ‘Good’
(green) (see, for example, Figure 6 or Figure 8).
In case certain extremely negative Parameter responses are given, the score for
all related matrix items (PIs, Objectives, SoC) for that system is automatically
‘bad’. Table 4 summarizes these so called ‘killer criteria’.
If one Parameter response is rated as ‘data not available’ the affected average
is labelled ‘Dataset incomplete’ for that system.
Parameters which often (i.e. for many systems) received positive or negative
scoring and which therefore often affect SoC scores, are listed in the
subchapters ‘Parameters with strong impact’.
Parameter textParameter response rated as ‘killer criteria’
Affected matrix items
SoC Objective PI
Do you observe flow or signs of recent flow (wet piping, etc.) at plant outlet?
No, because piping system is not connected to the plant
SoC 1SoC 2SoC 3SoC 4SoC 5SoC 6
All Objectives of:SoC 1SoC 2SoC 3SoC 4SoC 5SoC 6
All PIs of:SoC 1SoC 2 (2.1C only if sanitation facility is part of project)SoC 3SoC 4 (4.2B only if digester is part of the system)SoC 5Soc 6 (PIs de-pending on system setup)
Do you observe flow or signs of recent flow at plant inlet (wet piping, wet inlet chamber)?
No, because piping system is not connected to the plant
What building structure problems do you observe at treatment system (digester, settler, ABR, AF, HGF/PGF)?
System is clearly not connected to piping
What building structure problems do you observe at treatment system (digester, settler, ABR, AF, HGF/PGF)?
Major physical damage (e.g. large cracks, leakages, broken divider walls); affects operation or safety
5. Functioning Maintenance - systems are main-tained as intended
5.1 System is maintained - no major damage
5.1A No signs of structural damage compromising functionality or waranty aspects
What structural problems with the sanitation facili-ty(ies) (walls, roof and floor) do you observe?
Major physical damage (e.g. large cracks), affecting operation, use or safety.
Does a management entity (m.e.) exist?
No management entity exists
7. Effective man-agement: Existing, active and account-able management entity and operator
7.1 Active and accountable management entity
7.1A Existence of trained management entity with clarified responsibilities
Is there a person/caretaker/operator assigned and responsible for O&M activities? (in the following called ‘operator’)
No, nobody is assigned and responsible for O&M activities
7. Effective man-agement: Existing, active and account-able management entity and operator
7.2 Trained and equipped operator
7.2A Existence of trained, equipped operator who knows his responsibilities
What are the income sourc-es contributing to available O&M budget?
None 8. Sustainable Fi-nancing: Sufficient ongoing income to cover all short and long term costs
8.1 Regular income8.2A Regular operation and maintenance expenses (operator salary, material, equipment, elec-tricity and water costs) are covered by income
8.1A O&M budget and/or user fees have been agreed on and are collected8.2A Regular operation and maintenance expenses (operator salary, material, equipment, elec-tricity and water costs) are covered by income
Table 4: Killer criteria and affected matrix items
Table 3: Evaluation matrix structure
B O R D A’ S M & E M E T H O D O L O G Y
24 25
2.3 Assumptions made during evaluation method development
Assumptions had to be made during the development of this evaluation
methodology.
In order to best represent the widely varying field realities and project types
in the 11 countries represented in this study, they were developed together
with numerous BORDA senior staff and country staff members.
The authors are therefore confident that formulations which could lead to
misinterpretation of data have been avoided in most cases.
However, because of the challenges of long-range communication and the
complexity of certain project aspects, some readers may identify mismatches
between the evaluation-methodology/results and the project reality in their
countries. Please help remove potential mismatches by providing us feedback.
The following points represent a list of some major assumptions made during
method development:
Acceptable sludge, scum and solid waste disposal (SoC 1) See glossary for listing
of disposal practices assumed acceptable at DEWATS projects.
Acceptable reactor effluent quality (SoC 1, SoC 4) Effluent concentrations
benchmarks indicating acceptable performance quality are 200 mg COD/ l
for anaerobic effluent and 80 mg COD/ l for aerobic effluent. They are based
on feedback from BORDA Regional Directors and existing R&D data (see table
10 in ‘DEWATS Guide V1_2018-05-09’ for a summary of typical DEWATS reactor
effluent concentrations under tropical climates – the COD benchmark values
are chosen slightly higher to consider the effect of lower temperatures in
non-tropical applications) and discussions with senior engineers.
Acceptable system loading/ utilisation (SoC 4) System loading/ utilisation is
acceptable when the actual user number is between 66 and 133% of the user
number assumed for design.
Acceptable wastewater type discharged to system (SoC 4) The wastewater
type discharged to the DEWATS is acceptable when it matches the design
assumption. Any situation in which blackwater was designed to be treated
by the DEWATS but actually is not, is scored with ‘bad’ since it is assumed
that then, untreated blackwater is discharged to the environment. Most other
situation in which treatment or non-treatment of greywater or non-communal
wastewater does not match system design, is scored as ‘caution’.
M.e. responsibilities (SoC 7) The ‘complete’ list of responsibilities a m.e.
representative should be aware of is: financial administration, coordinating
& arranging financing of major repairs, employing, managing and training an
operator and deciding about additional connections.
Operator responsibilities (SoC 7) The ‘complete’ list of responsibilities an operator
should be aware of is: removing blockages, removing scum, cleaning facility,
desludging, reporting problems to m.e. and collecting fees.
O&M documentation (SoC 7, SoC 8) The ‘complete’ list of O&M related written
documentation existing at a well managed project site are documents for
financial administration (important), operator responsibilities (optional),
performed O&M activities (optional), m.e. meeting minutes (optional), legal
registration of m.e. (optional)
Project documentation (SoC 9) The ‘complete’ set of project documentation
contains at least design drawings, the design spread sheet and the feasibility
study.
M.e. training modules (SoC 9) Traning modules needed for comprehensive m.e.
training are financial training, O&M training and HHE training.
2.4 Data quality-assurance (QA)2.4.1 Data plausibility testingAll submitted surveys were tested for consistency and plausibility in mWater
(IT Platform) before being approved. Unplausibile data and inconsistencies
were then discussed with the respective M&E coordinators and corrected.
Only consistent datasets are included in this report.
2.4.2 Confirmation of draft results between HQ and regionsAfter the first evaluation of all approved data sets, HQ contacted all M&E
coordinators to confirm or rule out specific outliers or inconsistencies. In
cases that distinct results or tendencies could be identified as incorrect, e.g.
due to misunderstanding of the question and answer options, these mistakes
were corrected by HQ referencing the M&E coordinator.
2.4.3 QA for wastewater concentration measurementsSubmitted concentration data was considered reliable enough to be used in
this report if it fulfilled the following criteria:
i Measurements done by BORDA team not by external laboratory (exception:
Cambodia and Tanzania where data from the used external laboratory are
assumed to have been trustworthy in 2017)
i Samples were not taken on days with strong rain (there was no such case
in this dataset)
i Data is plausible: extreme differences between duplicate measurements
as well as extremely high values were double-checked and confirmed or
corrected by M&E coordinators
B O R D A’ S M & E M E T H O D O L O G Y
27E VA L U A T I O N R E S U L T S
3.1 Global overview across all SoCs To start with, we visualized the results for all SoCs over all visited projects
(n=140), i.e. on global level (Figures 6 and 7), to point out key tendencies which
are then elaborated in detail in each SoC Section.
0% 50% 100%
1. Environmental health (n = 81)2. Improving living conditions (n = 100)
3. Project specific impact (n = 75)4. Functioning technology (n = 42)
5. Functioning maintenance (n = 127)6. Sustaining demand (n = 92)
7. Effective management (n = 118)8. Sustainable financing (n = 119)
9. Planning, design, construction (n = 118)
%age of systems with complete datasets
Good Caution Bad
The overall messages conveyed by Figure 6 are that
i for more than 50% of projects with complete datasets, scores are ‘good’
for SoCs 2, 3, 4, 5, 6 and 8 (SoC 4 scores however, are only available for 56
systems)
i ‘Good’ project performance concerning SoCs 1, 7 and 9 occurs in only 40%
or less of visited projects with complete datasets. Strategies for improvement
may therefore be most important concerning these SoCs
i badly performing systems exist (although being a minority in this dataset)
and would require (rather urgent) corrective action.
Figure 7 displays the same information while indicating the magnitude
of incomplete datasets for each SoC. Overall, data collection has
been rather comprehensive for many SoCs already (acknowledging
that this is the first year using a new methodology across 11 countries).
However, the evaluation of SoC 1 and 4 is particularly affected by incomplete
datasets.
0 20 40 60 80 100 120 140
1. Environmental health2. Improving living conditions
3. Project specific impact4. Functioning technology
5. Functioning maintenance6. Sustaining demand
7. Effective management8. Sustainable financing
9. Planning, design, construction
Number of systems
Good Caution Bad Dataset incomplete Not relevant
Figure 6: Average SoC scores of investigated systems with complete datasets (‘Good’: score ≥ 2.5, ‘Caution’: 1.5 ≤ score < 2.5, ‘Bad’: score < 1.5)
Figure 7: Average SoC scores of all investigated systems (‘Good’: score ≥ 2.5, ‘Caution’: 1.5 ≤ score < 2.5, ‘Bad’: score < 1.5) (n=140)
3 Evaluation results
28 29E VA L U A T I O N R E S U L T S
‘Incomplete datasets’ mean that ‘data was not available’ – which is neither a
positive or negative weighting.
However, it means that a comprehensive evaluation of these 2 SoCs - for
those projects - is not possible for the time being, and, that the global results
(Figure 6, 7) are based on comparably smaller datasets.
The large numbers of incomplete datasets shown for SoCs 1 and 4 (see Figure
7) are mainly due to missing…
i effluent concentration measurements (SoC 1 and 4)
i information on actual user numbers derived from interviews on site (SoC 4)
! Suggested discussion on GD level D1: Should COD measurements be considered mandatory for
each M&E visit? Without ate least 1 proxy for treatment efficiency, the evaluation of system operation and project impact is bound to be incomplete.
We consider the approach to retrieve information on ‘user numbers’ by asking
the responsible person/management entity still more or feasible than by
conducting flow measurements.
Therefore, this proxy parameter should remain in place and even more emphasis
must be put in gaining answers during the M&E field visits (For details to
understand the precise parameter and response options, see SoC4 and Annex A).
17
36
25
62
Figure 8: Distribution of average scores for SoC 1: Environmental health
3.2 SoC 1: The sanitation service maintains or improves environmental health
3.2.1 Statement of Change structureThis SoC evaluates the project impact on environmental health by investigating
effluent quality and safe waste disposal.
Statement of Change
ObjectivesPriority Indicators
Parameters
1. The sanita-tion service maintains or improves environmen-tal health
1.1 Effluent meets discharge standards
1.1A The most recent effluent analysis complies with BORDA Standards
If data available and reliable6, does the average effluent COD meet relevant BORDA discharge standards (200 mg/l or 80 mg/l depending whether anaerobic or aearobic effluent)?
1.1B The most recent effluent analysis complies with Local Standards
If data available and reliable does the average efflu-ent, COD meet relevant local discharge standards.7
1.2 Removed waste is safely disposed or reused
1.2 Evidence that waste or by-products (sludge, scum, trash, biogas) is safely disposed or reused
Where is the sludge disposed to after desludging?
Where is the scum disposed to after descumming?
Where is the solid waste disposed to after removal from reactors or piping?
For BGD, is unused biogas burned off?
34
! SoCs 1 and 2 indirectly address the BMZ program objective ‘protection of freshwater resources’ (if relevant) by investigating
effluent quality, the disposal of other DEWATS waste streams and the exposure to faecal pathogens of surrounding communities. Specific in-depth investigations on this topic - e.g. on the distance between discharge point and ground water table, soil properties etc. – are currently beyond the scope and technical possibilities of ME field visits
3.2.2 Result overview on Statement of Change levelFigure 8 shows the average SoC 1 scores of all 140 visited systems, globally.
For all details on Objectives, Priority Indicator and Parameter results, refer
to Appendix A.
The large number of incomplete data sets is mainly due to lack of effluent
sampling (40 cases) and laboratory analyes deemed unreliable (15 cases).
Red = ‘Bad’ (score < 1.5), Yellow = ‘Caution’ (1.5 ≤ score < 2.5), Green = ‘Good’ (score ≥ 2.5),Grey = ‘Dataset incomplete’; numbers indicate number of systems per score range
6 Data are considered unreliable if measured in external labs, except in Cambodia and Tanzania
7 Present standards checked in April, 2018
Table 5: Structure of SoC 1 - The sanitation service maintains or improves environmental health.
Picture: DEWATS effluent discharging into river in close vicinity to local communities (Yogyakarta; Indonesia)
30 31
0
100
200
300
400
500
600
700
800
900
1000
Anaerobicsystem
effluentconcentration(m
gCO
D/l)
Afghanistan India Indonesia Laos
0
100
200
300
400
500
600
700
800
900
1000
Aerobicsystem
effluentconcentration(m
gCO
D/l)
Afghanistan Cambodia India Lesotho Tanzania
Concerning safe disposal of removed waste:
i Out of the 94 cases in which solid waste is removed from systems, in 21
cases (22%) solid waste is disposed unsafely.
i Out of the 77 cases in which systems have been reported to be descummed,
in 16 cases (21%) scum is disposed unsafely.
i Out of the 31 cases in which systems have been reported to be desludged,
in 11 cases (35%) sludge is disposed unsafely.
i Concerning solid waste removal, descumming and desludging, there is
no clear correation with country or system type.
i Biogas, when unused, is almost never flared off (12 out of 13 cases).
There is no clear relationship between system type and safe disposal for
removed waste.
Figure 9 shows the per country distribution of average SoC scores.
0 10 20 30 40 50
AfghanistanIndiaNepal
LesothoTanzaniaZambia
CambodiaIndonesia
LaosPhilippines
Vietnam
Number ofsystems
Good Caution Bad Datasetincomplete
3.2.3 Parameters with strong impactThe following Parameters had the strongest negative or positive impact.
Concerning effluent standard compliance:
i In 39 out of 58 cases (67%) measured effluent COD concentrations do not
meet local standards (see Figure 10). Concerning this aspect, there is no clear
reation with system type.
i Out of the 78 available system effluent concentration measurements, 35
(45%) do not meet the BORDA standard8 5 (200 mg COD/ l for anaerobic effluent,
80 mg COD/ l for aerobic effluent, Figure 11 and Figure 12). I The dataset shows
no relation between this parameter and system type.
0 10 20 30 40 50
AfghanistanIndiaNepal
LesothoTanzaniaZambia
CambodiaIndonesia
LaosPhilippinesVietnam
Number ofsystems
Belowstandard <20%abovestandard ≥20%abovestandard
Nodata Datanotreliable Nolocalstandards
8 ‘Effluent concentrations benchmarks indicating acceptable performance quality are 200 mg COD/ l for anaerobic effluent and 80 mg COD/ l for aerobic effluent. These are based on existing R&D data (see table 10 in ‘DEWATS Guide V1_2018-05-09’ for a summary of typical DEWATS reactor effluent concentrations under tropical climates – the COD benchmark values are chosen slightly higher to consider the effect of lower temperatures in non-tropical applications) and discussions with senior engineers.’
Figure 9: Per country distribution of average scores for SoC 1: Environmental health
Figure 10: Compliance of measured DEWATS effluent COD concentrations to local standards
Figure 11: Measured system effluent concentrations at DEWATS without aerobic treatment step. Each data-point represents one or the average of two measurements per system effluent. 2 datapoints are > 1000 mg COD /l (1675 mg COD/ l, Ammankulam Housing Colony Cluster 2 & 3/ India)
Figure 12: Measured system effluent concentrations at DEWATS with HGF. Each data-point represents one or the average of two measurements per system effluent
E VA L U A T I O N R E S U L T S
BORDA-internal standard
BORDA-internal standard
32 33E VA L U A T I O N R E S U L T S
3.2.4 Recommendations regarding strong impact ParametersChallenges concerning this SoC were observed at systems from all countries.
Possible follow-up actions:
i in-depth investigation (with country engineers or involved R&D staff) to
identify probable reasons for high and low effluent concentrations.
i in-depth investigation to identify probable reasons for low desludging,
descumming and solid waste removal frequencies and reasons for inadequate
disposal
i BORDA internal discussion and definition of proper and safe handling of
waste streams to update training material for system managers.
i For a large number of visited systems (62 out of 140, 44%) existing effluent
concentration values were either not considered reliable because they were
measured by external laboratories of questionable quality (15 out of 140,
11%) or no effluent samples were taken in the first place because of budget
constraints (47 out of 140, 34%).
Therefore an internal protocol for the quality assessment of external laboratory
data needs to be developed. Additionally, the practice of sample stabilization
(not followed for most data presented in this report) will need to be adhered to.
! Suggested discussion on GD level D2: How important is the monitoring (cost-benefit) of the effluents’
nutrient and/or pathogen concentrations with regard to the future importance of re-use aspects & environmental health of small scale sanitation systems and BORDAs positioning (Mission &Vision)?
3.3 SoC 2: The sanitation service improves the living conditions of communities
3.3.1 Statement of Change structureThis SoC evaluates the project impact on the living conditions of underserved
and disadvantaged9 connected communities as well as the project impact on
the living conditions of surrounding communities.
9
31
58
42
Figure 13: Pie charts with distribution of average scores for SoC 2: Improving living conditions
Picture: Test of submersible pump in Chachchamal Healing Center; highlighting the need for appropriately treated wastewater prior to discharge into the environment (Chamchamal, Kurdistan)
Statement of Change
ObjectivesPriority Indicators
Parameters
2. The sani-tation service improves the living conditions of communities
2.1 Underserved people are connected to the sanitation service
2.1A Majority of sanita-tion users previously had no or basic access to sanitation
Before this system existed, what was the sanitation option used (or if SME, the wastewater discharge method) by the majority of users connected to the system?
2.1B Majority of sanita-tion users are classified as disadvantaged (ie. Income)
What proportion of users are classified/registered as low income according to Project planning/FS?
2.1C Unrestricted CSC access for disabled and elderly
Is access to the sanitation facility difficult or impossible for disabled/ elderly?
2.2 Potential exposure to faecal pathogens for surrounding communities is managed
2.2 Exposure to faecal pathogens for sur-rounding communities is managed
Is the exposure to faecal pathogens for surrounding communities managed?
3.3.2 Result overview on Statement of Change level
Figure 13 shows the average SoC 2 scores for all 140 visited systems, globally.
For all details on Objectives, Priority Indicator and Parameter results, refer
to Appendix A.
Red = ‘Bad’ (score < 1.5), Yellow = ‘Caution’ (1.5 ≤ score < 2.5), Green = ‘Good’ (score ≥ 2.5),Grey = ‘Dataset incomplete’;
numbers indicate number of systems per score range
Table 6: Evaluation structure for SoC 2 - The sanitation service improves the living conditions of communities.
1
9 ‘Underserved and disadvantaged’ refers to low income households without access to basic sanitation
34 35E VA L U A T I O N R E S U L T S
Figure 14 shows the per country distribution of average SoC scores.
0 10 20 30 40 50
AfghanistanIndiaNepal
LesothoTanzaniaZambia
CambodiaIndonesia
LaosPhilippines
Vietnam
Number ofsystems
Good Caution Bad Datasetincomplete
3.3.3 Parameters with strong impactThe following Parameters had the strongest negative or positive impact.
i In some cases systems don’t cater for underserved and disadvantaged
] In 15 out of 82 (18%) cases where systems were designed for
communities (mostly systems connected to several households - 13 out
of 15), most community members are not part of the low income segment.
This is the case for 12 out of 13 investigated systems in Indonesia (see
Figure 15).
! ‘Background information’: ‘Low income’ population in Indonesia BORDA guided DEWATS implementations in Indonesia are part of
the national SANIMAS program. BORDA’s role is an advisory one, both on local and/or national governmental level, but final decision making in terms of where and what kind of projects to implement is up to the local government bodies.
Slum areas with – by definition - a low income population are not part of the SANIMAS program due to their illegal status and ineligibility to receive public infrastructure funding, in most cases.
0 10 20 30 40 50
AfghanistanIndia
NepalLesotho
TanzaniaZambia
CambodiaIndonesia
LaosPhilippines
Vietnam
Number of systems
Majority (> 50%)
Some (25 - 50%)
Few (<25%)
Info not available
No proj. doc.
Not relevant, no human "users" -e.g. some SME
] The previous sanitation option was already classified as improved
in 12 out of 119 (10%) cases. This concerns significant fractions of visited
systems in Tanzania and the Philippines (see Figure 16). This aspect does
not clearly depend on system type – it is observed at residential, SME
and certain institutional systems.
] In 11 out of 24 (46%) visited sanitation centres, there is no easy access
for elderly and disabled (Tanzania 2; Zambia 9). All visited systems in
Afghanistan (1), India (1), Lesotho (1), Laos (1) and Cambodia (9) had good
access.
0 10 20 30 40 50
Afghanistan
India
Nepal
Lesotho
Tanzania
Zambia
Cambodia
Indonesia
Laos
Philippines
Vietnam
Number ofsystems
ImprovedsanitationoptiondidNOTexistbeforeprojectstartImprovedsanitationoptionexistedbeforeprojectstartNotrelevant,wastewater-stream didnotexistbeforeNodata
i The exposure to fecal pathogens for the surrounding communities is
mostly well managed. All seven situations in which exposure is not well
managed as well as most of the 18 cases in which exposure management is
scored as medium occur in Afghanistan and Indonesia at residential systems
(appartments and multiple houses) because:
] Systems with post-treatment and discharge to stream or river used
by community close downstream (100m): 4 cases
] Systems without post-treatment and reuse for fish farming: 1 case
] Systems without post-treatment and reuse for vegetable production:
18 cases
] Systems without post-treatment and discharge to stream or river used
by community close downstream (100m): 2 cases
] Systems without post-treatment and discharge to stream or river
which is dry during some parts of the year: 6 cases
Figure 16: Occurance of improved sanitation options before project implementations
Figure 15: Responses given to the question ‘What proportion of users are classified/registered as low income according to project documentation?’
Figure 14: Per country distribution of average scores for SoC 2: Improving living conditions
36 37E VA L U A T I O N R E S U L T S
3.3.4 Recommendations regarding strong impact ParametersPossible follow-up actions:
i Confirm or adapt BORDAs mission statement at managerial level and
make sure most projects are aligned with it at local level.
] Indonesian SoC scores (those of Afghanistan, Nepal and Laos to
smaller extent) are affected in this data-set due to systems at which
small fractions of system-users are classified/registered as low ‘income’ .
] Tanzanian and Philippines SoC scores (those of India, Lesotho, Cam-
bodia and Laos to smaller extent) are affected in this data-set due to
systems at which sanitation options were classified as ‘improved’ before
project start.
i Improve access to the sanitation facilities for (elderly and) disabled in
Tanzania and Zambia. Define ‘improved access’ design options depending on
local and social factors; exchange with the technical team in Cambodia (all
9 visited systems had good access in Cambodia).
i Discuss within country teams the actual pathogen exposure risks for
surrounding communities in the identified projects and – if necessary – find
solutions to minimize the exposure risks for system users and their neighbours.
3.4 SoC 3: The service achieves project specific impact
3.4.1 Statement of Change structureThis SoC identifies project impacts which are detailed in the project
documentation, as embedded into a country’s program, and go beyond BORDA’s
main mission statement of creating liveable spaces while improving the living
conditions of beneficiaries and cater for environmental protection. This SoC
therefore complements SoC 1 and SoC 2.
Table 8: Evaluation structure of project specific objectives
Picture: BORDA – gardening next to the ...slaugtherhouse, creating direct and additional benefits for the community members („Rastro Municipal“ Léon; Nicaragua)
Table 7: Evaluation structure for SoC 3 – Achieving project specific impact
Statement of Change Objectives Priority Indicators Parameter
3. The service achieves project specific impact
3.1 Sanitation service achieves intended Objectives
3.1A Compared with the project documents the original Objective(s) of the sanitation service was (were) achieved
Was/ were the Objective(s) of the sanitation service achieved?
Note: This SoC also processes (draws from) PIs from other SoCs, depending on
the projects’ specific objectives. Table 8 lists all considered project specific
objectives and indicates which PIs are used to assess their impact.
Specific project objectives can vary across countries and project types.
Considered here are: production and usage of biogas, re-use of wastewater,
pre-treatment of wastewater before discharge to solid-free sewer, reduction
of O&M costs).
Project specific objectivesAssociated Priority Indicators
Parameter
Produce and use biogas10
4.2B There are obvious signs of biogas production from the biogas digester
When you open the gas valve furthest away from the BGD, do you hear or smell gas release?
6.2B For biogas systems, biogas is used to full capacity
How often is the biogas of this system being used?
Re-use of wastewater Where does the effluent go?
Pre-treat wastewater before discharge to solid-free sewer
4.1A Users within acceptable range of design11
Is the utilisation (design/actual connected user) within acceptable range (66 – 133%)?
Is the intended wastewater type discharged to treatment?
5.1A No signs of structural damage compromising func-tionality or warranty aspects
What building structure problems do you observe at treatment system (digester, settler, ABR, AF, PGF)?
Do you observe problems with existing pumps (wastewater or water pumps)?
What structural problems with the sanitation facility(ies) (walls, roof and floor) do you observe?
Reduction of O&M costsIf the Objective was reduction of desludging costs, were desludging costs significantly reduced by this project?
67
10 This project specific objective is assumed for every system with a biogas digester in its set-up
11 This Indicator is included based on the assumption that incorrect loading or wastewater type would lead to reduced treatment and – in the case of underload - discharge of completely untreated wastewater to the environment
38 39E VA L U A T I O N R E S U L T S
! ‘Background information’ on the objective ‘reduction of O&M costs’
In Lesotho DEWATS are often implemented because of the comparably low operation costs for households: due to local soil conditions (very rocky and clayey) septic tank effluent does not easily infiltrate with frequent clogging and overflow of soak-aways – therefore requiring regular and costly emptying. DEWATS effluent on the other hand has undergone better treatment and can be recycled for gardening or better infiltrated – reducing the need of reactor emptying by desludging services. This objective may not exist in situations with different soil conditions.
In South Africa, reduction of O&M costs would rather refer to minimized electric costs when applying DEWATS, for comparison.
3.4.2 Result overview on Statement of Change level
Table 9 contains the average scores for project specific objectives (e.g. 51
projects aim for the re-use of wastewater. Out of these 51 projects, 45 achieve
this aim).
Note: Not all objectives are part of all projects
Systems with this project specific objective
Average scores per project specific objective
Produce and use biogas 29
4
19
51
Re-use of wastewater 51
45
06 0
Pre-treat ww before discharge to solid-free sewer
19
4
54
6
Reduction of O&M costs 40
35
005
3.4.3 Project specific objective ‘Produce and use biogas’3.4.3.1 Result overviewFigure 17 shows the per country distribution of average scores for the objective
‘Produce and use biogas’.
0 5 10 15 20
AfghanistanIndiaNepal
LesothoTanzaniaZambia
CambodiaIndonesia
LaosPhilippinesVietnam
Number ofsystemsObjectivemet Objectivesomewhatmet Objectivenotmet Infonotavailable
3.4.3.2 Parameters with strong impactPlease refer to Table 8 for the evaluation structure of this project specific
objective.
The following Parameters had the strongest negative or positive impact.
In 6 out of 34 cases (18%) biogas is not used at all (Nepal 1 out of 4, Lesotho
3 out of 9 and Tanzania 2 out of 6) (see Figure 18), which coincides in most
countries with the number of systems where no biogas reaches the appliances
(see Figure 19).
There was no clear relation between this observation and system type.
! 'Background information' on the objective ‘reduction of O&M costs'
The responses recorded by ME team Zambia indicate in all implemented systems biogas usage as well as no gas pressure on the piping systems. This may be due to the fact that coincidentaly all gas had been used up by the users just before the field-visits took place. Due to the responses given on gas usage, it is assumed here that biogas digesters are in good working order in Zambia.
! Suggested discussion on GD level D3: How relevant is the appropriate and rigorous use and promotion
of biogas for BORDA and its positioning regarding climate friendly technologies? If considered relevant, what are the required next steps?
Figure 17: Average scores of systems with the project specific objective ‘Produce and use biogas’
Table 9: Overview of average system scores per project specific objectives
40 41E VA L U A T I O N R E S U L T S
0 2 4 6 8 10 12 14
Afghanistan
India
Nepal
Lesotho
Tanzania
Zambia
Cambodia
Indonesia
Laos
Philippines
Vietnam
Number ofsystems
Dailybiogasusage
Notdailybutregularly
Seldom(lessthanonceaweek)
Nobiogasusage
0 2 4 6 8 10 12 14
Afghanistan
India
Nepal
Lesotho
Tanzania
Zambia
Cambodia
Indonesia
Laos
Philippines
Vietnam
Number ofsystems
Yes,biogasreachedappliance
Noaccesstogasvalve
Biogaspipinghasnotbeeninstalled
No,biogasdoesnotreachapplianceandNOleakages orblockagesinpipingareobserved
3.4.3.3 Recommendations regarding strong impact ParametersPossible follow-up action:
i Identify why in many cases biogas is not used and does not reach appliances
(formulate steps to improve hardware and/or maintenance issues)
3.4.4 Project specific objective ‘Reuse of wastewater’3.4.4.1 Result overviewFigure 20 shows the per country distribution of average system scores for
the objective ‘Reuse of wastewater’.
0 5 10 15 20
AfghanistanIndia
NepalLesotho
TanzaniaZambia
CambodiaIndonesia
LaosPhilippines
Vietnam
Number of systems
Objective met Objective not met Info not available
3.4.4.2 Parameters with strong impactPlease refer to Table 8 for the evaluation structure of this project specific
objective.
This objective is evaluated with only one parameter. If effluent is being reused
(fish, agriculture and/or horticulture for food production, non-food horticulture,
other) this objective is scored ‘Objective met’ – which is the case in 45 out
of 51 systems (88%). Otherwise this objective is scored ‘Objective not met’.
There was no clear relation between this parameter and system type.
3.4.4.3 Recommendations regarding strong impact ParametersPossible follow-up action:
i Emphasize the relevance of accurate definition of project objectives in
Nepal, where, in many cases in which reuse is stated as one of the project
objectives, wastewater is actually not being reused at all.
3.4.5 Project specific objective ‘Pre-treat ww before discharge to solid-free sewer’
3.4.5.1 Result overviewFigure 21 shows the per country distribution of average system scores for the
objective ‘Pre-treat ww before discharge to solid-free sewer’.
0 5 10 15 20
AfghanistanIndiaNepal
LesothoTanzaniaZambia
CambodiaIndonesia
LaosPhilippinesVietnam
Number ofsystemsObjectivemet Objectivesomewhatmet Objectivenotmet Infonotavailable
3.4.5.2 Parameters with strong impactPlease refer to Table 8 for the evaluation structure of this project specific
objective.
! ‘Background information’ The project specific objective ‘Pre-treat wastewater before discharge to solid-free sewer’ is scored with the help of parameters which investigate
• the volume of wastewater to be pre-treated (an underloaded system scores bad because it would in effect not pre-treat large volumes of wastewater it was designed for)
• the technical ability of the system to adequately pre-treat wastewater (load, wastewater type discharged to system, state of building structure of reactors and sanitation centres)
Figure 18: Biogas usage as observed in the different countries
Figure 19: Responses given to the question ‘When you open the gas valve furthest away from the BGD (e.g. at the stove), do you hear or smell gas release?’
Figure 20: Average scores of systems with the project specific objective ‘Reuse of wastewater’
Figure 21: Average scores of systems with the project specific objective ‘Pre-treat ww before discharge to solid-free sewer’
42 43E VA L U A T I O N R E S U L T S
The following Parameters had the strongest negative or positive impact.
The yellow and red scores shown for Zambian systems are mainly due to
i Parameter responses with red scores: utilisation rates below 66% (one
system) and above 133% design value (two systems)
i Parameter responses with yellow scores: minor physical damage of reactor
building structure (digester, settler, ABR, AF, PGF) which does not affect
operation or safety (six out of nine systems) but which may impede future
treatment
i Parameter responses with yellow scores: minor physical damage of
sanitation centre building structure which does not affect operation or safety
(seven out of nine systems) but which may lead in future to reduced usage
The red score for the Vietnamese system is mainly due to the fact that no
black-water is discharged to the system although it was designed for it – it is
therefore assumed that significant volumes of untreated blackwater reach
the environment.
3.4.5.3 Recommendations regarding strong impact ParametersPossible follow-up actions:
i repair systems with minor building structure damages in Zambia
3.4.6 Project specific objective ‘Reduction of O&M costs’3.4.6.1 Result overviewFigure 22 shows the per country distribution of average system scores for
the objective ‘Reduction of desludging costs’.
0 5 10 15 20
AfghanistanIndiaNepal
LesothoTanzaniaZambia
CambodiaIndonesia
LaosPhilippinesVietnam
Number ofsystemsObjectivemet Objectivesomewhatmet Objectivenotmet Infonotavailable
3.4.6.2 Parameters with strong impactPlease refer to Table 8 for the evaluation structure of this project specific
objective.
This objective is evaluated with only one parameter. The question ‘Were
desludging costs significantly reduced by this project?’ was confirmed in all
cases in which responses were given.
3.4.6.3 Recommendations regarding strong impact ParametersNo follow-up action required.
Picture: Re-use of biogas for cooking purposes (Tanzania)
Figure 22: Average scores of systems with the project specific objective ‘Reduction of desludging costs’
44 45
3.5 SoC 4: Functioning technology – systems are operating as intended
3.5.1 Statement of Change structureThis SoC evaluates the project outcome ‘functioning technology’ by considering
system loading and wastewater treatment.
Statement of Change
Objectives Priority Indicators Parameters
4. Functioning Technology - systems are operating as intended
4.1 System operat-ing as designed - acceptable loading and system hydraulics
4.1A Users within acceptable range of design
Is the utilisation (design/actual connected user) within acceptable range (66 – 133%)?
Is the intended wastewater type discharged to treatment?
4.1B Influent flow to the treatment system is observed
Do you observe flow or signs of recent flow at plant inlet (wet piping, wet inlet chamber)?
4.1C The system does not experience severe flow surges
Does the system experience any flow surge issues?
4.1D Evidence of effluent flow
Do you observe flow or signs of recent flow (wet piping, etc.) at plant outlet? (RESPONSES INCLUDE KILLER CRITERIA)
4.2 Systems operating as de-signed - treatment meets BORDA requirements
4.2A Anaerobic, and where applicable aerobic, effluent quality meet BORDA requirements
Does the average COD concentration of the anaerobic effluent comply to BORDA design value (200 mg/ l)?
Does the average COD concentration of the aerobic effluent comply to BORDA design value (80 mg/ l)?
4.2B There are obvious signs of biogas production from the biogas digester
When you open the gas valve furthest away from the BGD, do you hear or smell gas release?
3.5.2 Result overview on Statement of Change levelFigure 23 shows the average SoC 4 scores for all visited systems, globally.
The large number of incomplete data sets is mainly due to
i lack of anaerobic and (where relevant) aerobic effluent sampling (55 and
57 cases respectively).
i lack of information on the actual user number (36 cases)
Red = ‘Bad’ (score < 1.5), Yellow = ‘Caution’ (1.5 ≤ score < 2.5), Green = ‘Good’ (score ≥ 2.5),Grey = ‘Dataset incomplete’;numbers indicate number of systems per score range
For all details on Objectives, Priority Indicator and Parameter results, refer
to Appendix A.
Figure 24 shows the per country distribution of average SoC scores.
0 10 20 30 40 50
AfghanistanIndiaNepal
LesothoTanzaniaZambia
CambodiaIndonesia
LaosPhilippines
Vietnam
Number ofsystems
Good Caution Bad Datasetincomplete
3.5.3 Parameters with strong impactThe following Parameters had the strongest negative or positive impact.
i 7 systems (5%) do not treat any wastewater at the time of field-visit.
! Corrective action motivated by ME field visits7 visited systems (1 in Zambia, 6 in India) were found to not treat any
wastewater at all. Motivated by the findings of the ME field visits in 2017, repairs were done at 2 of them (1 in Zambia and 1 in India) and at the time this report is written both are fully functional.
This highlights the importance of • ME-type quality assessment of implemented DEWATS • The elaboration of procedures for the financing and implementation
of corrective actions
i The utilitsation rate in 43 out of 103 (42%) cases is either below 66% or
above 133% design value (see Figure 25). Out of these 43 systems, by far most
(35) have a too low utilisation rate.
This issue is observed in all countries and is not dependent on age. However, the
largest fractions of systems with low loading are found in India (see Figure 26)
where by far most systems cater for less people than assumed during design.
The utilitsation rate for system type ‘CSC’ in 10 out of 11 cases is below the
benchmark of 66% design value. For all other system types there appears to
be no clear relation with this parameter.
i Effluent concentrations:
] The internal BORDA standard for acceptable anaerobic reactor
effluent of 200 mg COD/ l is not met in 16 out of 76 (21%) cases (see Figure
27) when allowing for 20% tolerance (actual benchmark: 240 mg COD/ l).
] Available data shows no relation with system type.
] In some cases the anaerobic effluent is further treated aerobically
(usually by a HGF).
Figure 24: Per country distribution of average scores for SoC 4: Functioning technology
Figure 23: Pie charts with distribution of average scores for SoC 4: Functioning technology
Table 10: Evaluation structure for SoC 4 – Functioning technology
711
3884
E VA L U A T I O N R E S U L T S
46 47E VA L U A T I O N R E S U L T S
] The internal BORDA standard for acceptable aerobic effluent of 80
mg COD/ l is not met in 16 out of 39 (41%) cases (see Figure 12) even when
allowing for 20% tolerance (actual benchmark: 96 mg COD/ l). Countries
with the highest fraction of high effluent concentrations are Lesotho
followed by India. Available data shows no relation with system type.
i Observations on flow surge issues (signs of water fluctuations inside reactor
chambers, observed changes during strong rain, backflow from waterbody
system discharges to) indicate that out of 134 cases, 19 systems (14%) experience
severe flow surges and 26 systems (19%) experience some form of related
issues. These issues are found across all countries and most system types.
i Two thirds of all visited systems show no signs of exposure.
i In 11 out of 35 cases (31%) biogas does not reach biogas appliances.
i Positive: The wastewater type discharged to the systems matches design
assumptions in 129 out of 134 cases (96%).
0%
50%
100%
150%
200%
250%
300%
350%
400%
450%
500%
Util
isat
ion
rate
(a
ctua
l vs
desi
gn u
nit n
umbe
r)
Estimated utilisation rates of 103 systems
0 10 20 30 40 50
AfghanistanIndiaNepal
LesothoTanzaniaZambia
CambodiaIndonesia
LaosPhilippines
Vietnam
Number ofsystemsBelow (<66%) Acceptable (≥66%,<133%) Above(≥133%) Nodata
0
100
200
300
400
500
600
700
800
900
1000
Anaerobicreactore
ffluentconcentration
(mgCO
D/l)
Afghanistan Cambodia India Indonesia Laos Lesotho Tanzania
3.5.4 Recommendations regarding strong impact ParametersPossible follow-up actions:
i identify reasons for low utilisation rates – especially in India. Consider
technical solutions to make systems more adaptable to varying loads (e.g.
designs with several treatment trains which can easily be by-passed).
i Increase number of COD sampling data (acidification and accredited
laboratories) by stricter adherence to ME SOPs
i Implement storm-water diversion options for large scale roll-out
i identify why in many cases biogas does not reach appliances to provide
corrective action, i.e. repair the hardware and/or address maintenance issues
i investigate (with/by country engineers or involved R&D staff) probable
reasons for high and low effluent concentrations.
Figure 26: Utilitsation rates as estimated for 103 systems depending on country
Figure 27: Measured effluent concentrations of anaerobic treatment12 Each data-point represents one or the average of two measurements per system effluent. Two datapoints are beyond 1000 mg COD /l (both values are 1675 mg COD/ l, Ammankulam Housing Colony Cluster 2 & 3/ India)
1 2 This Figure differs from Figure 11 by showing all measured anaerobic reactor effluents, also those which are further treated aerobically
Figure 25: Utilitsation rates as estimated for 103 systems, values are plotted in accending order
Picture: Structural damage and system hydraulics inspection of ABR (Indonesia).
48 49E VA L U A T I O N R E S U L T S
3.6 SoC 5: Functioning maintenance – systems are maintained as intended
3.6.1 Statement of Change structureThis SoC evaluates the project outcome ‘functioning maintenance’ by
investigating signs of major damage and regular system maintenance.
Statement of Change
ObjectivesPriority Indicators
Parameters
5. Functioning Maintenance - systems are maintained as intended
5.1 Systems maintained - no major damage
5.1A No signs of structural damage compromising functionality or warranty aspects
What building structure problems do you observe at treatment system (digester, settler, ABR, AF, PGF)?(RESPONSES INCLUDE KILLER CRITERIA)
Do you observe problems with existing pumps (wastewater or water pumps)?
What structural problems with the sanitation facili-ty(ies) (walls, roof and floor) do you observe?(RESPONSES INCLUDE KILLER CRITERIA)
5.2 Maintenance activities occurring as intended
5.2A Maintenance is adequate
Do you observe flow or signs of recent flow at plant inlet (wet piping, wet inlet chamber)?
In the past, were maintenance issues with the sewer network fixed (clogging, blockage, bad smell or overflow)?
In the past, were major issues with the sewer network fixed (broken pipes or manhole, leakages, other major damage)?
In the past, were maintenance issues with the house-hold grease-traps fixed (clogging, bad smells)?
Can the manhole covers of the treatment system be opened? Please try at least 5 manhole covers. In case they were opened before the start of the field visit, please ask the m.e. or operator.
Could a local desludging service provider access the treatment system with his cart, truck, etc.?
Has the system been desludged?
How thick is the scum in the second (or if too difficult to open, third) ABR chamber?
Do you see much plastic waste (more than 20 items) inside the second (or if too difficult to open, third) anaerobic reactor chamber (can be settler, ABR or AF chamber)? (single selection)
Do you observe problems concerning the planted gravel filter?
What functional problems do you observe about the sanitary installations?
Do you observe problems with the water trap?
When you open the gas valve furthest away from the BGD (e.g. at the stove), do you hear or smell gas release?
Is the biogas appliance working (lamp/ heater/ stove)?
Do you observe problems with the pressure gauge?
For BGD, is unused biogas burned off?
Table 11: Evaluation structure for SoC 5 – Functioning maintenance
Picture: DEWATS effluent sampling at Beedi-workers-colony for subsequent laboratory analysis at CDD Society (Bangalore, India)
50 51E VA L U A T I O N R E S U L T S
0 10 20 30 40 50
AfghanistanIndiaNepal
LesothoTanzaniaZambia
CambodiaIndonesia
LaosPhilippines
Vietnam
Number ofsystems
Noproblem Minordamage Majordamage
] Of 24 visited sanitation facilities, 11 (46%) show some signs of minor
physical damage (e.g. small cracks), not affecting operation, use or safety.
i Maintenance related Parameters relevant to all system types:
] The typical desludging interval of normal loaded ABRs is about 3
years (desludging periods for normal loaded settlers have been revised
and shortened; see DEWATS Spread sheet). 46 out of 78 visited systems
(56%) which have been operating longer than 3 years have never been
desludged.
Figure 31 presents the per country distribution: all or almost all systems
visited in Lesotho, Tanzania and Zambia are concerned as well as most
systems in India and Afghanistan.
Out of these 46 systems, 22 systems are underloaded (actual user number
is below the threshold of 66% of design user number) which could explain
why desludging did not happen after 3 years of operation. However, 22
systems (50%) are normal to highly loaded (actual user number is above
66% of design user number).
Most systems in SEA are operating less than three years.
Available data shows no relation with system type.
! Appropriate desludging intervalsFor this evaluation, desludging intervals of 3 years – an estimation
often used in BORDA documents and supported by R&D results for tropical applications – is taken as benchmark.
Appropriate desludging is used as one of several proxies for ‘functioning maintenance’ because of its importance for adequate system operation.
What limits the use of this proxy is the underlying assumption that all reactors are designed with a similar 'load to reactor volume' ratio – an assumption which was found to be wrong during the review of this report (e.g. reactors in Lesotho are often designed larger than in other countries).
Also, ideally actual system load per m³ reactor volume would need to be calculated to ensure that no underloaded systems are scored ‘bad’ (a first attempt was done to identify underloaded systems by comparing actual and design user number).
Figure 30: Physical damage observed at treatment system building structure (digester, settler, ABR, AF, HGF/PGF). If minor and major damages were observed at the same system, only the occurance of major damage is shown in this graph
3.6.2 Result overview on Statement of Change levelFigure 28 shows the average SoC 5 scores for all visited 140 systems, globally.
Red = ‘Bad’ (score < 1.5), Yellow = ‘Caution’ (1.5 ≤ score < 2.5), Green = ‘Good’ (score ≥ 2.5),Grey = ‘Dataset incomplete’;numbers indicate number of systems per score range
For all details on Objectives, Priority Indicator and Parameter results, refer
to Appendix A
Figure 29 shows the per country distribution of average SoC scores.
Most system scores are ‘good’ in Afghanistan, Cambodia, Indonesia, Laos
and Philippines.
On the other hand, system scores in Zambia and Lesotho are very often ‘caution’
and partly ‘bad’ for all visited systems. Details are provided by below graphics.
0 10 20 30 40 50
AfghanistanIndiaNepal
LesothoTanzaniaZambia
CambodiaIndonesia
LaosPhilippines
Vietnam
Number ofsystems
Good Caution Bad Datasetincomplete
3.6.3 Parameters with strong impactThe following Parameters had the strongest negative or positive impact.
i In most cases (132 systems; 94%) there is no or only minor structural
damage to the reactor building structure
] Major structural damage compromising functionality or warranty aspects
was however observed at 11 systems (major physical damage observed 6
times: India, 2; Lesotho, 1; Nepal, 1; Tanzania, 1; Vietnam, 1), severely damaged
or missing manhole cover observed 3 times (India, 1; Lesotho, 1; Zambia, 1),
7 systems were found not to be treating any wastewater))
Available data shows no relation with system type.
] 26 instances (19%) of minor physical damage (e.g. cracks in building
structure) which do not affect operation or safety (but would probably
require some reparation work in the medium term) were observed (digester,
settler, ABR, AF, HGF/PGF). See Figure 30 for the distribution across
countries: in Lesotho, Zambia and Tanzania large fractions of visited
systems have minor physical damage.
] In 103 systems (73%) there was no noticeable damage at all.
Available data shows no relation with system type.
Figure 29: Per country distribution of average scores for SoC 5: Functioning maintenance
11
5165
13
Figure 28: Pie charts with distribution of average scores for SoC 5: Functioning maintenance
52 53E VA L U A T I O N R E S U L T S
i Parameters relevant to sewer system maintenance:
] Often, maintenance of sewer network is insufficient: in the past, 42 out
of 73 visited systems connected to a sewer line experienced maintenance
issues (clogging, blockage, bad smell or overflow) which were not fixed
in 10 (23%) cases.
Figure 33 shows the per country distribution. In Nepal, maintenance of all
sewer systems is reported to be inadequate. In India a very large fraction
of interviewees did not know whether the sewer system ever required any
maintenance at all – which probably indicates poor maintenance.
] 38 systems experienced major issues in sewer networks (broken
pipes or manhole, leakages, other major damage) of which 11 cases (28%)
were not corrected
Figure 34 shows the per country distribution and indicates an identical
trend as for sewer network maintenance: In Nepal, repairs of all sewer
systems is reported to be inadequate. In India a very large fraction of
interviewees did not know whether the sewer system ever required repairs
at all – which can indicate poor maintenance.
0 10 20 30 40 50
AfghanistanIndiaNepal
LesothoTanzaniaZambia
CambodiaIndonesia
LaosPhilippines
Vietnam
Number ofsystems
Yes,allissuesfixedSome issesfixedNoissuesfixedInterviewee doesnotknowNomaintenanceissuessofarNosewernetwork- notrelevant
0 10 20 30 40 50
AfghanistanIndiaNepal
LesothoTanzaniaZambia
CambodiaIndonesia
LaosPhilippines
Vietnam
Number ofsystems
Yes,allmajor issuesfixedSomemajorissuesfixedNomajorissuesfixedInterviewee doesnotknowNomajorissuessofarNosewernetwork- notrelevant
] Out of 26 visited systems with grease traps, 6 responses (23%) indicate
maintenance issues which were never fixed.
Figure 33: Management entity or operator responses to the question: ‘In the past, were maintenance issues with the sewer network fixed (clogging, blockage, bad smell or overflow)?’, colors indicate response scoring, ‘Interviewee does not know’is scored ‘caution’
Figure 34: Management entity or operator responses to the question: ‘In the past, were major issues with the sewer network fixed (broken pipes or manhole, leakages, other major damage)?’, colors indicate response scoring, ‘Interviewee does not know’is scored ‘caution’
Since such tests go beyond the scope of this report (available loading information is scarce and difficult to interpret and reactor volumes are often missing in project documentation) the evaluation results of this parameter should be regarded as indicative only.
For future data evaluation, the scoring of this parameter will be adapted to better suit the existing, varying situations.
0 10 20 30 40 50
AfghanistanIndiaNepal
LesothoTanzaniaZambia
CambodiaIndonesia
LaosPhilippines
Vietnam
Number ofsystems
Yes,withinthelast3years Yes,butmorethan3yearsago
Never, systemisolderthan3years Interviewee doesnotknow
Never, systemisyoungerthan3years
] At 23 out of 140 systems (16%), reactor manhole covers could not be
opened at all or only after very much effort. This affects also the parameter
scores on scum and solid waste removal in PI 5.2A (see Table 11).
] Good access of local desludging service provider was reported in 132
out of 140 cases (94%).
i Parameters relevant to HGF maintenance:
] A total of 52 observations on maintenance were made at 45 out of
76 visited HGFs (59%). For details see Figure 32.
] The most critical aspect is filter blockage:
Filter material is considered blocked if water stagnates on filter surface
and swivel pipe is set correctly. This was observed at 11 systems (6, 1, 1, 2
and 1 systems in India, Nepal, Lesotho, Tanzania and Zambia, respectively
– the data set does not show an obvious relation with system type).
05101520
Otherproblems(e.g.systemsmellsbad.,manydeadleaves,fewplants)
Deadplants
Stagnatingwateronthesurface-probablereason:filterblockage
Stagnatingwateronthesurface-swivelpipetoohigh
Largeamountsofsolidwaste
Slimeonsurface
Swivelpipebroken,waterlevelverylow
Numberofobservations
Figure 31: Occurance of ABR/AF desludging
Figure 32: Observations made on the maintenance of 76 visited HGFs
54 55E VA L U A T I O N R E S U L T S
3.6.4 Recommendations regarding strong impact ParametersSystem maintenance is a challenge in all investigated countries (although
less pronounced in Afghanistan, Cambodia and Indonesia).
Possible follow-up actions:
i corrective action for all 11 systems with severe structural damage.
i investigate reasons for poor system maintenance concerning:
] Desludging
] PGF (blockages)
] Sanitation centres
] Small sewer systems
] Biogas systems
Picture: Manual desludging with sludge gulper (Daressalaam; Tanzania)
! Grease trapsMissing effluent flow at one Vietnamese DEWATS treating wastewater
from a restaurant lead to further investigations. It was found that the installed grease trap was too small and that therefore large amounts of grease reached the reactors and blocked pipes and the HGF.
This highlights the critical importance of correct grease trap dimensioning and maintenance – especially in the case of food industry and restaurants – and – should they not yet exist or not be accessible to all BORDA engineers - the need for the dissemination of detailed design and maintenance guidelines for grease traps.
i Parameters relevant to sanitation facility maintenance (all implemented
by the BORDA partner network):
] A total of 62 observations on various issues were made at 16 out of
24 visited sanitation facilities (66%). For details see Figure 35.
] Most frequently observed issues are missing O&M and HHE posters,
missing waste bins with covers inside cubicles, no functioning lights,
unused laundry area and missing handwashing facilities.
0 2 4 6 8 10 12
Water isnotavailable
Sometoilets/bathroomsseemunused
Facilitydoesnothavefunctioninglights
Blockedfloor-drains
Sometoilets/bathroomsareblocked/broken
Sometoilets/bathroomsareunclean
Brokenhandwashing-basinortap
Somedoorsaremissing/broken/can'tbelocked
Laundryareaseemsunused
Nohandwashingfacility
Electricity isnotavailable
Nowastebinswithcoversinsidecubicles
NoO&MorHHEpostersarehungup
Number ofobservations
i Parameters relevant to biogas system maintenance:
] Water trap installation is an issue at 31 out of 35 (89%) visited biogas
settlers: in 23 cases (66%) there is no watertrap; in 8 cases it cannot be
accessed (23%).
] Biogas often does not reach the stove (18 cases; 51%).
] Biogas, when unused, is almost never flared off (12 out of 13 cases;
92%).
] In many cases, pressure gauges are not installed (25 cases; 71%). In
3 cases pressure gauges are installed but don’t contain water (which can
lead to biogas loss).
] The stove was reported to be in working order in 22 cases (63%).
i About Parameters relevant to pump maintenance:
] Pumps generally work well: 44 visted projects included pumps, out of
which all worked in 38 cases (86%). At four projects the pump was found
to be broken, which however only affected the treatment in one case.
Figure 35: Observed maintenance issues at sanitation centres (24 visited sanitation centres)
56 57
3.7 SoC 6: Sustaining demand – system is available, used to capacity and acceptable
3.7.1 Statement of Change structureThis SoC evaluates the project outcome ‘sustaining demand’ by investigating
availability, utilisation rate and acceptability of the system.
Table 12: Evaluation structure for SoC 6 – Sustaining demand
Figure 36: Pie charts with distribution of average scores for SoC 6: Sustaining demand
7 9
76
48
Statement of Change
Objectives Priority Indicators Parameters
6. Sustaining Demand - system is available, used to capacity and acceptable
6.1 Service is adequately available to users
6.1A For systems with SSS, system receives and processes wastewater
Do you observe flow or signs of recent flow at plant inlet (wet piping, wet inlet chamber)?
(RESPONSES INCLUDE KILLER CRITERIA)
6.1B For systems with CSC, ratio of users to functioning toilets is acceptable
Acceptable ratio of users per functioning toilets (20 users per toilet)
6.1C DEWATS processes wastewater
Do you observe flow or signs of recent flow (wet piping, etc.) at plant outlet?
(RESPONSES INCLUDE KILLER CRITERIA)
6.1D For biogas systems, biogas can be used
When you open the gas valve furthest away from the BGD, do you hear or smell gas release?
6.2 Utilisation rate: Service is used to capacity
6.2A Utilisation is close to full capacity (ie. actual/design users)
Is the utilisation (actual/design users) wihin acceptable range?
6.2B For biogas systems, biogas is used to full capacity
How often is the biogas of this system being used?
6.3 Acceptability: Culturally ac-ceptable, users satisfied with system
6.3A Satisfaction indicated by high utilisation or no evident issues of low acceptance
Good acceptance and satisfaction is indicated by:
- utilization rate above 66% (actual vs. design value) or
- if users do not give reasons related to low ac-ceptability when asked why people don’t connect to/ use the DEWATS
3.7.2 Result overview on Statement of Change levelFigure 36 shows the average SoC 6 scores for all visited 140 systems, globally.
The large number of incomplete datasets is mainly due to lack of information
on the actual user number (37 cases).
Red = ‘Bad’ (score < 1.5), Yellow = ‘Caution’ (1.5 ≤ score < 2.5), Green = ‘Good’ (score ≥ 2.5),Grey = ‘Dataset incomplete’;numbers indicate number of systems per score range
For all details on Objectives, Priority Indicator and Parameter results, refer
to Appendix A.
Figure 37 shows the per country distribution of average SoC scores.Picture: Gravel washing with freshwater at Beedi-workers Colony, highlighting the partially work intensive O&M activities (Kengeri; India)
E VA L U A T I O N R E S U L T S
58 59
0 10 20 30 40 50
AfghanistanIndiaNepal
LesothoTanzaniaZambia
CambodiaIndonesia
LaosPhilippines
Vietnam
Number ofsystems
Good Caution Bad Datasetincomplete
3.7.3 Parameters with strong impactThe following Parameters had the strongest negative or positive impact.
i The utilitsation rate in 43 out of 103 (42%) cases is either below 66% or
above 133% design value (see 25). Out of these 43 systems, by far most (35)
have a low utilisation rate.
This issue is observed in all countries to a degree or another and is not
dependent on age or system type. However, the largest fractions of systems
with low loading are found in India (see Figure 26) where by far most systems
cater for less people than assumed during design.
The utilitsation rate for system type ‘CSC’ in 10 out of 11 cases (91%) is below
the benchmark of 66% design value. Other system types do not show a clear
relation with this parameter.
i Availability of sanitation service is reduced since:
] 7 systems don’t treat any wastewater.
] In 11 out of 35 cases (31%) biogas does not reach biogas appliances.
] Often high ratio of users per functioning toilet: in 9 out of 21 cases
(43%), more (at times far more!) than 20 users are using one toilet. 8 of
these 9 cases are schools with up to 300 users per toilets.
i Cultural acceptability appears high across all systems: in 136 out of 139
cases (98%) utilitsation rate is either high enough to indicate acceptance or
lower utilisation rates are due to other reasons than (cultural) inacceptance
(when asked the question ‘Why do people / HH not connect to DEWATS system?’
community representatives would answer ‘technical issues’, ‘community budget
does not allow for further connections’ or ‘community internal conflict’).
3.7.4 Recommendations regarding strong impact ParametersPossible follow-up actions:
i investigate reasons for the large number of systems with low utilisation
rates (especially in India). Consider technical solutions to make systems more
adaptable to varying loads (e.g. designs with several treatment trains which
can easily be bypassed).
i investigate why biogas does not reach appliances and improve hardware
and/or maintenance issues.
i discuss and define an adequate ‘users per toilet ratio’ depending on
project type and review design recommendations.
Figure 37: Per country distribution of average scores for SoC 6: Sustaining demand
Picture:: Explaining the importance of WASH to school kids (Kurdistan)
E VA L U A T I O N R E S U L T S
60 61
3.8 SoC 7: Effective management - existing, active and accountable management entity and operator
3.8.1 Statement of Change structureThis SoC evaluates the project outcome ‘effective management’ based on the
existence of an active and accountable management entity and of a trained
and adequately equipped operator.
In some project types the management entity and the operator are one and
the same person (e.g. in the case of single house systems: the house owner)
or the operator has other main duties for which he/she is employed (e.g. some
school systems where the teachers take on O&M activities).
Statement of Change
ObjectivesPriority Indicators
Parameters
7. Effective management: Existing, active and accountable management entity and operator
7.1 Active and accountable management entity
7.1A Existence of trained management entity with clarified responsibilities
Does a management entity (m.e.) exist?(RESPONSES INCLUDE KILLER CRITERIA)
Did the management entity receive the required trainings (Financial, O&M, HHE)?
Is the management entity aware of its responsibilities?
Does management entity have documentation of legal registration to carry out its responsibilities (e.g. officially registered with appropriate government department)?
7.1B Management entity is active and accountable
Does the management entity meet regularly?
Are O&M responsibilities of the operator defined and documented?
Is the operator regularly paid an agreed salary (in cash, in kind or both)?
Are income and/or expenses documented in financial administration logbook or elsewhere?
Is there documentation of conducted O&M activities?
7.2 Trained and equipped operator
7.2A Existence of trained, equipped operator who knows his responsibilities
Is there a person/caretaker/operator assigned and responsible for O&M activities? (in the following called ‘operator’)(RESPONSES INCLUDE KILLER CRITERIA)
Has the operator received O&M training?
Is the operator aware of his responsibilities?
Does the operator have all necessary tools required to perform O&M activities (opening manholes, deblocking sewer system, scum and solid waste removal from reactors)?
3.8.2 Result overview on Statement of Change levelFigure 38 shows the average SoC 7 scores for all visited 140 systems, globally.
For all details on Objectives, Priority Indicator and Parameter results, refer
to Appendix A.
Red = ‘Bad’ (score < 1.5), Yellow = ‘Caution’ (1.5 ≤ score < 2.5), Green = ‘Good’ (score ≥ 2.5),Grey = ‘Dataset incomplete’;`numbers indicate number of systems per score range
Figure 39 shows the per country distribution of average SoC scores.
0 10 20 30 40 50
AfghanistanIndiaNepal
LesothoTanzaniaZambia
CambodiaIndonesia
LaosPhilippines
Vietnam
Number ofsystems
Good Caution Bad Datasetincomplete
Concerning Afghanistan: this graph depicts a worse than actual situation since
five systems are managed by one management entity which (at the time of the
field visit, due to various external factors) was not able to employ an operator.
3.8.3 Parameters with strong impactThe following Parameters had the strongest negative or positive impact.
i 2 out of 136 cases (1%) have no management entity (both cases are in India).
i At 14 out of 140 systems (10%), nobody at all performs O&M activities (see
Figure 40). These systems are almost all residential systems, all located in
India (9 systems) and Afghanistan (in the case of Afghanistan: all 5 systems
are part of the same housing complex managed by one entity, thus 1 operator
is missing at time of field visit).
i At 14 out of 140 systems (10%), people without formal O&M responsibility
take over the O&M of the system to some degree.
0 10 20 30 40 50
AfghanistanIndiaNepal
LesothoTanzaniaZambia
CambodiaIndonesia
LaosPhilippines
Vietnam
Number ofsystemsYes,operatoriscommunitymember,employee (ofSMEorinstitution)ortheowner
Yes,operatorisexternal serviceprovider
No,butoneormorepeopleWITHOUTO&Mresponsibilitytakecareoffacility
No,nobodyisassignedandresponsibleforO&Mactivities
15
5548
22
Figure 39: Per country distribution of average scores for SoC 7: Effective management
Figure 38: Pie charts with distribution of average scores for SoC 7: Effective management
Figure 40: Operator existence at visited systems depending on country
Table 13: Evaluation structure for SoC 7 – Effective management
E VA L U A T I O N R E S U L T S
62 63
i Observations indicating low activity or accountability of management entities:
] No documentation of conducted O&M activities: 101 out of 140 cases (72%)
] No documentation on financial administration: 56 out of 140 cases (40%)
] No operator payment: 16 out of 95 known cases in which the operator
should have been paid (17%) (see Figure 41 for country details)
] The management entity does not meet regularly in 27 out of 107 cases
(25%)
] No documentation of the operators’ O&M responsibilities in 26 out of
123 cases (21%)
0 5 10 15 20 25 30 35 40 45 50
Afghanistan
India
Nepal
Lesotho
Tanzania
Zambia
Cambodia
Indonesia
Laos
Philippines
Vietnam
Number ofsystems
Operator ispaid
Notalwayspaidornotenough
Nopayment
There isnooperator
Operatornotavailable forinterview
Nodata
HHsystem- operatorisnotpaid
! O&M related documentation held by usersThis ME method includes checks on whether the following O&M
related written documentation exists at a project site:
– Financial administration– Operator responsibilities– Performed O&M activities– Meeting documentation of m.e. meetings ! – Legal registration of m.e.Missing documentation is currently scored as ‘bad’ in the case of financial administration, in all other cases ‘yellow’ or ‘green’.
] Limited awareness of responsibilities: when asked what their
responsibilities are, management entity representatives often exclude
the operator management although listing other responsibilities (55 out of
109, 50%, excluding single household systems138) (see Figure 42 for country
distribution). Also, ‘financial administration’ was only listed in 41 out of 132
cases (31%) (see Figure 43). In six cases the response by the management
entity representative indicates no awareness of responsibilities.
Available data indicates no relation with system type.
13 Single household systems are excluded from this analysis because existing data is difficult to interpret. At such systems, the management entity and the operator are typically one and the same person: the household owner. It would therefore seem unnatural for a household owner to consider ‘employing and managing operator’ as part of his responsibilities.
0 10 20 30 40 50
AfghanistanIndiaNepal
LesothoTanzaniaZambia
CambodiaIndonesia
LaosPhilippinesVietnam
Number ofsystemsYes,awareofallfive responsibilities
Aware ofatleastoneresponsibility(includingmanagingoperator)
Aware ofatleastoneresponsibility(butnotmanagingoperator)
Notawareofanyresponsibility
Aware ofotherresponsibilities
There isnomanagemententity
Managemententitynotavailable forinterview
Nodata
0% 20% 40% 60% 80%
Coordinating&arrangingfinancingofmajorrepairs
Employingandmanagingoperator
Financialadministration(generatingandadministratingincome)
Trainingoperator
Decidingaboutadditionalconnections
Other
Interviewee doesnotknow
M.e. thinkshe/shehasnoresponsibility
Projects atwhich responseoption wasgiven(%oftotal 123systems)
] Training of management entities generally occurs but in most cases
includes only O&M training:
] Training of management entities does often not include all three
modules (financial, O&M, HHE) defined in this evalution as the benchmark:
only in 18 out of 132 cases (14%) were all modules received – mainly in
Indonesia and Cambodia (see Figure 44).
Figure 41: Occurance of operator payment per country
Figure 42: Evaluation of question answered by the management entity: ‘What are your responsibilities concerning this project?’ Options (not shown to the m.e.): Financial administration, coordinating and arranging financing of major repairs, employing and managing operator, training operator, deciding about additional connections, other; colors indicate scoring
Figure 43: Given response options for the question: ‘What are your (the management entity’s) responsibilities concerning this project?’. Graph excludes the 17 visited sinlge household projects. Out of the 123 remaining projects, 2 did not have a management entity.
E VA L U A T I O N R E S U L T S
64 65
In most cases one or two modules were received (in 95 out of 132 cases,
72%). Financial and HHE training were the least received trainings. O&M
trainings were significantly more often received (see Figure 45).
Interestingly, a higher number of received training modules does not
correlate with increased management entity awareness of responsibilities
such as defined for this evaluation (compare Figure 42 and Figure 44).
The dataset does not indicate a clear relationship between this parameter
and system type.
0 10 20 30 40 50
AfghanistanIndiaNepal
LesothoTanzaniaZambia
CambodiaIndonesia
LaosPhilippines
Vietnam
Number ofsystemsYes,allthree trainingmoduleswere received
Oneortwooutofthreetrainingswere received
None- m.e.haschangedsincestartofoperation
None- m.e.hasnotchangedsincestartofoperation
M.e. receivedothertypeoftraining
There isnomanagemententity
Managemententitynotavailable forinterview
Nodata
0 20 40 60 80 100 120
None- m.e. samesincestartofoperation
None- m.e.haschangedsincestartofoperation
HHEtraining
Financialtraining
Othertraining
O&Mtraining
Number ofprojects atwhichm.e.trainingmodulewasreceived
i Operators are often not aware of most or any of the operator responsibilities
(as defined as: removing blockages, removing scum, cleaning facility, desludging,
reporting problems to management, collection fees, other): 48 out of 109 cases
(45%). This is especially the case for systems visited in India, Lesotho and
Indonesia (see Figure 46 for the per country distribution).
The dataset does not indicate a clear relationship between this parameter
and system type.
Figure 47 indicates which activities operators consider part of their
responsibility. Operators most often don’t consider ‘collecting user fees’,
‘desludging’ and ‘removing scum’ as part of their responsibility.
0 10 20 30 40 50
Afghanistan
India
Nepal
Lesotho
Tanzania
Zambia
Cambodia
Indonesia
Laos
Philippines
Vietnam
Number ofsystems
Aware ofall(5or6)responsibilities
Aware ofmost(3or4)responsibilities
Aware offew(1or2)responsibilities
Notawareofanyresponsibility
There isnooperator
Operatornotavailable forinterview
0% 10% 20% 30% 40% 50% 60% 70%
Cleaningfacility
Removingblockages
Reportingproblemstomanagemententity
Removingscum
Desludging
Nodata
Other
Collectingfees*
Interviewee doesnotknow
Operator thinkshehasnoresponsibilities
Projects atwhich responsewasgiven(%oftotal visitedsystems)
i In most (104 out of 109 cases, 95%) cases operators have received O&M
training. However this training is often described as limited or brief, especially
in India, Tanzania and Zambia (see Figure 48).
Interestingly, an (assumed) higher level of training does not correlate with
increased operator awareness of responsibilities such as defined for this
evaluation (compare Figure 46 and Figure 48).
The dataset does not indicate a clear relationship between this parameter
and system type.
Figure 44: Number of training modules received by management entity across countries
Figure 45: Given response options for the question: ‘What trainings did the management entity receive?’. Out of 140 projects, 2 did not have a management entity.
Figure 47: Given response options for the parameter: ‘What are your (the operator’s) responsibilities concerning this project?’. Out of 140 projects, 30 did not have an operator. *Percentage for response option ‘Collecting fees’ was calculated based on 29 systems at which user fees were reported to be part of the income
Figure 46: Evaluation of operator question: ‘What are your responsibilities concerning this project?’ Options (not shown to the operator): Removing blockages, removing scum, cleaning facility, desludging, reporting problems to management, collection fees, other; colors indicate scoring
E VA L U A T I O N R E S U L T S
66 67
0 10 20 30 40 50
Afghanistan
India
Nepal
Lesotho
Tanzania
Zambia
Cambodia
Indonesia
Laos
Philippines
Vietnam
Number ofsystems
Yes
Onlylimited/brief training
No,operatorhaschangedsincestartofoperationNo,operatorsamesincestartofoperation
There isnooperator
3.8.4 Recommendations regarding strong impact ParametersPossible follow-up actions:
i investigate why in two Indian cases no management entity exists and
initiate corrective action
i investigate reasons for non-existence of operators in 28 cases (Afghanistan,
India, Nepal, Tanzania, Indonesia), initiate corrective action and share lessons
learned
i discuss the importance of written documentation of conducted O&M
activities, O&M responsibilities, management entity meetings and financial
administration for effective management
i emphasize the relevance of trainings and m.e.’s/operators’ awareness for
project sustainability
i investigate
] reasons why m.e.’s in most countries don’t receive all three training
modules (especially financial and HHE training),
] links between missing training, limited anwareness of responsibilities
and actual O&M and financial management
] reasons why m.e.’s often have limited awareness of their responsibilities.
] reasons why in some cases, operators are not paid.
] reasons why operators are often not aware of most or any of the
commonly assumed operator responsibilities (see Figure 46)
Figure 48: O&M training received by operator across countries
Picture: Management workshop with educators at NPH orphanage starting a DEWATS project with BORDA (Nicaragua)
E VA L U A T I O N R E S U L T S
68 69
3.9 SoC 8: Sustainable financing – sufficient ongoing income to cover all short and long term costs
3.9.1 Statement of Change structureThis SoC evaluates the project outcome ‘sustainable financing’ by investigating
the existence of regular income sufficient to cover all short and long term costs.
Statement of Change
Objectives Priority Indicators Parameters
8. Sustainable Financing: Sufficient ongoing income to cover all short and long term costs
8.1 Regular income
8.1A O&M budget and/or user fees have been agreed on and are collected
Was a fixed O&M budget defined?
What are the income sources contributing to available O&M budget?(RESPONSES INCLUDE KILLER CRITERIA)
Were the user fees set by local authority or agreed on by users themselves?
Is someone responsible for the user fee collection process and is this documented?
Are user fees collected?
What proportion of users pay fees?
8.2 Sufficient income to cover all short and long term costs
8.2A Regular operation and maintenance expenses (operator salary, material, equipment, electricity and water costs) are covered by income
Is the operator regularly paid an agreed salary (in cash, in kind or both)?
Are regular expenses other than operator salaries (e.g. material, equipment, electricity, water) paid through the available income?
8.2B Irregular expenses (replacing major parts, desludging, structural damage) are covered by income
Do solutions exist for irregular expenses in the future (replacing major parts, desludging, structural damage)?
In the past, have irregular expenses been paid?
3.9.2 Result overview on Statement of Change levelFigure 49 shows the average SoC 8 scores for all visited 140 systems, globally.
For all details on Objectives, Priority Indicator and Parameter results, refer
to Appendix A.
Red = ‘Bad’ (score < 1.5), Yellow = ‘Caution’ (1.5 ≤ score < 2.5), Green = ‘Good’ (score ≥ 2.5),Grey = ‘Dataset incomplete’;numbers indicate number of systems per score range
Figure 50 shows the per country distribution of average SoC scores.
0 10 20 30 40 50
AfghanistanIndiaNepal
LesothoTanzaniaZambia
CambodiaIndonesia
LaosPhilippines
Vietnam
Number ofsystems
Good Caution Bad Datasetincomplete
3.9.3 Parameters with strong impactThe following Parameters had the strongest negative or positive impact.
i In 9 out of 140 cases (6%) there is no income source which would contribute
to available O&M budget (see Figure 51 for the per country and Figure 52 for
the per system-type split-up).
0 5 10 15 20 25 30 35 40
AfghanistanIndiaNepal
LesothoTanzaniaZambia
CambodiaIndonesia
LaosPhilippines
Vietnam
Number oftimesanincome sourcewasreportedUserfees
Regulargovernment/utilitysupport
IncomethroughSMEactivities
Incomethroughotheracitivies(e.g. sellingwater etc)
Other(pleasespecify)
None
Interviewee doesnotknow
9
30
80
21
Figure 49: Pie charts with distribution of average scores for SoC 8: Sustainable financing
Figure 50: Per country distribution of average scores for SoC 8: Sustainable financing
Figure 51: Income sources contributing to available O&M budget depending on country, there can be multiple sources per system
Table 14: Evaluation structure for SoC 8 – Sustainable financing
E VA L U A T I O N R E S U L T S
70 71
0 5 10 15 20 25 30 35 40
RES- ApartmentsRES- Multiplehouses
RES- SinglehouseRES- CSC
RES- CSC+MultiplehousesSME- FoodindustrySME- Hotel/Hostel
SME- OfficesSME- Industry(other)
INS- HospitalINS- Religiouscentre
INS- SchoolINS- University
INS- Otherinstitution
Number oftimesanincome sourcewasreportedUserfeesRegulargovernment/utilitysupportIncomethroughSMEactivitiesIncomethroughotheracitivies(e.g. sellingwater etc)Other(pleasespecify)NoneInterviewee doesnotknow
i In many cases O&M budget was not defined (59 out of 133 cases, 44%).
This is especially the case for systems visited in India, Nepal, Lesotho and
Vietnam (see Figure 53).
The project type distribution of the data (see Figure 54) indicates that most
project types are affected. Mostly, however, multiple and single households,
food industry, industry and institutions.
0 10 20 30 40 50
AfghanistanIndiaNepal
LesothoTanzaniaZambia
CambodiaIndonesia
LaosPhilippines
Vietnam
Number ofsystems
Yes,O&Mbudgetdefinedbylocalauthority
Yes,O&Mbudgetdefinedbymanagemententity,membersofcommunityorownerNoO&Mbudgetdefined
Interviewee doesnotknow
! 'Methodological background information'Note: Fig 53: During the review of this report, the following question
was raised: ‘Normal households don’t have a defined budget for car repairs, why should they have defined one for O&M?’ To clarify: this parameter does not investigate whether or not O&M is functioning (aspects covered in SoC5). Missing O&M budget definition is scored as ‘bad’ based on the assumption that any management entity (also a household owner) who knows what to expect in terms of O&M costs also better assures the ‘sustainable financing of the system’ (see formulation of this SoC).
Secondly, communicating the importance of O&M budget definition to the management entities has often been described as essential and should therefore be done by BORDA, e.g. during commissioning.
0 5 10 15 20 25 30 35
RES- ApartmentsRES- Multiplehouses
RES- SinglehouseRES- CSC
RES- CSC+MultiplehousesSME- FoodindustrySME- Hotel/Hostel
SME- OfficesSME- Industry(other)
INS- HospitalINS- Religiouscentre
INS- SchoolINS- University
INS- Otherinstitution
Number ofsystems
Yes,O&Mbudgetdefinedbylocalauthority
Yes,O&Mbudgetdefinedbymanagemententity,membersofcommunityorownerNoO&Mbudgetdefined
Interviewee doesnotknow
Nodata
i There is no solution available for irregular expenses (replacing major parts,
desludging, structural damage) in 19 out ot 140 cases (14%). Countries where
such solutions appear to be missing completely are Lesotho and Vietnam (see
Figure 55), also, in Nepal and Tanzania most systems are affected.
Dataset shows no relation with project type.
i In 17 out of the 76 cases in which major repairs ocurred (22%), these
irregular expenses could not be paid. This has mainly been observed in India,
Nepal and Lesotho (see Figure 56).
Dataset shows no relation with project type.
i In some cases (16 out of 95 cases, 17%) operator salaries are not paid (see
Figure 57 for per country split-up).
Dataset shows no relation with project type.
i Other regular expenses (e.g. material, equipment, electricity, water) are not
covered in 13 out of 140 cases (9%, see Figure 58 for per country split-up). This
last point concerns systems in almost all countries – especially taking into
account that in 20 additional cases, interviewees did not know the situation
at their system. Dataset shows no relation with project type.
0 10 20 30 40 50
Afghanistan
India
Nepal
Lesotho
Tanzania
Zambia
Cambodia
Indonesia
Laos
Philippines
Vietnam
Number ofsystems
Solutionexists
Nosolutionexists
Interviewee doesnotknow
Figure 52: Income sources contributing to available O&M budget depending on project type
Figure 53: O&M budget definition depending on country
Figure 54: O&M budget definition depending on system type
Figure 55: Given response options for the question: ‘Do solutions exist for irregular expenses in the future (replacing major parts, desludging, structural damage)?’ – per country distribution, colors indicate scoring
E VA L U A T I O N R E S U L T S
72 73
0 5 10 15 20 25
Afghanistan
India
Nepal
Lesotho
Tanzania
Zambia
Cambodia
Indonesia
Laos
Philippines
Vietnam
Number ofsystems
Yes,all
Yes,all- butafterlongwait
Yes, someirregularexpenseswerepaidNoirregularexpenseswerepaid
Interviewee doesnotknow
0 10 20 30 40 50
Afghanistan
India
Nepal
Lesotho
Tanzania
Zambia
Cambodia
Indonesia
Laos
Philippines
Vietnam
Number of systems
Regular operator payment
Not always or not enough
No operator payment
There is no operator
HH system - operator is not paid
No data
0 10 20 30 40 50
AfghanistanIndia
NepalLesotho
TanzaniaZambia
CambodiaIndonesia
LaosPhilippines
Vietnam
Number of systems
All regular expenses (other than op. salary) are covered
Some regular expenses (other than op. salary) are coveredNo income to pay regular expenses (other than op. salary)Interviewee does not know
i In those cases (29 out of 140, 21%) in which user fees are part of the income
sources, the associated processes work well:
] User fees discussed and agreed on by users (27 out of 29, 93%)
] Responsibility for fee collection is defined (28 out of 29, 97%), even
though mostly (25 out of 28, 89%) not in written
] Userfees are regularly collected (21 out of 29, 72%)
] Almost all community members are said to pay the fee (20 out of 29
cases, 69%)
3.9.4 Recommendations regarding strong impact ParametersPossible follow-up actions:
i Investigate reasons why some projects have no source of income to cover
operational costs.
i highlighting the importance of an O&M budget definition by system users
as part of the hand-over procedure or training
i investigate reasons why in some cases, operators are not paid.
i investigate reasons why in some cases costs for regular expenses cannot
be covered
i investigate reasons why in some cases solutions for irregular expenses
do not exist
Figure 56: Given response options for the question: ‘In the past, have irregular expenses (replacing major parts, desludging, structural damage) been paid? ‘ – per country distribution, colors indicate scoring, figure shows data from 76 systems at which irregular expenses were needed
Figure 57: Operator payment per country, colors indicate scoring
Figure 58: Regular expense coverage per country, colors indicate scoring
Picture: Administrative log book for orders; FSTP Context Kigamboni (Dar e Salaam, Tanzania)
E VA L U A T I O N R E S U L T S
74 75
3.10 SoC 9: Quality planning, design and construction
3.10.1 Statement of Change structureThis SoC evaluates the project output ‘quality planning, design and construction’
by investigating the existence of project specific and accurate design
documentation as well as acceptability of overall investment costs.
Statement of Change
Objectives Priority Indicators Parameters
9. Quality planning/ design and construction
9.1 Project design appropriate to context
9.1A Project documentation is availa-ble and specific to project
Are all important project documents available (design drawings, design spread sheet, feasibility study)?
9.1B Project documentation includes appropriate information on O&M costs
Does documentation include cost es-timations of required O&M activities?
9.2 Systems built to design
9.2A Constructed systems matches design
Does type and the number of reactor chambers (for settler, ABR and AF) observed in the field, match the design?
3.10.2 Result overview on Statement of Change levelFigure 59 shows the average SoC 9 scores for all visited 140 systems, globally.
For all details on Objectives, Priority Indicator and Parameter results, refer
to Appendix A.
Red = ‘Bad’ (score < 1.5), Yellow = ‘Caution’ (1.5 ≤ score < 2.5), Green = ‘Good’ (score ≥ 2.5),Grey = ‘Dataset incomplete’;numbers indicate number of systems per score range
0 10 20 30 40 50
AfghanistanIndiaNepal
LesothoTanzaniaZambia
CambodiaIndonesia
LaosPhilippines
Vietnam
Number ofsystems
Good Caution Bad Datasetincomplete
3.10.3 Parameters with strong impactThe following Parameters had the strongest negative or positive impact.
i Estimation of O&M costs is not included in project documentation in 87
out of 140 cases (62%). See Figure 61 for the per country distribution. Countries
in which O&M cost estimation is never contained in project documentation
are Lesotho, Zambia, Indonesia, Laos, Philippines and Vietnam. In India, for
most projects such information is not available through available project
documentation.
0 10 20 30 40 50
AfghanistanIndiaNepal
LesothoTanzaniaZambia
CambodiaIndonesia
LaosPhilippines
Vietnam
Yes No Noaccesstodesigndocumentation
i For 70 out of 140 visited systems (50%) 1 or no project documents (out of
design drawings, design spread sheet, feasibility study) are available. This
issue is observed for older (31 out of 35 visited systems, 86%, >5 years age)
but also younger (35 out of 100 systems, 35%, <5 years age) systems.
Most such cases are observed in India, Nepal, Lesotho and Zambia (see
Figure 62).
! Background information: Lesotho – Quotations In the case of Lesotho, project documentation only consists of a
“quotation to the client”.
Since all household system designs in Lesotho are standardised and identical, otherwise important project documents such as design drawings, design spread sheet and feasibility study are deemed not necessary by the local management. The M&E methodology will be adapted to better suit these circumstances.
Figure 60: Per country distribution of average scores for SoC 9: Quality planning, design and construction
Figure 61: Availability of O&M costs in design documentation
0
8335
22
Table 15: Evaluation structure for SoC 9 – Quality planning, design and construction
Figure 59: Pie charts with distribution of average scores for SoC 9: Quality planning/ design and construction
E VA L U A T I O N R E S U L T S
76 77
0 10 20 30 40 50
AfghanistanIndiaNepal
LesothoTanzaniaZambia
CambodiaIndonesia
LaosPhilippines
Vietnam
Number ofsystems
Threeoutofthree Twooutofthree Oneoutofthree Noneofthethree
i In most cases, the constructed treatment modules and reactor chamber
numbers match the design (116 out of 118; 98%).
Not part of this SoC evaluation – but obviously related to it – is the question
of implementation costs:
In any case, knowing the overall investment costs should be considered
important to evaluate a project’s accurate ‘planning' (ie. SoC9). However, a
detailed investigation would take into account various factors such as country,
system type, reactor setup, system size, age and inflation.
As a suggestion by the authors, Figure 63 presents the per user (design user
number) construction costs (defined as: ‘the total costs, direct and indirect,
associated with transforming the design plan into a DEWATS plant ready for
operation, Start: site preparation – End: Commissioning /Handover”) of the
38 visited residential systems for which data was available.
0
50
100
150
200
250
300
350
400
Cons
truction
costs(
EUR/
use
r)
Systems
Afghanistan
India
Indonesia
Lesotho
Nepal
Philippines
Tanzania
! Suggested discussion at GD level D4: Does BORDA see an added value in reporting ‘DEWATS
investment costs per user’ (see Figure 63) and in defining (and monitoring) benchmarks for it??
On one hand, BORDA has a non-profit driven approach to projects, which in the past and for various reasons may not have focussed on economic aspects. On the other hand, there is understandably an increasing pressure from donors to optimize implementation costs of sanitation infrastructure in order to reach a maximum number of beneficiaries with available funds. Depending on the decision regarding benchmarks, investment costs can either be displayed simply as existing ranges (No good-bad weighting; see Figure 63), or, in case benchmarks are considered useful or relevant, they need to be defined and can become part of the SoC9 evaluation in future reports.
3.10.4 Recommendations regarding strong impact ParametersPossible follow-up actions:
i highlight the importance of an O&M costs estimation as part of project
design and documentation.
i Investigate whether project implementation costs are contained in project
documentation as part of this SoC.
i Improve the availabilityof key project documents (design drawings, design
spread sheet, feasibility study) which are often missing (35% of investigated
cases younger than 5.5 years).
Figure 63: Per user (user number as assumed during design) construction costs of 38 visited residential systems
Figure 62: Availability of important project documents (design drawings, design spread sheet, feasibility study) at visited projects
Picture: Insertion of baffled pipes during construction work (Kabul, Afghanistan)
E VA L U A T I O N R E S U L T S
78 79
Published by BORDA e.V.
Am Deich 45
28199 Bremen
www.borda.org
Photos All rights reserved
by BORDA
Main authors Dr. Nicolas Reynaud
Elli Rodriguez
Dr. Alexander Wolf
Contact [email protected]
Design gegenfeuer.net
Release August 2018
with support from:
1
4a Appendix A: M&E Matrix structure and Parameter responses
In this Appendix, results are presented of each SoC with its Objectives, Priority Indicators and Parameters is shown (data from 2017 M&E investigations in 11 countries at 140 systems). Pie charts indicate average values as described in Chapter 2.2.
Note: SoC, Objective and PI scores depend on Parameter scores and killer criteria (see Table 4).
Color coding: Red = ‘Bad’, Yellow = ‘Caution’, Green = ‘Good’, Grey = ‘Dataset incomplete’ or ‘Data not available’, White = ‘Not relevant’
Statement of Change
Objective Priority indicator Parameter
1. The sanitationservice maintainsor improvesenvironmentalhealth
1.1 Effluent meets discharge standards
1.1A The most recent effluent analysis complies with BORDA Standards
If data available and reliable, does the average effluent COD meet relevant BORDA discharge standards (200mg/L or 80 mg/L depending whether anaerobic or aerobic effluent)? (Caution, yellow, means <20% above standard)
No <20% above
standard Yes No data Data not
reliable*
25 10 43 50 12
*Data not reliable if measured in external labs, except in Cambodia andTanzania
1.1B The most recent effluent analysis complies with Local Standards
If data available and reliable, does the average effluent COD meet relevant local discharge standards*? (Caution, yellow, means <20% above standard)
No No local standards
<20% above
standard Yes No data
Data not
reliable
32 20 7 19 50 12
Where is the sludge disposed to after desludging?
20
36
25
59
23
29
32
56
032
10
42
56
0
38
2719
56
0
2
Statement of Change
Objective Priority indicator Parameter
1.2 Removed waste is safely disposed or reused
1.2 Evidence that waste or by-products (sludge, scum, trash, biogas) is safely disposed or reused
Truc
ked
to S
TP -
no
man
ual h
andl
ing
Truc
ked
to s
olid
w
aste
site
- n
o m
anua
l han
dlin
g
Man
ually
col
lect
ed
with
sol
id w
aste
Reu
sed
Saf
ely
buri
ed
On
land
, no
t sa
fely
re
used
or
buri
ed
Wat
erbo
dy o
r ri
ver
Oth
er
Inte
rvie
wee
doe
s no
t kn
ow
Inte
rvie
wee
not
av
aila
ble
Slu
dge
neve
r re
mov
ed
9 0 2 7 6 2 1 0 8 0 106
Where is the scum disposed to after descumming?
Scu
m
neve
r re
mov
ed
Sea
led
into
do
mes
tic
solid
w
aste
co
llect
ion
Uns
eale
din
to
dom
estic
so
lid
was
te
colle
ctio
nSaf
ely
stor
ed
Saf
ely
buri
ed
On
land
, no
t bu
ried
Wat
erbo
dy
or
rive
r
Oth
er
Inte
rvie
wee
do
es n
ot k
now
Inte
rvie
wee
no
t av
aila
ble
63 9 2 8 40 8 6 6 3 0
Where is the solid waste disposed to after removal from reactors or piping?
Sol
id
was
te
neve
r re
mov
ed
Sea
led
into
dom
estic
so
lid w
aste
col
lect
ion
Uns
eale
d in
to
dom
estic
sol
id w
aste
co
llect
ion
Saf
ely
stor
ed
Saf
ely
buri
ed
On
land
, no
t bu
ried
Wat
erbo
dy o
r ri
ver
Oth
er
Inte
rvie
wee
doe
s no
t kn
ow
Dat
a no
t av
aila
ble
46 18 4 13 41 14 3 5 7 0
For BGD, is unused biogas burned off?
25
62
50
3 0
25
62
50
3 0
3
Statement of Change
Objective Priority indicator Parameter
No
BG
D -
not
re
leva
nt
Bio
gas
is
alw
ays
used
Yes,
un
used
bi
ogas
is
al
way
s bu
rned
of
f
Unu
sed
biog
as
is
som
etim
es
burn
ed o
ff
No,
un
used
bi
ogas
is n
ever
bu
rned
off
Inte
rvie
wee
do
esn'
t kn
ow
Inte
rvie
wee
no
t av
aila
ble
105 21 0 1 12 1 0
Statement of Change Objective Priority indicators Parameters
2. The sanitationservice improves theliving conditions ofcommunities
2.1 Underserved people are connected to the sanitation service
2.1A Majority of sanitation users previously had no or basic access to sanitation
Before this system existed, what was the sanitation option used (or if SME, the wastewater discharge method) by the majority of users connected to the system?
Open defecation
Pit/Unimproved latrine
Toilet directly to waterbody
Flush-toilet connected to soak pit
Other unimproved sanitation option
10 16 6 23 13
Other improved sanitation option
Flush-toilet connected to functional and sealed septic tank
Not relevant, wastewater-stream did not exist before
Info not available
Project documentation not available
12 0 39 17 4
2.1B Majority of sanitation users are classified as disadvantaged (ie. Income)
What proportion of users are classified/registered as low income according to Project planning/FS?
Not relevant, system does not have users, e.g. someSME
Majority (> 50%)
Some (25 - 50%)
Few (<25%)
Info not available
Project documentation not available
58 31 6 15 26 4
12
31
57
40
11
37
44
18
30 160
6718
39
20
31
283
58
4
Statement of Change Objective Priority indicators Parameters
2.1C Unrestricted CSC access for disabled and elderly
Is access to the sanitation facility difficult or impossible for disabled/ elderly?
Yes, difficult. Access by stairs (not ramp), rough or uneven path
Yes, difficult. Distance too far for easy access
Yes, difficult. Other observed/reported access issues
Good access. No issues observed
No sanitation facility
11 0 0 13 116
2.2 Potential exposure to faecal pathogens for surrounding communities is managed
2.2 Exposure to faecal pathogens for surrounding communities is managed
Problematic wastewaterdischarge of systems with post treatment
Stream or river used by community close downstream (100m)
Stream or river which is dry during some parts of the year
Pond/lake used by community
Percolation pit within 10m of a water well
4 0 0 0
Problematic wastewaterdischarge of systems without post treatment
Reu
se f
or f
ish
farm
ing
Reu
se f
or
agri
cultu
re a
nd/o
r ho
rtic
ultu
re f
or f
ood
prod
uctio
n
Reu
se f
or n
on-f
ood
hort
icul
ture
(e.
g.
law
n, la
ndsc
apin
g)
Str
eam
or
rive
r us
ed
by c
omm
unity
clo
se
dow
nstr
eam
(10
0m)
Str
eam
or
rive
r w
hich
is d
ry d
urin
g so
me
part
s of
the
ye
ar
Pond
/lak
e us
ed b
y co
mm
unity
Perc
olat
ion
pit
with
in 1
0m o
f a
wat
er w
ell
1 18 0 2 6 0 0
è Therefore: Is the exposure to faecal pathogens for surrounding communitiesmanaged?
Yes Somewhat No Information not available
88 18 7 27
110
13 0
116
1418
82
260
1418
82
260
5
Statement of Change Objective Priority indicators Parameters
3. The service achievesproject specific impact
3.1 Sanitation service achieves intended objectives
3.1A Compared with the project documents, the original objective(s) of the sanitation service was (were) achieved
Pro
du
ce a
nd
u
se b
iog
as
Re-
use
of
was
tew
ater
Pre
-tre
at
was
tew
ater
b
efo
re
dis
char
ge
to
soli
d-f
ree
sew
er
Red
uct
ion
of
O&
M c
ost
s
Systems with this objective
35 51 19 40
Objective met 4 45 4 35
Objective somewhat met
23 0 5 0
Objective not met 7 6 4 0
Info not available 1 0 6 5
Not objective 97 83 113 94
No info on objectives
4 4 4 4
Statement of Change Objective Priority indicator Parameter
4. FunctioningTechnology - systemsare operating asintended
4.1 System operating as designed - acceptable loading and system hydraulics
4.1A Users within acceptable range of design
Is the utilisation (design/actual connected user) equal or above acceptable range?
Below (<66%)
Acceptable (>=66%, <133%)
Above (>=133%)
No data
35 60 8 37
11
21
4315
50
11
21
4315
50
11
21
4315
50
6
Statement of Change Objective Priority indicator Parameter
Is the intended wastewater type discharged to treatment?
Yes
No,
gr
ey-w
ater
is
di
scha
rged
to
syst
em b
ut
it w
as n
ot d
esig
ned
for
it
No,
gr
ey-w
ater
is
no
t di
scha
rged
to
syst
em b
ut
it w
as d
esig
ned
for
it
No,
bl
ack-
wat
er
is
disc
harg
ed t
o sy
stem
but
it
was
not
des
igne
d fo
r it
No,
bl
ack-
wat
er
is
not
disc
harg
ed t
o sy
stem
but
it
was
des
igne
d fo
r it
No,
no
n-co
mm
unal
w
aste
wat
er is
dis
char
ged
to s
yste
m b
ut i
t w
as n
ot
desi
gned
for
it
No,
no
n-co
mm
unal
w
aste
wat
er
is
not
disc
harg
ed t
o sy
stem
but
it
was
des
igne
d fo
r it
Info
rmat
ion
not
avai
labl
e
127 2 1 1 2 1 0 6
4.1B Influent flow to the treatment system is observed
Do you observe flow or signs of recent flow at plant inlet (wet piping, wet inlet chamber)?
Yes No No, plant not conctd to piping
No access to plant inlet
TM not sure
126 6 0 8 0
4.1C The system does not experience severe flow surges
Are there signs of strong water level fluctuations inside anaerobic reactors (water, scum or small plastics on reactor walls above water level)? For details see GMF2 manual.
No Yes No access to inside of reactor
Settler, ABR and AF are not part of the design
TM not sure
98 13 19 3 7
Does the treatment system show changes during strong rain (overflow, bypass, smells)?
No strong rain in area
No, never Yes, changes observed
Yes, changes
Interviewee does not know
Interviewee not available
7 11
3884
711
75
47
07
37
59
37
0
90
123
8
24
26
86
40
7
Statement of Change Objective Priority indicator Parameter
sometimes
observed everytime
19 59 29 7 26 0
Does the treatment system experience backflow / inundation from the waterbody it discharges to?
No, never Yes, but only during major events (e.g. severe storm)
Yes, regularly
No discharge to waterbody
Interviewee does not know
Interviewee not available
118 4 1 2 15 0
Based on the above three parameters, does the system experience any flow surge issues?
*Only scored as ‘Information not available” if none of the above three parametersis answered
No Minor issues At least one type of issue
Information not available*
88 28 18 6
4.1D Evidence of effluent flow
Do you observe flow or signs of recent flow (wet piping, etc.) at plant outlet?
Yes, signs of recent flow
No signs of recent flow
No, because piping system is not connected to the plant
No access to plant outlet
TM not sure
117 5 2 14 2
70
117
160
8
Statement of Change Objective Priority indicator Parameter
4.2 Systems operating as designed - treatment meets BORDA requirements
4.2A Anaerobic, and where applicable aerobic, effluent quality meet BORDA requirements
No rain Short rain or drizzle
Rain Strong rain No field sampling
91 0 25 2 22
Does the average COD concentration of the anaerobic effluent comply to BORDA design value (200 mg/ l)?
*Data not reliable if measured in external labs, except in Cambodia and Tanzania**Considered as affected by rain only if ‘strong rain” was observed in the field
Yes Almost <20% above standard
No No data There is no anaerobic treatment step
Data-source not reliable* and/or data probably affected by rain**
53 7 16 40 10 14
Does the average COD concentration of the aerobic effluent comply to BORDA design value (80 mg/ l)?
*Data not reliable if measured in external labs, except in Cambodia and Tanzania**Considered as affected by rain only if ‘strong rain” was observed in the field
Yes Almost <20% above standard
No No data There is no aerobic treatment step
Data-source not reliable* and/or data probably affected by rain**
19 4 16 48 44 9
4.2B There are obvious signs of biogas production from the biogas digester
When you open the gas valve furthest away from the BGD (e.g. at the stove), do you hear or smell gas release?
21
17
44
58
0
24
13
45
58
0
9
Statement of Change Objective Priority indicator Parameter
Yes
No
acce
ss t
o ga
s va
lve
Bio
gas
pipi
ng
has
not
been
in
stal
led
No,
bio
gas
does
no
t re
ach
appl
ianc
e an
d le
akag
es o
r bl
ocka
ges
in
pipi
ng a
re
obse
rved
No,
bio
gas
does
no
t re
ach
appl
ianc
e an
d N
O le
akag
es o
r bl
ocka
ges
in
pipi
ng a
re
obse
rved
Not
rel
evan
t
11 6 1 0 17 105
222
110
105
10
Statement of Change Objective Priority indicator Parameter
5. FunctioningMaintenance - systems are maintained as intended
5.1 Systems maintained - no major damage
5.1A No signs of structural damage compromising functionality or warranty aspects
What building structure problems do you observe at treatment system (digester, settler, ABR, AF, PGF)?
Minor physical damage (e.g. small cracks); does not affect operation or safety
Major physical damage (e.g. large cracks, leakages, broken divider walls, floating bioballs in prefab-AF); affects operation or safety
Damaged manhole cover (cracked, partly missing); covering most of the manhole
Severely damaged or missing manhole cover
System is clearly not connected to piping
No noticeable damage
26 6 7 3 7 100
Do you observe problems with existing pumps (wastewater or water pumps)?
There are no pumps
No, all pumps work
Yes, at least one pump is broken but it does not affect plant treatment
Yes, at least one pump is broken and it affects plant treatment
No access to pumps
TM not sure
95 39 3 1 1 1
What structural problems with the sanitation facility(ies) (walls, roof and floor) do you observe?
11
5165
13 11
25
103
1 0
11
25
103
1 0
11
Statement of Change Objective Priority indicator Parameter
Minor physical damage (e.g. small cracks), not affecting operation, use or safety.
Major physical damage (e.g. large cracks), affecting operation, use or safety.
No noticeable damage
No CSC
10 0 14 116
5.2 Maintenance activities occurring as intended
5.2A Maintenance is adequate
See tables below
The following tables present parameter responses relevant to PI 5.2A
Do you observe flow or signs of recent flow at plant inlet (wet piping, wet inlet chamber)?
Yes No signs of recent flow
No, because piping system is not connected to the plant
No access to plant inlet
TM not sure
126 6 0 8 0
In the past, were maintenance issues with the sewer network fixed (clogging, blockage, bad smell or overflow)?
7
5962
120
7
5962
120
12
No sewer network - notrelevant
No maintenance issues so far
Yes, all issues fixed
Some isses fixed
No issues fixed
Interviewee does not know
Interviewee not available
67 31 17 4 10 11 0
In the past, were major issues with the sewer network fixed (broken pipes or manhole, leakages, other major damage)?
No sewer network - notrelevant
No maintenance issues so far
Yes, all issues fixed
Some isses fixed
No issues fixed
Interviewee does not know
Interviewee not available
67 35 13 3 11 11 0
In the past, were maintenance issues with the household grease-traps fixed (clogging, bad smells)?
No household grease traps - not relevant
No maintenance issues so far
Yes, all issues fixed
Some isses fixed
No issues fixed
Interviewee does not know
Interviewee not available
114 11 7 1 6 1 0
Can the manhole covers of the treatment system be opened? Please try at least 5 manhole covers. In case they were opened before the start of the field visit, please ask the m.e. or operator.
All can be opened within 10 min each
Majority (≥ 50%) can be opened within 10 min each
Majority (> 50%) can NOT be opened within 10 min each
Majority (> 50%) can NOT be opened at all
94 22 10 14
Could a local desludging service provider access the treatment system with his cart, truck, etc.?
Yes No TM not sure
13
132 6 2
Has the system been desludged?
Yes, within the last 3 years
Yes, but more than 3 years ago
Never, system is younger than 3 years
Never, system is older than 3 years
Interviewee does not know
Interviewee not available
28 3 60 46 3 0
How thick is the scum in the second (or if too difficult to open, third) ABR chamber?
There is no ABR
There is no scum
Less than 2 cm
2 cm or more
No access to chambers
24 44 42 11 19
Do you see much plastic waste (more than 20 items) inside the second (or if too difficult to open, third) anaerobic reactor chamber (can be settler, ABR or AF chamber)? (single selection)
No Some waste (< 20 items)
Yes, much waste (> 20 items)
No access to chambers
Settler, ABR and AF are not part of the design
92 25 6 13 4
Do you observe problems concerning the planted gravel filter?
No PGF Dead plants Slime on surface
Stagnating water on the surface - because swivel pipe is set too high
Stagnating water on the surface - probable reason: filter blockage
Swivel pipe broken, water level very low
No plants Large amounts of solid waste
Other problems (e.g. smell, dead leaves, few plants)
No problems
64 11 3 6 11 1 19 5 15 31
14
What functional problems do you observe about the sanitary installations?
Som
e to
ilets
/ ba
thro
oms
seem
unu
sed
Laun
dry
area
see
ms
unus
ed
Faci
lity
does
no
t ha
ve
func
tioni
ng li
ghts
Elec
tric
ity is
not
ava
ilabl
e
Wat
er is
not
ava
ilabl
e
Blo
cked
flo
or-d
rain
s
No
was
te b
ins
with
cov
ers
are
prov
ided
insi
de c
ubic
les
Som
e to
ilets
/ ba
thro
oms
are
bloc
ked/
brok
en
Som
e to
ilets
/ ba
thro
om
door
s ar
e m
issi
ng/b
roke
n/ca
n't
be
lock
ed
Som
e to
ilets
/ ba
thro
oms
are
uncl
ean
(rub
bish
, di
rty,
not
cl
eane
d in
pas
t 2
d)
No
O&
M o
r H
HE
post
ers
are
hung
up
Bro
ken
hand
was
hing
-bas
in
or t
ap
No
hand
was
hing
fac
ility
No
prob
lem
s
No
CSC
/Not
rel
evan
t
2 7 2 9 1 2 10 2 4 2 11 3 7 8 116
Do you observe problems with the water trap?
There is no water trap
No access to water trap
No problem, the current water trap setting allows free biogas flow
Yes. Water trap has not been emptied, although required by design
Yes. Water trap is not at the lowest point of the biogas piping
Yes, other problem
Not relevant
23 8 4 0 0 0 105
When you open the gas valve furthest away from the BGD (e.g. at the stove), do you hear or smell gas release?
Yes No access to gas valve
Biogas piping has not been installed
No, biogas does not reach appliance and leakages or blockages in piping are observed
No, biogas does not reach appliance and NO leakages or blockages in piping are observed
Not relevant
11 6 1 0 17 105
15
Is the biogas appliance working (lamp/ heater/ stove)?
There is no appliance and/or biogas piping
Yes Not accessible, cannot be checked
No, appears broken, corroded or clogged
No, because gas does not reach
No, because of other problem
Not relevant
5 22 2 1 3 2 105
Do you observe problems with the pressure gauge?
No pressure gauge installed
No. it contains water and is well calibrated
Yes. it contains water but is not well calibrated
Yes. it is empty (no water)
Not relevant
25 7 0 3 105
For BGD, is unused biogas burned off?
No BGD - not relevant
Biogas is always used
Yes, unused biogas is always burned off
Unused biogas is sometimes burned off
No, unused biogas is never burned off
Interviewee doesn't know
Interviewee not available
105 21 0 1 12 1 0
Statement of Change Objective Priority indicators Parameters
6. SustainingDemand - system isavailable, used tocapacity andacceptable
6.1 Service is adequately available to users
6.1A For systems with SSS, system receives and processes wastewater
Do you observe flow or signs of recent flow at plant inlet (wet piping, wet inlet chamber)?
Yes No No, plant is clearly not connected to piping system
No access to plant inlet
TM not sure
126 6 0 8 0
16
Statement of Change Objective Priority indicators Parameters
6.1B For systems with CSC, ratio of users to functioning toilets is acceptable
Acceptable ratio of users per functioning toilets (20 users per toilet)
Defined ratio and below
Above defined ratio
No data No CSC
11 10 3 116
6.1C DEWATS processes wastewater
Do you observe flow or signs of recent flow (wet piping, etc.) at plant outlet?
Yes, signs of recent flow
No signs of recent flow
No, because piping system is not connected to the plant
No access to plant outlet
TM not sure
117 5 2 14 2
6.1D For biogas systems, biogas can be used
When you open the gas valve furtherst away from the BGD, do you hear or smell gas release?
7 9
76
48
720
89
240
70
125
80
100
11 3
116
90
116
150
Statement of Change Objective Priority indicators Parameters
Yes No access to gas valve
Biogas piping has not been installed
No, biogas does not reach appliance and leakages or blockages in piping are observed
No, biogas does not reach appliance and NO leakages or blockages in piping are observed
Not relevant
18 6 11 0 0 105
6.2 Utilisation rate: Service is used to capacity
6.2A Utilisation is close to full capacity (ie. actual/design users)14
Is the utilisation (actual/design users) equal or above acceptable range?
Below (<66%)
Acceptable (>=66%, <133%)
Above (>=133%)
No data
35 60 8 37
6.2B For biogas systems, biogas is used to full capacity
How often is the biogas of this system being used?
No BGD - notrelevant
Daily Not daily but regularly
Seldom (less than once a week)
Never Interviewee does not know
Interviewee not available
105 20 2 5 7 1 0
14 May be affected by ‚killer criteria’: scored as ‘bad’ if piping is not connected to system. 17
19 5
110
105
38
13
54
35
0
47
059
34
0
8 5
211
105
18
Statement of Change Objective Priority indicators Parameters
6.3 Acceptability: Culturally acceptable, users satisfied with system
6.3A Satisfaction indicated by high utilisation or no evident issues of low acceptance
If the utilisation is below 66% design value, what are the reasons?
High utilisation rate (> 66%) (=3)
Low utilisation rate (<= 66%) but reported reasons do not indicate low acceptance
Low utilisation rate (<= 66%) and reported reasons indicate low acceptance
Low utilisation rate (<= 66%) but no information available about reasons
No information on utilisation rate but reported feedback does not indicate low acceptance
No information on utilisation rate and reported feedback indicates low acceptance
No data
68 32 2 1 35 2 0
110
128
1 011
0
128
1 0
19
Statement of Change Objective Priority indicators Parameters
7. Effectivemanagement:Existing, active andaccountablemanagement entityand operator
7.1 Active and accountable management entity
7.1A Existence of trained management entity with clarified responsibilities
Does a management entity (m.e.) exist?
Yes, members of the served community
Yes, members of the institution (e.g. school) or SME
Yes, external agency
Yes, other (e. g. owner)
Yes, but not available for personal / phone interview
No management entity exists
25 67 1 0 41 2
Did the management entity receive the required trainings (Financial, O&M, HHE)?
Yes, all required trainings were received
One or two out of three trainings were received
None -m. e.haschangedsincestart of operation
None - m. e.has notchangedsincestart ofoperation
M.e.received othertype oftraining
There is no management entity
Management entity not available for interview
19 95 9 4 5 2 2
Is the management entity aware of its responsibilities (financial administration; coordinating & arranging financing of major repairs - pumps, broken piping, structural damage to treatment system; employing and managing operator; training operator; deciding about additional connections)?
Yes, aware of all responsibilities above
Aware of some responsibilities (including managing operator)
Aware of some responsibilities (but not managing operator)
Not aware of any responsibility
Aware of other responsibilities
There is no management entity
Management entity not available for interview
1 58 59 7 7 2 2
15
5548
22
6
59
52
230
2
45
87
60
20
Statement of Change Objective Priority indicators Parameters
Does management entity have documentation of legal registration to carry out its responsibilities (e.g. officially registered with appropriate government department)?
Not applicable as no legal registration necessary
Yes and legal documentation is shown (photograph must be taken)
Yes, but legal documentation is not shown
No Interviewee does not know
There is no management entity
Management entity not available for interview
59 4 57 9 3 2 2
7.1B Management entity is active and accountable
Does the management entity meet regularly?
No, but m. e. exists ofonly 1 person (owner household system)
Yes and meting documention is shown (photograph must be taken)
Yes, but meeting documentation is not shown
No, m. e. does not meet regularly
There is no management entity
Management entity not available for interview
27 5 75 25 2 2
Are O&M responsibilities of the operator defined and documented?
Yes, as part of a contract
Yes, in a printed document
No documents, but interviewee can provide detail on operator role
No documents and interviewee cannot provide details on operator role
There is no operator
Operator not available for interview
22 9 66 12 14 3
Is the operator regularly paid an agreed salary (in cash, in kind or both)?
20
62
35
230
21
Statement of Change Objective Priority indicators Parameters
Yes Not always or not enough
No payment
There is no operator
Operator not available for interview
No data
HH system - operator is not paid
77 2 16 13 3 11 18
Are income and/or expenses documented in financial administration logbook or elsewhere?
Yes, documentation is shown
Yes, but documentation NOT shown
No documentation exists
There is no management entity
Management entity exists but nobody is available for interview
6 78 54 2 0
Is there documentation of conducted O&M activities?
Yes, logbook or documentation shown (photograph must be taken)
Yes, but documentation NOT shown
No documentation exists
There is no management entity
Management entity exists but nobody is available for interview
8 31 99 2 0
7.2 Trained and equipped operator
7.2A Existence of trained, equipped operator who knows his responsibilities
Is there a person/caretaker/operator assigned and responsible for O&M activities? (in the following called ‘operator’)
Yes, operator is community member, SME employee or owner
Yes, operator is external service provider
No, but one or more people WITHOUT O&M responsibility take care of facility
No, nobody is assigned and responsible for O&M activities
107 5 14 14
Has the operator received O&M training?
16
67
54
3 0
16
67
54
3 0
22
Statement of Change Objective Priority indicators Parameters
Yes Only limited/ brief training
No, operator has changed since start of operation
No, operator same since start of operation
There is no operator
Operator not available for interview
No data
74 30 4 1 28 3 0
Is the operator aware of his responsibilities?
Yes, aware of all responsibilities
Aware of most responsibilities
Not aware of most responsibilities
Not aware of any responsibility
There is no operator
Operator not available for interview
No data
13 48 44 4 28 3 0
Does the operator have all necessary tools required to perform O&M activities (opening manholes, deblocking sewer system, scum and solid waste removal from reactors)?
Yes, all tools
Some tools
No tools
Interviewee does not know
No, nobody is assigned and responsible for O&M activities
Interviewee not available
48 63 11 4 14 0
23
Statement of Change Objective Priority indicators Parameters
8. SustainableFinancing: Sufficientongoing income tocover all short andlong term costs
8.1 Regular income 8.1A O&M budget and/or user fees have been agreed on and are collected
Was a fixed O&M budget defined?
Yes, O&M budget defined by local authority
Yes, O&M budget defined by m.e.
No O&M budget defined
Interviewee does not know
Interviewee not available
26 49 49 9 0
What are the income sources contributing to available O&M budget?
Use
r fe
es
Reg
ular
(lo
cal)
go
vern
men
t/ u
tility
su
ppor
t
Inco
me
thro
ugh
SM
E ac
tiviti
es
Inco
me
thro
ugh
othe
r ac
itivi
es (
e.g.
se
lling
wat
er e
tc)
Oth
er (
plea
se
spec
ify)
Non
e
Inte
rvie
wee
doe
s no
t kn
ow
Inte
rvie
wee
not
av
aila
ble
31 44 25 12 34 8 1 0
Were the user fees set by local authority or agreed on by users themselves?
Yes, user fees set by local authority
Yes, user fees discussed and agreed on by users
User fees discussed but not agreed on by users
Interviewee does not know
No set user fees
Interviewee not available
11 18 2 0 102 0
Is someone responsible for the user fee collection process and is this documented?
9
30
80
21
9
52
72
70
9
52
72
70
24
Statement of Change Objective Priority indicators Parameters
Yes, responsibility is defined and documentation is shown (photograph must be taken)
Responsibility is defined but documention is NOT shown
Responsibility is not defined
Interviewee does not know
No set user fees
Interviewee not available
3 27 1 0 102 0
Are user fees collected?
Yes, regularly
Yes, but irregularly/ on demand
No Interviewee does not know
No set user fees
Interviewee not available
23 3 5 0 102 7
What proportion of users pay fees?
Almost all (> 66%)
Some/half (33 - 66%)
Few/None (< 33%)
Interviewee does not know
No set user fees
Interviewee not available
20 3 0 8 102 7
8.2 Sufficient income to cover all short and long term costs
8.2A Regular operation and maintenance expenses (operator salary, material, equipment, electricity and water costs) are covered by income
Is the operator regularly paid an agreed salary (in cash, in kind or both)?
Yes Not always or not enough
No payment
There is no operator
Operator not available for interview
No data
HH system - operator is not paid
77 2 16 13 3 11 18
Are regular expenses other than operator salaries (e.g. material, equipment, electricity, water) paid through the available income?
Yes, all regular expenses are paid
Yes, some regular expenses are paid
No income to pay regular expenses
Interviewee does not know
Interviewee not available
84 23 13 20 0
17
29
80
140
25
15
86
140
25
Statement of Change Objective Priority indicators Parameters
8.2B Irregular expenses (replacing major parts, desludging, structural damage) are covered by income
Do solutions exist for irregular expenses in the future (replacing major parts, desludging, structural damage)?
Yes (e.g. via regular income, extra community contribution, local government support)
No solution exists
Interviewee does not know
Interviewee not available
102 19 19 0
In the past, have irregular expenses been paid?
Yes,
all
Yes,
all
- bu
t af
ter
long
wai
t
Yes,
som
e ir
regu
lar
expe
nses
wer
e pa
id
No
irre
gula
r ex
pens
es w
ere
paid
No
maj
or r
epai
rs
Inte
rvie
wee
do
es
not
know
Inte
rvie
wee
no
t av
aila
ble
37 3 10 17 64 9 0
30
1991
00
26
Statement of Change Objective Priority indicators Parameters
9. Quality planning/design andconstruction
9.1 Project design appropriate to context
9.1A Project documentation is available and specific to project
Are all important project documents available (design drawings, design spread sheet, feasibility study)?
Yes, all three
Two out of three
One out of three
None of the three is available
51 19 35 35
9.1B Project documentation includes appropriate information on O&M costs
Does documentation include cost estimations of required O&M activities?
Yes No No access to any design documentation
53 83 4
9.2 Systems built to design
9.2A Constructed systems matches design
Does type and the number of reactor chambers (for settler, ABR and AF) observed in the field, match the design?
Yes No No access to count reactor chambers
No access to documentation or documentation insufficiently detailed
116 2 2 20
0
8335
22 49
55
36
00
70
19
51
00
83
4
53
00
20
116
220
20
116
220
27
4b Appendix B: Per project average SoC scores
This Appendix presents the per system average SoC scores sorted by countries (scale 1 to 3, 1 representing bad performance, 2 caution, 3 good performance). Grey cells indicate missing scores because of incomplete datasets or, in the case of some SoC 3 scores, non-relevance of the SoC for these projects.
Country Name 1.Environmental
health
2.Improving
livingcond.
3.Projectspec.
impact
4.Functioning
technology
5.Functioning
maintenance
6.Sustaining
demand
7.Effective
management
8.Sustainable
financing
9.Planning,
design,constr.
Killer
Criterion(ia)
Afghanistan ACTcampus 2.3 3.0 3.0 2.7 2.6 2.5 2.7 3.0 3.0Afghanistan BanaiMUDA 3.0 2.4 2.4 3.0 3.0Afghanistan AbdulRahmanMosque 2.8 2.0 3.0 3.0 3.0 3.0 2.7 2.9 3.0Afghanistan AUWSSC 2.6 2.5 3.0 2.8 3.0 3.0 2.6 3.0 3.0Afghanistan Qasabahousingcomplex(I) 2.0 1.0 2.8 1.0 2.4 3.0 YesAfghanistan Qasabahousingcomplex(II) 2.3 1.0 2.8 1.0 2.1 3.0 YesAfghanistan Qasabahousingcomplex(III) 2.0 1.0 2.8 1.0 2.1 3.0 YesAfghanistan Qasabahousingcomplex(IV) 1.7 1.0 2.8 1.0 2.1 3.0 YesAfghanistan Qasabahousingcomplex(V) 1.7 1.0 2.8 1.0 2.1 3.0 YesAfghanistan BORDAoffice 2.0 3.0 3.0 2.6 2.6 3.0 2.8 3.0 3.0Afghanistan Naeemhouse 2.4 2.0 3.0 2.8 2.9 3.0 2.3 2.5 2.3Afghanistan NouriHouse 2.6 2.5 2.7 2.8 2.5 2.5 3.0Afghanistan ShamsaChildrenvillage 2.5 2.5 3.0 3.0 3.0 3.0 2.5 3.0 3.0Afghanistan NadirHouse 2.5 3.0 3.0 3.0 2.5 3.0 2.3 2.5 2.0Afghanistan FaisalHouse 2.5 3.0 3.0 3.0 2.5 3.0 2.2 2.5 2.3Afghanistan MukhlesHouse 2.6 3.0 3.0 2.7 2.7 2.5 2.3 2.5 2.0Afghanistan AbdullahIbneMasoudMosque 3.0 3.0 3.0 2.3 2.6 2.5 3.0 2.3Afghanistan ZainulabedinLaghmaniMosque 2.5 3.0 3.0 3.0 2.7 3.0 2.5 3.0 3.0Afghanistan RasulHouse 2.2 3.0 3.0 2.3 3.0 2.5 2.4 2.5 2.3Afghanistan Emalhouse 2.5 3.0 3.0 2.7 2.6 2.5 2.4 2.6 2.0Afghanistan ImamAzamMosque 3.0 3.0 2.7 2.3 3.0 2.0Afghanistan MukhtarHouse 2.5 3.0 3.0 2.7 2.7 2.5 2.3 2.5 2.3Afghanistan IndiraGhandiinstituteofchildshealth(IGICH)Phase1 2.8 2.5 3.0 2.8 2.9 3.0 2.7 3.0 3.0Afghanistan IndiraGhandiinstituteofchildshealth(IGICH)Phase2 2.8 2.5 3.0 3.0 2.9 3.0 2.5 3.0 2.7Cambodia KeoMonyPrimarySchool 1.8 2.8 2.4 2.8 2.6 2.7 2.8 3.0Cambodia ThnoutKhpousPrimarySchool 1.0 3.0 2.8 2.5 2.5 3.0Cambodia TekhakPanhNhorPrimarySchool 2.7 2.6 2.5 2.8 3.0Cambodia BithMeasPrimarySchool 3.0 2.9 2.7 2.9 3.0
28
Country Name 1.Environmental
health
2.Improving
livingcond.
3.Projectspec.
impact
4.Functioning
technology
5.Functioning
maintenance
6.Sustaining
demand
7.Effective
management
8.Sustainable
financing
9.Planning,
design,constr.
Killer
Criterion(ia)
Cambodia SamakiPrimarySchool 1.8 3.0 2.6 2.6 2.2 2.5 3.0 3.0Cambodia 7MakaraPrimarySchool 2.7 3.0 3.0 2.9 2.9 2.6 2.3 2.6 3.0Cambodia 24Kanha1993 3.0 2.7 2.6 2.6 3.0 3.0Cambodia Chrava 2.5 3.0 2.9 2.6 2.7 2.8 3.0Cambodia Kosang 3.0 2.8 2.6 2.8 2.7 3.0India GoodEarthFootprints 1.0 1.0 1.0 1.0 1.0 1.0 2.3 2.7 YesIndia T.Chintalacherry 1.3 2.4 2.0 2.5 2.5 3.0 1.7India T.Mettupatty 2.0 2.5 3.0 1.7India Krishnampatty 1.7 2.7 2.1 2.5 2.5 3.0 1.7India Lakshminayakampatty 2.3 3.0India GoodEarthMalharMosaic 2.3 2.3India GoodEarth,Terraces 1.3 1.0 3.0 2.3 3.0India GoodEarth,Resonance 1.3 2.6 2.5 2.1 2.7India RajivNagar 1.0 1.0 1.0 1.0 1.0 1.0 1.7 YesIndia GoodEarthPalmGrove 2.7 2.9 2.2 1.0 YesIndia Vaikalpatty 2.6 2.5 2.5 3.0 1.7India Manikkapuram-ADColony 3.0 2.3 2.5 2.5 3.0 2.7India Ponnamalpatty 1.7 2.4 2.6 2.5 2.5 3.0 1.7India M.Subulapuram 2.6 2.5 3.0 1.7India T.Muthurengapuram 1.0 2.5 2.2 2.2 1.7 YesIndia Tekkampatty 2.4 2.5 2.6 1.7India Sathagally 1.0 1.0 1.0 1.0 1.0 1.0 1.0 YesIndia AnumaPrecisionTool 2.3 3.0 3.0 1.9 2.3India GokaldasImages 3.0 1.0 2.5 2.7 2.3India AcuprintPvt.Ltd., 1.6 3.0 1.0 2.7 3.0 3.0 1.7India GokuldasImages2 3.0 1.0 2.4 2.3India SatGuru 2.4 2.5 2.3India KalmeshwarWeeklyVegetableMarket 3.0 1.0 2.2 2.2 1.7India HarvestInternationalSchool 2.0 3.0 1.0 2.6 2.3India ResidenceofAnthyaMadith 2.0 1.7 1.0 2.6 2.3 YesIndia AssosciationofPeoplewithDisability(APD) 1.3 3.0 2.6 2.3 2.3 2.7India AnneManeFarmHouse 2.7 3.0 3.0 3.0 2.4 3.0 1.6India KamalSolarFactory 3.0 1.0 2.6 2.5 2.7 1.7India Mr.NarasingaRao'sresidence 1.3 3.0 1.0 2.7 2.6 3.0 2.3 1.7India AmmankulamHousingColony-Cluster1 1.7 2.4 1.0 2.5 3.0 Yes
29
Country Name 1.Environmental
health
2.Improving
livingcond.
3.Projectspec.
impact
4.Functioning
technology
5.Functioning
maintenance
6.Sustaining
demand
7.Effective
management
8.Sustainable
financing
9.Planning,
design,constr.
Killer
Criterion(ia)
India AmmankulamHousingColony-Cluster2 1.0 2.4 1.0 2.5 2.7 YesIndia AmmankulamHousingColony-Cluster3 1.0 2.4 1.0 2.0 2.7 YesIndia SilverOakSchool 2.7 3.0 1.0 2.3 2.7 2.3India PositiveLabels1 1.4 2.0 3.0 2.4 2.8 2.5India NurtureInternationalschool 1.3 2.6 1.0 2.5 1.7 YesIndia IITGandhinagar 3.0 3.0 2.5 2.5 2.4 2.0 3.0India PositiveLabels2 2.0 1.0 2.3 2.5India MoazariVillage.Part-1 2.3 2.1 1.7India MoazariVillagePart2 2.3 2.1 1.7India ShendolaVillage 2.3 2.1 1.0 1.7 YesIndia WarkhedVillage 2.4 1.0 2.5 1.7 YesIndia FlipKartWarehouse 1.3 3.0 1.0 2.0 1.7 YesIndia MarvelTechnology&ToolPvtLtd 2.3 3.0 1.0 3.0 2.8 3.0 1.9 2.0 1.7India GoodEarthOrchid 2.4 1.0 2.3 2.7Indonesia LempuyangwangiHospital 3.0 2.4 3.0 2.3Indonesia KSMBeriman 1.5 2.0 2.3 2.3 2.5 2.3Indonesia KompanSehat 1.3 2.0 2.7 2.4 3.0 2.6 2.8 2.3Indonesia TegalSehat 1.8 2.3 2.8 2.5 3.0 2.7 2.8 2.3Indonesia KSMIkhlas 3.0 2.0 3.0 2.5 3.0 2.6 2.7 2.3Indonesia KSMMersudiWaras 3.0 2.3 3.0 2.8 3.0 2.7 2.8 2.3Indonesia MahesoKridho 2.8 2.3 3.0 2.8 3.0 2.5 2.7 2.3Indonesia PasebanManunggal 2.5 1.7 2.7 3.0 3.0 2.8 2.8 2.3Indonesia GebangMakmur 1.0 2.0 2.3 3.0 3.0 2.3Indonesia Miguno 2.0 2.0 3.0 3.0 3.0 2.5 2.8 2.3Indonesia Selancar 2.0 2.3 3.0 3.0 3.0 2.6 2.8 2.3Indonesia KaryaKrida 2.0 1.7 3.0 2.7 3.0 2.5 2.7 2.3Indonesia ManunggalingKarso 2.7 2.3 2.7 2.7 3.0 2.2 2.8 2.3Laos LuangphabangNightMarket 2.0 2.3 2.3 2.6 2.8 2.3Laos WorldBankofficeLaos 2.0 2.6 3.0 2.3Laos ParkhaotaiPrimarySchool 2.3 3.0 3.0 2.6 2.6 2.5 2.3 2.3Lesotho Setlolela 1.5 2.0 2.4 2.3 2.7 1.8 1.8Lesotho Putsoa 1.5 2.0 2.0 2.3 1.7 2.5 1.8 1.5Lesotho MaitumelengNtsihlele 3.0 2.4 3.0 1.8 1.0 1.7 YesLesotho LipholoMakhetha(oldCouple) 1.8 2.0 2.4 2.0 1.6Lesotho Lerotholi 1.8 2.0 2.1 1.8 2.3 1.8 1.5
30
Country Name 1.Environmental
health
2.Improving
livingcond.
3.Projectspec.
impact
4.Functioning
technology
5.Functioning
maintenance
6.Sustaining
demand
7.Effective
management
8.Sustainable
financing
9.Planning,
design,constr.
Killer
Criterion(ia)
Lesotho NapoSefali 1.8 2.0 1.0 2.0 1.8 YesLesotho BanaProject 2.4 1.8 1.3Lesotho MeMaiboiteloJobo 2.0 2.0 3.0 2.3 2.1 2.0Lesotho NonyaneAssociates 2.2 1.8 1.8Lesotho God'sLoveOrphanage 2.0 1.9 1.8 1.5Nepal DhulikhelHospital 2.7 2.5 3.0 2.5 2.7 1.7Nepal SushmaKoiralaHospital 1.0 1.0 1.0 1.0 1.0 1.0 2.3 YesNepal KathmanduUniversity 1.0 1.0 1.0 1.0 1.0 2.3 YesNepal ENPHOOffice 2.5 2.0 2.8 3.0 2.2 2.6 1.7Nepal MalpiInternationalSchool 3.0 1.0 2.3 3.0 1.7Nepal SungaCommunity 1.0 1.0 1.0 1.0 1.0 1.0 2.4 1.7 2.3 YesNepal Srikhandapur 1.0 2.1 2.3 2.5 2.6 2.3Nepal SanoKhokanacommunity 3.0 1.0 2.2 2.5 1.7Nepal Amaghar 3.0 2.0 2.5 2.3 2.2 2.3Nepal ICIMOD,Khulmatar 3.0 3.0 2.8 3.0 2.3Philippines RosarioDistrictHospital 2.0 2.0 3.0 2.5 2.8 3.0 2.3Philippines CenterforCommunityTransformationMagdalenaCampus 2.3 2.4 3.0 2.8 2.6 2.3Philippines BagbagCommunity 3.0 3.0 2.3 2.3 2.3Tanzania CCBRTextension_2 2.8 2.0 2.0 1.0 2.5 2.5 2.0 YesTanzania ISTHousingGreyWater_2 2.7 2.0 2.7 2.6 2.5 2.4 2.5 2.7Tanzania ISTHousingBlackWater_2 1.7 2.3 2.4 2.5 2.7Tanzania Majaniyachaisec.School_2 1.5 2.3 2.0 2.6 2.2 2.3 2.3 2.2 2.3Tanzania BORDA-SADCoffice_2 2.8 2.0 2.0 2.8 2.7 2.7 2.6 3.0 2.7Tanzania ShokonyPr.School_2 3.0 1.0 2.3 2.7Tanzania SchoolofSt.Jude_2 2.2 3.0 2.0 2.6 2.2 2.5 2.3 2.8 2.7Vietnam VuDinhThipig-farm 1.0 2.5 2.0 1.0 2.0 YesVietnam NguyenVanTuypig-farm 1.0 2.1 1.0 2.0 YesVietnam NguyenVanThinhpigfarm 2.6 2.0 1.0 2.3 YesVietnam FieldrestaurantHoiAnCity 1.0 2.4 2.2 YesVietnam VoThiSauschool 2.7 2.4 1.0 2.0 YesZambia PestalozziDEWATS 2.5 2.0 2.2 2.7 2.3 2.7 2.3Zambia LivingstoneLibuyu-BG40digester 2.5 2.0 2.2 2.4 2.6 3.0 1.7Zambia LivingstoneLibuyu-BG30digester 2.5 2.0 2.2 2.4 2.6 3.0 1.7Zambia LivingstoneLibuyu-BG20marketdigester 2.5 2.0 2.2 2.7 2.6 3.0 1.7Zambia LivingstoneLibuyu-BG20contraversial 2.5 2.0 2.2 2.7 2.7 3.0 1.7
31
Country Name 1.Environmental
health
2.Improving
livingcond.
3.Projectspec.
impact
4.Functioning
technology
5.Functioning
maintenance
6.Sustaining
demand
7.Effective
management
8.Sustainable
financing
9.Planning,
design,constr.
Killer
Criterion(ia)
Zambia LivingstoneLibuyu-BG20Mr.Tembo 2.5 2.0 2.2 2.7 2.6 3.0 1.7Zambia LivingstoneLibuyu-BG10marketdigester 2.5 2.0 2.3 2.7 2.6 3.0 1.7Zambia LivingstoneLibuyu-BG20Guesthouse 2.0 2.2 2.1 2.6 3.0 1.7Zambia LivingstoneLibuyu-BG10 2.5 2.0 2.3 2.7 2.6 3.0 1.7Zambia LivingstoneLibuyu-BG20 2.5 2.0 2.1 2.7 2.6 3.0 1.7Zambia LivingstoneEEPSatelitedigester 1.0 1.0 1.0 1.0 1.0 1.0 2.1 1.0 1.7 YesZambia OldMacDonaldsfarm 2.3 2.0 2.3 2.7 1.9 1.8 2.3