Terms of Reference

Improving Sanitation Planning and Efficiency Improvement Impact

Evaluation of the Urban Sanitation and Rural Infrastructure Project

A. Background• The Government of Indonesia has received a loan from the Asian Development

Bank (ADB) for the Urban Sanitation and Rural Infrastructure (USRI) Support to PNPM Mandiri Project. The Project aims at (i) upgrading basic rural infrastructure in poor villages in four provinces, and (ii) promoting community-driven sanitation development to contribute to achieving the Millennium Development Goal (MDG) target to halve, by 2015, the proportion of people without sustainable access to basic sanitation.• The USRI project includes an associated capacity development technical

assistance (TA 7843-INO) to assist the Government to institutionalize sanitation related advocacy and training programs addressing the limited understanding of clean water use, hygiene practice and health. The TA is also expected to address the lack of knowledge on appropriate technologies for establishing community-driven sanitation facilities in poor urban neighborhoods.

• The Directorate General of Human Settlements (DGHS) is the Executing Agency (EA) of the USRI Project and the TA. The DGHS has established a central project management unit (CPMU). The DGHS as the key implementing agency and through the CPMU is responsible for the day-to-day TA management and implementation, including TA supervision, monitoring, and reporting. This consulting services assignment will be financed from the TA proceeds.• The project is implemented in about 800 rural communities in four provinces, and

1,400 poor urban neighborhoods in 34 cities in five provinces. The project design monitoring framework is in Appendix 1, project locations are in Appendix 2.• The impact of the Project is reduced poverty in the project areas; the expected

outcome is improved access to service delivery and healthy live conditions for the poor, near poor, and women in the project communities. The project has three outputs: (i) strengthened capacity for community planning and development, (ii) improved rural basic infrastructure, and (iii) improved sanitation services.

Output 1: Strengthened Capacity for Community Planning and Development.

• The project adopts a community driven development (CDD) approach. Under this output, the project provides support to strengthen the capacity of communities to prioritize, design, implement, and maintain infrastructure projects. • The project deploys community facilitators (CFs) to assist communities to carry

out:1. poverty mapping, 2. identify problems and needs, 3. evaluate community implementation capacity,4. develop efficient planning and decision-making processes;5. establish and run community implementing organizations (CIOs); 6. formulate development plans and specific investment plans to be financed by

block grants; 7. prepare technical designs and implement civil works, and 8. formulate and implement operation and maintenance (O&M) plans to ensure

sustainability of completed facilities.

• Each participating village and neighborhood has established a community implementation organization (CIO). CIOs comprise of at least five members elected by the communities. On average, at least 30% of the CIO members are women. CIOs are approved by the village heads/neighborhood officials and registered with the district/city satker,1 which is the prerequisite for opening a CIO bank account.

1. Regional working unit of MPW

Output 2: Improved Rural Basic Infrastructure

• Block grants for basic infrastructure upgrading are provided to 800 villages.1. 400 villages received three cycles of block grants, 2. 200 villages received 2 cycles of block grants, and 3. 200 villages received 1 cycle of block grant2.• These villages are located at the 145 project subdistricts (kecamatans) in Jambi,

Lampung, Riau and South Sumatra that were previously covered under the RIS PNPM II. About 200 villages, selected under the Project, have received one cycle of block grants under another PNPM project financed by the government own resources (category A village). The remaining 600 villages (category B village) are villages that have not received any PNPM block grants.• Infrastructure upgrading financed by the project include building and repairing

rural roads, pathways, small bridges, irrigation facilities, water supply systems and public sanitation facilities. Supported by CFs, the CIO takes the lead in developing village investment plans in close cooperation with the community members.

2. The original target was 600 villages to receive 2 cycles of block grants

• After the village investment plans, which include the final technical designs and cost estimates, have been approved at a village assembly and submitted to the DPIU for approval; a contract will be executed between the CIO and the district Satker and the first installment of the block grant will be released. Civil works are undertaken by communities with technical guidance from the CFs, DPIUs, and consultants. • Block grants are transferred directly to community bank accounts managed by

the CIOs.The first installments of 40% of the approved block grant are transferred directly into the CIO’s bank account as an advance payment. The remaining funds, subject to certification of progress by the DPIUs, are disbursed to communities in two additional installments.

Output 3: Improved Sanitation Services

• To ensure that community sanitation facilities established under the Project are integrated in city development plans, only neighborhoods located in cities with an approved City Sanitation Strategy (CSS) are eligible to participate in the Project. 1,439 neighborhoods3 in 34 cities in the provinces of Central Java, DI Yogyakarta, East Java, South Sulawesi and North Sulawesi, will benefit from block grants to finance (i) construction of public bathing, toilets and washing facilities, (ii) improvement of communal sewerage systems and waste treatment, and (iii) improvement of disposal/reuse systems.

3. Originally 1,350 neighborhoods were targetd to be coverd, the minor charge added 89 more villages to be target

B. Objectives of the AssignmentThe objective of the assignment is to assess and evaluate the overall impact, outcomes and outputs of the project compared to the targets and performance indicators specified in the DMF, gender action plans and other targets specified in project documents.

C. Scope of Services, Tasks and Expected Deliverables

The main expected results of the assignment are: • An independent assessment of the USRI performance, with particular attention

to the achievements of the project against its targets and objectives; • An independent assessment of the USRI in terms of relevance, effectiveness,

efficiency, sustainability, impact; • Key lessons and practical recommendations for the exit strategy of the USRI

project and consolidation and replications of achievements; • Recommendations for ADB on future involvement in the sanitation sector

(particularly community-based system) and rural development activities (e.g. second stage USRI);

• Advices to the government to expediting the achievements of the target of 100% coverage of sanitation services by 2019.

In addition to assessments on the achievements of the project quantitative targets and objectives, the evaluation should include: • An evaluation of quality of community empowerment and facilitation; • An evaluation of the quality of planning documents such the community action

plans, community sanitation improvement action plans (CSIAPs), mid-term poverty reduction (MTPR) plans, city sanitation strategies (CSS);

• An evaluation on the project implementation organization/implementation at all levels;

• An evaluation of the performance of all project stakeholders at all levels; • An assessment of the quality of infrastructure built, including the appropriateness

of the used technology; • An evaluation of the project design, approaches and monitoring framework; • Project impacts on the participating cities/districts’ sanitation

performance/targets, sanitation institutions/agencies; • An evaluation of the project gender action plan;

• An evaluation of project impacts on environment, poverty, gender, governance, and sanitation sector development;

• The expected deliverables are: 1. Evaluations of the project results, impacts, lessons and detailed

recommendations for the project exit strategy including the consolidation and if applicable the replication of the project approach and achievements;

2. Advice on ADB’s future involvement in the sector. The findings and recommendations should be presented in a reader friendly and professional manner and may include illustrations and photos, interviews with stakeholders/beneficiaries, etc.

• The team will also be required to present results of the assignment during a national workshop. The consultant should ensure that data from a proper number neighborhoods and villages (proper sampling method) are obtained so that conclusions are validated.

D. Period of Services and Staffing Requirements

• The services of the consultant will be required for a total of 3.5 months. The consultant team will be selected and engaged in accordance with the ADB’s Guidelines on the Use of Consultant (2012, as amended from time to time). • The NMC will be recruited through selection of national firms. The quality and

cost-based selection method, which requires the submission a simplified technical proposal within 25 days,4 will be used to select the consultant. A total minimum of 18 person-months of consultant inputs will be required, including 14.5 person-months of national consultants and 3.5 international consultant inputs are required for this assignment.

4. ADB’s procedures of recruitment of consultant allow shorter period for preparation proposals, if the electronic system is applied

The assignment schedule is estimated as follows:

Activity Month-1 Month-2 Month-3 Month-41 2 3 4 1 2 3 4 1 2 3 4 1 2 3 4

Preparation of survey

Data collection

Data analysis

Report preparation

Workshop

Report finalization

E. Team Composition and Qualification Requirements of Key Experts:

1. Sanitation Impact Evaluation Specialist/Team Leader (International, 3.5 person-months) a. Roles, Responsibilities and Tasks

The main responsibilities of the TL include: i. lead and manage the team and ensure that all team members deliver his/her

high quality outputs in a timely manner; ii. lead the team in preparing and submitting all the reports and ensure a high

quality of the deliverables; iii.lead in the preparation of the evaluation methodology/survey tools, including

the determination of samples, control groups; iv. supervise and monitor data collection process and ensure that the collected data

are valid;

v. ensure that all the required data/information to complete the assignment are collected;

vi. ensure that the collected data are properly and timely analyzed; vii. ensure the timely recruitment, mobilization and training of the field

investigators (surveyors); viii. present the overall impact evaluation results to project stakeholders (a

national workshop); ix. lead in coordination with ADB and the Executing Agency (EA) in obtaining

approvals and comments on the draft of deliverables; x. lead in data analysis process; xi. carry out analysis and review of city sanitation strategies (CSS) of sample

cities participating in the project. The review should include the quality of the CSS compared to the standard CSS;

xii. review the quality of community sanitation improvement action plans (CSIAPs) and other community sanitation plan documents;

xiii. assess the quality of project manuals (JUKLAK/JUKNIS etc.), technical/design standards, particularly for the sanitation part;

xiv. assess the sanitation block grant size and effectiveness of their fund channeling;

b. Required Key Qualificationsi. The team leader/evaluation specialist is expected to have extensive

experience in project impact evaluation activities, and significant experience in leading teams.

ii. He/She is preferable to hold a master degree in environmental engineering/health or other related fields.

iii. Experience in evaluating urban sanitation and rural infrastructure related projects, applying the community-driven development approach is preferable.

iv. The specialist is expected to have experience in working in Indonesia. Basic knowledge of Bahasa Indonesia will be an advantage.

v. The TL is expected to have excellent English oral and writing skills, to be a good team player and have excellent communication skills.

2. Rural Infrastructure/Development Specialist/Co-Team Leader (national consultant, 3.5 person-months) a. Roles, Tasks and Responsibilities

The main responsibilities of the rural infrastructure/development specialist include: i. Support the TL and other team members in preparing the survey

tools/methodology, particularly in the rural infrastructure part of the project;

ii. Carry out analysis and review the quality of MTPR, CAP etc.; iii. Support the TL to ensure that all the required data/information to

complete the assignment are collected; iv. Support the TL to ensure the timely data collection; v. Support the TL in the recruitment, mobilization and training of the field

investigators (surveyors); vi. Present the overall impact evaluation results to project stakeholders

(national workshop);

vii. Assess the quality of built rural infrastructure; viii. Assess the quality of project manuals (JUKLAK/JUKNIS etc.),

technical/design standards, particularly for the rural infrastructure part; ix. Assess the rural infrastructure block grant size and effectiveness of their

fund channeling; x. Assess the selection process of participating districts and villages; xi. Assess the quality of training for rural infrastructure community facilitators; xii. Assess the quality of community facilitator selection; xiii. Support the TL in report preparation/submission of deliverables; xiv. Assess project’s impact on rural/village development sector; xv. Provide a brief analysis on impact of the enactment of the new village law

on the assess the quality of project monitoring and supervision by the Executing Agency, participating cities/districts, consultants and other related stakeholders;

vii. Assess the quality of management consultants supporting DGHS in implementing and managing the project;

viii. Implement other tasks to ensure the timely completion of the assignment and submission of high quality reports/deliverables.

ix. rural/village development sector; b. Required Key Qualifications

i. The specialist is expected to have extensive experience in project impact evaluation activities.

ii. He/She is preferable to hold a bachelor degree (S1) in civil/environmental engineering or other related fields.

iii. Experience in evaluating rural infrastructure/development related projects, applying the community-driven (e.g. PNPM Rural) development approach is preferable.

iv. The expert is preferable to have experience in working in project areas (villages in southern part of Sumatra).

v. The expert is expected to have good English oral and writing skills, to be a good team player and have excellent communication skills.

3. Statistician/Data Specialist (National consultant, 2 person-months) a. Roles, Tasks and Responsibilities

The main responsibilities of the statistician/data specialist include: i. Lead in the preparation of data collection methodology/survey tools,

including the determination of samples, control groups; ii. Lead in the selection, mobilization and training of surveyors; iii. Lead in the supervision of data collection process; iv. Ensure that all required data are collected timely; v. Lead in quality assurance of collected data and data analysis; vi. Support TL in the preparation of all reports and deliverables; vii. Support TL and other team members in delivering their outputs; viii. Present data analysis results to project stakeholders; ix. Undertake other related tasks to ensure the timely completion of the

assignment.

b. Required Key Qualifications • The specialist is expected to have extensive experience in data collection

(surveys) and analysis assignments. • Experience in evaluation of similar project impacts would be an advantage. • He/She is preferable to hold a bachelor degree (S1) statistics or other

related fields. • The expert is preferable to have experience in working in project areas. • The expert is expected to have good English oral and writing skills, to be a

good team player and have excellent communication skills.

4. Non-Key Experts (Data Collection Supervisors/Survey Coordinators)

Nine (9) non-key experts will be required to supervise and coordinate the data collection process in the nine project provinces. The total allocated input for the non-key experts is 6 person-months. Each coordinator will be responsible to ensure the completion of data collection in his/her assigned province. The coordinators are tasked to support the TL in mobilizing, training and assigning survey areas of surveyors. He/She will be responsible to ensure the high quality of collected data. The consulting firm should properly allocate the inputs of supervisors in each project province considering the coverage and number of villages and neighborhoods to be visited by surveyors (e.g. in South Sulawesi and North Sulawesi only one city participating in the project, while in Central Java 19 cities/districts).

• The following are the required key qualifications for data collection supervisors: i. Hold a bachelor degree in engineering (civil or sanitary engineering

is preferable) or other related fields, ii. Have experience in data collections/surveys, iii. Have excellent communication skills particularly verbal

communication skills, iv. Familiar with local culture and structure. • It should be noted that non-key experts are NOT subject to technical

proposal evaluations. However, payments of non-key experts will be included under the remuneration category (FIN-3 form).

F. Reporting Requirements and Time Schedule for Deliverables

The following reports should be prepared and submitted by the consultant: • An inception report outlining the evaluation work plan and field work methodology

and analytical framework. The report should be submitted within 2 weeks of the issuance of the notice to proceeds (NTP). • Interim report providing progress of assignment implementation including preliminary

analysis of project impacts. The report is to be submitted within 1.5 months after the issuance of the NTP. • Draft Final Report, to be submitted within 2.5 months after the issuance of the NTP.

This report contains a comprehensive account of project evaluation results and outcomes, lessons-learned, experiences gained, and conclusions as well as recommendations, as specified in the scope of assignment. The draft final report must be presented to and discussed with the ADB, Executing Agency and other project stakeholders in a national workshop. • Final Report, to be submitted within 2 weeks after the completion of the assignment.

The final report should contain comments and inputs provided during the discussion of the draft final report. • Other reports as required by ADB related to the assignment.

G.Provisional Sums Allocation The following activities are included in the provisional sums allocation:

• Training of surveyors and • National Workshop.

Term of references and estimated budgets for activities under the provisional sums will need to be endorsed by ADB prior to their implementation.

H.Client’s Input and Counterpart Personnel

• Office spaces and other equipment will not be provided. ADB cost estimate under the OPE category has included budgets for office rentals, the running and maintenance cost of the office including electricity, telecommunications and cleaning cost etc. The

• consulting firm should also allocate sufficient budget for duty trips/travels to cities/neighborhoods/villages required in carrying out the assignment.

• DGHS and ADB will provide at no costs, access to all project documents and data available at DGHS and ADB to support the implementation of the assignment. DGHS will also assign counterpart staff at the CPMU and at the national Satker to work with the consultant. Costs for these counterpart staff will be borne by the DGHS. At the local level, the project cities/districts will appoint staff at the DPIUs and city/district satkers as counterpart staff at the city/district level. Costs for these counterpart staff will be borne by the respective project city/district. The project cities/districts will also provide access to data/information/documents to support the execution of the work at no costs.