3ie Delhi Seminar Series
Shame or SubsidyWhat explains the impact of Total Sanitation
Campaign
Sumeet Patil 1
S. K. Pattanayak 2, K Dickinson 3, J-C Yang 3, C. Poulos 3
1 NEERMAN, Mumbai (formerly RTI International)
2 Duke University, USA (formerly RTI International)
3 RTI International, USA
2
Study Team Partners
Rajiv Gandhi Drinking Water Mission, Govt of India
Orissa State Water and Sanitation Mission and Department of Rural Development –
World Bank – Funders
WHO, USAID, UNICEF, ICMR - Multidisciplinary technical advisory group
RTI International – Principal investigating agency
Duke University – Analysis and publication phase
NEERMAN – Analysis and publication phase
TNS Mode – Survey agency
3
Overview of Presentation
Policy Context for Study (4 slides)
Study Objectives (1 slide)
Intervention (3 slides)
Methodology and Implementation (9 slides)
Results (12 slides)
Approximately, 45 minutes
4
Race against Time
Source: WSP 2009 calendar
5
Policy Context for Study
Child Diarrhea - key underlying link for India’s MDG targets
Sanitation is expected to break fecal-oral transmission and thus improve health
Universal access to toilets (no open defecation) by 2022 is a goal of Nirmal Bharat Abiyaan (NBA)
Heated comparison between supply “pushed” subsidy based TSC and CLTS based demand driven “no subsidy” based approaches
Limited evidence to guide implementers and policy makers
Govt M&E/MIS data highly unreliableSource: Chambers and Von Medeazza (2013): working paper
6
7
Policy Context for Study
Evidence to make determination is very thin Impact evaluation in sanitation sectors (are (were) fewCross-sectional assessments (lacking baseline, control,
statistical power)
Need for rigorous impact evaluation2005 RCT. Hammer and Spear (2013). Working Paper2006 RCT. Pattanayak et al. (2009). This paper2011 RCT. Patil et al. (2013). Working Paper2011 4-arm QE. We hope that endline happens
8
Overview of Presentation
Policy Context for Study
Study Objectives Intervention
Methodology and Implementation
Results
9
Study Objectives
Whether CLTS based behavior change coupled with subsidy based intervention (TSC) impacts latrine use and child health?Track the logic chain from inputs to intermediate outputs
to outcomes to health impactsGenerate operational knowledge to guide policy
Ability to study the effect of Shame only and shame + subsidy because of the TSC program design feature
10
Overview of Presentation
Policy Context for Study
Study Objectives
Intervention Methodology and Implementation
Results
11
Community-Led Total Sanitation in Bhadrak
Knowledge Links
12
Intervention: Community-Led Total Sanitation (Kar, IDS)
Knowledge alone does not change behavior; need to create “triggering events” and intensive Behavior Change campaign “walk of shame” “defecation mapping” “fecal calculation”
TSC related Incentives for BPL for latrine construction (Rs 1500)
Supply side: masons, rural sanitation mart, know how, motivation, monitoring
Immediate outputs: Out of 20 villages, 9 resolved to end OD, 2 agreed in principle, 5 decided to meet, and 4 were unable to reach a consensus
13
CLTS Program – Logic Model
Subsidized labor & materials
Number of IHL constructed
Use of IHL
Improvements in child health
Satisfaction with IHL
Personal benefits
Broader welfare impacts
PROGRAM INPUTS PROGRAM OUTPUTS PROGRAM OUTCOMES PROGRAM IMPACTS
Knowledge
Attitudes
Preferences
MEDIATING FACTORS
Community-ledTotal Sanitation (CLTS)
Emphasis on dignity & privacy
IHL construction know-how
Communication on water-washed diseases
Budging social norms
14
Overview of Presentation
Policy Context for Study
Study Objectives
Intervention
Methodology and Implementation
Results
15
Study Design
Randomized Control ExperimentWell controlled. Random and blind assignment of
treatment
Sample Size: 20 CLTS villages + 20 control villages and 25 HHs per village (with u5 children)
Baseline (2005) and Endline (2006) Panel Surveys
2 rounds, same season, same households
Difference in Difference (DID) estimation of impactsDifference: Before and After and With and Without
16
Sample Selection
Selected a district (Bhadrak) with adequate water
Selected blocks (Tihidi & Chandbali) without prior TSC
Restricted villages to have >70 HHs and < 500 HHs
Restricted to 1 village per GP to reduce spill over
Selected 40 villages & randomly assigned 20 to treatment
Listed and mapped all households in 40 villages
Randomly selected and surveyed 25 households with child < 5 yrs in each village
17
Study Villages
10 0 10 Km.
N
EW
S
20.- Controlled Villages of RTI - WB Study
LEGEND:
VillagesControl village
Baliarpur
Dhurbapahalipur
Taladumka
Sasankhas
Balipada
Begunia
Nuasahi
Birabarpur
Talabandha
Rajnagar
Madhupur
Barikpur
Hatapur
Ambolo
Padisahi
Bhimpur
Badapimpali
Budhapur
Gouriprasad
Sansamukabedhi
T I H I D I B L O C K
C H A N D B A L I B L O C K
OraliHengupati
Bhuinbruti
Satuti
Baincha
JashipurSanasingpur
Haripur
Nayananda
Balisahi
Arjunbindha
Jaladharpur
Mangrajpur
Guanal
Deuligaan
Bahu
Amarpur
Jaydurgapatna
Aigiria
Tentulida
Tentulida
18
Data: Measurements
Outputs, Outcomes, Impacts: Household pit latrines (IHL): constructed, operational and in-use Diarrhea frequency & severity (> 3 episodes in 24-hr, 2-week recall) Child growth (anthropometrics – MUAC, weight, height)
Additional parameters: Individual - sex, age, class, caste, religion Household - family size and composition, education, housing
conditions, asset holdings, occupation and expenditures, services Community – roads, electricity, environmental sanitation,
employment, clinics, schools, credits, markets Institutional - main governmental and NGO programs, local
government size and composition Water quality (E. coli & total coliform) – community sources (all), in-
house (50%)
19
Data: Household Survey
Respondent - Primary Care Giver Water samples collected from approx 50% of surveyed households Modular questionnaire
Knowledge, Attitudes Household SEC Sanitation Behaviors – outputs and outcomes Hygiene Behaviors Water Sources and their use Water Treatment/safety behaviors Food safety behaviors Environmental conditions – HH and community Budget constraints Community Participation
20
Data: Community Survey
Respondent – sarpanch, GP member, Informal leader, Doctor, etc Water samples collected from up to 10 in-use drinking water
sources Modular questionnaire design
Background: population, households, area, arable land, major crop grown Public infrastructure: roads, water supply, sanitation, hygiene,
electricity, clinics, schools, STD booths, telegraph offices, post offices, credits and markets
Environmental sanitation: general cleanliness, drainage, animal and household waste, use of water sources, open defecation practices
WSS scheme: Swajaldhara, piped water, hand pumps, etc Development Programs: Health, education, women support etc Economy: employment opportunities, major governmental and NGO
programs, prices Local government: structure, composition, activities
21
Survey Implementation - I
Schedule & Resources 1 month of data collection to catch the monsoons!! Field Teams – RTI (3 + 1 consultant) and TNS (30 field
people + 2 researchers)
Focus groups
Pre-testing (2 rounds of 50 household surveys)
Training (8 days. Mix of in-class and field practice)(manuals prepared)
Supervision: Supervisors executives Managers Researchers. Back checks, spot checks.
22
Fieldwork
23
Survey Implementation - II
Data Processing On field editing, 100% scrutiny before data entry CSPro based data entry Cross-tabulation based cleaning
WQ Samples 50% HHs and up to 10 in use sources. Sterilized bottles Cold chain transport to lab within 24 hours
24
Overview of Presentation
Policy Context for Study
Study Objectives
Intervention
Methodology and Implementation
Results
25
Baseline Balance - I
T C p-valueSEC
From scheduled caste 28 26 0.858From other backward classes 29 24 0.449Below poverty line 60 61 0.91
WASHUsed improved water sourced 37 42 0.602Boiled or treated drinking water 9 13 0.192Adults washed hands at 5 critical instances 11 9 0.564Dumped garbage outside of house 68 69 0.794Threw wastewater in the backyard 46 48 0.705With individual household latrine 6 12.7 0.03
26
Baseline Balance - II
T C P-valueAttitudes
Completely dissatisfied with current sanitation 72 61 0.011Water supply is most important improvement 7 12 0.149Sanitation is most important improvement 5 8 0.264Women lack privacy during defecation 32 30 0.82Women are not safe defecating in the open 29 29 0.463Government should bear the cost of sanitation 53 50 0.561
Health U5 diarrhea in past 2 weeks 28 23 0.218(MUAC)-for-age z-score for U5 -1.3 -1.3 0.677height-for-age z-score for U5 -2 -1.9 0.687weight-for-age z-score for U5 -2.2 -2.3 0.341
Estimation
27
28
% Households owning & using Toilets(by intervention and year)
I indicates the 95% confidence interval.
6%
32%
13% 13%
0%
10%
20%
30%
40%
2005 2006 2005 2006
CLTS Villages Control Villages
0%26%
DID= 26%-0%= 26%***
29
E Coli Levels in HH Drinking Water
0
5
10
15
20
25
2005 2006 2005 2006
CLTS Control
Elusive Health Impacts
30
BL/EL T C T-C DID
U5 diarrhea Prevalence0 27% 23% 4.30%
-4.90%1 14% 15% -0.60%
MUAC-for-age z-score0 -1.34 -1.33 -0.011
0.1331 -1.2 -1.32 0.123
Height-for-age z-score0 -1.95 -1.94 -0.007
0.2811 -2.01 -2.29 .273*
Weight-for-age z-score0 -2.16 -2.25 0.088
-0.1921 -2.22 -2.3 0.069
Shame or Subsidy?
Triple Difference to get the relative effect of shame and subsidy
BPL = Subsidy + Shame and APL = Shame alone
DID for BPL – DID for APL = 34.2 – 20.7 = 13.5%
13 % effect (about 1/3rd) by the “subsidies”
Full sample BPL APLImpact (mean test with EL) 19 23.7 12Impact DID 28.7 34.2 20.7
31
Is this result replicable elsewhere?
Another RCT in Madhya Pradesh
A scaled up and more “realistic” program
50% to shame + “less subsidy” and 50% to shame and “more subsidy (by Rs 2700)
32
Control Treatment N Mean N DifferenceOverall 1514 0.22 1525 0.19 (0.035) ***Poor 375 0.17 300 0.32 (0.046) ***Non-poor 1139 0.24 1225 0.15 (0.037) ***
33
Are effects sustainable?
Percent of Households Owning a Latrine, Treatment Villages
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Ambo
la
Badap
impa
la
Baliar
pur
Balipa
da
Barikp
ur
Begun
ia
Bhimpu
r
Biraba
rapu
r
Bodha
pur
Dhrub
apah
alipu
r
Goura
pras
ad
Hatap
ur
Mad
hupu
r
Nuasa
hi
Padhi
sahi
Rajnag
ar
Sanas
amuk
aved
i
Sasan
khas
Talab
andh
a
Talad
umuk
a
2004 GoO Data2005 HH Survey2006 HH Survey2006 Community Survey2007 GoO Data
34
Findings from Mixed Methods - I
Some factors indicate “possibility of sustainability” Increased satisfaction with sanitation situation Increased belief that improving sanitation is the family’s
responsibility
Lack of knowledge of the “germ theory” is not the most important BUT privacy and dignity are key
Households prioritize. Toilets may be “our” priority, not theirs: 80% want health dispensary, 59% roads. Compare to 7-9% for water supply and sanitation
Findings from Mixed Methods - II
Support structure – NGOs, district officials, involvement of triggering team, village institutional capacity are important success factors
Subsidies are tricky business may have created an incentive for NGOs to “cut
corners” and produce lower quality latrines Concern that subsidies in general defeat the sense of
self-reliance Will subsidy be counteractive in long term?
How and when you give subsidies will matter Community based incentives (e.g. NGP) instead of
individuals? Is “post” incentives practical for poor population?
35
7 years later…
Credible evidence that “shaming” works BUT, so do subsidies BUT, does the relative contributions depend on
“intensity” of CLTS or amount of subsidy?
Seems to be continued increase in toilet coverage BUT, what about use? And toilet maintenance? BUT, will we reach 100% open defecation free status?
BUT, what about health impacts?
7 year later, we still stare at above critical questions without credible answers36
Thank You
Sumeet Patil: [email protected] Other papers
Pattanayak et al. (2010), “ “How valuable are environmental health interventions?...” Bull WHO, 88:535-542.
Pattanayak et al. (2009), “Shame or subsidy revisited:…” Bull WHO, 87:1-19.
Pattanayak et al. (2009), “Of taps and toilets….”, J of Water and Health, 7(3): 434–451.
World Bank (2011). “Of Taps an Toilets”. WB report on Evaluation of CDD program in RWSS.
37
