Impact EvaluationImpact Evaluation
Low-Income Housing Low-Income Housing Finance in Sri LankaFinance in Sri Lanka
Binh T. NguyenBinh T. Nguyen
Independent Evaluation Department Independent Evaluation Department
Asian Development BankAsian Development Bank
17 Oct 201017 Oct 2010
ContentsContents
Overview of ADB Assistance in Low-Income Overview of ADB Assistance in Low-Income HousingHousing
Case Project: Urban Development and Low-Case Project: Urban Development and Low-Income Housing Project in Sri LankaIncome Housing Project in Sri Lanka
The Impact EvaluationThe Impact Evaluation
The Importance of Housing for Achieving the Millennium Development Goals
Millennium Development Goals (MDGs) Role of Housing in Achieving MDGs
Goal 1: Eradicate extreme poverty and hunger
Residential activities can provide job opportunities and income and thereby allow urban poor to invest in food and other basic needs.
Improved housing conditions raise worker productivity. Residential activities improve a nation’s wealth (e.g. taxes and
savings) and allow governments and agencies to invest in social oriented programs to reduce poverty.
Goal 2: Achieve universal primary education
Improved, and access to, housing in appropriate locations lowers absenteeism from school.
Improved, and access to, housing increases educational productivity. Secure tenure allows parents to engage in income-generation
activities allowing them to cater for educational expenses. Goal 3: Promote gender equality and empower women
Secure tenure contributes to household stability and provides women with peaceful atmosphere to engage in economic-generating activities.
Good housing reduces stress and contributes to women’s productivity. Goal 4: Reduce child mortality
Good housing and related services (e.g. water, electricity, and sanitation) reduces the risk of diseases among children.
Goal 5: Improve maternal health
Improved housing lowers the need for health services for women. Secure tenure reduces stress among slum dwellers, especially
women. Safeguard procreation and nurturing of the young.
Goal 6: Combat HIV/AIDS, malaria and other diseases
Access to housing reduces homelessness and risks of social vices associated with street people.
Good housing brings comfort, reduces overcrowding and limits the transmission of communicable diseases (e.g. tuberculosis), it facilitates and enhances care-giving.
Health conditions depend largely on good living environment. Goal 7: Ensure environmental sustainability
Good housing conditions and related services contribute to a good environment.
Use of environmentally friendly building materials, including energy-efficient materials, contributes to environmental protection.
Good housing and urban design are cornerstones for mitigating ecological footprints of settlements and reducing vulnerability to climate change.
Goal 8: Develop a global partnership for development
Partnership between national government and international development agencies creates synergy and reduces duplication of programs.
Partnership between national government and international development agencies for housing ensures realistic policies and programs, and sharing of best practices.
Programs that involve partnerships among national governments, international development agencies, local communities and slum dwellers have a better chance of long-term sustenance.
Source: Tibaijuka, Ana Kajumulo, “Building Prosperity: Housing and Economic Development”, UN-Habitat and Earthscan, London and Sterling, Virginia, 2009.
ADB Loans and Grants for LIUHADB Loans and Grants for LIUH
Year of Year of ApprovaApprova
ll
LIUH Loans & GrantsLIUH Loans & GrantsAll ADB Loans & All ADB Loans &
GrantsGrants
Amount Amount
NumbeNumberr
Amount Amount
($, M)($, M)% of All % of All LoansLoans
NumbeNumberr
Amount Amount
($, M)($, M)
1966-1966-19691969 - -
- - - -
21 21 99.68 99.68
1970-1970-19791979
4 4
98.80 98.80 2.452.45
239 239 4,040.57 4,040.57
1980-1980-19891989
7 7
269.60 269.60 2.412.41
262 262 11,206.53 11,206.53
1990-1990-19991999
9 9
612.00 612.00 1.921.92
323 323 31,915.58 31,915.58
2000-2000-20102010
19 19
346.30 346.30 0.390.39
2,046 2,046 88,968.94 88,968.94
TotalTotal 39 39
1,326.7 1,326.7 0.940.94
2,891 2,891
141,513.21 141,513.21
Project Performance RatingsProject Performance Ratings
RatingRating NumberNumber %%
Highly Highly SuccessfulSuccessful
11 4.54.5
SuccessfulSuccessful 1313 59.159.1
Partly SuccessfulPartly Successful 44 18.218.2
Unsuccessful Unsuccessful 11 4.54.5
Not RatedNot Rated 33 13.613.6
Total Total 2222 100100
Case ProjectCase ProjectLoan 1632-SRI: Urban Development Loan 1632-SRI: Urban Development
and Housing Projectand Housing ProjectBasic Data:Basic Data:
Approved 1998 Completed 2005Approved 1998 Completed 2005
Total cost $102.99M ADB loan $67.02M Govt/Banks $35.97MTotal cost $102.99M ADB loan $67.02M Govt/Banks $35.97M
Four Components: Four Components:
(i)(i) Urban Development: roads, traffic improvement, water Urban Development: roads, traffic improvement, water supply, drainage, etc.supply, drainage, etc.
(ii)(ii) Community Development: basic infra, tenure regulations Community Development: basic infra, tenure regulations
(iii)(iii) Housing Finance: housing loans to low-income householdsHousing Finance: housing loans to low-income households
(iv)(iv) Institutional Development: training on staff skills in municipal Institutional Development: training on staff skills in municipal management, environment management, etc.management, environment management, etc.
Project Expenditure ($, Million)Project Expenditure ($, Million)
ComponentComponent ADBADB GovtGovt Total Total
PercenPercentt
Urban Urban InfrastructureInfrastructure
40.2840.28 23.623.6 63.8863.88 62.062.0
Community Community DevelopmentDevelopment
1.411.41 1.61.6 3.013.01 2.92.9
Housing Housing FinanceFinance
19.919.933
77 26.9326.93 26.126.1
Institutional Institutional DevelopmentDevelopment
3.533.53 3.773.77 7.37.3 7.17.1
ChargesCharges 1.871.87 00 1.871.87 1.81.8
TotalTotal 67.067.022
35.9735.97 102.99102.99 100100
Housing Finance ComponentHousing Finance Component
Objectives:Objectives:
• Increase access of low-income households (LIHs) to Increase access of low-income households (LIHs) to market-based housing finance through the formal market-based housing finance through the formal sector;sector;
• Facilitate improvements of housing conditions and Facilitate improvements of housing conditions and quality of life; andquality of life; and
• Promote formal banking sector interest in financing Promote formal banking sector interest in financing low-income housing market segment. low-income housing market segment.
LIHs = households with monthly income below the 55LIHs = households with monthly income below the 55thth income percentile, i.e., below Rs12,500 (appr. $200) income percentile, i.e., below Rs12,500 (appr. $200) per month. per month.
Housing Finance Component Housing Finance Component (Cont.)(Cont.)
Basic Data:Basic Data:
• Total Amount = $26.93M Total Amount = $26.93M
ADB loan = $19.93M PCIs = $7MADB loan = $19.93M PCIs = $7M
• Total Borrowers = 28,378Total Borrowers = 28,378
• PCIs = 7 participating credit institutions PCIs = 7 participating credit institutions
- Housing Finance Development Corporation = Housing Finance Development Corporation = 68.6%68.6%
- 3 Regional Development Banks = 27.9%3 Regional Development Banks = 27.9%
- 3 Commercial Banks (BOC, Hatton, National) = 3 Commercial Banks (BOC, Hatton, National) = 3.5%3.5%
Housing Finance Component Housing Finance Component (Cont.)(Cont.)
Loan Disbursement by ProvinceLoan Disbursement by Province
Province Province
NumbeNumber r
of of Loans Loans
(%) (%)
Loan Loan Amount Amount
(SLRs (SLRs million)million)
(%) (%)
Western Western 5,097 5,097 18.0 18.0 604.6 604.6 25.1 25.1
Southern Southern 8,109 8,109 28.6 28.6 561.2 561.2 23.3 23.3
Central Central 7,051 7,051 24.9 24.9 500.0 500.0 20.8 20.8
SabaragamuSabaragamuwa wa
2,154 2,154 7.6 7.6 216.8 216.8 9.0 9.0
North North Western Western
1,658 1,658 5.8 5.8 182.5 182.5 7.5 7.5
North Central North Central 2,506 2,506 8.8 8.8 179.5 179.5 7.5 7.5
UVA UVA 1,420 1,420 5.0 5.0 125.3 125.3 5.2 5.2
North and North and East East
383 383 1.3 1.3 38.3 38.3 1.6 1.6
Total Total 28,378 28,378 100.0 100.0 2,408.2 2,408.2 100.0 100.0
Housing Finance Component Housing Finance Component (Cont.)(Cont.)
Loan Disbursement by Income Loan Disbursement by Income Group Group
Income Group Income Group
(SLRs per (SLRs per month) month)
NumbeNumber r
of of Loans Loans
(%) (%) SLRsSLRs
MillionMillion (%) (%)
less than 2,500 less than 2,500 362 362 1.3 1.3 12.6 12.6 0.5 0.5
2,501–5,000 2,501–5,000 4,198 4,198 14.8 14.8 219.6 219.6 9.1 9.1
5,001–7,500 5,001–7,500 9,630 9,630 33.9 33.9 719.6 719.6 29.9 29.9
7,501–10,000 7,501–10,000 8,948 8,948 31.5 31.5 825.8 825.8 34.3 34.3
10,001–12,50010,001–12,500 5,240 5,240 18.5 18.5 630.6 630.6 26.2 26.2
Total Total 28,378 28,378 100.0 100.0 2,408.2 2,408.2 100.0 100.0
Housing Finance Loan Housing Finance Loan (Cont.)(Cont.) Loan Disbursement by PurposeLoan Disbursement by Purpose
Purpose Purpose NumberNumber
of of Loans Loans
(%) (%)
Construction of New House Construction of New House and Extensions and Extensions
19,751 19,751 69.6 69.6
Renovation of Existing Renovation of Existing Houses Houses
6,130 6,130 21.6 21.6
Purchase of Land for New Purchase of Land for New House Construction House Construction
2,128 2,128 7.5 7.5
Service Connections Service Connections 312 312 1.1 1.1
Purchase of House Purchase of House 57 57 0.2 0.2
Total Total 28,378 28,378 100.0 100.0
Why This Project Was Chosen?Why This Project Was Chosen?
• Findings of the evaluation will provide insights Findings of the evaluation will provide insights and lessons for urban development operations and lessons for urban development operations guidelines being prepared by the Urban COPguidelines being prepared by the Urban COP
• The project had clear assignment rule: The project had clear assignment rule: Households with income below the 55Households with income below the 55thth income income percentile (this was confirmed during the percentile (this was confirmed during the Reconnaissance Mission in preparing for the Reconnaissance Mission in preparing for the evaluation)evaluation)
• The project appeared to be the best among ADB The project appeared to be the best among ADB housing projects in terms of baseline data: loan housing projects in terms of baseline data: loan applicants were required to submit a detailed applicants were required to submit a detailed household profile, and these are kept in PCIshousehold profile, and these are kept in PCIs
The Impact EvaluationThe Impact Evaluation
Objectives: Objectives: In addition to assessing the extent to which In addition to assessing the extent to which the housing finance component met its stated objectives the housing finance component met its stated objectives by usingby using the standard evaluation criteria, this impact the standard evaluation criteria, this impact evaluation will:evaluation will:
• Empirically assess the welfare change of the Empirically assess the welfare change of the beneficiaries that can be attributed to the housing beneficiaries that can be attributed to the housing finance componentfinance component
• Identify factors (social, economic, project design and Identify factors (social, economic, project design and implementation) influencing the project outcomesimplementation) influencing the project outcomes
• Propose a sensible set of outcome indicators and Propose a sensible set of outcome indicators and benchmarks that can be used in future project benchmarks that can be used in future project design, monitoring and evaluationdesign, monitoring and evaluation
Evaluation FrameworkEvaluation Framework
Hypothesis: Hypothesis:
Improved housing conditions will lead to increased Improved housing conditions will lead to increased labor productivity, income, and overall social well-labor productivity, income, and overall social well-being of the project beneficiaries.being of the project beneficiaries.
Logic Model:Logic Model:
Inputs Inputs Activities Activities Outputs Outputs Outcomes Outcomes ImpactsImpacts
Impact IndicatorsImpact Indicators
Follow IADB study (2008) and Field and Kremer (2006)Follow IADB study (2008) and Field and Kremer (2006)
Household-level outcomes:Household-level outcomes:• Housing quality index (HQI),Housing quality index (HQI),[1][1]
• Per capita household consumption expenditure (per year),Per capita household consumption expenditure (per year),• Household completeness (presence of spouse and formally Household completeness (presence of spouse and formally
married),married),• Occupation ratio (percent of working household members),Occupation ratio (percent of working household members),• School attendance (of school age children), andSchool attendance (of school age children), and• Nourishment ratio (percent of children under 6 who are not Nourishment ratio (percent of children under 6 who are not
under-nourished).under-nourished).
[1][1] , where , where aaii equals unity if the house has condition equals unity if the house has condition ii, and zero, and zero
otherwise; andotherwise; and i i runs through the seven dwelling quality indicators: potable runs through the seven dwelling quality indicators: potable water access, sewage connection, electricity connection, walls, floors, ceilings, water access, sewage connection, electricity connection, walls, floors, ceilings, and overcrowding problems (more than 2 persons living per room).and overcrowding problems (more than 2 persons living per room).
i
iaHQI7
Impact Indicators (Cont.)Impact Indicators (Cont.)
Community-level outcomes:Community-level outcomes:
• Poverty rate (percent of households below Poverty rate (percent of households below the poverty line),the poverty line),
• Housing shortage (percent of households Housing shortage (percent of households without a house),without a house),
• Loan default ratio, Loan default ratio,
• Crime rates, andCrime rates, and
• Net migration (difference between migration Net migration (difference between migration in and out).in and out).
Estimation MethodsEstimation Methods
Regression Discontinuity Design:Regression Discontinuity Design:
Before the treatment After the treatmentBefore the treatment After the treatment
+++++++++++ ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
********************
******************
***************
**********
Y
Participants Non-participants
+++++++++++ ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
****************************
***************
*******************
************
Y
Participants Non-participants
Cutoff ScoreCutoff Score
RDD (cont.)RDD (cont.)
The program impact is estimated by the mean difference The program impact is estimated by the mean difference in outcomes for persons above and below the cut-off point. in outcomes for persons above and below the cut-off point.
Probability of BorrowingProbability of Borrowing
Before treatment:Before treatment:
After treatment:After treatment:
.55 income
.55 income
Expected OutcomeExpected Outcome
Before Treatment:Before Treatment:
After Treatment:After Treatment:
.55 income
.55 income
Fuzzy Regression Discontinuity Fuzzy Regression Discontinuity DesignDesign
Estimator:Estimator:
where where EE[.] is the expectation operation; [.] is the expectation operation; xx* is the cutoff (i.e., the 55th income * is the cutoff (i.e., the 55th income percentile); percentile); YYii and and XXii are the outcome and forcing (treatment-determining) are the outcome and forcing (treatment-determining) variable of household variable of household ii, respectively; and is the assignment variable , respectively; and is the assignment variable with 1[.] being the index function taking value 1 if the condition in the square with 1[.] being the index function taking value 1 if the condition in the square brackets is correct, and zero otherwise. The denominator represents the jump in brackets is correct, and zero otherwise. The denominator represents the jump in borrowing probability due to treatment assignment.borrowing probability due to treatment assignment.
We will follow We will follow Imbens and Lemieux (2008)Imbens and Lemieux (2008)[1][1] to use the local linear regression to use the local linear regression estimation method, where both the difference in the outcome (numerator) and estimation method, where both the difference in the outcome (numerator) and the difference in the borrowing probability (denominator) will be estimated by the difference in the borrowing probability (denominator) will be estimated by the fitted values of the fitted values of YY and and WW at both sides of the cutoff. at both sides of the cutoff.
[1][1] Imbens, G. and T. Lemieux. 2008. Regression discontinuity designs: a guide to practice. Imbens, G. and T. Lemieux. 2008. Regression discontinuity designs: a guide to practice. Journal of Journal of EconometricsEconometrics, vol. 142 (2): 615 – 635. , vol. 142 (2): 615 – 635.
] x X | W[ E lim - ] x X | W[ E lim
] x X |Y [ E lim - ] x X |Y [ E lim
i i* x↓x
i i* x↑x
i i* x↓x
i i* x↑x
*][1 xXW ii
Other Estimators: PSM and DDOther Estimators: PSM and DD
The RDD estimator gives an unbiased estimate The RDD estimator gives an unbiased estimate of the project impact. However, it only estimates of the project impact. However, it only estimates the project effect near the cutoff. the project effect near the cutoff.
If the project effect is constant, this poses no If the project effect is constant, this poses no problem. problem.
To give an estimate of the overall average To give an estimate of the overall average treatment effect, we will also use the propensity treatment effect, we will also use the propensity score matching (PSM) method combined with a score matching (PSM) method combined with a double/single difference estimator, pending data double/single difference estimator, pending data availability and quality. availability and quality.
Data RequirementData Requirement
Province Province TreatmenTreatmen
ttComparisoCompariso
nn
CommunitCommunityy
SurveySurvey FGDFGD KIIKII
SouthernSouthern 495495 495495 1010 1010
CentralCentral 420420 420420 1010 1010
WesternWestern 300300 300300 1010 1010
North CentralNorth Central 150150 150150 1010 1010
SabaragamuSabaragamuwawa 135135 135135 1010 1010
TotalTotal1,5001,500 1,5001,500 5050 5050
101000
Household SampleHousehold Sample
TimeTime Treatment GroupTreatment Group Comparison GroupComparison Group
19981998 T0 = 1,500 households with T0 = 1,500 households with income below the cutoff; income below the cutoff; households actually households actually borrowed will be drawn from borrowed will be drawn from client profiles in PCIs’ client profiles in PCIs’ databases; households that databases; households that did not borrow will be drawn did not borrow will be drawn from household profiles kept from household profiles kept in the village statistics in the village statistics archivesarchives
C0 = 1,500 households with C0 = 1,500 households with income above cutoff drawn from income above cutoff drawn from household profiles in the village household profiles in the village statistics archivesstatistics archives
20102010 T1 = the same 1,500 T1 = the same 1,500 households of T0 to be households of T0 to be surveyed for the studysurveyed for the study
C1 = the same 1,500 households C1 = the same 1,500 households of C0 to be surveyed for the of C0 to be surveyed for the study study
Implementation PlanImplementation Plan
MilestonesMilestones OCTOCT NOVNOV DECDEC JANJAN FEBFEB MARMAR APRAPR MAYMAY JUNJUN JULJULAUAUGG SEPSEP
PreparationsPreparations
Data CollectionData Collection
DataData AnalysisAnalysis
Drafting ReportDrafting Report
Review/RevisionReview/Revision
CompletionCompletion
Thank you.Thank you.
For more information:For more information: http://www.adb.org/Evaluation/resources.asphttp://www.adb.org/Evaluation/resources.asp