Date post: | 16-Dec-2015 |
Category: |
Documents |
Upload: | rosemary-dawson |
View: | 216 times |
Download: | 0 times |
DIME – FRAGILE STATESDUBAI, MAY 31 – JUNE 4
Sudan Community Development Fund:Preliminary Slice I Impact Evaluation Results and Needs for Future Evaluations
Abdulgadir Turkawi, Krishna Pidatala, Tei Fujiwara, Ryan Sheely
Community Development Fund (CDF) National program that uses CDD approach to rapidly provide basic social infrastructure and services to war-affected and underdeveloped areas of North Sudan – i.e. In 4 states out of 15 states
Two Phases Slice-1 (2006 -2008) - $25 million Slice-2 (2008 -2011) - $50 million
CDF – Project Background
Project Background (2)
Slice I (2006-2008) - US$ 25 million 4 states out of 15 states; 10 neediest
localities within these 4 states; 20 communities per locality = 200 total communities
Slice II (2008-2011) - US$ 50 million 4 states out of 15 states; 6 more
communities added to the original 10 Slice-I localities; 19 new localities added in the 4 states in Slice-II to increase coverage within the states.
Total Slice-I communities = 260 Total Slice-II communities = 380
Implementation Overview Baseline 1 – May 2007 (3 treatment
communities per locality; 2 control communities per locality; 27 random Households in each community )
Follow-up Survey/Baseline Survey 2 – June 2008
Follow-up survey for Slice1 - panel survey for same households as in baseline 1 for Slice1 ;
Baseline survey for Slice2 - 4 treatment communities per each new locality; 2 control communities per each new locality; 24 random Households in each community
Going Forward – Final survey expected in 2011
Evaluation Strategy and Survey Design Targeting – based upon poverty &
population assessments; 20 lowest ranked communities in each locality were selected to receive the CDF program. 21st and 22nd lowest communities in each locality picked as the control group
Treatment & Control groups – selected communities with similar characteristics
Cluster Random Sampling – Households in Treatment and Control groups were selected randomly
Selection of Treatment Communities
Total of 20 Communities per Locality Needed to Ensure that there was at least one
treatment Community per Administrative Unit Within each Administrative Unit, communities
were ranked based on poverty, population, availability and condition of Basic Services and Population
Number of Communities Chosen for Treatment Per Administrative Unit - based on above ranking (Poverty, population, availability & condition of basic infrastructure & services)
Selection of Control Group
To construct a control group, all communities within each locality were ranked based on the number and condition of basic services and population
The 21st and 22nd communities on the list were selected as a control group These were the communities that were
not selected that were most similar to the selected communities
Survey Methodology
Community and Household Questionnaires 50 Communities Chosen from all 10
Localities – 5 from each locality 3 Treatment Communities Randomly
Chosen From Each Locality 2 Control Communities From Each Locality
Selection of Households 27 Households Randomly Selected from
Each of the 5 Communities Sampling Frame-Household Lists where
available, “Spin the Pen” method where not available
Evaluation Questions
The Slice 1 Baseline and Follow-Up Surveys were designed to assess the overall effectiveness of the project at meeting its objectives:
Measurements : Access to Education, Health, and Water? Good Governance? Participation and Social Capital?
Results of Slice I Impact Evaluation
Results Estimated Using Difference-in-Difference Approach
Education Gains in enrollment, reduction in
dropouts▪ Decline in female dropouts
Increases in number of classrooms, toilets, benches, and teachers dormitories
Treatment communities 34% more satisfied with education after intervention, compared to control group
Results of Slice I Impact Evaluation Health
Fewer statistically significant increases in health center functionality
Increase in frequency of health center visits Satisfaction with health facilities significantly
increased Water
Fewer statistically significant increases in water quality
Increased consumption of water Increased Number of Pump sets Increased Satisfaction with Access to Water
Results of Slice I Impact EvaluationGovernance
Increase in Reported Rates of Leader Compliance with Community Needs and Leader Responsiveness
Decrease in Ease of Changing Leader Participation and Social Capital
Increase in Community’s Ability to Solve Development Problems
No significant increase in Participation in Community Activities or Meetings
Evaluation Challenges & Lessons Learned
Sample attrition – possibility that some households could have moved by 2011
Data Management – difficulty in matching of some households from baseline & follow-up surveys
Gender sensitivity & participation – 1st baseline survey did not have any female respondents. Addressed this shortcoming in 2nd baseline survey for Slice-II.
Survey questionnaires too long – Need to condense follow-up questionnaires. Interview takes 1 hour 15 minutes.
Looking forward – Impact Evaluation
2011 - Final survey expected to be undertaken
Impact Evaluation – Need for continued support from DIME for facilitation & technical expertise to the project
Phase-II – Dependent upon the Referendum in January 2011 and the political landscape there after. We expect a more rigorous IE design for the next phase/project. Evaluation design - possible sub treatment
interventions Survey design and management Sampling
Questions & Feedback Needed:Slice II Follow-Up Survey
Budget constraints – For the final survey, should we reduce – (a) the number of households per community ,or (b) the number of communities per locality ?
Preparing for Future Impact Evaluations
Build Local capacity – involve local counterparts in IE technical design & analysis (as far as possible)
Gender Sensitivity/Participation – IE expert on team to be a woman (based upon past experience)
Focus on Project/Program – Develop project/program questions to be answered by IE
National Statistics Bureau – Look at possibility to involve them in some way to build their capacity (most projects ignore them)
DIME Support– need continued support from them. DIME to provide technical expertise, oversee IE work, ensure quality of work, etc.
Questions For Future Evaluations
What is the effect of installation of solar electricity on health, education, and security outcomes?
What is effect of social accountability mechanisms on Infrastructure Functionality?
Due to implementation in progress, may not be able to evaluate until Phase II
Will look for opportunities for evaluations in Slice II
THANK YOU