Girls’ scholarship program
Often small/no impacts on actual learning in education research◦ Inputs (textbooks, flipcharts) little impact on learning◦ De-worming affected attendance but not test scores
What is often most important in education policies and programs? Incentives
What happens if we offer direct incentives for student learning?
What happens if only offer this for a disadvantaged subgroup? Girls
The debate over cash incentives◦ “Pros”
Incentives to exert effort Helps with self-control problems Externalities to effort
◦ Possible “cons” Exacerbate inequality Weaken intrinsic motivation (short or long run) Gaming the system (cramming, cheating)
Merit awards could affect◦ Eligible students’ own effort◦ Other students effort & teacher effort could be either
complements or substitutes
The Girls Scholarship Program Randomized evaluation in Kenyan primary schools
63 treatment & 64 comparison schools Balanced treatment groups
Announced an award for girls in treatment schoolsBased on end of year standardized test scoresTop 15% of grade 6 girls in program schools win award
1000 KSh (US$12.80) for winner and her family500 KSh (US$6.40) for school feesPublic recognition at an award ceremony
Two cohorts of scholarship winners, 2001 & 2002 Survey data on attendance, study habits, attitudes
Program implemented in two districts: Teso & Busia
Randomization and awards stratified by district◦ Historical and ethnic differences in the two
districts◦ NGOs have poor relations with some Teso
communities◦ Tragic lightning incident early 2001
C
CC
CC
C
C
CC
C
CC
CC
C
C
C
C
C
C
C
CC
C
C
CC
CCC
C
C
CC
C
C
C
CC
C
C
CC
C
C
C
CC
CC
CC
C C
C
C
C
C C
C
C
C
C
C
T
T
T T
TT
T
TT
TT T
TT
T
T
T
TT
TTT
T T
TT
TTT
T
T
T
T
T
T
T
TT
T
TT
T
T
T
T
T
T T
T T
T T
T
T
TT
T
T
T
TT
T
T
T
r
ÿ
GSP DistrictsTesoBusia
School Attrition
School Pulled Out
GSP SchoolsC ComparisonT Treatment
Effects of Lightning
ÿ Lightning
r Winner Refused
10 0 10 20 Miles
N
EW
S
School attrition: five Teso schools pulled out in 2001
Test attrition: treatment vs. comparison with complete 2001 data:◦ Teso 54% vs. 66%, Busia 77% vs. 77%
Differential test attrition: significantly more high-achieving students took the 2001 exam in comparison schools relative to program schools, likely to bias estimated program impacts toward zero in Teso
How to deal with it?Make sure that no one drops out from your original
treatment and control groups. If there is still attrition…
Check that it is not different in treatment and control. Also check that it is not correlated with observables.
If there is differential attrition1.Impute outcome variable based on baseline covariates2.Bounds: run the analysis under the “best-case” and
“worst-case” scenario. Either the best or the worst students are the ones that drop out at a rate that is equal to the rate of differential attrition
Estimate total effects and district effects Estimate effects for treatment schools (T):
istsistsist XTTEST
Cheating is likely not a concern Evidence of “learning”: consistent effects
over two years and two cohorts No effect on tutoring, household textbook
purchases, self-esteem, attitudes toward school, amount of chores at home
Teachers report more parental support in Busia
Student and teacher attendance increased
Important to think through programmatic issues when designing interventions – incentives must be aligned (teachers, parents, students…)
Randomizing by school can help to pick up within class/school externalities
Things can go wrong – need to monitor attrition Large and persistent gains in learning are
possible to achieve
What if instead of linking student performance to students, we made the teachers responsible?
Randomized evaluation in Kenya (Glewwe, Ilias and Kremer (2004))
Offered teachers prizes based on schools’ average scores◦ Top scoring schools and most improved schools
(relative to baseline)◦ Each category 1st, 2nd, 3rd and 4th prizes were
awarded (21% to 43 % of teacher monthly salary)◦ Penalized teachers for dropouts by assigning low
scores to students who did not take the exam
What was affected:Treatment scores 0.14 sd above controlStrongest for geography, history, and religion (most
memorization)Exam participation roseExtra-prep sessions
What was NOT affected:Dropout/ repetition ratesTeacher attendance Homework assignment or pedagogyLasting test score gains
Conclusions:Teachers’ effort concentrated in improving short-run
outcomes, rather than stimulating long-run learning
Busia: Overall (0.18 – 0.20 s.d.)◦ Persistent effect for girls the next year◦ Spillover effect for boys
Teso: Scholarship less successful: either no significant program effect or unreliable estimates
Merit-based scholarships can motivate students to exert effort◦ Test score and attendance gains among girls in the medium-run
Positive classroom externalities◦ Initially low-achieving girls, boys, and teachers
Possible multiple equilibria in classroom culture Cost-effective way to boost test scores Equity concerns – may wish to restrict to particular areas
or populations