Post on 26-Jul-2020
transcript
Amanda Beatty, Emilie Berkhout, Luhur Bima, Thomas Coen, Menno Pradhan, Daniel Suryadarma
15 Years of Education in Indonesia: Rising Enrolment and Flat Learning
Profiles
21 June 2018
Education expenditures as a percentage of total government expenditureshave almost doubled
Equivalent to a threefold increase in total education expenditures inreal termsNOTE: Data not available for 2006.Source: World Bank DataBankDiop, Ndiame; Gil Sander, Frederico. 2018. Indonesia Economic Quarterly: Learning more, growing faster (English). Washington, D.C. : World Bank Group.
0 %
5 %
1 0%
1 5%
2 0%
2 5%
2 00 1 2 00 2 2 00 3 2 00 4 2 00 5 2 00 6 2 00 7 2 00 8 2 00 9 2 01 0 2 01 1 2 01 2 2 01 3 2 01 4 2 01 5
Perc
enta
ge o
f gov
ernm
ent e
xpen
ditu
res
Primary school enrolment has been universal, while secondary school enrolment has been rising
Source: IFLS 3, 4 and 5
0 %
1 0%
2 0%
3 0%
4 0%
5 0%
6 0%
7 0%
8 0%
9 0%
1 00 %
P rima ry Ju n io r se co n d ar y S en io r s ec o nd a ry
Perc
enta
ge e
nrol
led
School level
2 00 0 2 00 7 2 01 4
Getting to OECD levels in PISA will take generations
300
350
400
450
500
550
2000 2015 2030 2045 2060 2075 2090M ath Rea ding Pr ojectio n: M athPr ojectio n: Re ading OECD Ave rage: M at h OECD Ave rage: R eading
Source: World Bank, World Development Report 2018: LEARNING to Realize Education’s Promise
TIMSS results even show a negative trend
300
350
400
450
500
1999
2000
2001
2002
2003
2004
2005
2006
2007
2008
2009
2010
2011
2012
TI MSS math score (non-Islamic schools)TI MSS math scoreLinear (TIMSS math score)
500=International mean
We present Indonesian learning profiles
• We seek to better understand the Indonesian learning
crisis by assessing learning by grade
• This study and Afkar et al. (forthcoming) are first to show
learning profiles for Indonesia
• Afkar et al. (forthcoming) use school-based test in 2011 and
2012
• We find flat learning profiles using an almost nationally
representative dataset covering 2000 to 2015
• Our findings are consistent with the results of PISA,
TIMSS and Afkar et al. (forthcoming)
The Indonesia Family Life Survey allows us to generate learning profilesfor numeracy skills
• Indonesia Family Life Survey (IFLS): panel survey in 2000, 2007
and 2014 representative of 83% of Indonesian population
• Two sets of multiple choice numeracy tests, covering Grades 1 –
5 curriculum
• Correct for guessing: ! = 1 − % × '( + %×1
• Substantial group answered both versions of the test
• Those above 14 years old who answered the easy version in the
previous survey round
• About 60 percent of 15 year olds+ respondents
Test items for 7-14 y.o. Grade level49-23 1
267+112-189 2
(8+9)*3 3
56/84 4
1/3-1/6 4
Test items for >=15 y.o. Grade level5684
4
(412+213):(243-118) 3
0.76-0.4-0.23 4
(100-65)% of 160 million (in text) 5
5% interest on Rp. 75,000 (in text) 5
Little learning between the age of 7 and 14
Source: IFLS 5
0 %
1 0%
2 0%
3 0%
4 0%
5 0%
6 0%
7 0%
8 0%
9 0%
1 00 %
7 8 9 1 0 1 1 1 2 1 3 1 4
Perc
enta
ge c
orre
ct
Age
4 9-2 3 2 67 + 11 2 -1 89 (8 +9 )* 3 5 6/ 84 1 /3 -1/ 6
Those above 14 years old still struggle with the easiest questions
0 %
1 0%
2 0%
3 0%
4 0%
5 0%
6 0%
7 0%
8 0%
9 0%
1 00 %
1 5 1 6 1 7 1 8 1 9 2 0 2 1 2 2 2 3 2 4 2 5 2 6 2 7 2 8
Perc
enta
ge c
orre
ct
Age
4 9-2 3 2 67 + 11 2 -1 89 (8 +9 )* 3 5 6/ 84 1 /3 -1/ 6
Source: IFLS 5
Grade level competency of 18-28 y.o. lags far behind curriculum
0 %
1 0%
2 0%
3 0%
4 0%
5 0%
6 0%
7 0%
8 0%
9 0%
1 00 %
P rima ry Ju n io r se co n d ar y S en io r s ec o nd a ry o r h ig h er
Perc
enta
ge a
t gra
de le
vel c
ompe
tenc
y
Highest school level enrolled or completed
< g ra d e 1 g rad e 1 g rad e 2 g rad e 3 g rad e 4 g rad e 5
Source: IFLS 5
We calculate one numeracy score over grades
1. Impute missing values
2. Item Response Theory using 2 parameter logistic modelto generate a numeracy score
• Takes into account difficulty levels and discrimination power• Use group that answered both versions for test equation• Predict probability of correct answer for each item
3. Take mean of probabilities4. Correct for guessingInterpretation: Mean probability of knowing the answer toany of the items
IFLS 5 7–14 15–30
Percentage generated with at least one imputed item 17.9 8.2
Flat learning profiles irrespective of the imputation method for currentlyenrolled students
Source: IFLS 5
0 %
1 0%
2 0%
3 0%
4 0%
5 0%
6 0%
7 0%
8 0%
9 0%
1 00 %
1 2 3 4 5 6 7 8 9 1 0 1 1 1 2
Prob
abili
ty o
f kno
win
g th
e an
swer
to a
ny it
em
Currently enrolled school grade
Im pu ta tion - no n -m is sing i tem s a n d R av e n Up p er b o u nd Lo w er b o un d
Similar findings for 18-30 y.o.
Source: IFLS 5
0 %
1 0%
2 0%
3 0%
4 0%
5 0%
6 0%
7 0%
8 0%
9 0%
1 00 %
< P rim ar y P rima ry < J un io r
se co n d ary
Ju n io r
se co n d ary
< S e nio r
se co n d ary
> = S en io r
se co n d ary
Pro
ba
bil
ity
of
kn
ow
ing
th
e a
nsw
er
to a
ny
ite
m
Education completed
Im pu ta tion - no n -m is sing i tem s a n d R av e n Up p er b o u nd Lo w er b o un d
Numeracy skills deteriorated between 2000 and 2014 for currently enrolled students in all grades
2000 2014
Mean 38.8 33.6
Coefficient -5.9 (p=0.000)
0 %
1 0%
2 0%
3 0%
4 0%
5 0%
6 0%
7 0%
8 0%
9 0%
1 00 %
1 2 3 4 5 6 7 8 9 1 0 1 1 1 2
Prob
abili
ty o
fkno
win
g th
e an
swer
Currently enrolled school grade
2 00 0 2 01 4
Deteriorating numeracy skills of 18-24 y.o. confirm downward trend
2000 2014
Mean 31.2 31.4
Coefficient -3.2 (p=0.000)
0 %
1 0%
2 0%
3 0%
4 0%
5 0%
6 0%
7 0%
8 0%
9 0%
1 00 %
< P rim ar y P rima ry < J un io rse co n d ary
Ju n io rse co n d ary
< S e nio rse co n d ary
> = S en io rse co n d ary
Prob
abili
ty o
f kno
win
g th
e an
swer
Education level completed
2 00 0 2 01 4
Conclusions
• Flattening learning profiles• Numeracy skills did not improve between 2000 and
2014
• Limitations• Instrument contains few items• Respondents of a household survey might not take the test
seriously
• Robustness checks do not reject our results• Findings in line with literature• Children enrolled in primary school in IFLS data mostly
score better than enrolled children in Afkar et al. test data (BERMUTU, 2011)
Expenditures on education have increased threefold between 2000 and 2015 in real terms
NOTE: LHS IDR trillion, RHS percentage of GDP and spending
Source: Diop, Ndiame; Gil Sander, Frederico. 2018. Indonesia Economic Quarterly: Learning more, growing faster(English). Washington, D.C. : World Bank Group.
The instrument has acceptable validity, but would benefit from more items
Validity• Unidimensional based on factor analysis
Reliability• Cronbach’s alpha is slightly too low (0.67, at least 0.7 preferred)
• Shows need for more items, as item-test correlations are between 0.42 and 0.63