Impact of ICTs-trained school teachers on educational outcomes: evidence from
Colombia.
Fabio Snchez
Tatiana Velasco
School of Economics
Universidad de los Andes
March, 27th.
2015
Since 2004, Computadores para Educar, a Colombian nationwide program that equipped
schools with computers giving access to technology to the students, has also trained
teachers in ICT usage for instruction. By 2014, over 41,000 schools of the 46,000 existing
in Colombia had received computers by the program. Meanwhile, of the 386,000 teachers
in the public school system, about 74,000 have been trained in ICTs. This research aims to
assess the effect that these ICTs-trained teachers have on dropout rates and performance on
standardized tests. Particularly, we use a unique dataset that allow us to track students and
teachers in all the public schools of the country by year and school since 2005, and we
combined these data with information about the program implementation. We find that that
greater proportion of ICTs trained teachers in a particular subject improve the performance
of students in that subject (math, chemistry, physics, etc.) and reduce dropout. To correct
for endogeneity, we use as instrumental variable the proportion of ICT-trained teachers by
subject or school level in the neighboring municipalities in t-1. Our results indicate that an
increase in one standard deviation in the proportion of ICT-trained teachers within a school
level reduces students drop out by 0.20 standard deviations. Similarly, an increase in one
standard deviation of proportion of ICT trained teachers in a particular subject increases
students performance on standardized test in that subject by 0.70 standard deviations.
These results are evidence that what matters for educational performance is not student
access to ICT and computers but rather the training of teachers in the usage of ICT in the
classroom.
JEL Classification: I21, I28
Key words: ICT, teachers training, instrumental variables, drop out, standardized test
I. MOTIVATION
Nowadays, computer-based learning programs are a popular educational
intervention around the world (MacLeod, 2008). Nevertheless the evidence about whether
they improve or not educational outcomes is rather mixed. This is an issue, because many
of these programs combine two kind of interventions at the same time: they provide ICT
equipment to students and they train teachers on how to use this equipment. As a
consequence, when evaluating the impact of this particular group of programs, researchers
have failed in differentiating which of the interventions conduces the effect found.
This has been the case of Computadores para Educar (CPE). This is a Colombian
program that has provided ICTs to public schools in all the country since 2001. Since 2004,
the program included a formal training in ICT of 150 hours to teachers in public schools
that received CPE equipment. By 2013, 74,000 teachers had received the ICT training,
which represents 23% of the public schools teachers. The CPE program has been
empirically evaluated two times. The first one by Barrera-Osorio and Linden (2009) whom
did not find a significant effect of the program on educational outcomes, and the second
one by Rodriguez, Snchez and Mrquez (2011) whom found a significant impact of CPE
for reducing dropout and for improving Saber 11 performance and access to higher
education. In any of these cases, the researchers were able to property identify the channels
that conduced to the found result. Specifically, they did not explain which component of the
program conducted the effect: the ICTs, the teachers training, or both.
Literature on educational policy interventions had provided insights about the
elements of these mixed programs that may conduced the effect. For example, MacLeod
(2008) argues that while teachers training programs seem to have attracted less attention
than computer-based programs, [] the content of the training programs seems to matter
more for changes in student performance than the structure of the program itself.
(MacLeod, 2008:5). Kennedy (1998) reviewed several papers on teachers intervention
programs and concluded that those programs whose content focused in teachers
knowledge of the subject, on the curriculum, or on how students learn the subject are the
ones with the bigger impact on educational outcomes. Thus, we hypotheses that ICT-
training for teachers in the main channel that conduces CPEs impacts, because it aims to
provide pedagogical and practical tools to improve classroom and teaching practices as
teachers appropriate ICTs.
In this paper we attempt to identify the channel that conduces to CPE impacts on
educational outcomes. For that end, we combined instrumental variables with fixed effects
models to identify the causal effect of the ICT-trained teachers, isolated of the CPE
equipments effect, We find that an increase of one standard deviation in the proportion of
ICT-trained teachers within a school conduces to a significant impact in dropout rate, grade
retention rate and performance. Particularly, it reduces dropout rate in 0.20 standard
deviations, grade retention rate in 0.76 standard deviations and increases Saber 11
performance in 0.7 standard deviations. Our preliminary robustness checks indicate that the
CPE impacts goes through teachers training and not through CPE equipment.
This paper is organized as follows. Section II. Presents the literature review that
frameworks our research question, section III. Provides a description of CPE program, with
a particular focus on the ICT training of teachers, section IV describes the empirical
strategy used for this papers, section V describes the information sources we used and how
we processed them, section VI present our results and section VII presents our preliminary
conclusions.
II. LITERATURE REVIEW
There is an extensive literature about ICT interventions both in developing and
developed countries with mixed results on students performance. However, teacher
training programs had attracted less attention than ICT interventions in developing
countries, while the evidence is mixed in developed countries (He, Leigh & MacLeod,
2008). In a meta-analysis conducted by Kennedy (1998), it is concluded that the content of
the training programs is more important than the structure of the program itself for changes
in student performance (He, Leigh & MacLeod, 2008). Thus, we divide the literature
between two big groups: those based on programs with only ICT interventions/dotation and
those based on programs that include both ICT dotation and teacher training.
In the group of programs with only ICT interventions/dotation, there is an extensive
literature with mixed results on students performance. For example, Angrist & Lavy
(2002) show, using a 2SLS estimation, that a large-scale program in Israel that provided PC
to elementary and middle schools between 1994 and 1998 had a positive effect on computer
use by the students but it had no effect for eight graders neither in math nor Hebrew, and
even a negative effect for fourth graders in math. Also, Rouse & Krueger (2004) evaluate
the short-term effect of a well-defined use of a computer program in the United States
known as Fast ForWord. They used OLS and IV estimation and found that there was no
statistically significant impact on the language and reading skills of the students.
Conversely, other programs had found positive effects of ICT
interventions/dotation. For example, Banarjee, Cole, Duflo & Linden (2005) use a
difference-in-difference estimation to evaluate a computer-assisted learning (CAL) program
focused on children at the bottom of the class in Vadodara, urban India. They found that the
program increased treatment math scores by 0.35 standard deviations the first year, and
0.47 the second year. In the same way, Barrow, Markman & Rouse (2008) used an IV
approximation and found positive and statistically significant effects of 0.25-0.42 standard
deviations on students performance in pre-algebra and algebra test scores of a program
known as I Can Learn in three urban districts in The United States. As well, Machin,
McNally & Silva (2007) found positive effects on English and Science test scores of
computers dotation in the United Kingdom using, as a natural experiment, a policy change
in the UK in 2001. Finally, Fuchs & Woessman (2004,) based on observational data,
concluded that once it is controlled for family background and school characteristics, there
is an inverted U-shape relationship between student achievement and computer/internet use
at school.
In the group of papers that evaluate programs that include both ICT dotation and
teacher training, there is less literature and it is also mixed. In this literature, programs were
integrated with a teacher training that sought for the appropriation of the ICT on teaching
and learning. On the one hand, Barrera-Osorio & Linden (2009) found no statistically
significant effect in students performance of random assigned program Computadores
para Educar (same program evaluated in this paper), which aimed to integrated computers
into teaching of language in public schools in Colombia. Also, Sharma (2014) evaluates the
One Laptop per Child program in Nepal that gave training to teachers about how to teach
using laptop-based materials. Sharma (2014) used a difference-in-difference estimation and
found no statistically significant impact on math and a negative impact on English.
On the other hand, Rodrguez, Snchez & Mrquez (2011) evaluate a large-scale
version of Computadores para Educar in Colombia (same program evaluated in this
paper) with an OLS and IV approximation. They found that the program reduces drop-out
rates, increases standardized students test scores and increases the probability of accessing
higher education. In the same way, He, Linden & MacLeod (2008) evaluate an Indian
PicTalk program for teaching English to children in grades 1 to 5 that include both
machine-based implementation and activities based on flash cards and teacher manuals and
training. They found positive effects of PicTalk program and that lower performing
students benefit more from interventions with activities implemented by teachers, while
higher performing students benefit more from interventions with self-paced machines only.
However, all this literature has failed to identify in a casual and empirical way the
channels through by which those effects took place. Nevertheless, some of the papers
mentioned above had some intuitive reasons about why their specific programs had no
impact. For example, Rouse & Krueger (2004) argues that the absence of impact may be
due to that teachers failed in learning how to use ICT to enhance instruction in an effective
way. Barrera-Osorio & Linden (2009) say that the CPE program failed to integrate
computers and the educational programs because the teachers did not incorporate the
computers into the curriculum. Finally, Rodrguez, Snchez & Mrquez (2011) conclude
that access to technology is effective only if it comes together with teacher's formation
processes.
This latter idea is important for two reasons. First, the literature about teacher
formation has found positive effects on student performance. For example, Banarjee et. Al
(2005) evaluates a remedial education program in urban India, in which a local young
woman (balsakhi) teaches basic skills to lagged children. They use a difference-in-
difference evaluation and find a positive impact on test scores of 0.14 standard deviations
the first year, and 0.28 standard deviations the second year.
Second, the evaluation of Computadores para Educar made in this paper identify
the channel through the program has the positive effects we observe. So, this paper is able
to identify whether the impact observed is due to computer dotation or teachers training.
III. COMPUTADORES PARA EDUCAR AND THE ICT-TRAINING COURSE
The Colombian Program Computadores Para Educar (CPE) began operating in
2000 under the leadership of the Ministry of Technologies of the Information and the
Communications - MTIC -. The main objective of CPE is to reduce the gap in access and
knowledge in ICTs of students of public schools in the country. For that end, CPE
undertakes various activities. Thus, CPE gathers and readapts laptops and desktops and
allocates the equipment to public schools1, provides ICT-training for teachers from those
schools and disposes electronic remains in order to reduce environmental impact.
By 2013, 82% of public schools in Colombia had received the equipment provided
by CPE. As shown in Figure 1 by 2005 only 9% of schools in the country had obtained
1 The schools selected must meet the following criteria: not having equipment, having equipment but out of
date and, having some equipment but not enough to cover the number of students attending to that school.
CPE equipment and by 2010 that number jumped to 45%. As a consequence, the number of
students receiving CPE equipment also increased over time. By 2005 percentage of senior
high schools students with CPE equipment was 25% and by 2010 such percentage reached
70%.
Since 2004, CPE reoriented its strategy by giving a fundamental role to the ICT-
training of teachers. Before 2004, CPE offered short instruction (about 20 hours) to
teachers and school staff on the basic usage and functions of the provided equipment.
Nonetheless, after 2004 the program authorities replaced such basic instruction for certified
training course offered to teachers of CPE schools.
The teachers ICT-training is a crucial stage of the program. It consists of a 150
hours course (120 hours of on-site classes and 30 hours of virtual classes) that provides
practical, theoretical and motivational skills thus teachers can efficiently incorporate ICTs
in their teaching practices. The course can be summarized in 3 steps: first, teachers learn
the basics of ICT-infrastructure management and usage. Once teachers finish this stage,
they certified as Digital Citizens. Second, teachers deepen their knowledge on ICTs
through theoretical and practical lectures. Here, they formulate and develop a classroom
project that incorporates ICT usage. The project must be related to the classroom
curriculum. Once the classroom project is properly formulated the course moves toward the
third stage. Here, teachers bring their project into classroom practice, receive feedback of
their students and proceed to evaluate the project. The course finish at the end of this stage
and teachers receive a certification as ICT-trained teachers. Then, they can present the
projects in regional and national meetings called Teach Digital where the best projects
are awarded.
In order to properly augment the number of ICT-trained teachers, CPE developed a
strategy at national level that encompasses four steps. First, CPE calls for universities to bid
for carrying out the ICT courses for teachers in eight Colombian regions previously
defined2. The selected universities are hired by CPE to offer the ICT-training course for
two years. Every year, and before the beginning of the course the selected universities and
CPE design the ICT-training strategy and curriculum. Then, the second step begins and a
team of experts from the universities visit the schools and the local educational authorities
in order to recruit teachers for the ICT-training course. Every year, each university must
meet a minimum number of trained teachers on its assigned region. This minimum number
has increased every year. The third stept is the execution ICT course itself described above.
The fourth and final step is the socialization of the projects in regional and national
meetings called Teach Digital. In these meetings, the teachers that finished the ICT-
training courses present their project and experiences to their colleagues. The best project in
every region receives an award and is invited to present their findings at the Teach
Digital national meetings.
As a consequence of the teachers ICT-training implementation, the proportion of
trained teachers across municipalities has increased over time and expanded across the
national territory. Figure 2 represents the proportion of ICT-trained teachers in each
municipality in 2005, 2008, 2011 and 2013. In 2005 the proportion of them was under 20%
for most of the municipalities growing steadily over the years. By 2013, most of the
2 In 2014 the 8 regions grouped the following departments. Group 1: Atlntico, Bolvar, Crdoba, San Andrs
and Sucre; group 2: Cesar, la Guajira, Magdalena and Norte de Santander; group 3: Caldas, Quindo,
Risaralda and Valle del Cauca; group 4: Arauca, Boyac, Casanare, Santander and Vichada; group 5;
Caquet, Guaviare, Huila and Tolima; group 6: Cauca, Nario and Putumayo; group 7: Amazonas, Cundinamarca, Distrito Capital, Guaina, Meta and Vaups; and group 8: Antioquia and Choc. Each group has one university in charge of the training except group 1 that has four universities because of the number of schools, students and teachers.
municipalities had at least 20% of their teachers with ICT-training and some had reached
over 60% .
Between 2004 and 2013 74,434 teachers received the ICT-training certification
under the described scheme. This represents nearly 24% of the teachers in Colombian
public schools. As the teachers self-select or the ICT-training course the ICT-trained
teachers might be different from non-trained in their observable characteristics. Table 1
presents by trained and non-trained the teachers characteristics for selected years. As
observed, the teachers who enroll in the ICT-training are younger, and exhibit lower
education level (secondary or less and normalista).
Moreover, the proportion of ICT-trained teachers differs by education level and
subject. Figure 3 and 4 depicts the proportion of teachers by level and subject respectively.
Thus, figure 3 shows that a growing proportion of ICT-trained teachers teaches in primary
rather than in secondary school. Graph 4 indicates there is a significant variation of the
proportion of ICT-trained teachers by subject and year. Physics, math and social sciences
have the biggest proportion while philosophy has the lowest.
IV. DATA
The data used in this paper comes from four different sources, organized in two
different ways: the first one for dropout and grade retention measures and the second for
performance measures. In the following, we proceed to explain each of our sources and
then, we explain how we organized them in order to apply our proposed identification
strategy.
Data sources
Our main source is the administrative information of CPE program. CPE authorities
have meticulously gathered information of the three strategies of CPE implementation: the
gathering, readaption and allocation of laptops and desktops to public schools; the certified
ICT course to teachers; and the management of electronic remains. For this paper we
focused on the first two sources. From each one of them, we can track the year in which
each school received CPE equipment and if it have been replaced or thicken within a
certain school. We can also identify each ICT-certified teacher and the year in which that
teacher received the certification. Both information sources are available since the
beginning of each strategy: 2001 and 2004, respectively.
Second, we use the information of the Annex 6a from the Resolution 166 gathered
by the National Ministry of Education. This is administrative data available since 2005
until 2013 that yearly gathers information of every student in the public schools system.
Annex 6a provides information of students school and grade. It also provides socio-
economical information such as sex, age socioeconomic strata and mothers education.
With this information, we are able to identify whether a student dropped out or was retained
in a grade for a certain year.
Our third source of information allows us to characterize teachers. In this case, we
use the Annex 3a data also from the Resolution 166 which is available since 2008 until
2013. This source yearly gathers information of every teacher in the public schools system.
Annex 3a allows us to identify the school in which the teacher teaches each year, the level
in which she teaches and the subject she teaches. It also provides information of teachers
education level, age, sex and date of hired.
Finally, we use information of the Colombian standardized test Saber 11. This test is
compulsory for students finishing the last year of secondary education. The exam evaluates
students performance in math, language, social science, biology, chemistry, physics,
English and biology and includes an elective subject chosen by the student or the school.
Additionally, it gathers socioeconomic information of the student such as age,
socioeconomic strata and mothers education. This test also allows us to identify the
student's school. Saber 11 is available since 2000 until 2012.
Dataset for estimation of ICT teachers training impact
For dropout and grade retention estimations we used the Resolution 166 datasets for
students, teachers and CPE intervention. First, each dataset is reduced to one register by
educational level (either primary or secondary), year and school. Thus, we obtained
averages of the students and teachers characteristics, and dropout and grade retention rates.
Then, we combined the three datasets using the school id. As a result, we obtained a panel
by educational level, school and year. Thanks to the administrative information for CPE,
we can identify the exact year in which CPE entered the school and the exact year in which
each teacher was certified as ICT-trained. As result, we observed the average dropout and
grade retention by school, educational level and year between 2005 and 2013. One
important feature of this data is that we can identify whether a teacher has remained in the
school where she received the ICT-training or moved to another school, with or without
CPE equipment.
For the estimations for performance in Saber 11 we used the Resolution 166 dataset
for teachers, the dataset for Saber 11 and again, the dataset of CPE intervention. In this
case, we reduced each dataset to one register by subject area (or taught area in the case of
the dataset for teachers), year and school. Thus, we obtained averages of the students and
teachers characteristics, and average performance in each subject of the Saber 11 test. Then,
we combined the three datasets using the school id. As a result, we obtained a dataset that
allows us to track schools since 2004 until 2013.
In both cases, we made an important adjustment to the dataset for teachers. As
explained before, this dataset is available only since 2008. But, the ICT-training began in
2004 and the information of CPE allows us to identify the teachers that were trained since
that year. Thus, between 2004 and 2007, we fixed the total number of teachers in each
school level or area with the total number of teachers in 2008. Under this assumption for
the information between 2004 and 2007 is how we calculated the proportion of ICT-trained
teachers.
V. EMPIRICAL STRATEGY
Our objective is to assess the impact of ICT-training for teachers on educational
outcomes. Particularly, we focus on three major outcomes: Dropout rate, grade retention
rate and subject performance on the secondary exit test Saber 11. Thus, we reduce our
sample to the public schools that have the computers of CPE program, because we are
interested in measure the effect of teachers ICT-trained isolated of the computers
endowment. Under this scenario, our initial framework is described by the following
equation:
In equation ,
represents dropout rate, grade retention rate or performance
on Saber 11 in school , in the year and in municipality . Our interest variable is
which represents the proportion of ICT-trained teachers in school
, in the year and in municipality . Also, we control for socio-economical characteristics
of the students. Thus,
represents the average age of students in school ,
in the year and in municipality . Similarly, represent the average proportion
of women and
represents the average education level of students
mothers. Furthermore, we include fixed effects at the school level ( and year fixed effect
( to capture difference that could emerge over time and across schools. We also include
CPE years at the schools fixed effects ( to capture all non-observables
that may change along the CPE intervention. Finally,
represents the error term.
Nevertheless, this model has endogeneity issues that restrict us of obtaining
unbiased estimators. The main issue is that ICT-trained and non-trained teachers coexist
within schools. Furthermore, teachers self-enrolled in the ICT-training course which means
that ICT-training is non-random within teachers. Thus, although equation allows us to
control for many of the unobserved factors that may vary between schools, this framework
does not correct by within schools variation. For example, if there is an unobserved change
in the school curriculums that affects students performance, we would not be able to
correct for it and we could be assigning the effect of that change to the ICT-training. As a
consequence, we would be obtaining a non-causal and biased estimator.
In order to control for within school variation, we follow Brutti and Snchez (2015)
and exploit two facts observed in figures 3 and 4: first, the proportion of ICT-trained
teachers varies between levels and within schools; and second, the proportion of ICT-
trained teachers varies within taught subjects. For the latter, we take advantage of the fact
that all students take a secondary exit exam that evaluates seven different subjects. As all
schools must teach at least these subjects to secondary students and each one is taught by
different specialized teachers, we include a within schools variation that accounts for the
ICT-trained teachers proportion in each of the evaluated subjects and the Saber 11
performance in each of the taught subjects. For the between educational levels case, even
though we cannot observed the exactly same student in primary and secondary level
simultaneously, we can use the within educational level variation to correct for
unobservable school characteristics that may change between educational levels.
For that end, we propose the following estimation framework suitable in a time
panel dataset by school and educational level, and by school and taught subject.
In the equation ,
represents dropout rate, grade retention rate or
performance on Saber 11. As mentioned before, one distinctive characteristic of this
framework is that it allows for variation within school. Depending on the outcome
measured, we observe our variable either at the educational level or at the taught subject
level. Thus, if represents dropout rate or grade retention rate, we say
is either of
these outcomes in school , in the year , in municipality and in the educational level .
Here, we take two levels into account: primary and secondary education. On the other hand,
if represents the performance on Saber 11, we say
is the average performance on
Saber 11 in school in the year in municipality and in the taught subject . In this case,
we take into account the same subjects measured by the Saber 11 test: Mathematics,
Physics, Chemistry, Language, English, Social Science, Biology and Philosophy. This
framework allows us to control for unobservable characteristics that may change within
schools.
Our interest variable is
. This variable represent the ICT-
trained teachers proportion in school , in the year , in municipality and in the
educational level or taught subject In the educational level case, we calculated the
proportion of ICT-trained teachers either at primary or secondary level in relation to the
total of teachers in the respective level, for the school , to year and in municipality . In
the taught subject case, we only take into account teachers at the secondary level as they are
the potential teachers of the senior students taking the Saber 11 test. Under this assumption,
we calculated the proportion of ICT-trained teachers in each taught subject in relation to
the total number of teachers in the respective subject, in school , to year and in
municipality .
As in equation , we control for socio-economic characteristics of the students
and we allow for variation by educational level. We preserved the fixed effects proposed
for equation and we added fixed effects by level or subject ( that control for
unobservable and constant differences across them. Finally,
represents the error term.
The empirical strategy presented in equation allow us to correct an important
part of endogeneity issues. But, we need to take into account other sources of endogeneity
such as self-regression. It is possible that, even after controlling for within school variation,
the proportion of ICT-trained teachers is highly correlated with the previous performance
by educational level and by taught subjects. In order to check if that is the case, we proceed
to present a set of endogeneity exercises that confirm that self-regression is an issue in the
equation (2). Specifically, Figure 5 and 6 present the correlation between dropout rate and
grade retention rate by educational level before CPE program entered to the school, with
the proportion of ICT-trained teachers by level. On one hand, there is a correlation between
the proportion of ICT-trained teachers by educational level and the previous to CPE
dropout rate. Particularly, the school levels with the lower dropout rates have a higher
proportion of ICT-trained teachers. Similarly, the school levels with higher grade retention
rate have a higher proportion of ICT-trained teachers. This issue is supported by the
correlations between the mentioned variables presented in table 1. Except for the
correlation between previous to CPE dropout rate in primary and proportion of trained
teachers in that level, it is observed that all correlations are statistically significant and with
a high magnitude t-statistic. Figure 6 confirms that the auto-regression issue also exists for
the Saber 11 estimation. Lower performance in a subject before CPE is correlated with a
higher proportion of ICT-trained teachers. The results in table 2 confirm the direction of the
correlation for all subjects except Social Science and Philosophy were the t-statistic is non-
significant.
In order to correct for the auto-regression and other endogeneity issues that are not
being taken into account at this point, we look for a source of exogenous variation in the
program design that allows us to explain the proportion of ICT-trained teachers and that is
not correlated with our outcomes. Here, we recall the ICT-training strategy description of
the section III. Once the universities that will offer the course in each region are selected,
they hire a team of consultants that must look for teachers to enroll in the course. Every
year, a number of enrolled teachers must be met and this number has increased over time.
Additional to figure 3 and 4 that exhibit how the number of trained teachers increases every
year, figure 2 describes how the ICT-training course has expanded geographically along
municipalities and years since 2005. Thus, we attempted to exploit that particularity of the
program. Specifically, we calculated the average years of experience as ICT-trained
teachers of the teachers in the neighbor municipalities by educational level and by taught
area till the previous year. We prefer this variable instead of the proportion of ICT-trained
teachers for two reasons: first, it captures the geographical expansion of the training as well
as the increased in the time of implementation; second it also captures increases in the
proportion of ICT-trained teachers, which allow us to control for possible peer-pressure to
enroll in the ICT course.
The average years of experience as ICT-trained teachers of the teachers in the
neighbor municipalities by educational level and by taught area till the previous year can be
a good instrumental variable if it accomplishes two minimum criteria: the instrument is
correlated with our variable of interest and it is not related with the outcome. Figures 7 and
8 indicate that the first criteria is met, because the correlation between the suggested
instruments and the endogenous variable is positive. Furthermore, the estimations presented
in the results section display the F statistics of the first stage estimation which is highly
significant. The second criterion is the exclusion restriction. Nevertheless, in the estimation
framework that we suggest, it is not feasible that the proportion of ICT-trained teachers in
the neighbor municipalities by level or subject till the previous year may explain a within
school difference either in dropout or grade retention rate, or Saber 11 performance by
subject.
As a result, our identification strategy can be summarized in the equations and
. The former, represents the first stage of our estimation. Thus,
represents the average years of experience as ICT-
trained teachers of the teachers in the neighbor municipalities by educational level
and by taught area till the previous year . Equation represents the second stage
estimation with the same controls described for equation .
VI. RESULTS
Effects of ICT-training on performance on Saber 11
First, we present the results of ICT-trained teachers on performance on Saber 11.
Thus, we estimated the equation (2) and (3) and present the results in table 4. The
coefficients must be interpreted as the average effect of ICT-trained teachers on each
subject within school. Our OLS results suggest that the ICT-training has no significant
impact on students performance. But, once we correct the endogeneity issues with the
instrumental variable, ICT-trained teachers affect students performance on Saber 11
positively and significantly. In order to facilitate the interpretation, we present the mean
and standard deviation of the outcome and of the Proportion if ICT-teachers. Thus, we
deduct that an increased in one standard deviation in the proportion of ICT-trained teacher
in a certain taught subject within a school increases students performance in that subject by
0.82 standard deviations.
As observed, the IV coefficient exhibits an important increased compared to the
OLS coefficient. This is due to the negative biased estimator which underestimates the
effect of ICT-training. The biased can be explained by the negative correlation between the
omitted variables and the proportion of ICT-trained teachers and the positive correlation
between the omitted variables and the outcome variable. The figure 7 and the table 3
illustrate the first part of this argument. Specifically, they represent the correlation between
the proportion of ICT-trained teachers by subject, with the average performance in Saber 11
by subject of the school previous to CPE. Even though this correlation represent only one
possible endogeneity problem self-regression -, it clearly illustrates the existence of sub-
estimation problems with the OLS model.
Dropout and grade retention
Tables 5 and 6 present the results of dropout and grade retention estimates of the
OLS and IV models. In this case, the estimated coefficients must be interpreted as the
average effect of ICT-trained teachers in a certain educational level and within schools.
Again, we use the information of the average and standard deviation of the variables in
order to simplify the interpretation. Thus, with the OLS coefficient we estimate that an
increase of one standard deviation in the proportion of ICT-trained teachers within a school
level, reduces the dropout rate in 0.013 standard deviation. But, when estimated using the
IV strategy, this effect increases to 0.20 standard deviations. This is also the case for the
grade retention estimates presented in table 6. While with the OLS we predict that an
increase of one standard deviation in the proportion of ICT-teachers within a school level
reduces the grade retention rate by 0.03 standard deviations, the effect when we estimate
the IV model is 0.71 standard deviations.
As for the Saber 11 performance estimates, we argue that the change in the size of
the coefficient is due to the endogeneity issues corrected with the instrumental variable. For
example, for the grade retention estimates, we find evidence of sub estimation when using
the OLS model. As exhibit by figure 7 and table 2, the correlation between the grade
retention rate previous to CPE and the proportion of ICT teachers is positive. Intuitively,
we also say there is a positive correlation between the grade retention rate previous to CPE
and the proportion of ICT-trained teachers. Thus, these endogeneity problems are highly
corrected by our instrumental variable.
CPE equipment effect versus ICT-trained teachers effect
In this paper we also attempt to measure if the CPE effect found in the paper of
Rodrguez & Snchez (2015) is due to the effect of ICT-teachers training. Thus, we
replicated in our data the identification strategy they proposed in the most recent version of
their paper and estimate the following equation.
Equation (5) follows the same structure of the equation (1). Thus, instead of
measuring a within school effect, we measure a between schools effect. In this case, our
explicative variable is the number of years that CPE equipment has been on the school.
Particularly, we apply this equation only to the school that eventually receive the CPE
intervention but that do not have ICT-trained teachers. As a consequence, we can ensure
that the observed effect we might find is only due to CPE equipment and not due to CPE
teachers.
As explained by Rodriguez and Snchez (2015), this strategy has endogeneity issues
that need to be attempted. For that end, they proposed to use the percentage of schools in
municipality m where school s is located that has been served by CPE k+1 years. This
strategy explodes the variation observed in the expansion strategy of CPE, particularly of
the intervention of equipment endowments. For our purposes we slightly modify the
instrument to the proportion of schools in neighbor municipalities m-1 to school s location
that has been served by CPE k+1 years. This is the particular value that the instrument in
equation (3)
will take in our estimations.
Table 7 presents the result of the equation (5). For the performance and grade
retention estimates, we observed that CPE has no significant impact these outcomes. In the
dropout case, we observe a slightly reduction of the dropout rate of 0.05 standard deviations
in the OLS model and of 0.02 standard deviations in the IV model.
VII. PRELIMINAR CONCLUSIONS
Since 2001, Computadores para Educar has served to the students of Colombian
public schools with ICT equipment. Even more, it has trained teacher in ICT to ensure that
the new infrastructure is properly integrated into the classroom activities. With this model,
CPE has proven to be effective in reducing dropout rate and school performance.
Nevertheless, we do not have evidence of which of the described aspects explain the
program impact: the ICT equipment, or the ICT-training for teacher, or both. In this paper
we attempt solve this question by using fixed effect techniques and instrumental variables.
This is an important question for the literature about educational policy intervention.
As explained by Kennedy (1998) and MacLeon (2008), there is evidence that indicates the
effectiveness of interventions that affect the classroom practices and teachers practices for
improving educational outcomes. As they discuss, these type of interventions have proven
to be more effective for improving educational outcomes than those that merely affect
classroom resources.
Our hypothesis is that CPE improves educational outcomes through teachers
training and not through ICT equipment. Our first estimations have provided us with
evidence that our hypothesis is not rejected. Particularly, we find that increasing by one
standard deviation the proportion of ICT-trained teachers by educational level reduces
dropout by 0.20 standard deviations and reduces grade retention by 0.76 standard
deviations. Even more, improving the proportion of ICT-trained teachers by taught subject,
increases the average performance by subject in Saber 11 test by 0.7 standard deviations.
Conversely, the estimations of the effect of ICT equipment on the students of the
schools that do not have trained teachers, do not exhibit a consistent effect on educational
outcomes. The estimation presented in table 7 indicate that CPE equipment has a small
effect on dropout reduction of 0,003 standard deviations, but it does not have a significant
impact on grade retention or Saber 11 performance.
Our estimations require further analysis. Particularly, we attempt to explore three
additional issues with our data. First, it is possible that our instrumental variable presents
additional endogeneity through a spatial correlation. Thus, we need to take into account
concepts and techniques of spatial econometrics in order to correct this issue. Second, we
must identify possible spillover effects of the ICT-training within school and between
subjects or educational levels. Third, we must look for non-linearities in the found effect.
VIII. REFERENCES
Angrist, Joshua and Victor Lavy. 2002. New evidence on classroom computers and
pupil learning. The Economic Journal 112 (October): 735-765.
Banarjee, Abhijit V., Shawn Cole, Esther Duflo and Leigh Linden. 2007.
Remedying Education: Evidence from Two Randomized Experiments in India. The
Quearterly Journal of Economics 122, no. 3: 1235-1264.
Barrera-Osorio, Felipe and Leigh L. Linden. 2009. The Use and Misuse of
Computers in Education: Evidence from a Randomized Experiment in Colombia. Working
Paper no. 4836, World Bank Policy Research Working Papers.
Barrow, Lisa, Lisa Markam and Cecilia E. Rouse. 2008. Technologys Edge: The
Educational Benefits of Computer-aided Instruction. Working Paper no. 14240, National
Bureau of Economic Research, Cambridge, MA.
Fuchs, Thomas and Ludger Woessmann. 2004. Computer and student learning:
Bivariate and multivariate evidence on the availability and use of computers at home and at
school. Working Paper no. 1321, CESifo.
He, Fang, Leigh L. Linden and Margaret MacLeod. 2008. How to Teach English in
India: Testing the Relative Productivity of Instruction Methods within the Pratham English
Language Education Program. Working Paper.
Kennedy, Mary. 1998. Form and Substance in Inservice Teacher Education.
Research Monograph no. 13. National Institute for Science Education.
Machin, Stephen, Sandra McNally and Olmo Silva. 2007. New technology in
schools: Is there a payoff? The Economic Journal 117 (July): 1145-1167.
Rodrguez, Snchez & Mrquez (2011) Impacto del Programa Computadores para
Educar en la desercin estudiantil, el logro escolar y el ingreso a la educacin superior.
Documento CEDE No. 15
Rouse, Cecilia E. and Alan B. Krueger. 2004. Putting computerized instruction to
the test: a randomized evaluation of a scientifically based reading program. Economics of
Education Review 23: 323-338.
Sharma, Uttam. 2014. Can Computers Increase Human Capital in Developing
Countries? An Evaluation of Nepals One Laptop per Child Program. Agricultural and
Applied Economics Associations 2014 Annual Meeting, Minneapolis, MN, July 27-29,
2014.
Figure 1. Accumulated Number of Schools with CPE equipment by year
Source: Resolution 166. Ministry of Education
Figure 2. Percentage of ICT-trained teachers by municipality
Source: Resolution 166. Ministry of Education
4,168 5,567 8,242
11,329 15,287
20,859
25,262 28,696
37,396
0,00%
10,00%
20,00%
30,00%
40,00%
50,00%
60,00%
70,00%
80,00%
90,00%
2005 2006 2007 2008 2009 2010 2011 2012 2013
Figure 3. Proportion of ICT-trained teachers by level
Source: Resolution 166. Ministry of Education
Figure 4. Proportion of ICT-trained teachers proportion by subject
Source: Resolution 166. Ministry of Education
0
0,05
0,1
0,15
0,2
0,25
0,3
2005 2006 2007 2008 2009 2010 2011 2012 2013
ICT-
trai
ne
d t
eac
he
rs
Year
Primary Secondary
0
0,02
0,04
0,06
0,08
0,1
0,12
0,14
2004 2005 2006 2007 2008 2009 2010 2011 2012
Biology Social Science Language English
Math Chemistry Physics Philosophy
Table 1. Teachers characteristics
2005 2008 2011 2013
Training Training Training Training
No Yes
t-
statistic No Yes
t-
statistic No Yes
t-
statistic No Yes
t-
statistic
Primary Age 43.43 41.29 *** 44.65 43.11 *** 45.35 45.01 *** 46.12 45.79 ***
Woman 0.77 0.75 *** 0.76 0.75 *** 0.75 0.73 *** 0.76 0.75 ***
Teacher's educative level
Secondary or less 0.19 0.18 *** 0.18 0.17 *** 0.26 0.31 *** 0.07 0.07 **
Normalista 0.09 0.11 *** 0.11 0.13 *** 0.09 0.14 *** 0.13 0.12 ***
Bachelor 0.56 0.56
0.57 0.56 ** 0.54 0.47 *** 0.56 0.52 ***
Graduate education 0.16 0.15 ** 0.14 0.14
0.11 0.11
0.24 0.29 ***
Teacher's experience
Years 15.22 13.33 *** 15.47 14.44 *** 15.42 15.62 ** 15.45 16.49 ***
Years since ICT-training
0.11
0.90
2.65
4.46
Change of school
Change of school between
2009-2013
0.23 0.23 ** 0.36 0.38 ***
Change of school after ICT-
training between 2009-2013
0.15
0.29
N 90,853 30,705 108,193 35,088 93,656 32,678 105,558 38,466
Secondary Age 44.51 41.00 *** 45.34 42.55 *** 45.50 44.48 *** 45.91 45.87
Woman 0.55 0.58 *** 0.54 0.57 *** 0.52 0.54 *** 0.53 0.56 ***
Teacher's educative level
Secondary or less 0.13 0.15 *** 0.13 0.14 ** 0.13 0.18 *** 0.02 0.03 ***
Normalista 0.03 0.04 *** 0.03 0.05 *** 0.02 0.04 *** 0.01 0.02 ***
Bachelor 0.61 0.62
0.65 0.65
0.69 0.64 *** 0.67 0.64 ***
Graduate education 0.23 0.19 *** 0.19 0.16 *** 0.16 0.14 *** 0.30 0.31 ***
Teacher's experience
Years 14.87 11.46 *** 14.60 12.14 *** 13.94 13.26 *** 15.06 14.44 ***
Years since ICT-training 0.00 0.12
0.88
2.78
4.65
Change of school
Change of school between
2009-2013
0.21 0.22
0.35 0.37 ***
Change of school after ICT-
training between 2009-2013
0.15
0.30
N 90,252 13,073 112,133 15,870 106,250 14,549 122,390 17,096
Figure 5. Endogeneity test for Dropout rate and grade retention rate by educational
level
Figure 6. Endogeneity test for Saber 11 score by subject
0.2
.4.6
Pro
p.
of
ICT
tra
ined
te
ach
er
by le
ve
l
0 .1 .2 .3 .4Dropout rate by level
Endogeneity graph Dropout
0.2
.4.6
.8
Pro
p.
of
ICT
tra
ined
te
ach
er
by le
ve
l
0 .1 .2 .3retention rate by level
Endogeneity graph retention
.2.4
.6.8
Pro
p.
of
ICT
tra
ined
te
ach
er
by d
iscip
lin
e
38 40 42 44 46 48Saber 11 score by discipline before CPE
Endogeneity graph Saber 11
Figure 7. Correlation between instrumental and instrumented variable by
educational level
Figure 8. Correlation between instrumental and instrumented variable by taught
subject
0.1
.2.3
.4
Pro
p.
of
ICT
tra
ined
te
ach
er
by a
rea
0 .5 1 1.5 2Theacher's experience in neighbour municipalities t-1
Instrumental variables Dropout
0
.05
.1.1
5.2
Pro
p.
of
ICT
tra
ined
te
ach
er
by d
iscip
lin
e
0 .2 .4 .6 .8Theacher's experience in neighbour municipalities t-1
Instrumental variables Saber 11
Table 2. Endogeneity test. Correlation between the proportion of trained teachers
and previous dropout rate and grade retention rate by educational level
(1) (2) (3)
All levels Primary Secondary
Dropout rate before intervention* -0.484*** 0.023 -0.327***
(-25.430) (0.783) (-10.561)
Observations 31,113 20,248 10,865
R-squared 0.026 0.012 0.032
Grade retention rate before intervention~ 0.549*** 0.240*** -0.448***
(18.788) (6.682) (-6.790)
Observations 23,712 15,346 8,366
R-squared 0.024 0.017 0.025
Years Fixed Effects Yes Yes Yes
Robust t-statistics in parentheses
*** p
35
Table 3. Endogeneity test. Correlation between the proportion of trained teachers and previous Saber 11 performance by subject
(1) (2) (3) (4) (5) (6) (7) (8) (9)
All subjects Biology
Social
Science Language English Math Chemistry Physics Philosophy
Saber 11 score before CPE
intervention* -0.019*** -0.017*** -0.002 -0.020*** -0.043*** -0.030*** -0.018*** -0.041*** -0.007
(-19.828) (-5.382) (-1.028) (-6.980) (-11.256) (-12.691) (-4.502) (-10.747) (-0.912)
Observations 22,513 2,929 4,450 3,232 2,189 3,660 1,675 3,866 512
R-squared 0.058 0.064 0.021 0.059 0.160 0.109 0.085 0.094 0.049
Year Fixed Effects Yes Yes Yes Yes Yes Yes Yes Yes Yes
Robust t-statistics in parentheses
*** p
36
Table 4. Estimation of the effect of ICT-trained teachers on Saber 11 performance by
subject
(1) (2) (3)
OLS IV First Stage
Proportion of ICT trained teachers by subject -0.011 11.433***
(0.026) (0.650)
Average years of experience of teachers ICT-trained in
neighbouring municipalities at t-1 by subject 0.161***
(0.007)
Saber 11 score before CPE intervention*
Kleibergen-Paap rk Wald F- statistic
528.487
Average score in Saber 11 test by subject 43,06
Standar deviation [2.68]
Average proportion of ICT trained teachers by subject 0.05
Standar deviation [0.18]
Observations 237,718 237,718 237,718
R-squared 0.359 -0.364 0.059
Number of schools 4,888 4,888 4,888
Students' characteristics Yes Yes Yes
Year fixed effect Yes Yes Yes
Years of CPE in the school fixed effect Yes Yes Yes
School fixed effect Yes Yes Yes
Discipline fixed effect Yes Yes Yes
Robust standard errors in parentheses
*** p
37
Table 5. Estimation of the effect of ICT-trained teachers on dropout rate performance by
level
(1) (2) (3)
OLS IV First Stage
Proportion of ICT trained teachers by level -0.006*** -0.091***
(0.002) (0.022)
Average years of experience of teachers ICT-trained in
neighbouring municipalities at t-1 by discipline
0.188***
(0.012)
Average dropout rate 0.13
Standard deviation 0.09
Average proportion of ICT trained teachers by level 0.10
Standar deviation 0.20
Kleibergen-Paap rk Wald F- statistic
262.89
Observations 125,436 125,400 125,400
R-squared 0.316 0.294 0.271
Number of Schools 11,034 10,998 10,998
Year Fixed Effect Yes Yes Yes
Years of CPE in the School Fixed Effect Yes Yes Yes
School Fixed Effect Yes Yes Yes
Educative level Fixed Effect Yes Yes Yes
Robust standard errors in parentheses
*** p
38
Table 6. Estimation of the effect of ICT-trained teachers on grade retention rate
performance by level
(1) (2) (3)
OLS IV First Stage
Proportion of ICT trained teachers by level -0.022*** -0.519***
(0.002) (0.032)
Average years of experience of teachers ICT-trained in
neighbouring municipalities at t-1 by level
0.242***
(0.014)
Average grade retention rate 0.10
Standard deviation 0.11
Average proportion of ICT trained teachers by level 0.13
Standar deviation 0.15
Kleibergen-Paap rk Wald F- statistic
313.06
Observations 127.436 127.356 127.436
R-squared 0.099 -1,287 0.256
Number of Schools 11.263 11.183 11.263
Year Fixed Effect Yes Yes Yes
Years of CPE in the School Fixed Effect Yes Yes Yes
School Fixed Effect Yes Yes Yes
Educative level Fixed Effect Yes Yes Yes
Robust standard errors in parentheses
*** p
39
Table 7. Estimation of the effect of Computers for Teaching on Educational outcomes. Only Schools with computers and without ICT-trained teachers
Saber 11 Dropout rate Grade retention rate
(1) (2) (3) (4) (5) (6)
OLS IV OLS IV OLS IV
Years of CPE intervention 0.025 0.035 -0.002*** -0.001** -0.000 0.001
(0.021) (0.021) (0.001) (0.001) (0.001) (0.001)
First Stage
Years of CPE intervention at the neighbour
mun
-0.323
0.028***
-0.275
0.002
0.001
(0.001)***
Kleibergen-Paap rk Wald F- statistic
1.50E+04
7.50E+04
7.50E+04
Average score in Saber 11 test by subject 43.25 0.13 0.10
Standar deviation 2.56 0.09 0.08
Average year of CPE intervention 1.28 1.25 1.53
Standar deviation 2.24 2.31 2.61
Observations 15,669 15,669 43,910 43,910 35.394 35.394
R-squared 0.003 0.002 0.174 0.174 0.046 0.046
Number of Schools 3,018 3,018 8,647 8,647 7.778 7.778
Year fixed effect Yes Yes Yes Yes Yes Yes
School Fixed effect Yes Yes Yes Yes Yes Yes
Robust standard errors in parentheses
*** p