Do differences in teacher contracts affect student ... · difficult to identify the effects of...

Do differences in teacher contracts affect student performance? Evidence from Togo

Emiliana Vegas The World Bank

Joost De Laat

Brown University

June 26, 2003

Abstract

Many poor countries are initiating teacher contract reforms to meet a growing demand for

primary education at a time of increasing government deficits. Key aspects of this reform include

reduced salaries and benefits for new, contractual teachers. Using data from Togo, we find that

students of regular teachers systematically outperform those of contractual teachers, even after

controlling for prior achievement, household-, school- and classroom characteristics. Variation in

teaching methods, absenteeism, and resentment over “unfair” pay across contract types do not

explain the performance gap. Instead, our findings suggest the reforms triggered a reduction in

supply of high quality teacher entrants.

Do differences in teacher contracts affect student performance? Evidence from Togo, p. 1

I. Introduction

Teacher salaries account for the bulk of public educational expenditures in most

countries, but the share of public expenditures spent on teacher salaries seems to be greatest in

the poorest of countries. As Table 1 shows, teacher salaries account for 74 to 96 percent of all

public expenditures on education in West Africa, with Togo exhibiting the highest share.

Motivated by budgetary pressures, many developing countries are adopting initiatives to

change teacher contracts to reduce teacher compensation and benefits. Understanding the

consequences of such policy initiatives, in particular their effects on student learning, is

important because a key concern for all countries and, especially low-income countries, is the

cost-effectiveness of alternative educational investments. One such initiative taken in Togo and

other West African countries is to hire contract teachers, who receive lower pay and benefits than

do regular teachers and whose job stability is not guaranteed. In this paper, we assess the impact

of this initiative on student achievement.

Although educators, economists, policy-makers and the public at large believe that

teachers are the key input in education, the relationship between teacher characteristics,

including wages, and student outcomes has been difficult to establish empirically (see

Hanushek’s (1986 and 1997) reviews of empirical research). Moreover, the more important

question of how to attract quality teachers, motivate them and retain them in the profession

remains unanswered.

A few recent studies are finding a relatively strong relationship between student

outcomes and teacher observed and unobserved characteristics. For example, Hanushek, Rivkin

and Kain (1998) find that at least 7 percent of the variance in student test scores may be

explained by variation in teacher characteristics. Loeb and Page (2000) find that raising the

wages of teachers by 50 percent would reduce high-school dropout rates by more than 15 percent


and increase college enrollment rates by close to 8 percent. Ballou and Podgursky (1998) find

that even though private school salaries in the U.S. are substantially below those in public school

systems, private school heads are more satisfied with the quality of their teachers and tend to be

more successful in retaining the best of their new teachers. They conclude that some of the

reasons include greater flexibility in structuring pay, more supervision and mentoring of new

teachers, as well as the freedom to dismiss teachers who underperform (Ballou and Podgursky,

1998). Our paper contributes to this line of research by (1) exploring the impact on student

achievement of implementing a teacher contract scheme whereby teachers are paid less and may

be dismissed due to poor performance; and (2) assessing the impact of such a reform in the

context of a very poor developing country.

We find that students of contractual teachers in Togo have lower achievement, even after

controlling for prior achievement, household characteristics, as well as a rich set of school,

classroom and teacher variables. Although this finding is consistent with selection on

unobservables, we believe this is not purely a story of selection on unobservables for at least two

reasons: (1) our data are uniquely rich in information on student background, school and

classroom characteristics, and teacher variables, which allows us to test the effect of contract

status on student learning after controlling for many usually unobserved student and school

variables; and (2) an analysis of the effect of contractual status on student learning among

teachers who likely entered the profession around the time of the policy change and thus were

not probably expecting to become contract teachers finds no effect, which suggests that a shock

to the quality of teacher entrants took place following the implementation of the contractual

teacher policy.

In the following section, we briefly review key aspects of the education sector in Togo

and the teacher contract reform. Then, we present our research question and discuss why

alternative teacher contracting mechanisms may affect student achievement. In Sections IV and


V, we describe our research strategy as well as our data. In Section VI, we present the results of

our analyses of the effects of differences in teachers contracts on student performance. In the

final section, we conclude with policy implications and directions fo r future research.

II. Background on the Education Sector in Togo

Togo initiated major public sector reforms in the 1980s and 1990s to cope with the

decline in government revenues resulting from a prolonged period of economic stagnation that

continues to persist. Between 1991 and 2000, per capita GDP dropped 10 percent, while at the

same time government revenue decreased from 17.4 percent of GDP in 1990 to 13 percent in

2000. In response to this sharp decline in revenues, the government froze public sector wages. In

fact, between 1982 and 1998 the government increased nominal base salaries of civil servants,

including teachers, on only two occasions, by 5 percent in 1990 and again in 1996, although

promotions, seniority premiums, and various other allocations increased average effective

salaries. Further, despite rapid population growth, budgetary pressures prevented commensurate

hiring of new teachers and building of new classrooms. Replacement of teachers quitting or

retiring was insufficient in a period which saw the number of primary school students grow by an

annual average of 4.5% between 1988 and 2000. This caused overcrowding in many schools and

a sharp rise in the number of pupils per teacher across the various education levels between 1984

and 1995. 1

The resulting deterioration in the quality of public education and its declining capacity to

absorb the increasing numbers of primary students led to major changes in the way primary

education is organized in Togo and resulted in an overhaul of teacher hiring practices. In

addition to wage freezes, the government shifted away from hiring relatively expensive civil

1 Source: IMF 1998 (p14) {their source: MENRS; and World Bank, Togo: Revue des Depenses Publiques, Feb 1997}


servant teachers, so-called fonctionnaires whose salaries in 1999 amounted to between 5,4 and

7,8 times per capita GDP, toward hiring non-permanent contractuels, or auxiliaires, instead.2

Most contractual teachers attended regular teacher training institutes and, in general, have at least

as many years of education as do regular teachers. Depending on qualifications, contractual

teachers receive much lower wages, equivalent on average to 40 percent of the civil servant

fonctionnaire wages, and have very limited promotion possibilities, pension rights, and non-wage

benefits. Today, only 45 percent of the primary public school teachers are fonctionnaires, while

55 percent are contractuels.

On their side, parents, particularly those living in urban areas, responded to the shortages

in public school places by sending their children to private primary schools, resulting in a 6.8

percent annual increase in private school enrollment between 1988 and 2000. Further, since the

mid 1990s, parents in certain rural areas where the provision of both public and private schools

has been either insufficient and/or unaffordable, began to organize themselves in community

organizations. Financed by the voluntary payments of fees, occasionally with the support of

nongovernmental organizations, these community organizations have created “local initiative

schools” (écoles d’initiative locale (EDILs)).

Both the surge in private primary education and the creation of these EDILs have

absorbed most of the overall growth in primary students. As Table 2 shows, net enrollment rates

in Togo increased from 74 to 91 percent between 1990 and 1999 (World Bank 2002). In 1995,

public schools enrolled 75 percent of all primary students compared to 25 percent in the private

sector; by 2001, public schools accounted for only 60 percent with 31 percent in private schools

and 9 percent in the EDILs. Finally, also in several public schools, local school management

supported by parents’ associations responded to understaffing by hiring contractual teachers at

substantially lower salaries. This group of teachers now constitutes about 8 percent

2 Mingat 2000.


(approximately 1200 in total) of primary pub lic school teachers and their salaries are usually

financed by voluntary school fees (Mingat 2002).

III. Research Question

Our main research question is: to what extent do differences in teacher contracts affect

student learning in Togo? Differences in teacher contracts are likely to affect student learning in

several ways. First, contractual teachers, by definition, have contracts that at least theoretically

may not be renewed. Such a threat of non-renewal may result in contractual teachers exerting

greater effort than tenured teachers.

Second, student learning among the students of contractual teachers may instead be

negatively affected by an induced “disgruntled worker” effect, similar to Akerlof and Yellen’s

“fair wage-effort hypothesis” (1990), which posits that workers proportionally withdraw effort

as their actual wage falls short of their perceived fair wage. Lower paid contractual teachers may

consider their compensation unfairly low given that their work requirements are identical to the

higher paid teachers and withdraw effort accordingly. We will explore the fair wage hypothesis

by investigating to what extent contractual teachers’ performance differs among teachers in

schools with different proportions of contractual teachers, holding constant other school

characteristics.

Third, changes in teacher contracts presumably work to purchase different types of

teachers by inducing variation among teacher entrants in the opportunity cost of alternative

employment opportunities. In particular, if certain attributes – such as, for example, motivation

and work effort – are rewarded by alternative employment opportunities and render some people

better teachers than others, a reduction in teacher compensation will likely reduce the supply of

qualified teachers and result in lower student learning. Unfortunately, directly testing this

opportunity cost hypothesis is almost impossible as Togo lacks detailed and reliable time series


information to infer trends in teacher pay relative to alternative employment opportunities.

Nevertheless, we are able to explore this question by analyzing the effect of contractual status on

student learning in different subsamples of teachers, taking into account the continued

deterioration of overall economic conditions in Togo.

IV. Research Strategy

Conceptual Approach

Because teachers and students may be distributed non-randomly among schools, it is

difficult to identify the effects of changes in teacher contract type on student learning. We follow

Todd and Wolpin (2001) and start with the following simple set-up of a general educational

production function. Suppose a student’s end-of-year achievement A1 is a function of the

student’s history of school and family inputs H0 until the beginning of the school year, the

student’s school inputs during the year S1 that are associated with the particular school chosen,

school inputs that were unanticipated at the time of the school choice 1S , the family inputs

during the year F1, and the student/household endowment µ:

(1) ),,,,( 11101 µFSSHfA =

School inputs associated with parental choice of school 1S include observable

characteristics such as school resources (e.g. facilities and materials), class size, and the extent to

which teacher quality is observable. 1S , on the other hand represents school inputs that were

unobservable at the time of the school decision, such as teacher illness or strike. The fourth

element, family inputs F1 includes school books and writing utens ils the student has access to at

home, whether the student has someone at home to help and motivate, as well as the student’s

time endowment to study. Particularly in poor countries like Togo, one would expect children to

spend significant time doing household tasks or working on the farm or for wages, which limits


their opportunity to do homework. Finally, the student’s endowment µ is a vector that includes

not only his/her “cognitive endowment” but also character traits that directly affect learning such

as the student’s and his/her parents’ motivation and effort in the pursuit of learning.

The nonrandom allocation of students to schools and classrooms results from the

formulation of the decision rules governing school inputs 1S and family inputs F1. Since parents

care about the cognitive achievements of their children, the choice of direct family inputs F1 and

the vector of school inputs 1S associated with the particular choice of school are really joint

decisions by parents. Abstracting from the effects of relative prices for family and school inputs,

we can write these joint decisions as:

(2) ),( 01 HWgS µ=

(3) ),,( 011 HSWhF µ=

where W are the family’s permanent resources. Given sufficient heterogeneity in resources W

and preferences/endowments µ, children will be sorted into different types of schools along these

characteristics. For instance, wealthy and/or more motivated families will likely send there

children to more expensive schools with better facilities and better teachers and are also likely to

purchase higher levels of family inputs at home.

Identifying the effect of an exogenous change in school inputs such as teacher

type/quality on student learning in the presence of such sorting would be a straightforward matter

if all inputs µ,,,, 1110 FSSH that enter the learning function directly were observed. However,

empirical identification is often complicated for three main reasons: (1) omitted variable bias; (2)

sample selection bias; and (3) attenuation bias. We address each in turn.

Omitted Variable Bias. In light of sorting of students by school type, the biggest

challenge facing estimation of student learning production functions is often omitted variable


bias. Surveys generally lack sufficient detail on family background and family inputs as well as

on school and teacher characteristics. Omitted variable bias arises, for example, if the same

parents that choose schools based on their attributes such as teacher quality also purchase more

family inputs that researchers do not observe, causing the coefficient estimate on teacher quality

in an educational production function predicting student learning to overstate the real effect. As

we will show below, our data are rich in information on a wide variety of school and family

inputs, decreasing the chances that the identification of the teacher contract effect suffers from

this kind of omitted variable bias even if parents choose schools based on the contractual status

of its teachers.

However, we do not observe the full vector µ, which in addition to serving as a sorting

mechanism in (2) and (3), is also a direct input into the student’s learning function. Although our

data from Togo lack information to reliably assess for each student the characteristics of

alternative schools not attended, we do not expect that school sorting driven by variation in

student and parental motivation to be an important source of bias, in particular with regards to

teacher contract type. In general, omitting µ in the above set-up would bias the coefficient

estimates on the other inputs only if the following two conditions hold: (i) there is sufficient

variation between parents in µ, and (ii) variation in µ is capable of inducing sufficient variation

(i.e. sorting) in school inputs S1 and family inputs F1 beyond variation induced in these two input

vector by differences in permanent resources between families. Although the first condition may

hold, evidence from neighboring Ghana and the economic situation in Togo itself suggests that

sorting is largely driven by differences in permanent resources, which themselves are not inputs

in the child’s learning function. Glewwe and Jacoby (1993), for example, find that school choice

in middle schools in Ghana is determined primarily by family resources and not by the imputed

ability (a proxy for µ) of the child conditional on these resources. They also find no evidence of


selection bias in their estimation of an educational production function.

We expect such ability- induced sorting to be even more uncommon among primary

school students in Togo. The reason is that the degree of sorting among different schools due to

variation in parental and child motivation is likely greater in middle than elementary school, and

more importantly, at roughly $310 GDP per capita, Togo is not only of the world’s poorest

countries, but even roughly 25% poorer than Ghana. With severely credit constrained parents,

the importance of household resources is likely to be much greater in determining school choice

than is motivation or ability, conditional on these resources. In addition, as we will discuss more

fully below, we do observe some elements of µ and elements which are likely to be proxying for

it. These include, for example, maternal literacy, whether French is spoken at home (since

students learn French in school, one would expect motivated parents to be more likely to speak

French at home, ceteris paribus), and whether the child has someone at home to help studying.

Finally, even though we observe detailed school and family inputs, and motivation/ability

induced selection in Togo is likely to be insignificant relative to wealth induced sorting, our

study could be prone to omitted variable bias that arises from not observing the entire history of

family inputs 0H . We will compensate for this by using a test score measure for each student at

the start of fifth grade. Although this is not necessarily a sufficient statistic for 0H , fixed but

unmeasured factors are eliminated and the only concern becomes allocations made on the basis

of unmeasured achievement influences that are unrelated to prior achievements (Hanushek,

2002). Further, by conditioning on initial test scores, an important concern that schools would

assign contractual teachers to low performing classes is also addressed.

Sample Selection Bias. A second challenge in identifying the effect of teacher contract

type on student learning is the issue of sample selection bias. Only roughly 80% of all children in

Togo enter primary school, and of those less than 80% finishes fifth grade (Mingat, 2002). Class


repetition in Togo is quite common and one would expect the children who finish fifth grade to

be better achievers, ceteris paribus, causing A1 to be left-censored. Since we lack data on

children in the same cohort but not attending school, we cannot directly calculate the usual

Heckman-style selectivity statistics. Nevertheless, because we have pre- and post-test data for

each student, we can focus on achievement gains over the course of the year and thus avoid this

type of sample selection bias.

Attenuation Bias. A third challenge facing proper estimation of education production

functions concerns measurement error. In almost all studies, student test score data are used to

proxy for an underlying latent ability/knowledge distribution whose measure is the ultimate

objective of interest. Because individual responses to questions are often not observed but final

scores are, the usual practice is to simply take these final scores and standardize this measure.

Using standardized final scores, however, ignores the fact that any two questions differ in the

probability that students at various points in the latent knowledge distribution will answer these

questions correctly. Such a method of standardizing the sum of “correct” responses may lead to

biased estimates of the underlying distribution whenever the set of questions does not accurately

distinguish between different points on the support of this distribution. In the estimation below,

we will (partially) correct for this kind of measurement error using maximum likelihood and

Bayesian estimation methods developed under the umbrella of Item Response Theory (see

Appendix for a descrip tion of IRT theory and our estimates).

Finally, there are two answers to the question of what is the effect of an exogenous

change in school inputs such as teacher quality on student learning (Todd and Wolpin, 2001).

The first,

11

1

1

1 *1

dSSA

dSdA

FF =∂∂

= ,

holds family inputs the same, whereas the second,


11

1

1

1

1

1

1

1 * dSSh

Ff

Sf

dSdA

∂∂

∂∂+

∂∂= ,

takes into account the indirect effect on student learning through the change induced in family

inputs. Our estimation will focus on answering the first question; what is the effect of an

exogenous change in teacher contract on student learning holding family inputs fixed. In theory

we could answer the second question too, by estimating the coefficient on teacher type, 11 Sh ∂∂ ,

in a family input equation. In practice, however, this is a formidable task unless the data come

from an experimental study in which teachers of different contracts were randomly assigned

across classrooms (and students were not allowed to change classrooms).

Empirical Strategy

Our empirical strategy is twofold. First, we decompose the variance in the variables of

interest by source to better understand the extent to which, for each variable, the proportion of

the observed variance comes from differences among students within a classroom/school or from

differences in the averages between classrooms/schools.

Second, we conduct robust ordinary least squares (OLS) regression to analyse the effects

of student, teacher, and school characteristics on student learning. Our empirical model can be

expressed as:

(4) ijijijijijijijtijt TSTSHAA εβββββββ +++++++= − 65432110

where Aijt is the end-of-year test score of student i in school j; Aijt-1 is the student’s

beginning-of-year test score; Hij is a vector of household characteristics specific to student i in

school j; Sij and Tij are a vector of school- and a vector of teacher and classroom characteristics,

respectively, of student i in school j observable at the beginning of the year, while ijS and ijT are

similarly vectors of school and teacher inputs of student i in school j not necessarily observed at

the beginning of the school year. These include, for example, teacher absenteeism and visits by


the government school inspector during the course of the school year. We estimate robust

standard errors to account for the clustering of students within classrooms/schools.

V. Data, Sample, and Variables Used in the Analysis

The data used in this study were collected during the 2000-2001 school-year as part of a

wider data collection effort by the Programme d’Analyse des Systemes Educatifs des Pays de la

CONFEMEN (PASEC). This is an initiative by the Confemen group of French speaking

countries in Africa whose aim is to develop more efficient and effective teaching sectors. As part

of its efforts, the organization has been carrying out school-based surveys since 1993 in Burkina

Faso, Central Africa, Congo, Djibouti, Guinea, Ivory Coast, Madagascar, Mali, Niger, Senegal,

and Togo, countries which have very similar educational systems dating back to the period of

colonialization under France. In each of the countries, samples of second grade (CP2) and fifth

grade (CM1) primary school students were selected from around 120 different classes in each

grade. Altogether, these surveys cover by now almost 1,500 schools, more than 2,300 teachers

and more than 40,000 primary school students.

In Togo, 12 second or fifth graders were randomly selected among the students in their

class from a random sample of 233 different schools. The second graders were distributed among

118 different schools, the fifth graders among 122 schools, covering 1406 second grade students

and 1440 fifth grade students. Of the 233 schools in our sample, only seven schools included

students from both grades. The students were administered tests at the beginning and at the end

of the 2000-2001 school year. At the time of the pre-test, students also were administered a

questionnaire to gather detailed information about their family and household backgrounds. At

the time of the post-test, teachers and school principals were administered questionnaires which

provide valuable teacher- and school- level data.


Our final sample consists of only the fifth-graders in the PASEC dataset. We chose to

focus on them exclusively because of problems encountered in merging the pre- and post-test

data for second graders. While information was collected on 122 fifth-grade classrooms, the

three data files for the student, teacher, and principal could only be merged for 85 schools

altogether. The means for schools for which there were not three data sets, but for which teacher

information was available, were not significantly different from the complete datasets. Our final

sample thus consists of 837 students within 78 classrooms/schools for whom information on all

the variables used in the estimation was available. Because we do not have more than one

classroom/teacher per school, we are not able to fully separate the teacher from the school

effects. However, one strength of our data is that it includes a great deal of information on

school and classroom characteristics, as well as on teachers, which allows us to compensate for

this shortcoming in the data collection design. Table 3 presents the names and descriptions of the

variables used in our analyses; Table 4 presents summary statistics for these variables.

Student- level variables

Student test scores. Our measures of student achievement consist of the item response

theory (IRT) score in the mathematics tests taken by the students at the beginning (pre-test) and

at the end (post-test) of the school year 2000-2001. These scores are generally preferred to using

the students’ raw test scores as they limit measurement error in proxying the latent “knowledge”

distribution of interest. We calculated these IRT scores using the 3-parameter logistic model,

which allows for guessing by students when the test contains multiple choice questions, as is the

case with the Togo tests (see the Appendix for a discussion of IRT scores and the estimation

procedure used). We use the post-test scores as the dependent variable in our analyses, and the

pre-test scores as one of the key predictors. By including the pre-test scores in our analyses, we

can to a great extent control for the student’s prior achievement and the many unobserved past


household, classroom and school inputs in the student’s background that may affect

achievement.

The first row of Table 4 presents standardized IRT scores for fifth grade students with

contract teachers and those taught by the higher paid civil servant teachers, the instituteurs.

Students of contractual teachers score significantly lower on the end-of-the-year test. Their mean

of –0.29 is equivalent to the score of students in the 39th percentile of the latent variable

distribution, versus a mean of 0.28 for the instituteur students, which corresponds to the 61st

percentile. Figure 1 provides kernel density estimates for the distribution of scores, which

indicate that the score distributions by teacher contract type are very similar, suggesting that the

difference in mean scores is not driven by a few outliers, but by a mean shift of the entire

distribution. Of course, the fact that the contractual students perform worse at the end of the year

does not by itself provide any prima facie evidence that the students’ underperformance is related

to systematic differences inherent to the two types of teachers they face. If these students

performed even worse at the beginning of the year, then having the lower paid contractual

teacher would suggest to boost test scores, not lower them.

However, students’ standardized beginning-of-the-year scores support the notion that

students taught by contract teachers are under-performing. These students also scored

significantly lower on the beginning-of- the-year test, but the gap was smaller. Their mean of –

0.14 was equivalent to the score of the students in the 44th percentile, as compared to a mean

0.15 for students of regular teachers, equivalent to the students in the 56th percentile of the latent

distribution. Thus, the initial spread of 12 percentile points had widened after one school year to

22 percentile points.

Student demographic characteristics. We control for student gender and age by including

an indicator variable for the students’ gender and a continuous variable representing his or her

age. Forty-eight percent of students of regular teachers are female, as compared to forty-two


percent of contractual teachers. But students of contract teachers tend, on average, to be slightly

older than those of regular teachers (11.0 and 11.8 years, respectively).

Student household characteristics. Using principal components analysis, we constructed

several composites. First, we developed an index of household assets (including 17 variables

such as the number of household members per room in the household, whether the household has

a refrigerator, a television set, electricity, a car, a bicycle, etc.). The higher the asset index, the

better the socio-economic conditions of the household. Second, we constructed a composite of

the extent to which the student has access to regular meals (students were asked whether they

rarely, sometimes, often or always eat a number of meals). The larger the composite, the less

frequently the student has access to regular meals. Third, we formed a composite of the number

of household tasks (such as cleaning, cooking, taking care of other children, etc.) that a student is

responsible for, outside of school work. The larger this composite, the more tasks around the

household that the student is responsible for outside of his regular school work.3 In addition, we

include in our analyses student background variables that prior research has shown may affect

student learning, such as mother’s literacy, access to books, whether the student studies at home,

and whether the student has someone at home to help with schoolwork.

Overall, the data indicate that students of contractual teachers tend to come from

households with lower average socio-economic background. The data also reveal important

differences in average socio-economic background between students of contractual and regular

teachers. For example, the average index of household assets, 0.54 for students of regular

teachers and –0.71 for students of contractual teachers, suggests that students from higher socio-

economic backgrounds are more likely to be in classrooms with regular teachers. Similarly,

students of contractual teachers are less likely to have literate mothers, to have access to

3 Interested readers may obtain more information on all principal components analyses included in this paper by contacting the authors.


mathematics textbooks, to study at home, and to have someone at home who can help with

school work than are students of regular teachers. Students of contractual teachers are also more

likely to be responsible for household tasks and tend to have less regular meals than are students

of instituteur teachers.

School- level variables

We explore whether student performance differs between private and public schools as

well as between schools in urban and rural areas. Students of contractual teachers are less likely

to be in private, urban schools. While more than 50 percent of instituteur students are in private

schools, this figure is only 10 percent for students of contractual teachers. Similarly, 55 percent

of students whose teachers are instituteur are in urban schools, whereas the comparable figure

for contractual teachers’ students is only 36 percent. We also control for school size (number of

students per school), though the average number of students per school is about the same (around

240) for students of regular and contract teachers.

In addition, we explore whether children in better equipped schools have higher

achievement. The principal’s questionnaire asked a number of questions (sixteen in total) related

to school facilities, such as whether the school has a library, a teacher conference room, an

infirmary, sports facilities, playground, etc. Using these data, we constructed a principal

components composite of school facilities. It is also possible that student performance is higher

in schools that receive more frequent inspections and/or support from pedagogic counselors.

Principals were asked to report how many times the inspector came to visit in the past year to

meet with parents, to provide advice to teachers, to provide advice to the principal, to motivate,

or for formal inspections. We constructed a principal components composite of quality of

inspection using this information. Similarly, principals were asked to report the frequency of

visits from pedagogic counselors to meet with parents, to counsel teachers, to guide the principal,

etc. Using this information, we constructed a principal components composite of the quality of


counseling to the school. These data suggest that students of contractual teachers tend to be in

schools with worse school facilities and in schools that receive less inspection and pedagogic

counseling. As shown in Table 4, the averages of the principal components composites of school

facilities, school inspection and pedagogic counseling to the school are substantially lower for

students of contractual teachers than they are for students of regular teachers.

Last, we explore the effect of the proportion of contractual teachers in the school on

student achievement to assess to what extent the data support the Akerlof and Yellen’s “fair

wages hypothesis.” The number of contract teachers as a share of all teachers in a school is 33

percent among students of contract teachers and 20 percent among students of regular teachers.

Teacher-level variables

Our predictor of interest is teacher’s contract status, an indicator variable representing

contractual teachers (0=fonctionnaires; 1=contractuels). If, holding student and school

characteristics constant, achievement of students of contract teachers is different (lower or

higher) than that of students of regular teachers, we need to understand why. In other words, to

what extent can we explain differences in student achievement between contract and regular

teachers by exploring the effects of teachers’ own characteristics and behavior in the classroom

as well as by looking at differences in the resources available in these classrooms? Our data are

rich in variables describing not only easily measurable teacher characteristics (e.g. years of

education and experience), but more importantly indicators of teacher behavior and the resources

available in each classroom.

Teacher characteristics. Prior research has shown that the relationship between teacher

experience and student achievement is strong though not linear. Very inexperienced teachers

tend to be less effective than those with at least a few years of experience (Rivkin, Hanushek and

Kain, 2000). Although the U.S. evidence on the effect of teacher education on student

achievement is mixed (Hanushek 1986 and 1997), in the developing country context, researchers


have found that additional education can be associated with better student outcomes (Harbison

and Hanushek 1992). We thus in our analyses a dichotomous variable representing five or less

years of teaching experience and a continuous measure of teachers’years of education.

The data indicate that contractual teachers have, on average, more years of education but

less years of experience than do regular teachers. The latter is not surprising given the recency

of the contractual hiring practices. As a result, in our estimations below, we include a two-way

interaction between experience and contract status in order to allow for the effect of experience

to vary between contractual and regular teachers.

Teacher behavior. As explained in the previous section, contractual teachers may exhibit

different levels of performance from regular teachers if they perceive their pay as insufficient or

if they receive pay on an irregular basis. Our analyses use information on teacher’s perceptions

of fairness of pay and teachers’ reports on the regularity of pay. Contractual teachers are more

likely to report that they receive their pay on a very irregular basis. While 36 percent of

contractual teachers report to receive their pay “very irregularly,” the comparable figure for

regular teachers is only 8 percent.

Another aspect of teacher behavior that may affect student achievement is teacher

absenteeism, and it may vary by teacher contract status. Our data contain teacher reports of the

total number of days absent in the school year and in the past month. We enter separately each

of these variables in our analyses. Teachers’ reports of absenteeism do not vary significantly

between contractual and regular teachers. Note that the average days absent, both in the school

year and in the last month are not significantly different for the two types of teachers.

Finally, a series of variables in our analyses attempt to capture differences between

classrooms in various dimensions of teacher quality, including teachers’ relationships with

parents, the extent to which teachers collaborate with other teachers in the school, the extent to

which the teacher provides time and help to students voluntarily outside of regular teaching


hours, and the extent to which the teacher is involved in leading, supervising, and grading

student work. For each of these dimensions, we use multiple indicators from the teacher

questionnaire to construct principal components composites. While contractual teachers report

to be slightly less willing to dedicate time outside of regular classtime to students, to report

meeting with parents regularly, and to tutor students for free outside the classroom, they are

slightly more actively involved in student homework (grading, asking students about lessons,

etc.) than are regular teachers. These differences are not, however, significant.

Classroom resources. There is an inconclusive debate in the U.S. and developing

countries regarding the effects of class size on student achievement. Although an analysis of the

effect of class size on student learning is beyond the scope of this study, we include class size as

a variable in our analyses to control for possible systematic allocations of teachers of different

contract status by class size. Indeed, the average class size of students of contract teachers is

slightly greater than that of students of regular teachers (36.6 v. 35.6).

Classrooms also differ in the availability of materials, such as chalkboards, desks,

electricity, and books. Teachers were asked a series of questions related to the materials

available in their classrooms. We used this information to construct a principal components

composite of classroom materials which suggests that contractual teachers tend to work in

classrooms with worse facilities and materials than those of regular teachers. The mean of the

principal components composite of classroom facilities for students in classrooms with

contractual teachers is –0.345, while it is 0.388 for students in classrooms with regular teachers.

VI. Findings

The analysis of variance shows considerable heterogeneity in student background among

the students within a classroom. To see this, we first decompose the observed variance in the

student- level variables into: (i) variation explained by differences in average student


characteristics between the sampled classes, and (ii) variation explained by differences in student

characteristics among students within each of these classes. These results, presented in Table 5,

show that although there is some sorting on household variables by classrooms (schools), the

majority of the variation in observed household variables comes from differences among

students within classrooms.

Forty percent of the variation in student test scores in the pre-test is explained by

variation between classrooms. Fifty percent of the variation in student test scores in the post-test

comes from differences between classrooms. This suggests that if parents in Togo are making

deliberate choices over school choice and family inputs, then student learning in Togo is

importantly affected by both types of inputs and, consequently, sorting by family background

variables (W, µ) is imperfect. To see this, first suppose that student test outcomes are solely

determined by school characteristics; i.e. variation in student backgrounds have no effect on

students’ test scores ( 0111 == µddAdFdA ). Under the scenario of the simple model above, if

schools were all that mattered for student learning, there would be great incentives for families to

choose the best affordable school for their child. This would induce a great amount of sorting by

family background variables (W, µ) of students across different schools and, independently of

such sorting, all the variation in test-scores would be between different classes. If, on the other

hand, school inputs had no effect on student test scores ( 011 =dSdA ), no deliberate sorting

would take place and all the variation in test scores would be between students within each of the

classes. Thus as the importance of family background variables in the production of student

learning A1 increases, the correlation between school inputs and family background variables

decreases. The proportion of total variance in household characteristics explained by differences

between classrooms, which ranges from 17 to 50 percent, also indicates considerable

heterogeneity within classrooms in Togo.


Table 6 presents the results from our OLS regressions with robust standard errors. The

most important predictor of student performance is previous performance on the pre-test.

Student age is negatively related to student performance, likely because older students tend to be

repeaters.

Recall that the simple framework presented above posits that student learning, equation

(1), is a function of family (F1) and school inputs (S1), which in turn are functions of wealth W1

and family endowment µ, conditional on the history of inputs H0 . The index of household assets

is indeed strongly and very significantly related to student performance. Introduction of family

and school inputs as well as any teacher characteristics that are observable to the student and her

family at the time of school choice and serve as school sorting indices by parental wealth should

reduce the significance of this index. The table shows that while inclusion of various indicators

of student household background and teacher and class characteristics indeed reduces the

coefficient size and statistical significance of the asset index, it nevertheless remains significant.

This suggests that the asset index captures a richer set of household background variables

associated with student achievement than are present in the data. Two indicators of household

income that do not seem to be fully captured by the asset index are the regularity with which a

student eats his/her meals and the extent to which the student is responsible for household tasks.

The estimated coefficients on these variables have the expected signs – with students who do not

have access to all meals on a regular basis having lower estimated student test scores and,

similarly, students who are responsible for household tasks having lower estimated achievement

– and are statistically significant.

The data do not support the notion that students in private, urban schools tend to perform

better. Similarly, other measures of school characteristics, such as school size and the quality of


school facilities, do not appear to affect student achievement. This is shown in Model 3 of Table

6.

Model 4 introduces teacher’s experience, education, and contract status. We also

incorporate a two-way interaction term between experience and contract status to account for the

recency of the implementation of the alternative contracting practice and the consequence that

contract teachers, by definition, will tend to have less experience than instituteurs. The

estimated coefficients on experience and education have the expected signs, indicating that

teachers with five or less years of experience are less effective while teachers with more years of

education are more effective.

The coefficient estimate on contractuel is large in magnitude, negative, and highly

statistically significant. This suggests that, holding constant student background variables and

controlling for teacher education and experience, contractual teachers are less effective than are

instituteurs. Since contractual teachers enjoy slightly more years of education, the positive and

significant coefficient on the teacher education variable indicates that the negative effect on

student learning of having a contractual in class is slightly mitigated by their greater education

levels. However, their currently lower experience levels tend to exacerbate their ineffectiveness.

The estimated coefficient on the two-way interaction between contractuel and experience

is positive, large and statistically significant, suggesting that the negative effect of contract status

on student achievement differs among teachers with less or more than five years of experience.

As Figure 2 shows, the negative effect of being a contract teacher is exacerbated among teachers

with more than 5 years of experience. One possible explanation for this finding is that more

experienced teachers who are contractuals resent their lower wages and worse working

conditions more than do contractual teachers with few years of experience. However, the data


do not seem to support this notion. 4 In light of the continued economic decline in Togo during

the 1990s, another possibility might be that after the initial wave of hiring contractual teachers,

alternative employment opportunities for potential teachers have worsened, thereby increasing

the average quality of the applicant pool of more recent contractual teachers relative to the older

pool of contractual teachers.

Model 5 in Table 6 adds to our previous model the proportion of contractual teachers in

the school. The coefficient estimate on the proportion of contractual teachers in the school,

which is large and negative, suggests that student performance is higher in schools where the

share of contractual teachers is lower. This is true for all students, whether their own teacher is a

contractuel or instituteur. Also, after inclusion of this variable, however, the coefficient estimate

on contractual teacher remains large and negative, suggesting that the contractual teacher

variable is not merely picking up school- level unobservables.

Further, even though as described above contractual teachers are more likely to report

feeling underpaid and receiving their pay on a very irregular basis, the estimated coefficients on

these variables are not statistically different from zero, and their inclusion does not affect the

coefficient estimate on contractual teachers5.

We did not find effects on student achievement of variables related to school or

classroom facilities, materials, and organization (other than those directly associated with teacher

characteristics). We also explored whether the effect of contract status on student achievement

varies by school- and classroom-level characteristics and found no evidence.

4 Teachers in our data were asked whether they consider their pay to be sufficient or insufficient. We explored whether the responses of contractual teachers with 5 or less years of experience differed from those of contractual teachers with more than 6 years of experience; we found that they did not. In addition, we also explored the effect of this dichotomous variable on student achievement , controlling for all other relevant student background, school, and teacher and classroom variables. We did not find a significant effect. 5 We further explored the disgruntled worker / fair-wage hypothesis by interacting the teacher’s contractual status with the proportion of contractuals in the school. If it holds, one might expect contractual teacher to underperform in schools in which the proportion of non-contractuals is high. The estimated coefficient on this interaction was not significant.


Given that our data do not support the hypothesis that the negative effect on student

achievement of teacher contract status is explained by differences in the schools or classrooms of

instituteur and contractual teachers, we asked whether the introduction of contractual hiring

practices led to a decline in the quality of the supply of teacher entrants. To explore this

question, we repeated the previous analysis on the subsample of teachers with between four and

six years of experience. These are teachers who likely had already entered teacher training

institutes when the policy was implemented and thus, in a sense, were “stuck” in the profession.

If there was a decline in the quality of teacher entrants, then we should not observe a negative

coefficient estimate on the contractual indicator among the subsample of teachers with four to six

years of experience, as they likey do not differ in systematic ways from instituteurs.

Indeed, the results, presented in Table 7, support the hypothesis that the implementation

of the alternative teacher hiring mechanism resulted in a decline in the quality of teacher

entrants. The coefficient estimate on the contractual indicator is not statistically different from

zero among teachers with 4-6 years of experience.

VII. Conclusions and Policy Implications

Experimenting with alternative teacher contracting practices is ever more necessary in

countries with low enrollment rates and constrained budgets. Togo, as many other West African

countries, initiated reforms in teacher contracting practices to meet a growing demand for

education at a time of growing government budgetary deficits. While this alternative system of

teacher contracting and compensation likely enabled Togo to expand access to primary

education, their effects on student learning are important to understand.

In this paper, we have analyzed rich data on students, schools and teachers to better

understand the impact of contractual teachers on student achievement in Togo. It seems likely

that contractual teachers, who are usually equally or better educated than regular teachers, would


be at least as effective as regular teachers in generating student learning, if not more – given the

potential threat of not having their contracts renewed. For this to occur, however, the threat of

not having their contracts renewed must be credible and the nonrenewal should be related to

teacher performance. Anecdotal evidence suggests that this is not the case in Togo, as

contractual teachers are rarely, if ever, let go.

Our analyses indicate that students of contractual teachers systematically underperform

(at least in mathematics) when compared to students of regular teachers, even after controlling

for prior student achievement, household characteristics and a variety of school and classroom

variables. Given the richness of our data regarding students, teachers, classrooms and schools,

we explored whether differences among contractual and regular teachers’ classrooms and

schools contributes to explain the gap in student achievement after controlling for student

background. We were unable to establish a relationship between these classroom and school

characteristics and the gap in achievement between students of contractual and regular teachers.

A preliminary exploration of whether the negative effect of contractual status is due to a

decline in the (unmeasurable) quality of teacher supply suggests that, in fact, the new policy may

have led to such a decline in teacher quality. When we investigated the effect of teacher

contract, controlling for student background, school and classroom characteristics, on student

achievement among teachers who likely entered teaching just around the time of the policy

change (and, thus, did not expect to become contractual teachers ex-ante), we find no

achievement gap between students of contractual and regular teachers. If, indeed, the negative

effect of contractual teachers is mostly due to a decline in the quality of those choosing to enter

teaching, the long-term negative effects of this policy change could be enormous. However, the

extent of such negative effects may importantly depend on the availability of alternative

employment opportunities for individuals considering to become teachers (Loeb and Page 2000).

The fact that the performance gap was smaller for less experienced contractual teachers who


were hired during increasingly worsening economic conditions suggests that the negative effects

of reducing teacher pay and benefits may be diminished during economically difficult times. In

addition, the positive effect of teacher education on student performance suggests a potentially

cost-effective way of further mitigating the negative effects on student learning arising from

switching contractual regimes.

Nevertheless, several important questions regarding the effects of alternative teacher

contracts remain unanswered by our research. For example, what aspects of teache r education

matter most for student learning, and which particular alternative employment opportunities

matter most for students in poor countries considering to become teachers? And, what are the

effects of such policy changes on teacher turnover? Countries in West Africa and elsewhere are

finding new and innovative ways of hiring the teachers they need to expand access to primary

and secondary education. Many rigid, expensive teacher salary structures prevailing in

developing countries are difficult to sustain in periods of budgetary crises. However,

understanding the effects of changes in teacher compensation and structures on teacher

performance and, ultimately, on student outcomes is critical. This paper is a contribution in this

direction.


References

Akerlof, George A., and Janet L. Yellen. 1990. “The Fair Wage-Effort Hypothesis and Unemployment,” Quarterly Journal of Economics, 105(2): 255-283.

Baker, Frank, 2001. The Basics of Item Response Theory. Wisconsin: Eric Clearing House on

Assessment and Evaluation. Ballou, Dale and Michael Podgursky. 1998. “Teacher Recruitment and Retention in Public and

Private Schools,” Journal of Policy Analysis and Management, 17(3): 393-417. Case, Anne, Angus Deaton. 1999. “School Inputs and Educational Outcomes in South Africa,”

Quarterly Journal of Economics, 114(3): 1047-1084. Glewwe, Paul, and Hanan Jacoby. 1994. “Student Achievement and Schooling Choice in Low-

Income Countries: Evidence from Ghana,” The Journal of Human Resources, XXIX(3): 843-864.

Government of Togo, 2002. “Le Système Educatif Togolais Eléments d’Analyse pour une

Revitalisation”, March. Hanson, Brad. 2002. IRT Command Language. Available at: (http://www.b-a-

h.com/software/irt/icl/). Hanushek, Eric A. 2002. “The Failure of Input-Based Schooling Policies”, NBER Working

Paper 9040. ________. 1997. “Assessing the Effects of School Resources on Student Performance: An

Update,” Educational Evaluation and Policy Analysis, 19(2): 141-164. ________. 1986. “The Economics of Schooling: Production and Efficiency in Public Schools,”

Journal of Economic Literature, 24 (3): 1141-77. Hanushek, Eric A., John F. Kain and Steven G. Rivkin. 1998. “Teachers, Schools and

Academic Achievement,” NBER Working Paper No. 6691. Harbison, Ralph W. and Eric A. Hanushek. 1992. Educational Performance of the Poor:

Lessons from Rural Northeast Brazil. New York: Oxford University Press. International Monetary Fund.1998. “Togo: Selected Issues”, IMF Staff Country Report No.

98/21.

Kennedy, Peter. 1992. A Guide to Econometrics (Third Edition). Cambridge, MA: The MIT Press.

Linden, W. van der, & R.K. Hambleton. 1997. Handbook of modern item response theory.

Heidelberg, Germany: Springer-Verlag.


Loeb, Susanna and Marianne E. Page. 2000. “Examining the link between teacher wages and

student outcomes: The importance of alternative labor market opportunities and non-pecuniary variation,” The Review of Economics and Statistics, 82(3): 393-408.

Todd, Petra, and Kenneth I. Wolpin. 2001. “On the Specification and Estimation of the

Production Function for Cognitive Achievement.” Mimeo, University of Pennsylvania.

UNESCO. 2002. “Statistics online.” Available at: (http://www.uis.unesco.org/en/stats/stats0.htm).

World Bank, 2002. World Development Indicators.


Table 1: Share of Current Public Education Expenditure by Purpose, Selected Countries in Western Africa

Year Teaching staff Non-teaching Teaching materials

Burkina Faso 1996 82.04% 17.96% 0.01% Central African Republic 1990 85.62% 14.38% 1.69% Chad 1996 87.86% 12.14% 6.91% Guinea 1993 83.37% 16.63% ./ Madagascar 1993 74.25% 25.75% ./ Mali 1993 60.90% 39.10% 4.63% Niger 1991 73.68% 26.32% 6.08% Togo 1996 96.08% 3.92% 3.06% Source: UNESCO 2002.


Table 2: Gross and Net Primary Enrollment Rates in Togo, 1990-1999.

Year Gross Enrollment Rate (%) Net Enrollment Rate 1990 109.4 74.7 1991 106.8 .. 1992 104.2 .. 1993 101.8 69.0 1994 113.3 77.8 1995 118.6 85.0 1996 119.6 81.3 1997 119.5 82.9 1998 133.3 94.9 1999 123.8 91.4 Source: World Bank 2002.


Table 3: Description of Variables Used in Analyses Variable Name Description

Irtscore_postm Mathematics IRT score in post-test Irtscore_prem Mathematics IRT score in pre-test Girl Dichotomous variable indicating the students’ gender (1= female,

0=male) Age Age of the student, in years Pc1asset First principal components composite of household assets Mother_lit Dichotomous variable indicating whether mother can read and write French Dichotomous variable indicating whether French is spoken at home Math_bk Dichotomous variable indicating whether the student has a math

textbook Study_hm Dichotomous variable indicating whether the student studies at home Help_hm Dichotomous variable indicating whether the student can get help for

his/her studies at home Pc1hhtask First principal components composite of student’s participation in

household tasks

Stud

ent-

leve

l var

iabl

es

Pc1hhmeal First principal components composite of student’s meals Private Dichotomous variable indicating whether the school is private Urban Dichotomous variable indicating whether the school is located in an

urban area Eff_tot Total number of students in the school Pc1schfac First principal components factor of school facilities Pc1schins First principal components factor of role of inspector to school Pc1schadv First principal components factor of role of counselor/advisor to school Sc

hool

-lev

el

vari

able

s

Contr Proportion of contractual teachers in the school (#contractual/#total) Contractuel Dichotomous variable indicating that the teacher is a contract teacher Experience Dichotomous variable indicating that the teacher has five or less years of

experience Education Teacher’s years of education Underpaid Dichotomous variable indicating the teacher’s report of whether s/he is

underpaid Pc1classfac First principal components factor of classroom facilities Abs_yr Total number of days absent in school year Abs_mo Total number of days absent in April (last month) Cls_size Class size (total number of students in the class) Virregular Dichotomous variable indicating that teacher reports to receive pay on a

very irregular basis Pc1pta First principal components factor of teacher’s relationship with parents Pc1collab First principal components factor of teacher’s collaboration with other

teachers in the school Pc1commit First principal components factor of teacher’s commitment to student

learning

Tea

cher

-leve

l var

iabl

es

Pc1hwinv First principal components factor of teacher’s involvement in student homework

Note: Teacher gender is not a variable included in our analyses because less than 4 percent of contractual and instituteur teachers are female.


Table 4: Means (Standard Deviations) of Variables Used in the Analyses

Variable Instituteur n=516

Contractual n=321

Proportion of explained variance

(if applicable)

Alpha reliability coefficient estimate

(if applicable) IRT post math score 0.278

(0.998) -0.288 (0.987)

IRT prior math score 0.151 (1.028)

-0.140 (0.949)

Girl (%)

0.475

0.417

Age 10.994 (1.974)

11.791 (1.878)

Household assets index (pc1)6

0.535 (2.398)

-0.707 (1.764)

0.262 0.633

Literate mother (%)

0.529

0.392

French spoken at home (%)

0.287

0.234

Math book at home 0.494

0.299

Studies at home (%) 0.946

0.903

There is someone at home to help study (%)

0.661

0.558

Household tasks index (pc1)

-0.124 (1.476)

0.174 (1.305)

0.254 0.543

Stud

ent-

leve

l var

iabl

es

Lack of regular meals at home index (pc1)

-0.276 (1.141)

0.091 (1.288)

0.434 0.508

Private school (%)

0.538

0.100

Urban school (%)

0.546

0.358

Total students at school 238.575 (82.914)

237.284 (79.322)

School facilities index (pc1)

0.390 (1.995)

-0.660 (0.847)

0.218 0.573

School inspector visits index (pc1)

0.060 (2.224)

-0.102 (1.272)

0.594 0.783

Teaching advisor visits index (pc1)

-0.021 (1.837)

-0.227 (1.354)

0.523 0.763 Scho

ol-le

vel v

aria

bles

Contractual teachers in school (%)

0.196 (0.187)

0.329 (0.180)

Experience (% with 0/5 yrs exp)

0.172

0.454

Education (years)

12.052 (1.561)

12.940 (1.792)

Underpaid (self-reported)

0.868

0.931

Class facilities index (pc1)

0.388 (1.519)

-0.345 (1.330)

0.305 0.608

Days absent per year 6.018 (7.615)

5.682 (4.838)

Tea

cher

-leve

l var

iabl

es

Days absent last month 1.502 (2.786)

1.676 (3.446)

6 Pc1: (first) principal component index


Class size 35.591 (15.402)

36.557 (13.029)

Receives salary very irregularly

0.083

0.358

Discuss student performance w/ parents index (pc1)

0.074 (1.384)

0.048 (1.346)

0.647 0.690

Teacher support and collaboration in school index (pc1)

0.064 (0.815)

.002 (0.805)

0.355 0.669

Teacher tutors students for free index (pc1)

0.089 (0.890)

-0.207 (0.847)

0.700 0.785

Teacher control / feedback over student work index (pc1)

0.107 (0.751)

0.204 (0.635)

0.323 0.797


Table 5: Decomposition of estimated variance in student characteristics by source, Togo 2001 – Upper Bounds on the Between Source Variation

Variable name Between classes (schools) as a percent

of total

Within classes (between students) as

a percent of total

Total variance

IRT post math score 0.50 0.50 1.00 IRT prior math score 0.40 0.60 1.00 Household assets index (pc1)

0.50 0.50 1.00

Literate mother 0.28 0.72 1.00 Studies at home 0.18 0.82 1.00 There is someone at home to help study

0.17 0.83 1.00


0.38 0.62 1.00


0.27 0.73 1.00


Table 6: Estimated coefficients (and robust standard errors) from OLS regressions of the effect of teacher contract on student learning, controlling for student background, school and classroom characteristics [n=837] Model Student background School

characteristics Teacher & classroom

characteristics 1 2 3 4(a) 5(b) Intercept 0.539*

(0.238) 0.002

(0.269) 0.343

(0.298) -0.793 (0.506)

-0.690 (0.505)

IRT prior math score 0.488*** (0.038)

0.476*** (0.039)

0.480*** (0.040)

0.434*** (0.041)

0.430*** (0.041)

Girl -0.051 (0.067)

Age -0.043* (0.020)

-0.021 (0.022)

-0.025 (0.020)

-0.018 (0.019)

-0.011 (0.019)

Household assets index (pc1)

0.097*** (0.018)

0.055** (0.017)

0.064** (0.020)

0.052** (0.015)

0.038* (0.014)

Literate mother 0.098 (0.064)

French spoken at home 0.095 (0.078)

Math book at home 0.082 (0.081)

Studies at home 0.113 (0.105)

There is someone at home to help study

0.089 (0.068)


-0.058* (0.027)

-0.040 (0.027)

-0.061** (0.023)

-0.048~ (0.023)


-0.069* (0.032)

-0.075* (0.034)

-0.051~ (0.029)

-0.041 (0.029)

Private school 0.093 (0.135)

Urban school 0.078 (0.145)

Total students at school

-0.000 (0.000)

School facilities index (pc1)

0.002 (0.027)

Experience (<6 yrs exp)

-0.090 (0.157)

-0.103 (0.147)

Education (years)

0.097** (0.034)

0.093** (0.034)

Contractual teacher -0.640*** (0.160)

-0.596*** (0.163)

Experience* Contractual

0.561** (0.223)

0.598** (0.219)

Contractual teachers in school (%)

-0.647* (0.270)

R-squared statistic 0.342 0.362 0.358 0.414 0.424 Notes: ~p-value<.10; *p-value<.05; **p-value<.01; ***p-value<.001 (a)A test of the null hypothesis that the joint effects of contractual, experience and their two-way interaction is zero is rejected at the 1 percent level. (b) We also tested the null hypothesis that the interaction between contractual status and the proportion of contractuals in the school is different from zero, but we were not able to reject it.


Table 7: Estimated coefficients (and robust standard errors) of the effect of contract status on student achievement controlling for student background and teacher characteristics in subsample of teachers with 4-6 years of experience Dependent variable : Post-test IRT math scores Intercept -0.812

(0.951) IRT prior math score 0.583***

(0.066) Age -0.041

(0.029) Household assets index (pc1) 0.065*

(0.030) Household tasks index (pc1) -0.061~

(0.030) Regular meals at home index (pc1) 0.024

(0.058) Experience (< 6 years) 0.364~

(0.175) Education (years) 0.075

(0.062) Contractual teacher 0.297

(0.272) Contractual*Experience -0.212

(0.261) R-squared Statistic 0.516 Notes: ~p-value<.10; *p-value<.05; **p-value<.01; ***p-value<.001


Figure 1: IRT End-of-Year Test Scores by Teachers’ Contractual Status

Ability/Knowledge

Test2 IRT Score - Contractuel Test2 IRT Score - Instituteur

-3.5 -3 -2.5 -2 -1.5 -1 -.5 0 .5 1 1.5 2 2.5 3 3.5

0

.05

.1

.15

.2

.25

.3

.35

.4

.45

.5


Figure 2: Prototypical Fitted Lines of Student IRT Math Post-Test Scores by Teacher Education, by Contract Status and Experience7

-0.8

-0.6

-0.4

-0.2

0

0.2

0.4

0.6

8 9 10 11 12 13 14 15 16

Teacher's years of education

Pre

dic

ted

IRT

mat

h p

ost

-tes

t sc

ore

Contractuel - 6+ Yrs of Exp. Contractuel - <5 Yrs. Of Experience

Instituteur - 6+ Yrs of Exp. Instituteur - <5 Yrs. Of Experience

7 To construct this plot, we use the sample average values and sample ranges for all variables in Model 4.


Appendix: Item Response Theory and Estimates

An extensive literature in psychological measurement has shown the superiority, in terms

of efficiency and bias, of the use of item response theory over the classical “raw score” approach

in measurements of latent knowledge distributions using test scores. a detailed discussion of item

response theory and its benefits can be found in the Handbook of modern item response theory

by van der Linden and Hambleton (1997).

We use IRT to compute test scores instead of raw test scores because it explicitly takes

into account that different test items are of varying difficulty. In IRT, each item is modeled as

having a unique item characteristic curve, which gives the probability of observing a correct

response at each point in the latent ability/knowledge distribution. Most educational test

measurement research uses 1,2, or 3 parameter distributions from the logistic family to model the

observed pattern of responses as a function of the latent trait for each item, although non-

parametric methods are increasingly employed too. The shape of this characteristic curve reflects

two main dimensions of the item; its difficulty, and its ability to discriminate between

individuals at different point on the support of the distribution. The more ‘difficult’ a question is,

the lower the probability that a student of a given ability will answer the item correctly.

Technically, the “difficulty of an item” is the point in the latent ability distribution at which the

probability of answering the item correctly is 0.5. Accordingly, in IRT, a student will obtain a

higher test score than another student with the same raw score, if she answered more difficult

items correctly than the other student. The precision with which the latent points are estimated

depends in large part on the degree to which the test items are capable of discriminating between

students below and above given points in the latent distribution, which in turn depends on the

slope of the item characteristic curve at given points. For example, if no students below some

latent ability point answer the item correctly, but all students above do, perfect discrimination at

that point of the distribution is achieved. An ideal test designed to accurately assess the entire

range of values in the latent distribution contains a series of items which differ in their difficulty

and cover a range of values of the latent points where the highest degree of discrimination (i.e.

the maximum slope) is achieved.

Since our Togo test contained multiple choice questions, we followed standard procedure

and estimated the item parameters and latent distribution under the assumption that the responses

to each item followed a 3-parameter logistic model, which allows for guessing by students when


the test contains multiple choice questions. Also called a three parameter Rasch model, the

probability of a correct response to item j as a function of the latent variable θ is modeled as:

)(1

1)|1Pr(

jj bDaj

jje

ccX −+

−+== θθ

where Xj is a random variable representing a response to item j (where 1 indicates a correct

response), aj (the “discrimination” parameter), bj (the “difficulty” parameter), and cj (the

“guessing” parameter) are item parameters, and D is a scaling constant. These three unique

parameters for each item k are estimated jointly with the individual specific parameters θi for all

N examinees. In light of the relatively small sample size, we took a Bayesian perspective and

calculated EAP (expected a posteriori) estimates of the latent variable distribution using the

mean of the posterior latent variable distribution for each examinee.8 The prior latent distribution

on θ is assumed to be standard normal.

Estimates of the latent knowledge distribution were computed using the ICL computer

program, which is freely available online at http://www.b-a-h.com/software/irt/icl/. It can

compute maximum likelihood or Bayes modal estimates of item parameters by finding the

maximum of the marginal likelihood or posterior distribution, where the marginalization is over

a discrete distribution of latent examinee proficiency in the population of examinees from which

the data were sampled. The EM algorithm is used to compute the maximum likelihood or Bayes

modal estimates (Hanson, 2002).

Once the latent distribution has been estimated together with the three parameters for

each of the items, one can draw the estimate of the item characteristic curve. Panels (a) and (b) of

Figure A1 present examples of two such curves. The first characteristic curve, which

corresponds to item three in the second test, is somewhat flat particularly at the high end of the

distribution. This suggest that this particular question adds little information to distinguish

between people over the entire support of the latent distribution, but particularly fails to

distinguish between people at the high end of the distribution. This is captured by the item

information function, drawn in the same graph, which corresponds to the slope of the item

characteristic curve. The second characteristic curve, which corresponds to item five in the same

test, is very steep in the center of the latent distribution, suggesting few people below the point θ

8 The IRT Command Language (ICL) , which is available at http://www.b-a-h.com/software/irt/icl/ , was used for this analysis.


where the gradient is highest have the correct answer, whereas most people above that point do

answer this item correctly.

The overall test information is captured in a similar graph, the test characteristic curve

and the associated test information curve, which are nothing more than the sum of the individual

item curves. Thus it can be readily seen that an ideal test would contain many questions, each

with high discrimination parameters, but with difficulty parameters over the entire support of the

latent distribution.

Figure A2 contains the test information of the beginning-of-year IRT test scores (before

standardizing) and the estimates of the test standard errors over a hypothetical support of θ . The

test standard errors can be obtained by taking the square root over the inverse of the test

information. The test information curve clearly indicates that standard errors fall quickly

between approximately θ = –0.5 and θ =2. The imputed knowledge for the 845 students used in

the empirical analysis ranged from θ = -2.73 and θ = 2.93.

The second test, which contains 76 questions instead of the 36 in the first test, is clearly

more informative about the underlying distribution, as measured by the greater range on the

support of the distribution over which standard errors are relatively low (see Figure A3). The

standard errors are fairly flat between –1 and 4. The imputed knowledge for the 845 students

used in the empirical analysis ranged from θ = -48 to θ = 2.77. To put the beginning-of-year

and end-of-year measures on the same scale, both scores were standardized (0,1) in the empirical

analysis.


Figure A1

(a) P

roba

bilit

y C

orre

ct

Characteristics of Item3Ability/Knowledge

Item

Info

rmat

ion

Item Characteristic Curve Item Information Curve

-4 -3.5 -3 -2.5 -2 -1.5 -1 -.5 0 .5 1 1.5 2 2.5 3 3.5 4

0

.1

.2

.3

.4

.5

.6

.7

.8

.9

1

.002572

.086962

(b)

Pro

babi

lity

Cor

rect

Characteristics of Item5Ability/Knowledge

Item

Info

rmat

ion

Item Characteristic Curve Item Information Curve

-4 -3.5 -3 -2.5 -2 -1.5 -1 -.5 0 .5 1 1.5 2 2.5 3 3.5 4

0

.1

.2

.3

.4

.5

.6

.7

.8

.9

1

7.8e-07

.650218


Figure A2

Sta

ndar

d E

rror

Test Characteristics Beginning-of-YearAbility/Knowledge

Test

Info

rmat

ion

Test Standard Error Test Information

-4 -3.5 -3 -2.5 -2 -1.5 -1 -.5 0 .5 1 1.5 2 2.5 3 3.5 4

.271981

1.82779

0

13.5183


Figure A3

Sta

ndar

d E

rror

Test Characteristics End-of-YearAbility/Knowledge

Test

Info

rmat

ion

Test Standard Error Test Information

-4 -3.5 -3 -2.5 -2 -1.5 -1 -.5 0 .5 1 1.5 2 2.5 3 3.5 4

.2183

2.47718

0

20.9842

Date post:	02-Jan-2020
Category:	Documents
Upload:	others
View:	0 times
Download:	0 times

Do differences in teacher contracts affect student ... · difficult to identify the effects of...

Documents