Education Finance and Policy Just Accepted MS. doi:10.1162 ... · PDF fileaccess to the data;...

1

Title: The Effects of Subgroup-Specific Accountability on Teacher Turnover and Attrition

Author: Matthew Shirrell

Running Head: Effects of Subgroup Accountability

Affiliation: Graduate School of Education and Human Development, the George Washington

University

Postal Address: 2134 G Street NW, Washington DC 20052 USA

E-mail Address: [email protected]

Acknowledgements: Work on this paper was primarily completed while the author was at

Northwestern University. The research was supported by funding from the Albert Shanker

Institute, the National Academy of Education/Spencer Foundation, and the Institute of Education

Sciences, U.S. Department of Education (through grant R305B080027 to Northwestern

University). The opinions expressed herein do not represent the views of the funders. The author

thanks the North Carolina Education Research Data Center at Duke University for providing

access to the data; David Figlio, Kirabo Jackson, Randall Reback, Jim Spillane, and two

anonymous reviewers for their valuable input; and Kara Bonneau and Bruce Foster for their help

with data.

Education Finance and Policy Just Accepted MS.doi:10.1162/EDFP_a_00227 by Association for Education Finance and Policy

2

Abstract:

The No Child Left Behind Act of 2001 required states to set cutoffs to determine which schools

were subject to accountability for their racial/ethnic subgroups. Using a regression discontinuity

design and data from North Carolina, this study examines the effects of this policy on teacher

turnover and attrition. Subgroup-specific accountability had no overall effects on teacher

turnover or attrition, but the policy caused black teachers that taught in schools that were held

accountable for the black student subgroup to leave teaching at significantly lower rates,

compared to black teachers that taught in schools not accountable for the black subgroup’s

performance. The policy also caused shifts in the students assigned to black teachers, with

schools that were held accountable for the black subgroup less likely to assign black students to

black teachers the following year. These findings demonstrate that subgroup-focused policies—

particularly those that use cutoffs to determine subgroup accountability—can shape the

composition of the teacher labor force in unintended ways, and have implications for the design

of future accountability systems that aim to close racial/ethnic gaps in achievement.


3

<A>1. Introduction

Significant gaps persist between the academic achievement of students of various races

and socioeconomic levels (Reardon and Robinson 2008). In recent decades, policies have

attempted to close these achievement gaps in a variety of ways, including by implementing

school desegregation orders, encouraging students and their families to move to different

neighborhoods or schools, and instituting promotion or graduation exams (Harris, 2008). Despite

these interventions, these achievement gaps have remained sizeable, and perhaps have even

widened in recent years (Reardon and Robinson 2008; Reardon 2011).

The No Child Left Behind Act of 2001 (NCLB) attempted to close these achievement

gaps in a novel way: by requiring schools to make yearly improvements not only in overall

student achievement, but also in the achievement of students of various subgroups, including

racial/ethnic minority subgroups. NCLB required that the students in each of these subgroups in

each school make yearly gains in achievement (what the law termed “Adequate Yearly

Progress,” or AYP), or the entire school would be labeled as failing. This “subgroup-specific

accountability” had major consequences for schools across the nation, as subgroups’ passage or

failure was a major contributor to whether schools passed or failed the accountability targets set

by NCLB, and thus the degree of sanction that they faced under the law (Reback, Rockoff, and

Schwartz 2014; Davidson et al. 2015).

Recent research has found mixed impacts of subgroup-specific accountability on student

achievement. While some work finds that the policy had the intended effects of raising minority

student achievement (Lauen and Gaddis 2012), and of narrowing at least some racial/ethnic

achievement gaps (Reardon et al. 2013), other work finds that the policy increased schools’

failure rates and lowered schools’ overall performance and the performance of minority students


4

in particular (Sims 2013). Despite these advances, however, we know little about the policy’s

effects on teachers, which is important to gain a more complete picture of the policy’s overall

impact. If the policy increased student achievement but resulted in the loss of many teachers, for

example, these benefits and costs should be weighed together when assessing the policy’s overall

effects.

There are a variety of reasons to believe that subgroup-specific accountability might have

affected teachers. State-level accountability policy affects teachers’ decisions about whether to

remain in or leave their schools or teaching (Clotfelter et al. 2004; Boyd et al. 2008; Feng, Figlio,

and Sass 2010), and NCLB altered teachers’ perceptions of their job security (Reback et al.

2014) and affected principals’ decisions about whether to remain in or leave their schools (Li

2012). NCLB also implemented subgroup-specific accountability using cutoff-based rules that

divided otherwise similar schools into those that were accountable for their racial/ethnic

subgroups and those that were not; prior studies demonstrate that setting cutoffs can affect

individuals whose scores fall either just above or, in some cases, just below those cutoffs (Papay,

Murnane, and Willett 2010; Domina, Penner, and Penner 2016). The implementation of NCLB’s

subgroup-specific accountability might thus have shaped teachers’ decisions to remain in or

leave their schools or teaching, as well as administrators’ decisions about which teachers to

retain, particularly in schools that fell near these cutoffs.

This study investigates whether the initial implementation of NCLB’s subgroup-specific

accountability affected the likelihood that teachers left their schools or teaching. Using data on

elementary school teachers in North Carolina, this study finds that when first implemented,

subgroup-specific accountability that targeted the black or white student subgroups had no

overall effects on teacher turnover or attrition. Accountability for the black student subgroup,


5

however, caused black teachers in schools that were held accountable for the performance of the

black subgroup to leave teaching at significantly lower rates than black teachers that taught in

schools not accountable for the black subgroup’s performance. It is unclear, however, whether

this discontinuity is driven by a decrease in black teacher attrition in schools just above the cutoff

or an increase in attrition of black teachers in schools just below. Further analysis suggests that

the policy caused changes in the teaching assignments of black teachers, particularly the

percentages of black students they were assigned. These results suggest that the implementation

of subgroup-specific accountability altered the composition of the teacher labor force in

unintended ways.

<A>2. Background

Subgroup-Specific Accountability

A central goal of NCLB was to close racial/ethnic achievement gaps (No Child Left

Behind Act 2001); to do so, the law required that the students in each racial/ethnic subgroup in

each school make yearly gains in achievement, or the entire school would be labeled as failing.

NCLB did not hold all schools accountable for the performance of their subgroups, however; to

address concerns about the reliability of AYP determinations, as well as about student privacy,

NCLB stipulated that a subgroup’s performance would not “count” towards AYP determinations

for schools with very small numbers of students in that subgroup. NCLB required states to set a

“minimum subgroup size,” which was the minimum number of students in a subgroup that were

required for that subgroup to count in a school’s AYP determination (Davidson et al. 2015).

Schools whose numbers of students in a subgroup were equal to or greater than their state’s

minimum subgroup size were held accountable for that subgroup’s performance, while schools


6

whose numbers of students in a subgroup fell below that cutoff were not held accountable for

that subgroup’s scores. In the first year of NCLB, these minimum subgroup sizes ranged from a

low of 10 to as high as 50 (Pierce 2003; Davidson et al. 2015). The state examined in this study,

North Carolina, set its minimum subgroup size at 40 students, and did not allow this number to

vary with the size of the school (Pierce 2003; Fulton 2006).

One consequence of these minimum subgroup size rules was that the difference of a

single student in a subgroup could affect whether that subgroup’s test scores counted in

determining a school’s AYP. In North Carolina, for example, the black student subgroup’s

performance did not count in AYP determinations for schools with 39 black students, but

counted for schools with 40 black students. NCLB thus required states to draw arbitrary lines

that divided otherwise similar schools, holding schools on one side of the line accountable for the

performance of a subgroup while leaving schools on the other side unaccountable. This study

investigates whether implementing such a policy caused teachers that worked in schools near

these cutoffs to leave their schools or teaching.

North Carolina is an ideal place to examine the effects of the initial implementation of

subgroup-specific accountability, since prior to NCLB, North Carolina had a strong state-level

accountability system, but no system of subgroup-specific accountability (North Carolina

Department of Public Instruction 2003; Lauen and Gaddis 2012). The effects of subgroup-

specific accountability in NCLB’s first year were therefore undiluted by a similar state-level

policy. Since North Carolina’s state-level accountability system remained in effect when NCLB

was implemented in 2002-03, the counterfactual to subgroup-specific accountability in this study

is a strong state-level accountability system that lacked a subgroup-specific component. The

rewards and sanctions faced by North Carolina schools did not change in NCLB’s first year:


7

under North Carolina’s state-level accountability system, staff at high-performing schools

received monetary awards, while low-performing schools were required to work with state

assistance teams and notify parents of their low performance (North Carolina Department of

Public Instruction 2003). These rewards and sanctions remained in place in 2002-03, but schools

additionally reported their overall AYP status in mandatory reports to parents. NCLB sanctioned

Title I schools if they did not make AYP, but only after schools failed to do so for two

consecutive years (North Carolina Department of Public Instruction 2003).

Holding Schools Accountable Versus Low Performance

Most prior studies of the effects of NCLB’s subgroup-specific accountability have

explored the effects of low subgroup performance, or the threat of such low performance, on

students and teachers (Lauen and Gaddis 2012; Reardon et al. 2013; Reback et al. 2014). Lauen

and Gaddis (2012), the only previous study to explore the effects of subgroup-specific

accountability in North Carolina, found that accountability pressure for the low-income, black,

and Hispanic subgroups increased the subsequent achievement of those students. The

“treatment” in the current study, however, differs from this prior work. Instead of investigating

the effects of pressure generated by the implementation of subgroup-specific accountability, this

study explores the effects of simply being held accountable for subgroups during NCLB’s first

year. More precisely, the treatment in this study can be thought of as a school being held

accountable for a subgroup while otherwise similar schools were not. Sims (2013) similarly

examined the impacts of being held accountable for an additional subgroup on both the

probability of schools’ failing to make AYP and their subsequent student achievement. He found

that schools that were held accountable for an additional subgroup were more likely to fail to

make AYP, and that such failure led to decreased future achievement for both the school overall


8

as well as the subgroup in question (Sims 2013).

Subgroup-specific accountability could have affected different groups of teachers in

different ways. Since subgroup-specific accountability was a policy that focused explicitly on

student race, it is particularly plausible that the policy had different effects on teachers of various

races. Prior research shows, for example, that black and white teachers respond differently to an

influx of black students into their schools (Jackson 2009). Research also suggests that an

organization’s orientation towards diversity is a key predictor of workers’ commitment to that

organization, particularly for minority workers (Foley, Kidder, and Powell 2002; Chrobot-Mason

2003), and minority workers’ perceptions of their workplaces’ “diversity climates” are key

predictors of their turnover (Griffeth and Hom 2001). The policy could also have impacted

teachers directly, by affecting teachers’ decisions about whether or not to remain in their schools

or teaching, or indirectly, by altering administrators’ decisions about teachers’ class assignments,

outside-of-classroom responsibilities, and which teachers to remove or keep.

There are four main ways that being held accountable for a subgroup might affect teacher

turnover or attrition. One is by altering the pressure that teachers felt to improve their students’

achievement. Since schools that were accountable for more subgroups were more likely to fail to

make AYP (Kane and Staiger 2002; Sims 2013), schools whose numbers of students in a

subgroup were just above the minimum subgroup size cutoff might have felt more accountability

pressure than teachers in schools just below the cutoff. Accountability pressure has been shown

to decrease teachers’ perceptions of their job security (Reback et al. 2014; Crocco and Costigan

2007), and the decreased job security for teachers in above-cutoff schools could have driven

these teachers from their schools or even from teaching. Changes in such pressure could also

have altered administrators’ decisions about classroom assignments or about which teachers to


9

keep or remove. If black teachers were more effective teachers of black students, for example,

administrators of schools that were newly accountable for the black subgroup could have

reassigned black students to those teachers in response to the increased pressure associated with

those students’ performance.

A second way that subgroup-specific accountability might have affected teachers is by

altering the allocation of resources either within or between schools. Prior evidence suggests that

increased resources often accompany accountability pressure, and that these resources can

potentially entice teachers to stay in their schools (Feng et al. 2010). Subgroup-specific

accountability could have led to increased resources—including materials, planning time, and

other supports—being devoted to schools just above the cutoff as compared to those just below;

these additional resources could have made teaching more appealing and enticed teachers to stay

in their schools or in teaching. Within schools, administrators may have made particular efforts

to direct resources to teachers that taught the students for which schools were newly accountable,

making these teachers particularly likely to remain. Students could also have been assigned to

teachers differently after the policy took effect. Again, this change could have affected teachers

of different races differently. If black teachers were more likely to teach black students, for

example, and increased resources were directed to black students (and their teachers) in schools

that were newly accountable for the black subgroup, these resources could have been particularly

beneficial to black teachers.

Third, subgroup-specific accountability could have affected teachers’ motivation to

remain in their schools or teaching. Again, these effects could have been different for teachers of

different races. For example, research suggests that some black teachers enter teaching because

of a “calling” prompted by their sense of duty to improve educational outcomes for black


10

students (Casey 1993; Irvine 2002; Dixson and Dingus 2008). In schools above the subgroup

accountability cutoff, black students’ performance “counted” towards AYP, reinforcing the

message that black students’ achievement was important, which may have been particularly

congruent with some black teachers’ motivations for entering and remaining in teaching. In

below-cutoff schools, however, black students’ performance did not “count,” a message which

could have conflicted with these teachers’ motivation to teach, perhaps driving them from their

schools or even from teaching.

Finally, subgroup-specific accountability could have impacted teachers by causing school

leaders to strategically manipulate their schools’ numbers of students in various subgroups. Prior

work finds that schools indeed manipulate their numbers of students in subgroups in order to

avoid NCLB accountability for those subgroups (Jennings and Crosta 2011). This finding

corroborates findings from other studies suggesting that some schools use suspensions and

classification into special education to remove potentially low-performing students from the

testing pool (Cullen and Reback 2006; Figlio 2006; Figlio and Getzler 2006). Teachers could

have directly observed this manipulation or heard of it second-hand, and knowledge of this

manipulation could have affected their motivation to remain in their schools or even teaching.

Such manipulation could also have been correlated with other practices that some administrators

engaged in in response to NCLB that made schools particularly bad places to work.

The impacts of subgroup-specific accountability on teachers also may not have been

confined to the schools above the cutoffs; recent research in education suggests that cutoffs that

divide otherwise similar individuals into groups can affect behavior for those that fall just short

of those cutoffs, as well. Domina, Penner, and Penner (2016), for example, examined the effects

of a policy that gave color-coded identification cards to high school students based on their test


11

scores; these cards were publicly displayed and conferred a number of advantages to students.

The authors found that students whose scores fell just short of the cutoff for a more prestigious

ID card had lower test scores and grades the following year (Domina, Penner, and Penner 2016).

Papay, Murnane, and Willett (2010) similarly found that low-income urban high school students

that barely failed a tenth-grade math exit exam were significantly more likely to drop out of

school the following year than similar students who just passed the exam. In both cases, falling

just short of the cutoff appeared to significantly affect behavior. Subgroup-specific

accountability could have similarly affected the turnover or attrition of teachers in schools that

fell just short of the cutoffs, perhaps by affecting teachers’ motivation, as discussed above.

Although the four explanations outlined in this section have some theoretical and

empirical support, they are largely speculative. However, they each lead to some testable

predictions. If changes in pressure explained any effects of subgroup-specific accountability on

teachers, we would expect teachers to leave their schools or teaching at higher rates from schools

that faced subgroup-specific accountability, compared to those that did not. We also might

observe that these effects were greatest in the schools where those subgroups were most likely to

fail. If resource allocation were most important, we would expect to see lower turnover and

attrition from schools that faced accountability for their subgroups. We might also expect to see

the teachers that worked most often with the focal students leaving their schools or teaching at

particularly low rates. If motivation were a main means by which subgroup-specific

accountability affected teachers, we might expect to see minority teachers remaining in their

schools or teaching at higher rates when their same-race subgroup “counted” in their schools, or

perhaps leaving their schools or teaching when they fell just short of those cutoffs. If strategic

manipulation took place, we might see particular effects for same-race teachers in schools that


12

fell just short of the subgroup accountability cutoff. After describing the data and methods, this

study examines the overall effects of subgroup-specific accountability on teachers, then turns to

testing these potential explanations.

<A>3. Methods

Data and Sample

Data for this study were taken from administrative data on North Carolina students,

teachers, and schools from the school years 1999-2000 through 2003-04. This five-year period

spanned the initial implementation of NCLB in 2002-03, allowing a comparison of teacher

outcomes in schools whose numbers of students in their subgroups fell just above or just below

the state’s minimum subgroup size cutoff in the periods before and after the law took effect.

These data included information on every public school teacher in the state, including

information on teachers’ race/ethnicity, education, and the racial/ethnic composition and

achievement of their students. For the purposes of these analyses, the sample was limited to

elementary school teachers, who had direct responsibility for specific students and therefore

might be expected to respond particularly strongly to subgroup-specific accountability.1 The

sample was also limited to black and white teachers, as these teachers comprised the vast

majority of North Carolina elementary school teachers during the study period (14% and 84%,

respectively, in 2001-02), and these analyses sought to explore the effects of accountability for

the same-race subgroup on these teachers.

Using this administrative data, measures of one- and two-year turnover and attrition were

constructed for both the pre- and post-NCLB periods. In the pre-NCLB period, each public

1 The sample was not limited to self-contained teachers, but instead included teachers of art, music, and other non-self-contained classes. 84% of teachers in the sample, however, taught self-contained classes.


13

elementary school teacher in North Carolina in 1999-2000 was searched for in the state data

from either 2000-01 or 2001-02 (depending on the particular outcome) to determine whether that

teacher (a) still taught in the same North Carolina public school, (b) taught in a different North

Carolina public school, or (c) had left public school teaching in North Carolina. Similar measures

were constructed for the post-NCLB period, beginning with all North Carolina teachers in 2001-

02 (the year before NCLB went into effect), and examining outcomes for these teachers in either

2002-03 or 2003-04.

Both one- and two-year outcomes were examined because the implementation of

subgroup-specific accountability took place over two years, and there was ambiguity about when

North Carolina schools and districts became aware of the state minimum subgroup size of 40

students. NCLB was signed into law in January 2002, but North Carolina’s minimum subgroup

size was not approved until April 2003 (Hickok 2003). Within this large window of time, it is

unclear exactly when North Carolina schools and districts became aware of the state minimum

subgroup size of 40 students. It is also unclear when teachers learned where their schools’

subgroups fell in relation to that cutoff. NCLB based its subgroup-specific accountability on the

number of tested students, but required students to have been enrolled in a school for nearly the

entire school year to count as a member of a subgroup.2 Preliminary test results were released

beginning in June of 2003, and press coverage in North Carolina at that time suggested that

NCLB had had positive impacts on student achievement, particularly for minority students

(North Carolina Department of Public Instruction 2003; Silberman 2003). Teachers therefore

could have been aware of where their schools’ subgroups fell in relation to the cutoff by this

2 NCLB allowed each state to define “continuous enrollment” (Davidson et al. 2015), and North Carolina required students to be continuously enrolled in their schools for 140 days at the time of spring testing (Lauen and Gaddis 2012). In the remainder of this study, references to the numbers of students in subgroups refer to the number of such continuously enrolled and tested students.


14

time, if not earlier. Given the uncertainty about the timeline of the policy implementation, it is

also possible that school administrators knew of the cutoff of 40 earlier than their teachers, and

were able to respond to that cutoff, perhaps by engaging in strategic manipulation, as described

above. Because of the ambiguity in the timeline of the policy’s implementation, as well as when

exactly teachers and administrators knew where they fell in relation to the minimum subgroup

size cutoff, analyses for this study examined both one- and two-year outcomes. Two-year

attrition was the preferred outcome, however, since it captured attrition from the year before

NCLB through the full initial implementation of the policy.

Data on the implementation of subgroup-specific accountability were taken from the

Barnard/Columbia NCLB database, a state-by-state compendium of information on states’

implementation of NCLB (Reback et al. 2011). These data, which were compiled from publicly

available records and personal correspondence with state officials, provided the number of

“continuously enrolled” and tested students by subgroup in each North Carolina school in the

first two years of NCLB (2002-03 and 2003-04), and were available for 94% of the elementary

schools in North Carolina during the study period.

Along with the numbers of tested students in each subgroup in each school, the

Barnard/Columbia NCLB data also contained the AYP outcomes (pass or fail) for each subgroup

in each school in both reading and math in 2002-03 and 2003-04, the first two years of NCLB. If

a school had an AYP outcome listed for a subgroup in both subjects, that school was deemed to

have been held accountable for that subgroup that year; any school without an AYP outcome for

a subgroup in both subjects was deemed to have not been held accountable for that subgroup.

North Carolina elementary schools were held accountable for the black and white subgroups far

more often for than any other racial ethnic groups: in the first year of NCLB, 64% of schools


15

were held accountable for the black subgroup, and 82% of schools for the white subgroup. (In

contrast, only 8% of schools were held accountable for the Hispanic subgroup, and 1% of

schools for the Asian subgroup.) Because large numbers of schools were held accountable for the

black and white subgroups, and because nearly all elementary school teachers in North Carolina

at the time of NCLB’s initial implementation were either black or white, analyses for this study

focused on the effects of being held accountable for these two subgroups.

In the first year of NCLB, North Carolina perfectly assigned schools to treatment or non-

treatment status based on the numbers of students in their black or white subgroups, holding

every school with 40 or more black or white students accountable for that subgroup, and not

holding any schools accountable for the black or white subgroup when the number of such

students was below 40. 95% of schools that were held accountable for the black or white

subgroups in the first year of NCLB were also held accountable for that subgroup in the law’s

second year, making it impossible to examine separate effects of subgroup-specific

accountability in NCLB’s first two years.

Table 1 presents descriptive statistics on the sample of elementary schools in North

Carolina in 2001-02, the year before NCLB took effect. Column A shows descriptive statistics

for all North Carolina elementary schools; Column B for North Carolina elementary schools

whose numbers of black students fell close to the state minimum subgroup size (between 35 and

44 tested black students) during the first year of NCLB; and Column C for elementary schools

whose numbers of white students were close to the state minimum subgroup size (between 35

and 44 tested white students). Schools whose numbers of black students were close to 40 were

generally more advantaged than other schools in the state. These schools had smaller percentages

of low-income students and higher state-level accountability performance than other schools.


16

These schools also had, on average, greater percentages of white students and smaller

percentages of black students than other North Carolina schools, and were more likely to be

located in rural areas and suburbs. Schools near the cutoff for the number of tested white

students, in contrast, were generally less advantaged than other schools in the state.

Table 2 shows information on whether or not North Carolina elementary schools were

held accountable for various subgroups during the first year of NCLB. In order to give an overall

sense of the implementation of subgroup-specific accountability during NCLB’s first year, this

table is not limited to the black or white subgroups. In NCLB’s first year, there was a great deal

of overlap in accountability for various subgroups. For example, 92% of schools near the cutoff

for the number of black students were held accountable for the white subgroup, while 86% of

schools near the cutoff for the number of white students were held accountable for the

performance of the black subgroup. Because of these overlaps in accountability for various

subgroups, some analyses controlled for whether schools were accountable for subgroups other

than the focal subgroup.

Some analyses examined whether the effects of subgroup-specific accountability differed

depending on the level of pressure schools faced for their subgroups. For these analyses, schools

were divided into groups based on the predicted probability their black or white subgroups would

make AYP, and indicators for each of these groups were interacted with each variable included

in the analyses.3 Other analyses explored whether the policy’s effects differed depending on the

percentages of black or white students in teachers’ classes. To accomplish this, the percentage of

3 For these analyses, the predicted probability of a subgroup making AYP in reading, and (separately) for making AYP in math, was computed by estimating school-by-subgroup logistic regression models where the dependent variable was an indicator for whether the black or white subgroup passed its accountability target in that subject during the first year of NCLB, and the independent variable was the prior year’s school-average reading or math test score for that subgroup. Results reported in the following section were robust to using the likelihood of making AYP in either subject.


17

teachers’ students that were black or white was interacted with each variable; in another analysis,

each variable was interacted with indicators for whether teachers’ percentages of black or white

students were more than a standard deviation above the mean for the entire sample, within a

standard deviation of the mean, or more than a standard deviation below the mean.

Table 3 presents descriptive statistics on the sample of North Carolina elementary

teachers used in this study. Teachers in schools with close to 40 tested black students were more

likely to be white than teachers in schools with close to 40 tested white students. They were also

more likely to have higher scores on licensure exams, standard teaching certification, and

National Board Certification. Teachers in schools with close to 40 tested white students were

also more likely to leave their schools or teaching within one or two years.

Research Design

This study used a regression discontinuity (RD) design to determine the effects of

subgroup-specific accountability on teachers. The identifying assumption of the analysis was that

schools with 39 tested students in a subgroup were identical to schools with 40 tested students,

except for the fact that NCLB held schools with 40 students in the subgroup accountable for that

subgroup’s performance. Any discontinuity in teacher turnover or attrition at the cutoff of 40

tested black or white students, therefore, was directly attributable to the policy.

Since the likelihood of receiving the treatment jumped from zero to one at the cutoff, this

study used a “sharp” RD design (Lee and Lemieux 2010; Bloom 2012). Linear probability

models were estimated for ease of interpretation, with standard errors clustered at the school

level to reflect teacher grouping in schools and the likely correlation of teacher outcomes within

schools. The bandwidth within which the analyses were conducted varied between 10, 20, and 30

tested students around the cutoff. Except for in a few cases noted below, the assignment


18

variable—the number of tested students in the black or white subgroups—was always from the

first year of NCLB (2002-03).4

The model for the RD analyses in the post-NCLB period was as follows:

𝑌𝑖𝑗𝑡 = 𝛽0 + 𝛽1𝑍𝑗𝑡−1 + 𝛽2𝑋𝑗𝑡−1 + 𝛽3𝑋𝑗𝑡−12 + 𝛽4𝑋𝑗𝑡−1

3 + 𝛽5𝑋𝑗𝑡−1𝑍𝑗𝑡−1 + 𝛽6𝑋𝑗𝑡−12 𝑍𝑗𝑡−1 +

𝛽7𝑋𝑗𝑡−13 𝑍𝑗𝑡−1 + 𝜀𝑖𝑗𝑡 (1)

where the outcome variable Yijt was an indicator for whether teacher i in school j left the base

year school, or public school teaching in North Carolina, in year t (2003-04); Zjt-1 was an

indicator for whether the number of tested black or white students in school j met or exceeded

North Carolina’s minimum subgroup size of 40 in year t-1, the first year of NCLB (2002-03); Xjt-

1 was the assignment variable, the number of tested black or white students in school j in year t-

1; and Xjt-12 and Xjt-1

3 were quadratic and cubic powers of the assignment variable. Xjt-1Zjt-1, Xjt-

12Zjt-1, and Xjt-1

3Zjt-1 were interactions between Xjt-1, Xjt-12, and Xjt-1

3 and the treatment dummy Zjt-

1. Values of Xjt-1 and its higher-order polynomials were centered at the cutoff of 40, which made

the intercept of the equation (β0) the estimate of teacher turnover or attrition at the cutoff in the

absence of subgroup-specific accountability. β1 was the treatment effect, and estimated the effect

of subgroup-specific accountability for the black or white subgroup on teacher turnover or

attrition in schools with exactly 40 tested students in that subgroup in 2002-03, the first year of

NCLB.5 Pre-NCLB analyses used 2001-02 as the base year.

Since the data spanned the pre- and post-NCLB periods, some analyses pooled pre- and

post-NCLB data and estimated a difference-in-differences RD model. This model not only

4 Some analyses tested whether accountability for subgroups in the second year of NCLB (2003-04) caused changes in teacher turnover and attrition. These analyses are discussed below. 5 For the pre-NCLB analyses, schools’ numbers of tested black or white students from 2002-03 were used as the assignment variable; these analyses were therefore limited to those pre-NCLB schools for which such information was available. 77 schools (7% of all North Carolina elementary schools) and 1,594 teachers (5% of elementary teachers) were lost from the pre-NCLB analyses because of the lack of NCLB-related data on their schools.


19

estimated the discontinuity in teacher turnover or attrition at the cutoff in both the pre- and post-

NCLB periods, but also estimated the difference in those discontinuities. This model was

estimated as follows:

𝑌𝑖𝑗𝑡 = 𝛽0 + 𝛽1𝑍𝑗𝑡−1 + 𝛽2𝑇𝑡 + 𝛽3𝑍𝑗𝑡−1𝑇𝑡 + 𝛽4𝑋𝑗−1 + 𝛽5𝑋𝑗𝑡−12 + 𝛽6𝑋𝑗𝑡−1

3 + 𝛽7𝑋𝑗𝑡−1𝑍𝑗𝑡−1 +

𝛽8𝑋𝑗𝑡−12 𝑍𝑗𝑡−1 + 𝛽9𝑋𝑗𝑡−1

3 𝑍𝑡−1 + 𝛽10𝑋𝑗𝑡−1𝑇𝑡 + 𝛽11𝑋𝑗𝑡−12 𝑇𝑡 + 𝛽12𝑋𝑗𝑡−1

3 𝑇𝑡 + 𝛽13𝑋𝑗𝑡−1𝑍𝑗𝑡−1𝑇𝑡 +

𝛽14𝑋𝑗𝑡−12 𝑍𝑗𝑡−1𝑇𝑡 + 𝛽15𝑋𝑗𝑡−1

3 𝑍𝑗𝑡−1𝑇𝑡 + 𝜀𝑖𝑗𝑡 (2)

where the difference between this equation and equation (1) was the addition of Tt, a dummy

variable that indicated whether an observation was from the post-NCLB period, and the

interaction of this variable with each variable in equation (1) to differentiate their effects in the

pre- and post-NCLB periods. The coefficient of interest was now β3, which estimated the

difference in the discontinuities in teacher turnover or attrition between the pre- and post-NCLB

periods at the cutoff of 40 tested black or white students. The assignment variable, Xjt-1, was still

measured in the first year of NCLB (2002-03).

In addition to the parametric analyses discussed above, nonparametric RD models were

also estimated, with bandwidths selected using the bandwidth optimization procedure of Imbens

and Kalyanaraman (2012). Applying this procedure, however, often produced optimal

bandwidths that were very small—usually between two and three, and sometimes less than two.

Nonparametric analyses were therefore conducted using bandwidths of two, three, and four

tested students around the cutoff, which are each very close to these optimal bandwidths, using a

triangle kernel to weigh observations within each bandwidth (McCrary 2008).

<C>Internal validity of research design.

Since the cutoff of 40 tested students was unique to NCLB, estimates of the policy’s

effects were not confounded with the effects of other policies. The institutional and statistical


20

evidence also suggested that schools did not manipulate their numbers of tested black or white

students in NCLB’s first year, nor did teachers sort themselves to schools on either side of the

cutoff in the period before the policy took effect.

<D>Institutional evidence against manipulation of the assignment variable.

Manipulation of schools’ numbers of tested students was unlikely to be substantial for

several reasons. One was that schools lacked precise control over their student enrollments. In

most public schools, additional students can enroll or withdraw at any time, giving schools only

imprecise control over the student enrollments of various subgroups. Schools could have

manipulated their numbers of tested students in the black or white subgroups by reclassifying

student race; student race/ethnicity, however, is recorded when students enroll in a particular

North Carolina school district, and this information remains in the district’s computer system

when students transfer public schools within districts. The only substantial opportunity to

reclassify student race, therefore, was when students were just entering a new district, and the

numbers of such students in any given year were unlikely to be great.

Another way that the number of tested students in subgroups could have been

manipulated was through the continuous enrollment provision of NCLB. Schools could have

manipulated their numbers of tested students in the black or white subgroups by enrolling

students just after the cutoff date for continuous enrollment, a practice which research suggests

some schools indeed attempt (Jennings and Crosta 2011). Such manipulation, however, could

only have affected students that were newly enrolling in a school near the cutoff date for

continuous enrollment, and the numbers of such students were again unlikely to have been great.

Schools could also have classified black or white students into special education, or used

suspensions, to push their numbers of tested students in these subgroups below the cutoffs. Prior


21

research suggests that schools indeed use suspensions and classification into special education to

remove potentially low-performing students from the testing pool (Cullen and Reback 2006;

Figlio 2006; Figlio and Getzler 2006). NCLB, however, required that 95% of the continuously

enrolled students in each subgroup be tested, or the entire school would fail to make AYP. Thus,

the only schools that could have excluded enough students from testing to push themselves

below the minimum subgroup size cutoff were those whose true numbers of continuously

enrolled students were just above the cutoff. In North Carolina, only schools with 40 or 41

continuously-enrolled students in a subgroup could have excluded enough students from test-

taking to push that subgroup’s numbers of tested students below 40 and still meet the 95%

participation requirement. Similarly, only schools with 38 or 39 tested students in a subgroup

could actually have had 40 or more continuously-enrolled students in that subgroup. This form of

manipulation was therefore only plausible in a very small number of schools. The following

sections explore whether the results presented are robust to excluding schools where such

manipulation was possible.

<D>Statistical evidence against manipulation of the assignment variable.

A common test for manipulation of the assignment variable in RD is to look for a

discontinuity in the density of cases at the cutoff (Lee and Lemieux 2010; Bloom 2012). Graphs

of the results from such a test are presented in Figure 1, using data from 2002-03. Results

showed no statistically significant discontinuities in the density of schools at the cutoff for either

the number of tested black students (p=0.15) or white students (p=0.32). For the number of

tested black students, the figure shows a greater density of schools just above the cutoff than just

below, contrary to what might be expected had manipulation indeed occurred.

Another test for manipulation is to examine the smoothness of pretreatment covariates


22

across the cutoff (Lee and Lemieux 2010; Bloom 2012). Figure 2 shows the smoothness of

several pretreatment covariates across the cutoff for the number of tested black students. At the

school level (Panel A), these included school size, the proportion of students qualifying for free

or reduced price school lunch, the proportion of schools located in an urban area, and the

proportion of schools rated “exemplary” under North Carolina’s state-level accountability

system. At the teacher level (Panel B), these included whether a teacher was National-Board-

certified, a high-scorer on teacher licensure exams, entering his/her first year in education, or

attended a less competitive college. Each of these covariates was from 2001-02, the year before

NCLB went into effect, while the x-variable (the number of tested black students) was from

2002-03, the first year of NCLB. All analyses were conducted at the school level. The graphs

each include a scatterplot, a local linear (nonparametric) regression line, and cubic, quadratic,

and linear fits, each fitted separately on both sides of the cutoff. Results for the white subgroup

were similar and are not presented here.

Panel A shows that there were no discontinuities in any pretreatment school covariates at

the cutoff for the number of tested black students. Panel B similarly shows no discontinuities in

teacher characteristics at the cutoff of 40 tested black students in the pretreatment year, 2001-02.

The findings of Panels A and B were confirmed in regressions that are not presented here, which

showed no pattern of significant discontinuities in any pretreatment covariates at any of the

bandwidths tested.6

<C>Inclusion of covariates.

Since prior research suggests that teacher turnover and attrition are associated with the

percentage of minority students in a school (Borman and Dowling 2008), one might expect an

6 Results of these analyses are available upon request.


23

increase in teacher turnover and attrition as the numbers of tested black students in schools

increases. This was indeed the case, even in a relatively small window around the cutoff.

Limiting the sample to teachers in the pre-NCLB period in schools that would later be within a

30 student bandwidth of the cutoff, an increase of one tested black student was associated with

an increased probability of leaving teaching of a tenth of a percentage point (p=0.02). Also

consistent with prior research (see Jackson 2009), this was true for white, but not black teachers.

The number of tested white students was not associated with the likelihood of attrition for either

black or white teachers.

The addition of school covariates, however, accounted for this trend. After including the

full set of school covariates in Table 1, schools’ numbers of tested black students was no longer a

statistically significant predictor of teacher attrition. This finding provided further reassurance

that there were not large underlying differences in teacher attrition across the cutoff in the period

before NCLB took effect, and those differences that were present could be accounted for by the

inclusion of school-level covariates. The following section, therefore, reports results of models

that both include and exclude covariates, to assess whether the inclusion of these school and

teacher characteristics had significant effects on the results.

<A>4. Results

Subgroup-specific accountability focused on the black or white student subgroups had no

overall effects on teacher turnover or attrition. This was true regardless of the predicted

probability that the black subgroup would make AYP, or the percentage of teachers’ students

that were black or white. At some bandwidths around the cutoff for the number of tested black

students, there was a significant negative discontinuity in the pre-NCLB period in teacher


24

turnover among teachers that taught the highest percentages of black students. Since there were

relatively few teachers that taught high percentages of black students in schools near the cutoff

for the black subgroup, however, this finding was likely to have been driven by random variation

in turnover among these teachers.7

Separate analyses for black and white teachers, however, found that accountability for the

black subgroup in the first year of NCLB had significant effects on the likelihood that black

teachers left public school teaching in North Carolina. Table 4 shows RD estimates of the effects

of accountability for the black subgroup on two-year teacher attrition, separating teachers into

black teachers (top panel) and white teachers (bottom panel). The table contains estimates of the

discontinuity in teacher attrition at the cutoff in the pre- and post-NCLB periods, as well as

estimates of the difference in these discontinuities. The columns of the table correspond to

various methods of analysis (nonparametric or parametric), bandwidths around the cutoff within

which the analysis was conducted (2, 3, 4, 10, 20, or 30 tested students), and functional forms

used to model the outcome-assignment variable relationship (cubic, quadratic, or linear). The

“mean attrition” in the leftmost column is the mean attrition in either the pre- or post-NCLB

period, depending on the row, and is provided to give a sense of the magnitude of the

discontinuities estimated across that row of the table.

Subgroup-specific accountability focused on the black subgroup caused a significant,

negative discontinuity in black teacher attrition at the cutoff of 40 tested black students. The

estimates in the top panel of Table 4 show that this discontinuity in teacher attrition was

consistent in magnitude and direction across both nonparametric and parametric analyses and a

variety of functional forms and bandwidths. (The exception is the linear model, whose estimates



25

are smaller in magnitude and only marginally statistically significant; this model, however,

generally provides a poor fit to the data.) The magnitude of these estimates suggests that

subgroup-specific accountability for the black subgroup had a large negative effect on the

likelihood that black teachers left teaching. Using a cubic functional form and a bandwidth of 20,

for example, there is a negative discontinuity of 0.39 in black teacher attrition at the cutoff,

representing a 68% drop in teacher attrition when compared to 0.57, the model’s estimate of

black teacher attrition at the cutoff in the absence of subgroup-specific accountability. This

negative discontinuity could be the result of a decrease in black teacher attrition in above-cutoff

schools, an increase in attrition in below-cutoff schools, or some combination of the two.

This discontinuity in black teacher attrition at the cutoff did not exist before NCLB took

effect, as can be seen in the second row of the top panel of Table 4. This suggests that the post-

NCLB discontinuity in black teacher attrition was not simply an artifact of an underlying

difference between schools on either side of the cutoff that predated the policy, but that this

discontinuity arose when NCLB took effect. Even after differencing out this small and

statistically-insignificant pre-NCLB discontinuity in black teacher attrition, the results of the

difference-in-differences RD analyses, presented in the third row of the top panel of Table 4,

show that the discontinuity in teacher attrition caused by accountability for the black subgroup

remained negative and was generally statistically significant, although at some bandwidths only

marginally so. These findings again suggest that subgroup-specific accountability for the black

subgroup significantly affected the attrition of black teachers from North Carolina public schools

in the period immediately after NCLB took effect. The bottom panel of Table 4 shows that

accountability for the black subgroup did not affect the likelihood that white teachers left

teaching, and analyses not presented here found that accountability for the white subgroup had


26

no effects on either black or white teachers.

Figure 3 presents a graphical version of these analyses, showing black teacher attrition in

the pre-NCLB period (top panel) and post-NCLB period (bottom panel), with the number of

tested black students in teachers’ schools in 2002-03 on the x-axis. The top panel of the figure

demonstrates that before NCLB went into effect there was no discontinuity in black teacher

attrition in schools that would later have 40 tested black students. The bottom panel of the figure,

however, shows a large, negative discontinuity in black teacher attrition at the cutoff in the post-

NCLB period.

The post-NCLB discontinuity in black teacher attrition appears to be driven, at least in

part, by an increase in black teacher attrition in schools with just fewer than 40 tested black

students. To further explore the changes in black teacher attrition on both sides of the cutoff,

Figure 4 shows the difference in average black teacher attrition between the pre- and post-NCLB

periods, by the number of tested black students in the school in the first year of NCLB. For the

purposes of this figure, teacher attrition is calculated at the school level and then averaged for all

schools with each number of tested black students. The figure shows that, relative to the pre-

NCLB period, black teacher attrition increased in the post-NCLB period in schools whose

numbers of tested black students fell just below the minimum subgroup size cutoff. At the same

time, the figure also shows that black teacher attrition decreased slightly, compared to the pre-

NCLB period, in schools just above this cutoff, as shown by the slight dip below zero just to the

right of the cutoff in the graph. Given the research design, it is not possible to determine whether

changes on one side of the cutoff or the other (or both) led to the discontinuity observed here.

To determine how much of this discontinuity in two-year attrition occurred during the

first year of the policy (between 2001-02 and 2002-03) and how much occurred in the second


27

year (between 2002-03 and 2003-04), similar analyses were conducted using one-year attrition as

the outcome, as well as for an additional year after the policy took effect (between 2003-04 and

2004-05), and three years prior to NCLB. Results of these analyses are presented in Table 5 and

Figure 5, and show no discontinuity at the cutoff throughout the pre-NCLB period. Between the

year that NCLB was signed into law and the first year the policy took effect (i.e. between 2001-

02 and 2002-03), there was a large, negative discontinuity at the cutoff in the likelihood that

black teachers left teaching, although this discontinuity could not be statistically distinguished

from zero. This discontinuity persisted, and in most specifications grew in magnitude, between

the first year that subgroup-specific accountability was implemented in schools (2002-03) and

the second year of the policy (2003-04), although it could generally not be statistically

distinguished from zero, before shrinking again the following year.8

Results for black teacher turnover (i.e. leaving the school, as opposed to leaving teaching)

show a similar pattern as those for black teacher attrition: after NCLB took effect there was a

large negative discontinuity in the likelihood that black teachers left their schools at the cutoff of

40 tested black students. Analyses limited to black teachers that remained in public school

teaching in North Carolina the following year, however, found no discontinuity in black teacher

turnover at the cutoff, suggesting that the turnover findings were driven by black teachers that

left teaching altogether.

Since schools received their AYP results in the summer of 2003 (North Carolina

Department of Public Instruction 2003; Silberman 2003), it was also possible that the black

subgroup’s contribution to the AYP status of the school may have impacted black teacher

8 Other analyses examined whether accountability for the black student subgroup in 2003-04—the second year of NCLB—affected black teacher turnover or attrition between 2001-02 and 2004-05, and between 2002-03 and 2004-05. In neither case did accountability in the second year of the policy have a significant effect on outcomes for black teachers.


28

attrition. To examine this possibility, a series of difference-in-differences analyses comparing the

attrition of black and white teachers in the pre- and post-NCLB periods were conducted. These

analyses were conducted separately for schools on each side of the cutoff, and for schools that

made or did not make AYP, both overall and for their black subgroups. These analyses found no

significant differences in black teacher attrition, except in schools that were held accountable for

their black subgroups and failed to make overall AYP. In those schools, black teacher attrition

increased in the post-NCLB period compared to white teachers, but only in schools where the

black subgroup did not contribute to the overall AYP failure (see Table 6). The results of these

difference-in-differences analyses are suggestive, and should not be interpreted causally, but

suggest that black students’ performance—at least in schools that failed to make overall AYP—

could have played a role in black teacher attrition.

Robustness Checks

<C>Excluding schools just below the cutoff.

The effects on black teachers of subgroup-specific accountability for the black subgroup

appeared to be driven by teachers in schools with 38 or 39 tested black students. Since prior

research has found effects of falling just short of cutoffs (Papay, Murnane, and Willett 2010;

Domina, Penner, and Penner 2016), this could represent the effect on black teachers of their

schools falling just short of the cutoff for accountability for the black subgroup. However, as

noted earlier, these schools are also those that could have potentially manipulated their numbers

of tested black students to push themselves just below the cutoff. To test the robustness of the

findings to the exclusion of teachers in these schools, these teachers were excluded from some

analyses to determine their effects on the results. When teachers in schools with 39 tested black

students were excluded, and the discontinuity in teacher attrition between black teachers in


29

schools with 38 tested black students and black teachers in schools with 40 tested black students

was estimated, the estimates of the discontinuity decreased in magnitude and were no longer

statistically significant, although they generally retained the negative sign (in parametric analyses

using a bandwidth of 30, for example, β=-0.158, SE=0.145). When schools with 38 tested black

students were additionally excluded from the analysis, estimating the discontinuity in attrition

between black teachers in schools with 37 tested black students and schools with 40, the

discontinuity decreased in magnitude to nearly zero and was not statistically significant (in

parametric analyses using a bandwidth of 30, β=-0.009, SE=0.150).

<C>Testing pseudocutoffs.

Since there were relatively few black teachers in the data, these teachers’ attrition varied

a great deal across schools with various numbers of tested black students, making it possible that

this study’s findings were simply an artifact of random variation. A common falsification test to

examine whether RD findings are unique to the cutoff is to conduct RD analyses using false

cutoffs, and examine whether discontinuities exist at these values of the assignment variable

(Imbens and Lemieux 2008). Following the procedure outlined by Imbens and Lemieux (2008),

separate analyses were conducted above and below the true cutoffs, so that the discontinuity at

the true cutoff would not influence the estimates, using the median value of the assignment

variable in the above- and below-cutoff samples as the false cutoffs. The same full range of

parametric and nonparametric analyses described earlier were conducted, and these analyses

found no evidence of discontinuities at any of the false cutoffs in either the pre- or post-NCLB

periods.9

<C>Including controls.



30

The inclusion of covariates, although not necessary to obtain unbiased regression

discontinuity estimates (Lee and Lemieux 2010), can increase the precision of the estimated

discontinuity, and can also provide a useful robustness test. If the estimated discontinuity

decreases in magnitude or becomes statistically insignificant with the addition of covariates, the

observed discontinuity may simply be an artifact of selection of units to one side or another of

the cutoff (Bloom 2012). To test this possibility, Table 7 presents results of RD models testing

the effects of accountability for the black student subgroup that include the full set of controls

from Tables 1, 2, and 3. For simplicity of presentation, these models focus on wider bandwidths

around the cutoff (20 and 30 tested black students) and the quadratic and linear forms of the

running variable. The first model in each panel reproduces the analyses from Table 4, which do

not include controls; the second model includes the control variables from Tables 1, 2, and 3; and

the third model in each section includes these controls as well as their interactions with a post-

NCLB dummy. Only difference-in-differences coefficients are presented.

Results presented in Table 7 show that the inclusion of control variables did not

significantly affect the estimates of the discontinuity at the cutoff. In the models with controls,

the magnitude of the estimated discontinuity was generally larger than that in the models without

controls, although the estimates from all three sets of models could not be statistically

distinguished from one another with 95% confidence. (The same was true for the year-by-year

analyses of black teacher attrition.) This suggests that the discontinuity at the cutoff in the main

analyses was not driven by differences in the covariates between schools just above and just

below the cutoff.

The inclusion of controls, however, did not diminish the dependence of the results on

schools with 38 and 39 tested black students. With the full set of controls included in the models,


31

the exclusion of schools with 39 tested black students (or of schools with 38 or 39 tested black

students) decreased the magnitude of the discontinuity in black teacher attrition at the cutoff and

rendered it statistically insignificant, as in prior analyses. The fact that these schools remain

highly influential to the results suggests that unique circumstances in these schools may account

for the results. The following examination of potential explanations for these findings, therefore,

both includes and excludes this group of schools, to determine their impact on the results.

<A>5. Exploratory Analyses of Potential Explanations

Since black elementary school teachers taught greater percentages of black students than

white teachers in their schools, it was possible that black teachers responded strongly to

accountability for the black subgroup simply because they worked with these students more than

other teachers.10 To explore this possibility, the RD analyses for black teachers in equations (1)

and (2) were interacted with the percentage of black students in black teachers’ classes (both as a

continuous and three-level categorical variable). Results showed no evidence of differential

effects of the policy depending on the percentage of black students in black teachers’ classes.

Other analyses examined whether the policy caused changes in black teachers’ teaching

assignments. Although these analyses did not shed light on the aspects of subgroup-specific

accountability that caused black teachers to leave teaching, they did enable an exploration of the

changes in black teachers’ working conditions that were caused by the policy, providing

suggestive evidence of what might have caused these teachers to leave. These analyses examined

black teachers’ class sizes, whether they taught self-contained classes or a tested grade, and the

10 The results of regressions using school fixed effects showed that black teachers’ classes had approximately 4 percentage points more black students than the classes of white teachers in their schools. In a class of 25 students, this translated to black teachers having, on average, nine black students in their classes, as compared to eight black students for their white colleagues.


32

percentage of their students that were black; they also compared black teachers’ classes in each

year to their prior year’s classes, examining whether they taught fewer students, a different

grade, were moved into a tested grade, or taught a greater percentage of black students compared

to the prior year. Each of these characteristics of black teachers’ teaching assignments was

examined using a full set of nonparametric and parametric RD analyses, using the bandwidths

and functional forms outlined earlier, for two years prior to NCLB and two years after the policy

took effect.

In 2003-04, the year after schools were first held accountable for their black students’

performance, black teachers in schools that fell just above the cutoff for the number of tested

black students the prior year taught significantly smaller percentages of black students than black

teachers in schools that fell just below the cutoff. Results from this analysis are presented in the

top row of Table 8, where the estimates are consistently negative and—particularly in the

parametric RD models—statistically significant. The magnitude of these estimates suggests that

in above-cutoff schools black teachers’ classes were comprised of about 12 percentage point

fewer black students than the classes of black teachers in schools just below the cutoff. In prior

years, including the first year of NCLB (2002-03), the discontinuities in the percentage of black

students in black teachers’ classes at the cutoff were imprecisely measured and not generally

statistically significant. This suggests that in 2003-04 there were changes in black teachers’

teaching assignments caused by accountability for the black subgroup the prior year. These

findings are robust to the inclusion of the full set of teacher and school covariates shown in

Tables 1, 2, and 3. Similarly, Table 9 shows that when schools with 38 or 39 tested black

students were excluded from the analyses, the results remained consistent in magnitude and

direction.


33

Results from analyses including all schools are depicted visually in Figure 6. In the two

years prior to NCLB, there are no discontinuities in the percentage of black students in black

teachers’ classes at the cutoff. The year that NCLB first took effect (2002-03), the imprecisely

estimated discontinuity at the cutoff appears to be driven by a particularly high value for schools

with 39 tested black students. In the first year after schools faced accountability for their black

subgroups (2003-04), however, the figure shows a clear discontinuity at the cutoff in the

percentage of black students assigned to black teachers. Subgroup-specific accountability clearly

caused changes in the way that black students were assigned to black teachers.

There was also a sizeable discontinuity in the likelihood that black teachers taught a

greater percentage of black students than the prior year at the cutoff, as can be seen in the bottom

panel of Table 8. This discontinuity was not consistently statistically significant across all

bandwidths and functional forms, but was large and negative in 2003-04, and increased a great

deal relative to the prior year (2002-03). Along with the results for the percentage of black

students in black teachers’ classes, this suggests that subgroup-specific accountability for the

black subgroup led to a shifting of black students away from black teachers in above-cutoff

schools, or a shifting of black students to black teachers in below-cutoff schools, in the year after

the policy took effect. It is important to note that these shifts in the teaching assignments of black

teachers happened after the policy caused the attrition of black teachers outlined previously. For

this reason, these changes in black teachers’ teaching assignments are not necessarily

explanations for the effects on black teacher attrition; instead, they can be seen as evidence that

the dynamics of black teachers’ teaching assignments were different in above- versus below-

cutoff schools after the policy took effect.

Other analyses examined whether schools that were accountable for the black subgroup


34

in the first year of NCLB subsequently hired more or fewer black teachers than schools that were

not accountable for the black subgroup’s performance. These analyses found no pattern of

significant discontinuities in the number of newly hired black teachers, or the percentage of

newly hired teachers that were black, at the cutoff in the pre-NCLB period, or in any of the three

years after the policy took effect.

<A>6. Discussion

The first section of this study proposed four mechanisms that might explain any effects of

subgroup-specific accountability on teacher turnover and attrition: job pressure, resource

allocation, teacher motivation, and strategic manipulation. Of these mechanisms, job pressure is

the least consistent with the results presented here. The job pressure explanation predicts that

accountability for the black subgroup would increase the pressure on teachers in above-cutoff

schools and increase those teachers’ turnover or attrition; instead, these analyses find that

accountability for the black subgroup caused black teachers in above-cutoff schools to leave

teaching at significantly lower rates than black teachers in below-cutoff schools. The resource

allocation explanation is more consistent with these results, since it predicts that teacher turnover

and attrition would decrease in above-cutoff schools; however, this explanation also predicts that

teachers that work most closely with the focal subgroup would be the most affected by the

policy, which these analyses do not find to be the case.

The motivation explanation is an intriguing possibility, particularly given that black

teachers were affected by accountability for the same-race subgroup. Seeing that the black

students “counted” in their schools, and that their schools were taking action to address the

achievement gap between black and white students, may have caused black teachers to remain in


35

teaching that might otherwise have left. In below-cutoff schools, black teachers might have been

discouraged by their schools falling just short of those cutoffs, and chosen to leave teaching.

Prior research has documented effects of falling just short of strict cutoffs for various policies

(Papay, Murnane, and Willett 2010; Domina, Penner, and Penner 2016), and it is possible that

the increased attrition of black teachers from schools that fell just short of accountability for their

black subgroups represents exactly such an effect. Prior research also suggests that minority

workers’ perceptions of the “diversity climates” of their workplaces strongly predict both their

commitment to their organizations and their turnover (Griffeth and Hom 2001; Foley et al. 2002;

Chrobot-Mason 2003). The specific mechanisms by which accountability for the black subgroup

affected black teachers is challenging to investigate using administrative data, however, and

surveys of teachers or qualitative research will likely be necessary to more fully explore these

effects.

The final potential mechanism discussed at the outset of this study, strategic

manipulation, is also consistent with the results presented here. The effects of accountability for

the black subgroup appear concentrated in schools with 38 or 39 tested black students, the very

schools where such manipulation could have occurred. Although teachers may have not had

knowledge of where their schools fell in relation to the cutoff, their principals could have had

knowledge of both this cutoff and their schools’ numbers of black students eligible for testing, as

well as the opportunity to manipulate their schools’ numbers of tested black students, and

teachers may have observed this manipulation.

This study’s findings on changes in black teachers’ class assignments, however,

demonstrate that the policy had significant effects on aspects of black teachers’ working

conditions. The year after subgroup-specific accountability was first implemented, black teachers


36

in schools that had been held accountable for the black subgroup were assigned significantly

smaller percentages of black students than black teacher in schools just below the cutoff. This is

consistent with black students being shifted away from black teachers in schools where the black

subgroup suddenly counted towards AYP. These results, in contrast to the results for turnover

and attrition, are not sensitive to the exclusion of schools with 38 or 39 tested black students.

The effects of accountability for the black subgroup on black teacher attrition was

cumulative, with some of the effect occurring between the announcement of the policy and its

first year of implementation, and these effects continuing in the year between the policy’s first

and second years. The small, insignificant, negative discontinuity in black teacher attrition at the

cutoff between 2001-02 and 2002-03 took place well before the testing that ultimately

determined which schools fell above the cutoff and which did not; given the ambiguities about

the timeline of the policy’s implementation, however, it is impossible to rule out that schools and

teachers had some knowledge of where they would fall in relation to the cutoff in the summer of

2002, and this information could have affected black teacher attrition during that time.

One interesting question raised by these analyses is whether black teacher attrition was

voluntary or involuntary. Black teacher attrition could have been driven by teachers’ decisions to

remain in or leave public school teaching in North Carolina; alternatively, accountability for the

black subgroup could have affected the likelihood that black teachers were let go from their jobs.

Analyses not presented here find that black teachers that left teaching from below-cutoff schools

after the policy took effect were more likely to be first-year teachers than black teachers that left

teaching from above cutoff schools (24% versus 4%, respectively, p< 0.05). This is consistent

with below-cutoff schools letting go of their most inexperienced black teachers, or with above-

cutoff schools holding onto those teachers.


37

This study suggests that policies that impose cutoffs and hold some schools accountable

for subgroup performance and others not may have unintended consequences for teacher labor

markets. Minority teacher attrition is a significant concern for schools and school districts

(Ingersoll and May 2011), and understanding the factors that drive that attrition—particularly the

role of large-scale policies such as NCLB—is an important direction for future research.


38

References

Bloom, Howard S. 2012. Modern regression discontinuity analysis. Journal of Research on

Educational Effectiveness 5(1): 43-82.

Borman, Geoffrey D., and N. Maritza Dowling. 2008. Teacher attrition and retention: A meta-

analytic and narrative review of the research. Review of Educational Research 78(3): 367-409.

Boyd, Donald, Hamilton Lankford, Susanna Loeb, and James Wyckoff. 2008. The impact of

assessment and accountability on teacher recruitment and retention. Public Finance Review

36(1): 88-111.

Casey, Kathleen. 1993. I answer with my life: Life histories of women working for social

change. New York: Routledge.

Chrobot-Mason, Donna L. 2003. Keeping the promise: Psychological contract violations for

minority employees. Journal of Managerial Psychology 18: 22-45.

Clotfelter, Charles T., Helen F. Ladd, Jacob L. Vigdor, and Roger Aliaga Diaz. 2004. Do school

accountability systems make it more difficult for low-performing schools to attract and retain

high-quality teachers? Journal of Policy Analysis and Management 23(2): 251-271.

Crocco, Margaret S., and Arthur T. Costigan. 2007. The narrowing of curriculum and pedagogy

in the age of accountability: Urban educators speak out. Urban Education 42(6): 512-535.

Cullen, Julie B., and Randall Reback. 2006. Tinkering towards accolades: School gaming under

a performance accountability system. In Improving School Accountability (Advances in Applied

Microeconomics, Volume 14), edited by Timothy J. Gronberg and Dennis W. Jansen, pp. 1-34.

Somerville, MA: Emerald.

Davidson, Elizabeth, Randall Reback, Jonah E. Rockoff, and Heather L. Schwartz. 2015. Fifty

ways to leave a child behind: Idiosyncrasies and discrepancies in states' implementation of


39

NCLB. Educational Researcher 44(6): 347-358.

Dixson, Adrienne D., and Jeannine E. Dingus. 2008. In search of our mother’s gardens: Black

women teachers and professional socialization. Teachers College Record 110(4): 805–837.

Domina, Thurston, Andrew M. Penner, and Emily K. Penner. 2016. ‘Membership has its

privileges’: Status incentives and categorical inequality in education. Sociological Science 3:

264-295

Feng, Li, David N. Figlio, and Tim Sass. 2010. School accountability and teacher mobility.

NBER Working Paper No. 16070.

Figlio, David N. 2006. Testing, crime and punishment. Journal of Public Economics 90(4-5):

837-851.

Figlio, David N., and Lawrence S. Getzler. 2006. Accountability, ability and disability: Gaming

the system? In Improving School Accountability (Advances in Applied Microeconomics, Volume

14, edited by Timothy J. Gronberg and Dennis W. Jansen, pp. 35-49. Somerville, MA: Emerald.

Foley, Sharon, Deborah L. Kidder, and Gary N. Powell. 2002. The perceived glass ceiling and

justice perceptions: An investigation of Hispanic law associates. Journal of Management 28(4):

471-496.

Fulton, Mary. 2006. Minimum size of subgroups for Adequate Yearly Progress (AYP): State

trends and highlights. Denver: Education Commission of the States.

Griffeth, Rodger W., and Peter W. Hom. 2001. Retaining valued employees. Thousand Oaks,

CA: Sage.

Harris, Douglas N. 2008. Educational outcomes of disadvantaged students: From desegregation

to accountability. In Handbook of research in education finance and policy, edited by David N.

Figlio and Helen F. Ladd, pp. 551-570. New York: Routledge.


40

Hickok, Eugene. Eugene Hickok to Michael E. Ward, June 30, 2003. Available

http://www2.ed.gov/admins/lead/account/letters/index.html#nc, retrieved 19 August 2013.

Imbens, Guido, and Karthik Kalyanaraman. 2012. Optimal bandwidth choice for the regression

discontinuity estimator. Review of Economic Studies 79(3): 933-959.

Imbens, Guido, and Thomas Lemieux. 2008. Regression discontinuity designs: A guide to

practice. Journal of Econometrics 142: 615-635.

Ingersoll, Richard, & Henry May. 2011. Recruitment, retention, and the minority teacher

shortage (CPRE Research Report #RR-69). Philadelphia, PA: Consortium for Policy Research in

Education.

Irvine, Jacqueline Jordan. 2002. African American teachers’ culturally specific pedagogy. In In

search of wholeness: African American teachers and their culturally specific classroom

practices, edited by Jacqueline Jordan Irvine, pp. 139-146. New York: Palgrave.

Jackson, C. Kirabo. 2009. Student demographics, teacher sorting, and teacher quality: Evidence

from the end of school desegregation. Journal of Labor Economics 27(2): 213-256.

Jennings, Jennifer L. & Peter M. Crosta. 2011. The unaccountables. Unpublished paper.

Kane, Thomas J., and Douglas O. Staiger. 2002. Volatility in school test scores: Implications for

test-based accountability systems. Brookings Papers on Education Policy 5: 235-283.

Lauen, Douglas Lee, and S. Michael Gaddis. 2012. Shining a light or fumbling in the dark? The

effects of NCLB's subgroup-specific accountability on student achievement. Educational

Evaluation and Policy Analysis 34(2): 185-208.

Lee, David S., and Thomas Lemieux. 2010. Regression discontinuity designs in economics.

Journal of Economic Literature 48(2): 281-355.

Li, Danielle. 2012. School accountability and principal mobility: How No Child Left Behind


41

affects the allocation of school leaders. Unpublished paper.

McCrary, Justin. 2008. Manipulation of the running variable in the regression discontinuity

design: A density test. Journal of Econometrics 142(2): 698-714.

No Child Left Behind Act of 2001, 20 U.S.C. § 6301 (2001).

North Carolina Department of Public Instruction. 2003. ABCs 2003 Accountability Report

Background Packet. Available http://www.ncpublicschools.org/abcs/02-03/03mediapacket.pdf,

retrieved 1 December 2015.

Papay, John P., Richard J. Murnane, and John B. Willett. 2010. The consequences of high school

exit examinations for low-performing urban students: Evidence from Massachusetts. Educational

Evaluation and Policy Analysis 32(1): 5-23.

Pierce, Jason. 2003. Minimum size of subgroups for Adequate Yearly Progress (AYP). Denver:

Education Commission of the States.

Reardon, Sean F. 2011. The widening academic achievement gap between the rich and the poor:

New evidence and possible explanations. In Whither Opportunity? Rising inequality, schools,

and children’s life chances, edited by Greg J. Duncan & Richard J. Murnane, pp. 91-115. New

York: Russell Sage.

Reardon, Sean F., and Joseph P. Robinson. 2008. Patterns and trends in racial/ethnic and

socioeconomic academic achievement gaps. In Handbook of research in education finance and

policy, edited by David N. Figlio and Helen F. Ladd, pp. 497-516. New York: Routledge.

Reardon, Sean F., Erica H. Greenberg, Demetra Kalogrides, Kenneth A. Shores, and Rachel A.

Valentino. 2013. Left behind? The effect of No Child Left Behind on academic achievement

gaps. Unpublished paper, Stanford University.

Reback, Randall, Jonah E. Rockoff, and Heather L. Schwartz. 2014. Under pressure: Job


42

security, resource allocation, and productivity in schools under No Child Left Behind. American

Economic Journal: Economic Policy 6(3): 207-241.

Reback, Randall, Jonah E. Rockoff, Heather L. Schwartz, and Elizabeth Davidson. 2011.

Barnard/Columbia No Child Left Behind database, 2002-2003 and 2003-04. Available

http://www.gsb.columbia.edu/nclb, retrieved 8 February 2013.

Silberman, Todd. 2003. Test scores leaping in N.C. Raleigh News & Observer, June 18.

Available http://archives.republicans.edlabor.house.gov/archive/issues/108th/education/nclb/

dc061903.htm, retrieved 3 January 2014.

Sims, David P. 2013. Can failure succeed? Using racial subgroup rules to analyze the effect of

school accountability failure on student performance. Economics of Education Review 32: 262-

274.


43

Table 1: North Carolina Elementary School Characteristics, 2001-02 A B C All North

Carolina elementary

schools

Elementary schools with

35 to 44 tested black students a


35 to 44 tested white students a

Fraction free or reduced lunch students 0.50 0.41 0.72 (0.24) (0.23) (0.21) Fraction Asian students 0.02 0.02 0.02 (0.03) (0.03) (0.02) Fraction black students 0.35 0.24 0.56 (0.26) (0.14) (0.17) Fraction Hispanic students 0.07 0.07 0.08 (0.07) (0.06) (0.08) Fraction Native American students 0.01 0.01 0.01 (0.05) (0.01) (0.02) Fraction white students 0.55 0.66 0.33 (0.28) (0.16) (0.18) Located in city 0.31 0.20 0.58 (0.46) (0.40) (0.50) Located in suburbs 0.17 0.26 0.00 (0.37) (0.44) (0.00) Located in town 0.14 0.10 0.25 (0.35) (0.30) (0.44) Located in rural area 0.39 0.45 0.17 (0.49) (0.50) (0.38) Charter school 0.01 0.02 0.03 (0.12) (0.15) (0.17) Number of students 511 521 414 (197) (200) (163) Exemplary (ABC Accountability)b 0.42 0.49 0.28 (0.50) (0.50) (0.45) Expected (ABC Accountability)b 0.37 0.34 0.19 (0.48) (0.48) (0.40) No recognition (ABC Accountability)b 0.20 0.17 0.53 (0.40) (0.38) (0.51) Predicted probability black subgroup made AYP 0.86 0.86 0.85 (0.03) (0.02) (0.02) Predicted probability white subgroup made AYP 0.98 0.99 0.98 (0.09) (0.04) (0.04) Schools, n 1,060 94 36

a In 2002-03, the first year of NCLB. b North Carolina’s state-level accountability system.


44

Table 2: Subgroup-specific Accountability in North Carolina Elementary Schools, 2002-03 A B C All North

Carolina elementary

schools


35 to 44 tested black

students


35 to 44 tested white

students Accountable for Asian subgroup 0.01 0.03 0.00 (0.11) (0.18) (0.00) Accountable for black subgroup 0.64 0.50 0.86 (0.48) (0.50) (0.35) Accountable for Hispanic subgroup 0.08 0.04 0.11 (0.28) (0.20) (0.32) Accountable for multiracial subgroup 0.00 0.00 0.00 (0.00) (0.00) (0.00) Accountable for Native American subgroup 0.01 0.00 0.00 (0.11) (0.00) (0.00) Accountable for white subgroup 0.82 0.92 0.47 (0.38) (0.28) (0.51) Accountable for econ. disadvantaged subgroup 0.91 0.88 0.89 (0.29) (0.32) (0.32) Accountable for limited English subgroup 0.04 0.02 0.06 (0.19) (0.15) (0.23) Accountable for special ed. subgroup 0.32 0.28 0.14 (0.47) (0.45) (0.35) Schools, n 1,060 94 36


45

Table 3: North Carolina Elementary Teacher Characteristics, 2001-02 A B C All North Carolina

elementary schools Elementary schools with 35 to 44 tested black students a

Elementary schools with 35 to 44 tested white students a

Mean SD Mean SD Mean SD Asian 0.00 (0.05) 0.00 (0.06) 0.00 (0.07) Black 0.14 (0.34) 0.08 (0.27) 0.23 (0.42) Hispanic 0.01 (0.09) 0.01 (0.09) 0.01 (0.09) Multiracial 0.01 (0.07) 0.00 (0.05) 0.00 (0.07) Native American 0.01 (0.09) 0.00 (0.06) 0.01 (0.09) White 0.84 (0.37) 0.91 (0.29) 0.75 (0.44) Female 0.94 (0.23) 0.95 (0.22) 0.92 (0.27) Entering first year in education 0.06 (0.24) 0.06 (0.24) 0.08 (0.27) Very competitive college 0.07 (0.26) 0.09 (0.28) 0.06 (0.23) Competitive college 0.47 (0.50) 0.51 (0.50) 0.41 (0.49) Less competitive college 0.39 (0.49) 0.35 (0.48) 0.44 (0.50) Holds advanced degree 0.27 (0.45) 0.28 (0.45) 0.24 (0.43) National Board-certified 0.04 (0.20) 0.05 (0.22) 0.02 (0.13) High-scorer on licensure exam 0.10 (0.30) 0.10 (0.30) 0.09 (0.28) Middle-scorer on licensure exam 0.78 (0.42) 0.82 (0.39) 0.75 (0.43) Low-scorer on licensure exam 0.12 (0.33) 0.08 (0.27) 0.16 (0.37) Holds standard license 0.93 (0.26) 0.93 (0.25) 0.90 (0.30) Taught self-contained class 0.84 (0.37) 0.85 (0.36) 0.87 (0.33) % students taught that are black 0.32 (0.27) 0.21 (0.15) 0.56 (0.19) % students taught that are white 0.54 (0.30) 0.65 (0.22) 0.28 (0.17) Left school within 1 year 0.22 (0.41) 0.22 (0.41) 0.26 (0.44) Left school within 2 years 0.35 (0.48) 0.36 (0.48) 0.44 (0.50) Left teaching within 1 year b 0.17 (0.37) 0.16 (0.37) 0.19 (0.40) Left teaching within 2 years b 0.26 (0.44) 0.28 (0.45) 0.30 (0.46) Teachers, n 31, 211 2,695 930

a In 2002-03, the first year of NCLB. b Public school teaching in North Carolina.


46

Table 4: Effects of Accountability for black Subgroup on Two-Year Attrition for Black and White Teachers Nonparametric Parametric (cubic) Parametric (quadratic) Parametric (linear) Panel A:

Black

teachers

Mean attrition 2 3 4 10 20 30 10 20 30 10 20 30

Post-NCLB 0.258 -0.360* -0.495+ -0.602** -0.715** -0.390* -0.311* -0.431* -0.238* -0.221* -0.213+ -0.141+ -0.062

(0.144) (0.270) (0.189) (0.243) (0.163) (0.132) (0.173) (0.114) (0.096) (0.110) (0.076) (0.070)

Pre-NCLB 0.259 -0.056 -0.270 -0.162 -0.088 -0.020 -0.090 -0.020 -0.037 -0.024 -0.019 -0.003 0.067

(0.141) (0.259) (0.184) (0.369) (0.211) (0.152) (0.243) (0.023) (0.101) (0.118) (0.074) (0.067)

Difference -0.628* -0.370* -0.222+ -0.411* -0.201+ -0.197* -0.195+ -0.138+ -0.129+

(0.255) (0.160) (0.131) (0.175) (0.114) (0.098) (0.111) (0.080) (0.071) n, post-NCLB

109 136 195 336 707 971 336 707 971 336 707 971

n, pre-NCLB 116 153 216 363 764 1,037 363 764 1,037 363 764 1,037 Panel B:

White

teachers

Post-NCLB 0.254 0.004 0.065 0.013 0.006 -0.074 -0.009 -0.019 0.005 0.014 -0.014 0.011 0.003

(0.040) (0.075) (0.056) (0.084) (0.060) (0.046) (0.063) (0.042) (0.034) (0.039) (0.027) (0.023)

Pre-NCLB 0.259 -0.018 -0.073 -0.029 0.036 0.028 -0.017 0.014 -0.007 -0.014 0.003 -0.018 -0.007

(0.038) (0.072) (0.055) (0.071) (0.046) (0.037) (0.049) (0.034) (0.028) (0.032) (0.023) (0.020)

Difference -0.030 -0.102 0.008 -0.033 0.012 0.028 -0.018 0.029 0.010

(0.092) (0.063) (0.051) (0.064) (0.046) (0.039) (0.044) (0.032) (0.027) n, post-NCLB

1,384 1,668 2,285 4,292 8,448 11,316 4,292 8,448 11,316 4,292 8,448 11,316

n, pre-NCLB 1,357 1,640 2,113 3,987 7,914 10,618 3,987 7,914 10,618 3,987 7,914 10,618

Notes: Each cell reports the estimated discontinuity in the probability that a teacher in a school with 40 black students (North Carolina's minimum subgroup size) left public school teaching in North Carolina. Each cell reports the results from a separate regression. Nonparametric estimates are from local linear regressions in the designated bandwidth around the cutoff, using a triangle kernel. Parametric analyses are linear probability models that include the given terms of the assignment variable (the number of tested black students), as well as the interaction of each polynomial with the treatment dummy. "Mean attrition" is the teacher attrition of black or white teachers in the entire sample, and is provided for reference. Nonparametric bandwidths of 2, 3, and 4 are near the optimal bandwidth given by the Imbens & Kalyanaraman (2012) procedure, which ranges between 2 and 3 for these regressions. Standard errors for parametric analyses are clustered at the school level. Analyses do not include covariates. +=p<0.10; *=p<0.05; **=p<0.01.


47

Table 5: Effects of Accountability for Black Subgroup on One-Year Black Teacher Attrition

Nonparametric Parametric (cubic) Parametric (quadratic) Parametric (linear) 2 3 4 10 20 30 10 20 30 10 20 30 PRE-NCLB 98-99 to 99-00 X -0.151 -0.066 0.044 0.006 -0.066 -0.067 -0.096 -0.102+ -0.013 -0.062 -0.035 X (0.204) (0.146) (0.176) (0.101) (0.077) (0.107) (0.067) (0.059) (0.069) (0.048) (0.046) 99-00 to 00-01 -0.002 -0.151 -0.024 0.118 0.079 -0.025 0.142 -0.008 -0.019 -0.020 -0.013 0.032 (0.108) (0.181) (0.140) (0.235) (0.129) (0.093) (0.160) (0.078) (0.066) (0.072) (0.052) (0.048) 00-01 to 01-02 -0.020 -0.129 -0.096 -0.109 -0.050 -0.057 -0.085 -0.022 -0.047 -0.022 -0.031 0.024 (0.141) (0.258) (0.182) (0.263) (0.163) (0.127) (0.180) (0.107) (0.086) (0.102) (0.067) (0.058) POST-NCLB 01-02 to 02-03 -0.205+ -0.107 -0.259 -0.340+ -0.193 -0.177+ -0.156 -0.121 -0.115 -0.097 -0.078 -0.041 (0.124) (0.248) (0.172) (0.184) (0.127) (0.103) (0.134) (0.092) (0.078) (0.089) (0.062) (0.056) 02-03 to 03-04 -0.274+ -0.478 -0.499* -0.522 -0.266 -0.139 -0.422 -0.095 -0.129 -0.088 -0.074 -0.009 (0.154) (0.299) (0.202) (0.429) (0.259) (0.191) (0.271) (0.154) (0.117) (0.146) (0.084) (0.071) 03-04 to 04-05 0.004 0.293 -0.063 -0.124 -0.063 -0.020 -0.043 0.032 -0.036 -0.007 -0.019 0.045 (0.102) (0.228) (0.136) (0.202) (0.131) (0.111) (0.139) (0.088) (0.074) (0.080) (0.057) (0.054)

Notes: Each cell reports the estimated discontinuity in the probability that a teacher in a school with 40 black students (North Carolina's minimum subgroup size) left public school teaching in North Carolina. Each cell reports the results from a separate regression. Nonparametric estimates are from local linear regressions in the designated bandwidth around the cutoff, using a triangle kernel. Parametric analyses are linear probability models that include the given terms of the assignment variable (the number of tested black students), as well as the interaction of each polynomial with the treatment dummy. Nonparametric bandwidths of 2, 3, and 4 are near the optimal bandwidth given by Imbens & Kalyanaraman (2012) procedure, which ranges between 2 and 3 for these regressions. Standard errors for parametric analyses are clustered at the school level. Analyses do not include covariates. +=p<0.10; *=p<0.05; **=p<0.01.


48

Table 6: Difference-in-Differences Analyses of Teacher Attrition in North Carolina Schools Accountable for Black Subgroup that Failed to Make Overall AYP in 2002-03 Pre-NCLB

(99-00 to 01-02) Post-NCLB

(01-02 to 03-04) Difference in differences

Black subgroup didn’t make AYPa Black teacher attrition 0.274 0.276 White teacher attrition 0.289 0.302 Difference (black - white) -0.015 -0.026 -0.012 Black subgroup made AYPa Black teacher attrition 0.205 0.258 White teacher attrition 0.265 0.245 Difference (black - white) -0.061** 0.013 0.073**

Notes: Standard errors clustered at school level. a In 2002-03, the first year of NCLB. +=p<0.10; *=p<0.05; **=p<0.01.


49

Table 7: Effects of Accountability for Black Subgroup on Two-Year Attrition for Black and White Teachers, with Controls Panel A: Black

teachers 20, quadratic 30, quadratic 20, linear 30, linear

Difference -0.201+ -0.237* -0.345** -0.197* -0.209* -0.299** -0.138+ -0.154+ -0.210* -0.129+ -0.113 -0.144+ (0.114) (0.115) (0.122) (0.098) (0.102) (0.109) (0.080) (0.082) (0.089) (0.071) (0.073) (0.076) Controls X X X X X X X X Controls x post X X X X N observations (teacher x year) 1,471 1,471 1,471 2,008 2,008 2,008 1,471 1,471 1,471 2,008 2,008 2,008

Panel B: White

teachers

Difference 0.012 0.028 0.029 0.028 0.029 0.040 0.029 0.031 0.036 0.010 0.017 0.019

(0.046) (0.045) (0.047) (0.039) (0.037) (0.040) (0.032) (0.030) (0.032) (0.027) (0.025) (0.025)

Controls X X X X X X X X Controls x post X X X X N observations (teacher x year) 16,362 16,362 16,362 21,934 21,934 21,934 16,362 16,362 16,362 21,934 21,934 21,934

Notes: Each cell reports the estimated discontinuity in the probability that a teacher in a school with 40 black students (North Carolina's minimum subgroup size) left public school teaching in North Carolina. Each cell reports the results from a separate regression. Controls are those listed in Tables 1, 2, and 3. Analyses are linear probability models that include the given terms of the assignment variable (the number of tested black students), as well as the interaction of each polynomial with the treatment dummy. Standard errors are clustered at the school level. +=p<0.10; *=p<0.05; **=p<0.01.


50

Table 8: Effects of Accountability for Black Subgroup on Classroom Characteristics for Black Teachers Nonparametric Parametric (cubic) Parametric (quadratic) Parametric (linear) 2 3 4 10 20 30 10 20 30 10 20 30 Percent black students in teachers’ class(es) 2003-04 -0.101* -0.074 -0.106+ -0.110 -0.164* -0.183* -0.161* -0.138* -0.107+ -0.163** -0.071 -0.025 (0.043) (0.136) (0.062) (0.114) (0.083) (0.082) (0.080) (0.066) (0.058) (0.057) (0.047) (0.053) 2002-03 -0.385** -0.759** -0.560** -0.634+ -0.341 -0.231 -0.418+ -0.185 -0.145 -0.190 -0.090 -0.055 (0.093) (0.193) (0.132) (0.351) (0.223) (0.171) (0.244) (0.140) (0.105) (0.123) (0.075) (0.067) 2001-02 X -0.449** -0.374** -0.410 -0.205 -0.125 -0.276 -0.103 -0.082 -0.097 -0.042 -0.021 X (0.166) (0.120) (0.314) (0.187) (0.144) (0.206) (0.116) (0.091) (0.105) (0.066) (0.060) 2000-01 -0.102 -0.385* -0.178 -0.145 -0.051 -0.031 -0.036 -0.030 -0.027 -0.037 -0.015 -0.013 (0.102) (0.177) (0.134) (0.344) (0.198) (0.144) (0.229) (0.118) (0.093) (0.109) (0.068) (0.060) n (2003-04) 91 127 169 322 702 986 322 702 986 322 702 986 Taught greater percentage black students than prior year 2003-04 -0.321 -0.230 -0.234 -0.387 -0.304 -0.280* -0.258 -0.246* -0.187+ -0.199+ -0.144+ -0.158* (0.218) (0.426) (0.283) (0.335) (0.196) (0.140) (0.226) (0.120) (0.098) (0.114) (0.079) (0.069) 2002-03 X 0.177 -0.039 -0.012 -0.037 -0.161 0.015 -0.155 -0.147 -0.132 -0.140 -0.132+ X (0.339) (0.239) (0.281) (0.179) (0.151) (0.184) (0.133) (0.109) (0.127) (0.088) (0.074) 2001-02 -0.081 -0.200 -0.153 -0.354 -0.301+ -0.101 -0.402* -0.096 -0.066 -0.098 -0.010 -0.015 (0.193) (0.360) (0.248) (0.248) (0.167) (0.141) (0.180) (0.125) (0.104) (0.116) (0.086) (0.074) 2000-01 X 0.470 0.492* 0.582* 0.392* 0.101 0.418* 0.074 0.069 0.134 0.042 0.042 X (0.289) (0.202) (0.236) (0.160) (0.143) (0.165) (0.126) (0.106) (0.114) (0.084) (0.073) n (2003-04) 78 109 146 274 587 822 274 587 822 274 587 822

Notes: Each cell reports the estimated discontinuity in the outcome for black teachers in a school with 40 black students (North Carolina's minimum subgroup size) in 2002-03, the first year of NCLB. Each cell reports the results from a separate regression. Nonparametric estimates are from local linear regressions in the designated bandwidth around the cutoff, using a triangle kernel. Parametric analyses are linear probability models that include the given terms of the assignment variable (the number of tested black students), as well as the interaction of each polynomial with the treatment dummy. Nonparametric bandwidths of 2, 3, and 4 are near the optimal bandwidth given by Imbens & Kalyanaraman (2012) procedure, which ranges between 2 and 3 for these regressions. Standard errors for parametric analyses are clustered at the school level. Analyses do not include covariates. +=p<0.10; *=p<0.05; **=p<0.01


51

Table 9: Effects of Accountability for Black Subgroup on Classroom Characteristics for Black Teachers, Excluding Schools Just Below the Cutoff

Quadratic Linear Bandwidth 20 Bandwidth 30 Bandwidth 20 Bandwidth 30

All w/o 39 w/o 38, 39 All w/o 39 w/o 38,

39 All w/o 39 w/o 38, 39 All w/o 39 w/o 38,

39 Percent black students in teachers’ class(es)

2003-04 -0.138* -0.134 -0.165+ -0.107+ -0.111+ -0.135+ -0.071 -0.071 -0.085 -0.025 -0.020 -0.027 (0.066) (0.082) (0.095) (0.058) (0.066) (0.073) (0.047) (0.051) (0.055) (0.053) (0.056) (0.058)

2002-03 -0.185 -0.199 -0.230 -0.145 -0.156 -0.179 -0.090 -0.095 -0.108 -0.055 -0.057 -0.066 (0.140) (0.158) (0.166) (0.105) (0.115) (0.121) (0.075) (0.080) (0.083) (0.067) (0.070) (0.072)

2001-02 -0.103 -0.118 -0.139 -0.082 -0.093 -0.109 -0.042 -0.048 -0.057 -0.021 -0.025 -0.031 (0.116) (0.128) (0.135) (0.091) (0.099) (0.103) (0.066) (0.069) (0.072) (0.060) (0.063) (0.065)

2000-01 -0.030 -0.043 -0.052 -0.027 -0.038 -0.045 -0.015 -0.021 -0.026 -0.013 -0.017 -0.021 (0.118) (0.129) (0.134) (0.093) (0.100) (0.103) (0.068) (0.071) (0.072) (0.060) (0.063) (0.064)

n (2003-04) 702 692 685 986 976 969 702 692 685 986 976 969

Taught greater percentage black students than prior year 2003-04 -0.246* -0.282* -0.313* -0.187+ -0.201* -0.214+ -0.144+ -0.147+ -0.149+ -0.158* -0.158* -0.157*

(0.120) (0.124) (0.136) (0.098) (0.101) (0.109) (0.079) (0.081) (0.085) (0.069) (0.071) (0.074) 2002-03 -0.155 -0.163 -0.169 -0.147 -0.150 -0.153 -0.140 -0.143 -0.145 -0.132+ -0.134+ -0.136+

(0.133) (0.142) (0.154) (0.109) (0.114) (0.122) (0.088) (0.090) (0.095) (0.074) (0.076) (0.079) 2001-02 -0.096 -0.058 -0.093 -0.066 -0.037 -0.062 -0.010 0.013 0.001 -0.015 0.004 -0.006

(0.125) (0.140) (0.152) (0.104) (0.113) (0.122) (0.086) (0.090) (0.095) (0.074) (0.077) (0.080) 2000-01 0.074 0.056 0.058 0.069 0.056 0.058 0.042 0.034 0.036 0.042 0.037 0.039

(0.126) (0.144) (0.151) (0.106) (0.117) (0.122) (0.084) (0.090) (0.093) (0.073) (0.077) (0.079) n (2003-

04) 587 581 574 822 816 809 587 581 574 822 816 809

Notes: Each cell reports the estimated discontinuity in the outcome for black teachers in a school with 40 black students (North Carolina's minimum subgroup size) in 2002-03, the first year of NCLB. Each cell reports the results from a separate regression. Analyses are linear probability models that include the given terms of the assignment variable (the number of tested black students), as well as the interaction of each polynomial with the treatment dummy. Standard errors are clustered at the school level. Analyses do not include covariates. +=p<0.10; *=p<0.05; **=p<0.01


52


53

Appendix A.1: Description of Data

Data used in this study were drawn from two sources. The first was a series of data files

on North Carolina teachers and schools, which are available to researchers through the North

Carolina Education Research Data Center (NCERDC) at Duke University. The specific data files

used in this study were the following (all from years 1999-2005):

School Activity Report (SAR) Personnel files

School Activity Report (SAR) student directory files

Student End-of-Grade (EOG) exam files

ABC accountability growth files

Teacher education files

Teacher licensure files

Teacher testing files

Teacher National Board Certification files

Common Core of Data (CCD) Public School Universe (PSU) files

The second source of data for this study was the Barnard/Columbia No Child Left Behind

Database. These data are publicly-available for download at the following website:

http://www8.gsb.columbia.edu/nclb/files#North%20Carolina

For this study, two data files were used: NorthCarolina_2003.dta, and NorthCarolina_2004.dta.

Stata do-files used to prepare the data and conduct the analyses are available from the

author upon request.


54

Figure 1: Discontinuities in Density of Assignment Variable

Panel A: Number of tested black students

Panel B: Number of tested white students


55

Figure 2: Estimated Discontinuities in Pretreatment School and Teacher Covariates

Panel A: School Covariates


56

Panel B: Teacher Covariates


57

Figure 3: Two-Year Black Teacher Attrition, by Number of Tested Black Students in School


58

Figure 4: Pre-Post-NCLB Difference in Two-Year Black Teacher Attrition, by Number of Tested

Black Students in School


59

Figure 5: Discontinuities in One-year Black Teacher Attrition at the Subgroup-Specific

Accountability Cutoff Across the Pre- and Post-NCLB Periods


60

Figure 6: Discontinuities in the Percentage of Black Students in Black Teachers’ Classes

Date post:	04-Mar-2018
Category:	Documents
Upload:	phamtuyen
View:	218 times
Download:	2 times

Education Finance and Policy Just Accepted MS. doi:10.1162 ... · PDF fileaccess to the data;...

Documents