Electronic copy available at: http://ssrn.com/abstract=2561048
USC Dornsife Institute for New Economic Thinking
Working Paper No. 15-08
Doing it Twice, Getting it Right?
The Effects of Grade Retention and Course
Repetition in Higher Education
Darjusch Tafreschi and Petra Thiemann
February 2, 2015
Electronic copy available at: http://ssrn.com/abstract=2561048
Doing it Twice, Getting it Right?
The Effects of Grade Retention and Course
Repetition in Higher Education∗
Darjusch Tafreschi†and Petra Thiemann‡
February 2, 2015
Abstract
This paper is the first to study the effects of grade retention and course repetition onstudent performance in higher education. We apply a sharp regression discontinuitydesign to administrative data from a Swiss university. First-year undergraduates whofall short of a pre-defined performance requirement have to repeat all first-year coursesbefore they can proceed to the second year. We find that grade retention increasesdropout probabilities after the first year by about 10 percentage points. Repetition ofa full year persistently boosts grade point averages by about 0.5 standard deviations,but does not affect study pace and major choices.JEL codes: I21, I23, J24Keywords: grade retention, course repetition, higher education, dropout, academicachievement, regression discontinuity
∗A special thanks goes to Sharon Pfister who appeared as a coauthor on an earlier version of thispaper. Her insights were very valuable during the initial stages of this project. We thank Michael Lechner,Hans Fricke, Martin Huber, Sacha Kapoor, Charles Manski, Giovanni Mellace, Marco Manacorda, Hansvan Kippersluis, Conny Wunsch, and Ludger Wössmann, as well as conference participants at the EALEconference in Bonn, the SMYE in Mannheim, the SSES conference in Zurich, the YSEM in Berne, and theRGS conference in Duisburg-Essen for helpful comments and discussions. The usual disclaimer applies.†Erasmus University Rotterdam, Department of Applied Economics, PO Box 1738, 3000DR Rotterdam,
The Netherlands, phone: +31 (0)10 40 81383, [email protected].‡University of Southern California, USC Dornsife Institute for New Economic Thinking and Department
of Economics, 3620 S Vermont Ave, Los Angeles, CA 90089, USA, phone: +1 (213) 821-2035, [email protected].
Electronic copy available at: http://ssrn.com/abstract=2561048
1 Introduction
At many universities and colleges around the world, students who fall short of certain
performance standards are forced or encouraged to repeat failed courses, sometimes a
full semester, or even a full year.1 These (grade) retention policies are commonly designed
to improve the match between students’ skill levels and the level of teaching, and thus to
help initially low-performing students succeed in college.
Grade retention, however, is a controversial policy, as little is known about its effec-
tiveness to persistently boost students’ performance in higher education.2 Does retention
improve students’ attainment levels by confronting them again with the course material?
Or does retention rather damage students’ performances, for example through negative
side-effects on soft factors like confidence and social interactions? Does retention lead
to higher student dropout rates? Retention does not only potentially harm the retained
students, but it also imposes high monetary costs on institutions: Public post-secondary
institutions in OECD countries spend on average 9,000 USD annually per student on core
educational services like instruction (OECD, 2014).
The main obstacle to the study of retention is (self-)selection into retention and course
repetition. Students often self-select into retention or are retained by their institutions.
Retained students may thus differ systematically from their fellow students, for example,
in terms of prior performance. Hence, uncorrected differences in educational outcomes
between retained and non-retained students are likely to reflect differences in student
characteristics rather than the effects of retention.
This paper is the first to study grade retention in higher education. We exploit a grade1These policies are widely implemented. For example: Course repetition exists at California Commu-
nity Colleges, the University of California at Berkeley, the Massachusetts Institute of Technology, HarvardUniversity, Northwestern University, New York University. Repetition of a full semester exists at PrincetonUniversity. Retention of a full year occurs at the Federal Institute of Technology in Zurich, the University ofSt. Gallen (both Switzerland), and Queen Mary University of London. Institutions have idiosyncratic rulesfor retention, and for how repeated courses count towards grade point averages. Unfortunately, we are notable to provide a comprehensive overview over retention policies around the world, because of the lack ofdata.
2An alternative policy is “remedial education” at the college level, which refers to the repetition of high-school-level courses. Bettinger and Long (2009) find a positive impact of remedial education.
2
retention policy at the University of St. Gallen (Switzerland) in a sharp regression discon-
tinuity design. First-year students who do not achieve a strict performance requirement
after the first year have to repeat all first-year courses before they can proceed to the
second year. We exploit local variation in the retention status around the performance
threshold (henceforth also “cutoff”) in a sharp regression discontinuity design and esti-
mate the effects of (1) grade retention on dropout probabilities and (2) repetition of all
first-year courses on outcomes that occur after repeating: grade point averages, credits
obtained per semester, and major choice. Both performance and choice outcomes are
important for later labour market success (c.f. Altonji et al. (2012), Arcidiacono (2004)).
This paper contributes to both the empirical literature and the policy debate on retention
policies at different schooling levels. The effects of grade retention on individual student
outcomes are difficult to predict theoretically (c.f. Manacorda (2012)), which motivated a
number of empirical studies (see below for a review). While positive effects may arise be-
cause of learning gains and a better match between students’ knowledge and the level of
teaching, negative effects can occur because of stigmatisation by teachers or classmates,
lower self-confidence, and slow adjustment to a new classroom environment.
We are not aware of any empirical study on the effects of grade retention in the context
of higher education. By contrast, researchers have so far focused on retention in primary
and secondary education. Recent studies find positive effects on grades, especially for
primary school children.3 Yet, grade retention increases dropout rates of children during
high school.4 Overall, the effects of grade retention appear to be age-dependent, with
rather positive results for primary school, and rather negative results in high school.
The effects of grade retention and course repetition in higher education may differ
from results in primary and secondary education for at least three reasons. First, mature3An unequivocally positive effect on test scores seems to exist for retained 3rd graders in the US. Three
independent studies find a positive effect for Chicago (Jacob and Lefgren, 2004), Texas (Lorence andDworkin, 2006), and Florida (Greene and Winters, 2007). This result, however, does not hold for 6th graders(Jacob and Lefgren (2004)). Roderick and Nagaoka (2005) find even negative effects on test scores of 6thgraders in Chicago. All outcomes examined in these studies are short-term outcomes, that is, measured1–3 years after grade retention.
4This result has been confirmed by Jacob and Lefgren (2009) for 6th graders in Chicago, by Ou (2010)for 9th graders in New Jersey, and by Manacorda (2012) for 7th to 9th graders in Uruguay.
3
students may cope better with negative events, and thus, the negative effects may in
general be less pronounced for older students. Moreover, stigmatisation by fellow students
and instructors may be mitigated because university students are less exposed to small-
classroom settings. Bonding with initial cohort members may be stronger or weaker in a
university environment, too. Second, the effects of grade retention on dropout probabilities
may be higher for university students. In contrast to high school education, university
education is voluntary. Furthermore, university dropouts have better outside options than
high school dropouts. Third, university students may benefit especially from repeating the
first year. Some students need additional time to develop new study habits, such as self-
guided learning. For these students, grade repetition may provide a valuable chance of
adjusting.
We derive the following conclusions in a higher education setting: Retention increases
dropout probabilities significantly. Students who have to repeat a full year are 10 percent-
age points more likely to drop out after the first year than students who are immediately
promoted. We find large and persistent effects on subsequent academic performance due
to the repetition of first-year courses. By the time of on-time graduation, repeaters outper-
form non-repeaters by about 0.5 standard deviations in grade point averages. The result
is robust across specifications. In contrast to grades, credits obtained per semester as
well as major choice after the first year remain unaffected.
2 Institutional setup
The University of St. Gallen offers three-year undergraduate degree courses in Business
Administration, Economics, International Affairs, and Legal Studies. It plays an important
role for the education of Managers and Economists in Switzerland: In the past years,
around 30% of all Swiss graduates in Economics and Business Administration received
their degree from St. Gallen (Table A.1). According to federal law, the university must
4
admit all students with either a Swiss high school degree (“matura”) or a Swiss nationality.5
Following the general trend, the number of students has increased strongly over the last
two decades: Between 1990 and 2013, the number of first-year undergraduates more
than doubled (from 582 students to 1328 students) (Table A.2).
To maintain control of the number of enrolled students, the university introduced a
probation period, or “assessment year”, in 2001: Students have to fulfil a strict perfor-
mance requirement in order to continue their studies after the first year. This requirement
is based on course performance across the bundle of all compulsory first-year classes
(see details below). Students who either do not meet the performance requirement or do
not complete all compulsory courses are allowed to repeat the first year once. Over the
years 2001-2008, approximately 70% of all entering undergraduate students passed the
first year at their first attempt, and approximately 80% passed in total (first and second
attempts combined, see Table A.5).
Students in the same specialisation, or track, of the first year take the same set of com-
pulsory courses, and have identical exam schedules (Table A.3). In the following, we focus
on the Business/Economics track, which accounts for about 90% of all students (see Ta-
ble A.4).6 The core curriculum of this track consists of courses in Business Administration,
Economics, Legal Studies, and Mathematics, which are tested during central examination
periods of the first and second semesters. Furthermore, students submit a term paper in
one of the core subjects, which they hand in before they register for the second central
examination period. In addition to the core subjects, students take courses in leadership
skills and critical thinking. Moreover, they have to prove sufficient skills in one foreign
language.
Students accumulate “minus credits” for every course they fail. The total amount of
minus credits determines whether they pass or fail the first year. The grading scheme5Foreign students are admitted based on an entrance test. The admission quota for foreign students
varies by year, but is usually around 25%.6There are two subgroups of students for which the first year differs. First, students who intend to spe-
cialise in legal studies follow a different curriculum during the first year. Second, students of non-Germanmother tongue can chose to complete the first-year courses within two years instead of one (“extendedtrack”). Because of their special status, both groups are excluded from all analyses in this paper (seeTable A.4).
5
and the computation of minus credits work as follows: Each course is graded. Grades
range from 1 to 6 in steps of 0.25, with 1 being the lowest grade, 4 being the minimum
passing grade, and 6 being the highest grade.7 Credit points are granted for each course,
no matter if the student fails or passes the course.8 Minus credits for each failed course
are defined as the difference between the obtained grade and 4, multiplied by the number
of credits for this course. If, for example, a student receives a grade of 3.5 in a course
worth 4 credits, he obtains 2 minus credits for this course. The total sum of minus credits
(MC) is then calculated as
MC =S∑
s=1
(4−Gs) ∗ Cs ∗ 1(Gs < 4), (1)
where S is the number of compulsory subjects in the first year, Gs is the grade obtained
in course s, and Cs is the number of credit points associated with this course, 1(.) is the
indicator function.
Students can proceed to the second year if they complete all compulsory courses and
accumulate at most 12 minus credits during the first year; students who do not fulfil these
criteria can repeat the entire first year once (grade retention treatment). Enforcement of
this rule is strict, and students cannot compensate minus credits by grades greater than
4 in other subjects. If a student repeats and successfully passes the second attempt, he
also proceeds to the second year (see Figure A.1). In case of repeated failure, the student
is coercively ex-matriculated and cannot repeat again.
After the first year, students have to choose one out of four majors (“Business”, “Eco-
nomics”, “International Affairs”, or “Law and Economics”). In contrast to the curriculum
of the first year, the curriculum of the following years is major-specific. We therefore de-
note the phase of undergraduate studies which starts after the first year as “major-specific
level.” At the major-specific level, students can decide how many courses and credit points
they complete per semester. Students graduate on time if they complete the major-specific7Unauthorised absence from an exam results in the lowest possible grade, i.e. a grade equal to 1.8The curriculum of the first year consists of 60 credit points in total (55 in 2001).
6
level within four semesters. Only a minority of students (34%) completes their undergrad-
uate studies on time. The majority of students (84%) completes the major-specific level
within six semesters (Table 2).
The grade point average (GPA) that a student obtains at the end of his Bachelor’s
Degree is a weighted average of all exam grades obtained at the major-specific level
(weighted by the number of credits). Grades that a student obtains during the first year
neither count toward the final GPA, nor appear on the final grade sheet. GPA is one
of our main outcomes. It does not only reflect performance at the major-specific level–
not intermingled with first-year GPA–, but also serves as the main signal of students to
employers.
3 Data
3.1 Data sources, variables, and samples
We use anonymised administrative student records from the University of St. Gallen, that
is, enrolment, course performance, and graduation information for all students who start
their undergraduate degree between 2001 and 2008. In addition to individual background
characteristics, the data contain information on dropout decisions, major choice, and per-
formance (i.e., the number of credits and grades) for all completed courses up to the fall
semester 2013. Thus, we can track all students for at least 8 semesters, that is, we ob-
serve both repeaters and non-repeaters up to the point of on-time graduation. Because of
censoring in 2013, we exclude students who enter after 2008.
We further restrict the sample in two ways. First, we include only first-year students
with a German mother tongue who start the Business/Economics track. These students
have to complete identical courses. Different course requirements or exam procedures
apply to students with non-German mother tongue or to students in other tracks (in total
about 15%, see Table A.4). Second, we include only students who completed all first-year
courses. This criterion leads to a further 20% reduction in the sample size. We provide
7
details on this sample selection criterion and the characteristics of excluded students in
Section 3.2.
We construct all outcome measures from the course and enrolment data. In partic-
ular, we investigate the following outcomes: whether a student drops out after the first
year; whether a student enrols into the major-specific level; major choice after the first
year; the number of credits accumulated by the end of each major-specific semester; and
grade point averages accumulated by the end of each semester at the major-specific level.
Thus, we cover all important measures to track a students’ choices and performance. We
standardise grades at the level of the entry year into the major-specific level to account
for changes in grade distributions over time. To assess the effect of repetition, we com-
pare repeaters and non-repeaters always at the same stage of their studies. For example,
to investigate differences in grades at the major-specific level, we compare repeaters and
non-repeaters who are both in their first major-specific semester. Thus, at the point of time
of comparison, repeaters have completed already one additional year at the university, and
are on average one year older. This is the most frequently used method in grade reten-
tion studies (same-grade-comparison), especially in the absence of standardised testing
procedures.
To determine treatment status, we apply Equation 1 to the course data. The perfor-
mance cutoff is sharp, that is, students are retained if and only if they fail. We verify the
sharpness in our data (not shown, as the graph would be trivial). We define a student as
a repeater if he enrols again in the first year for at least one semester after being retained.
The data contain the following background characteristics: age at entry; gender; na-
tionality; mother tongue; country of origin as well as region of origin for students from
Switzerland; and country, region and date of a student’s high school graduation. From
these variables, we can derive whether the student had to take an entrance test to be ad-
mitted to the university.9 We use the background characteristics to assess the identifying
assumptions of the regression discontinuity design.9These are students with neither a Swiss high school degree nor a Swiss nationality. See Section 2 for
details.
8
3.2 What happens during the first year?
Our main estimation sample contains all freshmen entering, between 2001-2008, who
complete all compulsory first-year exams. These students are likely to differ from their
peers who do not complete all courses in their performance as well as background char-
acteristics. Thus, they present a selected sample. To understand the characteristics of
our selected sample versus the sample of excluded students, we define and describe the
different types of excluded and included students based on their performance during the
first year.
Table 1: Description of student types
Types Dropout decision Exams taken Minus credits Obs. % of1st sem. 2nd sem. 1st sem. sample
1 Drops out during 1st sem. Not all None - 392 7.32 Not admitted to 2nd sem. All None > 12 437 8.13 Drops out after 1st or during 2nd sem. All Not all ≤ 12 181 3.44 Completes full 1st year All All ≤ 12 4,382 81.3
Total 5,392 100
The table characterises different student types according to their behaviour and performance during thefirst year. Based on all first-year students with German mother tongue who enrolled into theBusiness/Economics track between 2001 and 2008 (5,392 students). Type 4 constitutes the estimationsample.
In particular, we exclude the following types of students: (i) students who have not
completed all compulsory first-semester courses (type 1, N=392); (ii) students who have
completed all compulsory first-semester courses, but have exceeded the threshold of 12
minus credits already during the first semester and are therefore not admitted to the sec-
ond semester (type 2, N=437); and (iii) students who have passed the first semester but
have dropped out voluntarily after the first or during the second semester (type 3, N=181).
Students who have completed all required first-year courses and who did not accumulate
more than 12 minus credits during the first semester form our estimation sample (type 4,
N=4,382). These students account for 81% of all students with a German mother tongue
who enter the Business/Economics track.
9
Figure 1: First-year grade distributions by student type
1 2 3 4 5 6Grade
Term paperLegal Studies 2
Business 2Economics 2
Mathematics 2Legal Studies 1
Business 1Economics 1
Mathematics 1
Drops out during 1st semesterType 1
1 2 3 4 5 6Grade
Term paperLegal Studies 2
Business 2Economics 2
Mathematics 2Legal Studies 1
Business 1Economics 1
Mathematics 1
Not admitted to 2nd semesterType 2
1 2 3 4 5 6Grade
Term paperLegal Studies 2
Business 2Economics 2
Mathematics 2Legal Studies 1
Business 1Economics 1
Mathematics 1
Drops out after 1st/during 2nd semesterType 3
1 2 3 4 5 6Grade
Term paperLegal Studies 2
Business 2Economics 2
Mathematics 2Legal Studies 1
Business 1Economics 1
Mathematics 1
Completes full first yearType 4
Note: The figure shows course performance distributions (absolute course grades, with 1 being the lowestand 6 being the highest grade) for different student types, based on first-semester core courses. The boxplots display the median values (thick black lines), 25th percentiles (p25, left margin of each box), 75thpercentiles (p75, right margin of each box) and lower/upper adjacent values (left margin of thewhiskers/right margin of the whiskers). The lower and upper adjacent values are defined asp25− 1.5 ∗ (p75− p25) and p75 + 1.5 ∗ (p75− p25), respectively. Based on the sample of all first-yearstudents with German mother tongue who enter the Business/Economics track (5,392 students). Detailedtype definitions are presented in Section 3.2.
10
Students who complete the full first year outperform students who drop out before
they complete all first-year courses. Figure 1 compares course performance distributions
across types, based on completed courses during the first year. Median comparisons re-
veal little differences between types 1 and 2, that is, between voluntary and involuntary
dropouts during the first semester. By contrast, median performance for type-3 students
lies above median performance for type-2 students in all four first-semester core courses.
In more than half of the first-year courses (five out of nine), however, the median perfor-
mance of type-3 students still remains below the passing grade of 4. Students who com-
plete all first-year courses (type-4 students) fare best: They outperform type-3-students in
terms of median performance in eight out of nine courses.
Table A.6 shows differences in background characteristics across student types. Stu-
dents who complete all first-year courses are more often male and on average slightly
younger than students who drop out prematurely. Moreover, non-Swiss students and stu-
dents who have taken an entrance test are over-represented among students who com-
plete the full first year. These are students with higher costs of starting their degree in
St. Gallen (e.g., they have to study for the entrance exam, they have to move to a for-
eign country, they have to pay higher student fees). Thus, the data are in line with the
hypothesis that students who face higher initial costs are initially more motivated or able
(c.f. Manski (1989)).
3.3 Descriptive statistics
Table 2 provides descriptive statistics for the estimation sample (type 4), and compares
those who passed and those who failed their first attempt. As the composition of the
student pool as well as grading schemes remain stable over the years (see Table A.7), we
focus on the pooled sample for all eight cohorts.
The large majority of students is male (73%), and the average age of students is just
above 20. Non-Swiss students account for only 24% of the sample, as separate admission
rules apply to non-Swiss students: The fraction of students with neither a Swiss high
11
Table 2: Estimation sample: Descriptive statistics (cohorts 2001-2008)
Full sample Pass Fail Difference (p-value) # Obs.
(1) (2) (3) (4) (5) (6)# Obs. 4,382 3,817 565Background characteristics
Age 20.26 20.22 20.50 -0.28 (0.000) 4,382Male 73% 74% 72% 2% (0.318) 4,382Non-Swiss nationality 24% 25% 16% 9% (0.000) 4,382Entrance test 20% 22% 8% 14% (0.000) 4,382High school St. Gallen 15% 14% 18% -4% (0.014) 4,382
Minus creditsFraction: MC > 0 57% 50% 100% -50% (0.000) 4,382# MC in first year 4.67 2.46 19.64 -17.19 (0.000) 4,382
Course performance 1st yearGrade Business 1 4.22 4.30 3.68 0.62 (0.000) 4,382Grade Economics 1 4.58 4.69 3.90 0.79 (0.000) 4,382Grade Mathematics 1 4.53 4.66 3.59 1.07 (0.000) 4,382Grade Legal Studies 1 4.47 4.58 3.74 0.84 (0.000) 4,382Grade Business 2 4.26 4.38 3.46 0.92 (0.000) 4,382Grade Economics 2 4.45 4.60 3.50 1.10 (0.000) 4,382Grade Mathematics 2 4.52 4.68 3.47 1.20 (0.000) 4,382Grade Legal Studies 2 4.33 4.47 3.41 1.05 (0.000) 4,382Grade Term paper 4.98 5.03 4.59 0.45 (0.000) 4,382
Outcomes: DropoutDropout after 1st year 2% 1% 14% -13% (0.000) 4,382Repeater 11% 0 86% -86% - 4,382
Outcomes: Major-specific levelMajor-specific level started 96% 99% 71% 28% (0.000) 4,382Major Business 69% 69% 73% -4% (0.084) 4,189Major Economics 16% 17% 9% 7% (0.000) 4,189Major International Affairs 15% 15% 13% 3% (0.166) 4,189Major Law and Economics 6% 6% 4% 3% (0.026) 4,189
Outcomes: Credits at the major-specific levelCredits by 1st major-specific semester 26.5 26.7 24.9 1.8 (0.000) 4,189Credits by 2nd major-specific semester 54.0 54.3 51.0 3.3 (0.000) 4,189Credits by 3rd major-specific semester 77.5 77.8 74.1 3.7 (0.000) 4,189Credits by 4th major-specific semester 103.2 103.7 98.0 5.7 (0.000) 4,189
Outcomes: Grade point averages (GPA)GPA (std) by 1st major-specific semester 0.08 0.15 -0.55 0.70 (0.000) 4,079GPA (std) by 2nd major-specific semester 0.10 0.18 -0.65 0.82 (0.000) 4,079GPA (std) by 3rd major-specific semester 0.11 0.19 -0.68 0.87 (0.000) 4,079GPA (std) by 4th major-specific semester 0.12 0.20 -0.67 0.87 (0.000) 4,079
Bachelor’s Degree completedWithin 4 major-specific semesters (on time)(1) 34% 36% 20% 15% (0.000) 3,522Within 5 major-specific semesters(1) 59% 61% 45% 16% (0.000) 3,522Within 6 major-specific semesters(1) 84% 86% 70% 16% (0.000) 3,522
The table shows descriptive statistics for the estimation sample, that is, for all students who havecompleted all first-year courses. Column (1) shows means for the full sample, and columns (2) and (3)show the means for the sub-samples of students who passed and who failed the first year. Column (4)displays the difference in mean characteristics, and column (5) shows the p-value of the t-test for the H0 ofno difference in mean characteristics. Variables at the major-specific level are missing for individuals whonever enrol into the second year. Variables for major choice do not sum to 100% due to individuals whocomplete double majors. Grade point averages are missing for individuals who do not complete any creditsduring the first major-specific semester in which they are enroled. Information on completion of theBachelor’s Degree is missing for the cohort of 2008 due to censoring.(1)Values not available for the cohort of 2008. 12
school diploma nor Swiss citizenship is restricted to at most 25% of the student body.
Students who pass are different from students who fail, both in terms of background
characteristics and in terms of their performance at all levels (Table 2). Students who pass
are younger, less likely to come from St. Gallen, and more likely to come from outside
Switzerland than students who fail. They also perform significantly better in all exams,
which translates into a low dropout rate of only 1% after the first year. By contrast, 14%
of students who fail decide not to repeat the first year, that is, they drop out. The superior
performance of students who pass carries over to the major-specific level. Students who
pass the first year in their first attempt collect on average more credits and better grades
than students who do not pass the first attempt. This translates into earlier graduation
dates, counted from the beginning of the major-specific level, compared to students who
repeat and pass the first year in their second attempt.
The raw comparison between repeaters and non-repeaters would thus lead to the
biased conclusion that repetition has negative effects. Repeaters perform significantly
worse than non-repeaters at the major-specific level; but repeaters are low-performing
students in the first place. For instance, 50% of students who pass never obtain any
minus credits.
4 Empirical strategy
4.1 Identification
A comparison between the outcomes of all retained with all non-retained students would
lead to biased estimates of the effects of interest. The bias occurs as retained students are
lower performing students on average. To solve this identification problem, we implement
a regression discontinuity design, that is, we rely on a comparison of first-year students
who tightly pass with first-year students who tightly fail.
The identification of the treatment effect in a regression continuity design requires
the assumption of “local continuity” at the performance cutoff (c.f. Imbens and Lemieux
13
(2008)). This assumption states that the distribution of observed and unobserved charac-
teristics in the student pool must change smoothly at the threshold. For example, the aver-
age ability level of students must not abruptly deteriorate at the threshold. The assumption
implies that students who tightly fail would achieve on average the same outcome as stu-
dents who tightly pass, if they were exposed to the same policy (e.g., if everyone was
directly promoted to the second year).
Local continuity is likely to hold in our setting as students have–if anything–only impre-
cise control over the number of minus credits they accumulate. Students may set their
effort level before or during the semester to target a specific number of minus credits.
But grades are disclosed exclusively at the end of each semester, and all course grades
are jointly disclosed on the same day by mail. Therefore, students cannot adjust their
effort level in one subject in response to an unexpectedly high number of minus credits in
another subject.
Thus, whether a student accumulates, for example, 13 instead of 12 minus credits may
result from bad luck on just one of the core exams (e.g., equivalent to failing a four-credit
exam by the minimal grade margin, which is 0.25). This may be unrelated to how well
the student prepared for the whole exam period. If this holds for all students, the pool of
students with 12 minus credits resembles the pool of students with 13 minus credits in
terms of their behaviour and characteristics (e.g., learning behaviour, motivation, gender).
We can continue this thought experiment for other values in the neighbourhood around
the threshold (for example, 12 versus 12.25, 12 versus 14, or 11 versus 13 credits). As
we move farther away from the threshold, we expect the groups on opposite sides of the
threshold to become less comparable.
We test for potential violations of the local continuity assumption in various ways. First,
the assumption is violated if some students who are in danger of failing study just enough
to ensure that they do not have to repeat. Then, we would observe significantly fewer
students who just fail than students who just pass. Students, however, cannot adjust their
effort in such a precise way, as the university reveals grades and final credits only at the
14
end of each central exam period. Consequently, Figure 2 shows no visual indication of
sorting around the threshold. Likewise, the McCrary test (McCrary, 2008) does not reject
the null-hypothesis of continuity of minus credits at the cutoff (log difference in height =
0.21, standard error = 0.14).
Figure 2: Histogram of the assignment variable
010
020
030
040
050
0F
requ
ency
−10 0 10 20Number of minus credits (recentred)
Note: The figure shows a histogram of minus credits for all students in the estimation sample who accu-mulated at least one minus credit during the first year (2,486 students). The assignment variable (minuscredits) is the re-centred sum of minus credits accumulated during the first year (i.e. MC - 12.25, where MCis computed according to equation 1).
Second, the assumption is violated if students can manipulate grades of single exams.
For example, students may try to influence their graders to adjust pivotal grades. Students,
however, have no control over the grading process, and cannot otherwise manipulate
single grades. Thus, average grades in all core courses change smoothly at the cutoff
(Figure 3).
Third, we investigate the local continuity of background characteristics around the
threshold. We compare the means of student characteristics (age, gender, nationality,
whether students completed an entrance test, whether students completed their high
school in the canton of St. Gallen) in a neighbourhood of 0.5 and 1 minus credits to
both sides of the cutoff. In addition, we fit parametric linear and higher polynomial models,
as well as nonparametric models to both sides of the cutoff to detect non-continuities (see
Section 4.2 for details). Table 3 presents the coefficient estimates and standard errors
from these estimations. For gender, nationality, and entrance-test participation, we find no
15
Figure 3: Smoothness of first-year course grades around the threshold (first attempt)
34
5G
rade
Bus
ines
s 1
−10 −5 0 5 10Number of minus credits (adjusted)
discontinuity coeff. (se): .06 (.04)
(1) Business 1
34
5G
rade
Eco
nom
ics
1
−10 −5 0 5 10Number of minus credits (adjusted)
discontinuity coeff. (se): .07 (.06)
(2) Economics 1
34
5G
rade
Mat
h 1
−10 −5 0 5 10Number of minus credits (adjusted)
discontinuity coeff. (se): −.04 (.07)
(3) Mathematics 1
34
5G
rade
Mat
h 1
−10 −5 0 5 10Number of minus credits (adjusted)
discontinuity coeff. (se): .05 (.05)
(4) Legal Studies 1
34
5G
rade
Bus
ines
s 2
−10 −5 0 5 10Number of minus credits (adjusted)
discontinuity coeff. (se): −.11 (.05)
(5) Business 2
34
5G
rade
Eco
nom
ics
2−10 −5 0 5 10
Number of minus credits (adjusted)
discontinuity coeff. (se): −.03 (.06)
(6) Economics 2
34
5G
rade
Mat
h 2
−10 −5 0 5 10Number of minus credits (adjusted)
discontinuity coeff. (se): .1 (.07)
(7) Mathematics 2
34
5G
rade
Mat
h 2
−10 −5 0 5 10Number of minus credits (adjusted)
discontinuity coeff. (se): .05 (.05)
(8) Legal Studies 2
34
5G
rade
r T
erm
pap
er
−10 −5 0 5 10Number of minus credits (adjusted)
discontinuity coeff. (se): .05 (.06)
(9) Term paper
Note: The figure illustrates the smoothness of first-year course grades. The black dots indicate means ofcourse grades in bins of a width of 1 minus credit. Grades range from 1 to 6 with 1 being the lowest grade,6 being the highest grade, and 4 being the passing grade. The solid lines present a linear fit to both sidesof the threshold. The number of minus credits is re-centred around the cutoff of 12.25 minus credits. Thedashed lines indicate 95% confidence intervals. The discontinuity coefficient indicates the distance of thetwo regression lines exactly at the cutoff. Standard errors are in parentheses. The sample consists of allindividuals in the estimation sample within a range of 10 minus credits to either side of the cutoff (n = 1967).
16
evidence of sorting around the threshold. For age and local high school degree, we find
at most weak evidence of sorting. As none of the differences at the threshold are robust
across specifications, we conclude that sorting patterns around the threshold are due to
chance variations. Figure A.2 visualises the non-systematic patterns on both sides.
Table 3: RDD estimates: Student background characteristics
(1) (2) (3) (4) (5) (6) (7) (8) (9)Male -0,04 -0.01 -0.09 -0.06 0.04 -0.03 -0.08 -0.01 -0.08
(0.12) (0.08) (0.18) (0.07) (0.11) (0.16) (0.11) (0.09) (0.19)Age -0.43 -0.10 -1.00* -0.37 -0.60 -0.33 -0.70* -0.75* -0.55
(0.38) (0.22) (0.51) (0.31) (0.49) (0.69) (0.40) (0.34) (0.22)Non-Swiss nationality -0.09 -0.04 -0.08 -0.02 0.00 -0.11 0.01 0.01 -0.07
(0.09) (0.06) (0.13) (0.05) (0.06) (0.14) (0.06) (0.06) (0.05)Entrance Test -0.12 -0.07 -0.10 -0.02 -0.04 -0.17 0.01 -0.02 -0.09
(0.08) (0.05) (0.11) (0.03) (0.05) (0.14) (0.05) (0.05) (0.06)High school St. Gallen -0.17 -0.10 -0.26 -0.04 -0.18* -0.26 -0.15 -0.08 -0.14
(0.11) (0.07) (0.16) (0.06) (0.11) (0.18) (0.10) (0.08) (0.15)
Observations 65 140 140 790 790 790 1967 2378 2378Estimation [-0.5;0.5] [-1;1] [-1;1] [-5;5] [-5;5] [-5;5] [-10;10] [-12;12] [-12;12]windowPolynomial 0 0 1 1 2 3 3 3 NPorder
Note: RDD estimates (average marginal effects for all outcomes except age) using student backgroundcharacteristics as outcome variables. The different columns use different estimation windows and varyingpolynomial orders of the underlying running variable (minus credits). The respective estimation window foreach specification is reported as the minus credit range on each side of the threshold. All parametricestimates are based on logistic regression models for binary outcomes (male, non-Swiss nationality,entrance test, high school St. Gallen) and OLS models for continuous outcomes (age). Following Imbensand Kalyanaraman (2012), the bandwidth for the local-linear nonparametric specification (NP) isdetermined by cross-validation. The standard errors for the NP estimates are computed using 5000bootstrap replications. In all regressions, we control for a student’s cohort.* Significant at 10%-level, ** Significant at 5%- level, *** Significant at 1%-level. Standard errors are inparentheses.
Missing outcomes further complicate the identification of the effects of repeating. In
particular, outcomes at the major-specific level (credits, grades per semester) are miss-
ing for students who drop out after being retained (i.e., they do not repeat, they fail the
second attempt, or they drop out before or during the first major-specific semester). If
dropout probabilities are correlated with observed or unobserved student characteristics,
the estimated effect of repetition on educational achievement may be biased. For exam-
ple, students who are highly motivated may be less likely to drop out, even if they have to
repeat. Thus, the pool of repeaters may contain more highly motivated students than the
17
pool of non-repeaters. As a result, effect estimates of repeating may reflect differences in
the level of motivation between the two groups rather than the true effect of repeating.
In the presence of these potential biases, we cannot identify point estimates of the ef-
fects of repetition without additional assumptions; yet, we can compute interval estimates,
or “bounds”, for the effect. We follow Lee (2009), who presents identification results for
bounds under the assumption of “weak monotonicity”. Weak monotonicity implies that
grade retention affects dropout probabilities only in one direction. In other words, no stu-
dent drops out if and only if he is not retained. Data as well as economic reasoning
support this assumption. First, only 1% of non-retained students drop out, whereas 14%
of retained students drop out (Table 2). Second, as repetition is costly, both in terms of
effort and in terms of opportunity costs, no student should continue only because he is
retained. The bounds presented by Lee (2009) are sharp, that is, one cannot tighten the
bounds without additional assumptions. The bounds, however, are not valid for the whole
student population, but only for a sub-population. This sub-population contains only stu-
dents who would continue to study under any circumstances, that is, regardless of their
retention status (“never drop-outs”).
We provide descriptive evidence to support the importance of accounting for miss-
ing outcomes. To test whether dropout probabilities among retained students depend on
observable characteristics, we regress an indicator for non-missing GPA at the major-
specific level on first-year course performances as well as on student characteristics, and
control for minus credits during the first year (Table A.8). Retained students with better
performance in the first-year, second-semester courses “Business 2”, “Economics 2”, and
“Legal Studies 2” are significantly less likely to drop out than retained students with lower
performance in these subjects. Moreover, retained students who are older are less likely
to drop out than retained students who are younger. By contrast, we do not find equivalent
patterns among non-retained students, partly as dropout probabilities among non-retained
students are close to zero (Table A.9). These findings emphasise the importance of ac-
counting for potential biases which may arise from missing outcomes.
18
4.2 Estimation
For continuous outcomes (e.g., GPA), we estimate models of the following type:
Y =K∑k=0
αk ∗MCk +K∑k=0
βk ∗MCk ∗ 1(MC ≥ 0) +X ′γ + ε, (2)
where Y represents the educational outcome , MC corresponds to the re-centred number
of minus credits collected at the end of the first year (re-centred around the cutoff of 12 mi-
nus credits), k is the polynomial order (between 0 and 3 in our application), X is a vector of
control variables (e.g., cohort dummies), and ε is an error term. β0 denotes the estimated
causal effect of grade retention and repetition, respectively. We use varying windows of
data around the threshold. Using higher order polynomials and interaction terms, we allow
for a non-linear relationship as well as different slopes on both sides of the cut-off and bet-
ter fits for larger estimation windows (Imbens and Lemieux, 2008). For binary outcomes
(e.g., whether a student drops out), we use logistic regression models, but the results are
robust to alternative specifications (linear probability models, probit models; results not
shown). In all parametric specifications we control for a student’s cohort to account for
time-varying conditions such as course contents and grading schemes.
We also provide non-parametric point estimates (Imbens and Lemieux, 2008), which
rely on local linear regressions to either side of the threshold. The optimal bandwidth is
derived by cross-validation, and standard errors are computed using the bootstrap method
as suggested by Imbens and Kalyanaraman (2012).10
The non-parametric estimation of bounds for the population of never drop-outs, that is,
students that continue regardless of their retention status, follows Lee (2009). We focus
on mean comparisons within small windows around the cutoff, that is, windows of 0.5, 1,
and 1.5 minus credits to either side of the cutoff.
The estimation procedure for the bounds is based on a trimming method. We observe
that more retained students than non-retained students drop out. The difference p in
dropout probabilities can be estimated from the data and signifies the fraction of students10We implement the analysis using the -rd- command in STATA (Nichols, 2011).
19
who drop out if and only if they are retained (“compliers”, c.f. Lee (2009)) among the
joint population of compliers and never drop-outs. To compute the first bound, we trim the
upper p percent of the outcome distribution of non-retained students. We then compute the
first bound as the difference between the trimmed mean outcome for the sample of non-
retained students and the (untrimmed) mean outcome for the sample of retained students.
To compute the second bound, we proceed as before, but now start by trimming the lower
p percent of the outcome distribution of non-retained students. Whether the first bound
is the upper or lower bound depends on the direction of the treatment effect. Because
of small sample sizes, we do not include further control variables. We report analytic
standard errors based on Lee (2009).11
5 Results
5.1 The effect of grade retention on dropout probabilities
Retained students are potentially more likely to drop out for two reasons. First, students
who failed the first year and choose to repeat may fail for a second time and thus never en-
ter the major-specific level. Second, conditional on passing their second attempt, retained
students will enter the labour market one year later. As a result, the utility from continuing
to study may be lower for retained students, which may lead to higher dropout rates (c.f.
Manski (1989)). Stigmatisation by fellow students or instructors, and the costs related to
re-adjustments to new peers are likely to influence the drop-out decision, too.
Dropout probabilities immediately after the first year are indeed higher among retained
students than among non-retained students. Figure 4 shows a jump in the dropout proba-
bility directly at the cutoff, and a further increase in dropout probabilities with the number
of minus credits. The dropout probability lies below 2% among non-retained students, but
increases to more than 10% at the cutoff. The regression discontinuity estimates confirm
this pattern (Table 4). Because of the further increase after the cutoff, our preferred spec-11We implement the analysis using the -leebounds- command in STATA (Tauchmann, 2012).
20
Figure 4: RDD estimates: Dropout probability after the first year and probability of entering themajor-specific level
0.2
.4.6
.81
Pro
babi
lity
of d
ropo
ut a
fter
the
first
yea
r
−10 −5 0 5 10Number of minus credits (adjusted)
discontinuity coeff. (se): .083 (.027)
(1) Dropout probability after the first year
0.2
.4.6
.81
Pro
babi
lity
of e
nter
ing
the
maj
or−
spec
ific
leve
l
−10 −5 0 5 10Number of minus credits (adjusted)
discontinuity coeff. (se): −.194 (.034)
(2) Probability of entering the major−specific level
Note: The panels above provide a graphical illustration of the effects of retention on dropout probabilitiesafter the first year and probabilities of entering the major-specific level. The dots represent proportions,computed within bins of 2 minus credits. The black lines display quadratic fits to either side of the cutoff,together with 95% confidence intervals (dashed lines). The discontinuity coefficient indicates the distanceof the two regression lines exactly at the cutoff. Standard errors are in parentheses. The sample consistsof all individuals in the estimation sample within a range of 10 minus credits to either side of the cutoff (n =1967).
ifications are either small windows of at most one minus credit to both sides around the
cutoff (columns 1 and 2), wider windows with at least a quadratic term (columns 7 and
8), or even non-parametric estimates (column 9). The estimates from these specifications
vary between three and ten percentage points, and are not always significant, due to the
large variance in dropout probabilities for retained students.
The negative effect of retention on the probability of entering the major-specific level
is larger than the effect on dropout probabilities immediately after the first year: It ranges
from -9 to -20 percentage points in our preferred specifications and is mostly statistically
significant (Table 4). At least three reasons account for the increase in dropout rates
between the end of the first year and the beginning of the major-specific level. First,
some of the retained students may enrol for their second attempt while looking for outside
opportunities. Second, retained students may update the costs of repeating only after
having started their second attempt. Third, students may fail the first year for a second
21
Table 4: RDD estimates: Dropout probabilities (average marginal effects)
(1) (2) (3) (4) (5) (6) (7) (8) (9)Dropout probability 0.03 0.10** -0.04 0.10** 0.00 -0.03 0.07 0.06 0.04after the first year (0.06) (0.04) (0.10) (0.04) (0.12) (0.19) (0.07) (0.08) (0.08)Probability of entering -0.09 -0.20*** -0.05 -0.19*** -0.10 -0.05 -0.16* -0.16* -0.15the major-specific level (0.07) (0.05) (0.13) (0.05) (0.13) (0.20) (0.09) (0.09) (0.14)
Observations 65 140 140 790 790 790 1967 2378 2378Estimation [-0.5;0.5] [-1;1] [-1;1] [-5;5] [-5;5] [-5;5] [-10;10] [-12;12] [-12;12]windowPolynomial 0 0 1 1 2 3 3 3 NPorder
Note: The table shows RDD estimates (average marginal effects) for the effect of grade retention onstudent dropout probabilities. Students with zero minus credits are excluded from the estimation sample.All parametric estimates are based on logit models and control for cohort effects. Following Imbens andKalyanaraman (2012), the bandwidth for the local-linear non-parametric specification (NP) is determined bycross-validation. Standard errors for NP estimates are computed using 5000 bootstrap replications.* Significant at 10%-level, ** Significant at 5%-level, *** Significant at 1%-level. Standard errors are inparentheses.
time and thus be forced to drop out. The chance of passing the second attempt, however,
is high: approximately 80% of all students who complete the second attempt pass the first
year (Table A.5).
5.2 The effect of repeating on major choice
Students decide upon their major at the end of the first year. Repetition of the first year may
thus induce students to update their preferences for certain subjects and majors. Major
choice is a relevant outcome for two reasons. First, major choice determines which skills
a student acquires, and thus influences future earnings (c.f. Altonji et al. (2012), Arcidia-
cono (2004)). Second, students’ performance (grades, credits) might differ across majors.
Thus, if we detect differences in major choice between repeaters and non-repeaters, we
should account for these differences in the further analysis.
Overall, repetition does not affect major choice at the threshold. Table 5 investigates
major choice for students who enter the major-specific level. Repeaters are less likely to
choose “Business”, “Economics”, and “Law and Economics” as majors, and are slightly
more likely to choose International Affairs. The coefficients do not sum to one as some
22
Table 5: RDD estimates: Major choice probabilities (average marginal effects)
(1) (2) (3) (4) (5) (6) (7) (8) (9)Major: Business -0.04 -0.06 -0.11 -0.07 -0.04 -0.02 -0.12 -0.10 -0.09
(0.12) (0.08) (0.19) (0.07) (0.12) (0.17) (0.11) (0.10) (0.20)Major: Economics -0.13 -0.02 -0.16 0.01 -0.02 -0.22 0.04 0.03 -0.08
(0.09) (0.05) (0.11) (0.04) (0.07) (0.17) (0.06) (0.05) (0.16)Major: International 0.10 0.08 0.12 0.05 0.10 0.09 0.07 0.07 0.14***Affairs (0.07) (0.06) (0.15) (0.05) (0.09) (0.10) (0.07) (0.07) (0.02)Major: Law and -0.06 -0.05 0.00 -0.06 -0.05 -0.04 -0.06 -0.05 -0.06Economics (0.06) (0.04) (0.10) (0.04) (0.06) (0.06) (0.06) (0.06) (0.08)
Observations 58 122 122 737 737 737 1857 2256 2256Estimation [-0.5;0.5] [-1;1] [-1;1] [-5;5] [-5;5] [-5;5] [-10;10] [-12;12] [-12;12]windowPolynomial 0 0 1 1 2 3 3 3 NPorder
Note: The table shows RDD estimates (average marginal effects) for the effect of retention on major choiceprobabilities. Students with zero minus credits are excluded from the estimation sample. The coefficientsdo not sum to one as some students choose double majors. All parametric estimates are based on logitmodels and control for cohort effects. Following Imbens and Kalyanaraman (2012), the bandwidth for thelocal-linear nonparametric specification (NP) is determined by cross-validation. Standard errors for NPestimates are computed using 5000 bootstrap replications.* Significant at 10%-level, ** Significant at 5%-level, *** Significant at 1%-level. Standard errors are inparentheses.
students take double majors. Yet, none of the regression coefficients is significant, with the
exception of the non-parametric specification for the major in International Affairs (column
9). Thus, performance effects at the major-specific level are unlikely to be biased because
of major choices.
5.3 The effect of repeating on study pace and grade point averages
Repeating may affect both study pace and grades throughout the remaining studies. At the
major-specific level students can choose the number of subjects they take per semester.
Repeaters may attend more courses per semester to make up for the time lost because
of retention. Students, however, also face a trade-off between the number of courses per
semester and the effort that they spend on each course.
Repetition barely affects study speed. In general, the number of credits per semester
at the major-specific level is uncorrelated with the performance of students during their
first year. As Figure 5 shows, the number of credits is almost constant at around 25 cred-
23
Table 6: RDD estimates: Credit points by the end of each major-specific semester
Credits by major- 1.58 3.03** 3.70 2.42* 3.03 5.22* 2.96 2.00 2.36**specific semester 1 (2.07) (1.43) (3.37) (1.33) (2.08) (2.89) (1.81) (1.61) (1.30)Credits by major- 0.85 4.88* 2.76 5.20** 3.50 6.01 4.58 4.39 4.47specific semester 2 (3.77) (2.63) (6.19) (2.28) (3.57) (4.95) (3.09) (2.75) (6.50)Credits by major- -1.66 2.56 -0.87 3.87 0.61 2.45 2.07 1.54 2.16***specific semester 3 (5.06) (3.51) (8.25) (2.85) (4.46) (6.19) (3.84) (3.42) (0.52)Credits by major- -1.52 1.25 -3.24 1.21 -1.10 0.17 -0.67 -1.17 0.05specific semester 4 (6.48) (4.51) (10.59) (3.62) (5.66) (7.85) (4.87) (4.32) (7.57)
Observations 58 122 122 737 737 737 1,857 2,256 2,256Estimation [-0.5;0.5] [-1;1] [-1;1] [-5;5] [-5;5] [-5;5] [-10;10] [-12;12] [-12;12]windowPolynomial 0 0 1 1 2 3 3 3 NPorder
Note: The table shows RDD estimates for the effect of retention on credits per semester. Students withzero minus credits are excluded from the estimation sample. All parametric estimates are based on linearmodels and control for cohort effects. Following Imbens and Kalyanaraman (2012) the bandwidth for thelocal-linear nonparametric specification (NP) is determined by cross-validation. Standard errors for NPestimates are computed using 5000 bootstrap replications.* Significant at 10%-level, ** Significant at 5%-level, *** Significant at 1%-level. Standard errors are inparentheses.
its per semester on average, regardless of the number of minus credits, and regardless of
the repeater status. Retained students close to the cutoff accumulate slightly more credit
points per semester. Table 6 confirms this pattern. The effect of course repetition on cred-
its per semester is significant only in the first two semesters. Moreover, the effect amounts
to at most 5 credit points by the end of the second major-specific semester. As the en-
tire major-specific curriculum consists of 120 credit points, the effects on study speed are
negligible. Furthermore, the differences in credits per semester between repeaters and
non-repeaters fade out by the fourth major-specific semester.
By contrast, course repetition has strong and significant effects on grade point aver-
ages at the major-specific level. Overall, repeaters perform consistently about 0.5 stan-
dard deviations better than non-repeaters (Table 7). The effect is robust across speci-
fications and ranges from between 0.42 standard deviations to 0.7 standard deviations.
Moreover, the gains persist throughout all semesters. Figure 6 provides visual evidence.
The performance of students at the major-specific level decreases overall in the number of
minus credits during the first year. Repetition leads to an upward jump in grade point aver-
24
Figure 5: RDD estimates: Credits accumulated by the end of each major-specific semester
2040
6080
100
120
Cre
dits
by
maj
or−
spec
ific
sem
este
r 1
−5 0 5Number of minus credits (adjusted)
discontinuity coeff. (se): 2.2 (1.37)
(1) Credits by major−specific semester 1
2040
6080
100
120
Cre
dits
by
maj
or−
spec
ific
sem
este
r 2
−5 0 5Number of minus credits (adjusted)
discontinuity coeff. (se): 4.74 (2.34)
(2) Credits by major−specific semester 2
2040
6080
100
120
Cre
dits
by
maj
or−
spec
ific
sem
este
r 3
−5 0 5Number of minus credits (adjusted)
discontinuity coeff. (se): 3.34 (2.89)
(3) Credits by major−specific semester 3
2040
6080
100
120
Cre
dits
by
maj
or−
spec
ific
sem
este
r 4
−5 0 5Number of minus credits (adjusted)
discontinuity coeff. (se): .68 (3.6)
(4) Credits by major−specific semester 4
Note: The panels above provide a graphical illustration of the effects of retention on the number of creditsaccumulated for the Bachelor’s Degree by the end of each of the first four major-specific semesters. Thedots represent the mean outcomes in bins of 1 minus credits. The solid lines display a linear fit to eitherside of the cutoff (dashed lines: 95% confidence intervals). The discontinuity coefficient indicates thedistance of the two regression lines exactly at the cutoff. Standard errors are in parentheses. The sampleconsists of all individuals in the estimation sample within a range of 5 minus credits to either side of thecutoff who have non-missing information on credits (n = 737).
25
Figure 6: RDD estimates: Grade point averages (GPA) by the end of each major-specific semester(standardised)
−1
−.8
−.6
−.4
−.2
0G
PA
(st
d) b
y m
ajor
−sp
ecifi
c se
mes
ter
1
−5 0 5Number of minus credits (adjusted)
discontinuity coeff. (se): .53 (.14)
(1) GPA (std) by major−specific semester 1
−1
−.8
−.6
−.4
−.2
0G
PA
(st
d) b
y m
ajor
−sp
ecifi
c se
mes
ter
2−5 0 5
Number of minus credits (adjusted)
discontinuity coeff. (se): .52 (.14)
(2) GPA (std) by major−specific semester 2
−1
−.8
−.6
−.4
−.2
0G
PA
(st
d) b
y m
ajor
−sp
ecifi
c se
mes
ter
3
−5 0 5Number of minus credits (adjusted)
discontinuity coeff. (se): .46 (.12)
(3) GPA (std) by major−specific semester 3
−1
−.8
−.6
−.4
−.2
0G
PA
(st
d) b
y m
ajor
−sp
ecifi
c se
mes
ter
4
−5 0 5Number of minus credits (adjusted)
discontinuity coeff. (se): .48 (.12)
(4) GPA (std) by major−specific semester 4
Note: The panels above provide a graphical illustration of the effects of retention on grade point averagesby the end of each of the first four major-specific semesters. The dots represent the mean outcomes inbins of 1 minus credits. The solid lines display a linear fit to either side of the cutoff (dashed lines: 95%confidence intervals). The discontinuity coefficient indicates the distance of the two regression lines exactlyat the cutoff. Standard errors are in parentheses. The sample consists of all individuals in the estimationsample within a range of 5 minus credits to either side of the cutoff who have non-missing information ongrades (n = 700).
26
Table 7: RDD estimates: Grade point averages (GPA) by the end of each major-specific semester
GPA (std) by major- 0.44** 0.38*** 0.61* 0.55*** 0.54** 0.57* 0.61*** 0.70*** 0.55***specific semester 1 (0.22) (0.13) (0.31) (0.15) (0.23) (0.32) (0.20) (0.18) (0.06)GPA (std) by major- 0.61*** 0.42*** 0.61* 0.55*** 0.48** 0.59* 0.58*** 0.64*** 0.51***specific semester 2 (0.20) (0.13) (0.31) (0.14) (0.22) (0.30) (0.18) (0.16) (0.19)GPA (std) by major- 0.61*** 0.43*** 0.58** 0.49*** 0.50*** 0.62** 0.56*** 0.63*** 0.53***specific semester 3 (0.18) (0.11) (0.27) (0.12) (0.19) (0.26) (0.17) (0.15) (0.13)GPA (std) by major- 0.55*** 0.46*** 0.43 0.53*** 0.51*** 0.53** 0.60*** 0.63*** 0.49specific semester 4 (0.19) (0.11) (0.27) (0.12) (0.18) (0.25) (0.17) (0.15) (0.46)
Observations 56 118 118 700 700 700 1,788 2,177 2,177Estimation [-0.5;0.5] [-1;1] [-1;1] [-5;5] [-5;5] [-5;5] [-10;10] [-12;12] [-12;12]windowPolynomial 0 0 1 1 2 3 3 3 NPorder
Note: The table shows RDD estimates for the effect of retention on grade point averages by the end ofeach major-specific semester. All parametric estimates are based on linear models and control for cohorteffects. Following Imbens and Kalyanaraman (2012), the bandwidth for the local-linear nonparametricspecification (NP) is determined by cross-validation. Standard errors for NP estimates are computed using5000 bootstrap replications.* Significant at 10%-level, ** Significant at 5%-level, *** Significant at 1%-level. Standard errors are inparentheses.
ages at the cutoff. The performance of retained students, however, remains relatively low
compared to the performance of all students. At the cutoff, the performance of repeaters
lies still more than 0.4 standard deviations below the median by their fourth major-specific
semester.
5.4 Accounting for missing outcomes
The analysis of bounds on the effect of repetition confirms the positive effects of grade re-
tention on grade point averages, at least for the third and fourth major-specific semesters.
We investigate estimation windows of 0.5, 1, and 1.5 minus credits to each side of the
cutoff (Table 8). For the first two major-specific semesters, the bounds are only signifi-
cantly positive for the smallest estimation window. For the third and fourth major-specific
semester, the bounds are significant for all windows. For the smallest window, the effect
ranges between 0.42 and 0.69 standard deviations by the fourth major-specific semester.
The effect of repetition on academic achievement is thus substantial, even if we only take
27
the lower bound into account.
Table 8: Bounds on the effects of repetition on grade point averages
Outcome Estimation Mean Bounds 95% CI # obs. Trimmingwindow comparison of bounds proportion
(1) (2) (3) (4) (5) (6)GPA (std) by major- [-0.5;0.5] 0.54 [0.42; 0.66]** [0.02; 1.07] 65 0.10specific semester 1 [-1;1] 0.39 [0.15; 0.62] [-0.08; 0.86] 140 0.19
[-1.5;1.5] 0.38 [0.18; 0.54] [-0.02; 0.74] 216 0.14
GPA (std) by major- [-0.5;0.5] 0.59 [0.48; 0.70]** [0.13; 1.04] 65 0.10specific semester 2 [-1;1] 0.40 [0.14; 0.60] [-0.09; 0.83] 140 0.19
[-1.5;1.5] 0.35 [0.15; 0.50] [-0.03; 0.69] 216 0.14
GPA (std) by major- [-0.5;0.5] 0.61 [0.52; 0.75]** [0.22; 1.04] 65 0.10specific semester 3 [-1;1] 0.43 [0.23; 0.61]** [0.03; 0.81] 140 0.19
[-1.5;1.5] 0.32 [0.16; 0.44]** [0.00; 0.61] 216 0.14
GPA (std) by major- [-0.5;0.5] 0.53 [0.42; 0.69]** [0.10; 1.02] 65 0.10specific semester 4 [-1;1] 0.45 [0.27; 0.62]** [0.06; 0.83] 140 0.19
[-1.5;1.5] 0.34 [0.18; 0.48]** [0.01; 0.65] 216 0.14
Note: The table shows estimates for the bounds on the effects of repetition of a full year on grade pointaverages by the end of each of the first four major-specific semesters, following Lee (2009). The estimatesare computed for three different samples, that is, for individuals within a range of 0.5, 1, and 1.5 minuscredits to each side of the threshold, respectively. Column (2) shows a raw comparison of means betweenrepeaters and non-repeaters, column (3) shows the bounds, and column (4) displays the 95% confidenceinterval around the bounds. Column (6) indicates the fraction of individuals who were trimmed from thesample of non-repeaters. See Section 4.2 for details on the estimation.* Significant at 10%-level, ** Significant at 5%-level, *** Significant at 1%-level. Standard errors are inparentheses.
5.5 Are gains from repeating consistent with learning gains during
the first year?
Grade repetition can improve student performance for at least two reasons. First, stu-
dents may gain knowledge from repeating the course material. We denote this channel as
“learning gains”. Second, students may persistently change their behaviour in response
to grade retention, for example by exerting more effort, by becoming more efficient in pro-
cessing the study material, or by changing their peer groups. In practice, it is likely that a
combination of these mechanisms leads to the grade improvements we find.
The data does not allow us to disentangle the myriad channels through which grade
28
Table 9: Performance improvements in minus credits and first-year course grades between the firstand the second attempt of the first year
(1) Estimation window: student at most 1 MC away from threshold
Mean Mean Difference 95% CI t-stat # Obs.1st attempt 2nd attempt of difference
Minus credits (MC) 12.73 6.05 -6.68 [-10.57; -2.79] -3.37 64Grade (std) in Business 1 -0.17 0.25 0.42 [0.28; 0.55] 6.05 63Grade (std) in Economics 1 -0.19 0.35 0.54 [0.35; 0.72] 5.58 64Grade (std) in Mathematics 1 -0.41 0.20 0.61 [0.41; 0.81] 6.05 60Grade (std) in Legal Studies 1 -0.17 0.35 0.52 [0.33; 0.71] 5.31 64Grade (std) in Business 2 -0.54 -0.01 0.52 [0.29; 0.76] 4.31 61Grade (std) in Economics 2 -0.61 -0.03 0.58 [0.33; 0.84] 4.47 60Grade (std) in Mathematics 2 -0.69 -0.12 0.56 [0.34; 0.79] 5.03 57Grade (std) in Legal Studies 2 -0.49 0.03 0.52 [0.29; 0.75] 4.46 61Grade (std) in Term paper -0.09 -0.04 0.05 [-0.29; 0.40] 0.31 61
(2) Estimation window: student at most 5 MC away from threshold
Mean Mean Difference 95% CI t-stat # Obs.1st attempt 2nd attempt of difference
Minus credits (MC) 14.54 5.57 -8.97 [-10.33; -7.61] -12.95 234Grade (std) in Business 1 -0.14 0.19 0.33 [0.23; 0.42] 6.81 232Grade (std) in Economics 1 -0.21 0.26 0.47 [0.36; 0.57] 8.70 234Grade (std) in Mathematics 1 -0.41 0.09 0.51 [0.41; 0.60] 10.51 224Grade (std) in Legal Studies 1 -0.26 0.28 0.54 [0.44; 0.64] 10.62 234Grade (std) in Business 2 -0.59 -0.04 0.55 [0.44; 0.65] 10.47 225Grade (std) in Economics 2 -0.70 -0.11 0.58 [0.47; 0.70] 9.68 224Grade (std) in Mathematics 2 -0.70 -0.16 0.54 [0.43; 0.65] 9.38 214Grade (std) in Legal Studies 2 -0.66 -0.05 0.61 [0.51; 0.70] 12.18 224Grade (std) in Term paper -0.33 -0.11 0.22 [0.06; 0.38] 2.72 225
(3) Estimation window: student at most 10 MC away from threshold
Mean Mean Difference 95% CI t-stat # Obs.1st attempt 2nd attempt of difference
Minus credits (MC) 16.28 5.97 -10.31 [-11.39; -9.24] -18.79 364Grade (std) in Business 1 -0.20 0.18 0.38 [0.31; 0.46] 10.05 361Grade (std) in Economics 1 -0.21 0.21 0.42 [0.34; 0.50] 10.56 364Grade (std) in Mathematics 1 -0.44 0.11 0.56 [0.48; 0.63] 14.42 350Grade (std) in Legal Studies 1 -0.32 0.22 0.55 [0.47; 0.62] 13.79 364Grade (std) in Business 2 -0.66 -0.10 0.56 [0.47; 0.64] 12.80 351Grade (std) in Economics 2 -0.77 -0.16 0.61 [0.51; 0.70] 12.74 350Grade (std) in Mathematics 2 -0.79 -0.20 0.59 [0.50; 0.68] 12.83 336Grade (std) in Legal Studies 2 -0.78 -0.14 0.64 [0.56; 0.72] 15.50 349Grade (std) in Term paper -0.38 -0.09 0.29 [0.17; 0.41] 4.79 351
The table shows performance differences between the first and the second attempt of the first year (minuscredits, standardised grades in core courses). In panel 1 (panel 2, panel 3), the sample is restricted torepeaters who are at most 1 (5, 10) minus credit(s) away from the cutoff after their first attempt of the firstyear. Minus credits are reported in absolute, non-recentred, values. Course grades are reported instandard deviations. Column “t-stat” refers to the value of the t-statistics for a test for significantimprovements between the two attempts (H0: no difference in means between the two attempts).
29
retention affects educational outcomes. However, we can assess whether the effects are
consistent with learning gains through repetition. On that account, we compare exam
grades and minus credits from the first and second attempt for all repeaters who complete
at least one course during their second attempt.
On average, repeaters who were close to the cutoff in their first attempt exceed the
goal of passing by a large margin in their second attempt. Table 9 shows substantial
improvements in minus credits, both for repeaters close to the cutoff (at most one minus
credit away, panel 1) and for repeaters further away from the cutoff (5 or 10 minus credits
away, panel 2 and 3). Repeating students finish their second attempt with only six minus
credits on average. This implies that the average repeater would pass his second attempt
even if his minus credits doubled. The average gain in minus credits is largest for repeaters
who performed worst in their first attempt (panel 3). The improvements in minus credits
are statistically significant across all samples.
Lower amounts of minus credits correspond to higher grades (Table 9). Repeaters
who fail their first attempt by at most one minus credit point show significant grade im-
provements of between 0.42 and 0.61 standard deviations on average, with the exception
of the “Term paper” (panel 1). Similar patterns are observed for students covered by pan-
els 2 and 3. The findings suggest that repeaters master the first-year course material far
better after repetition. Grade improvements beyond the first year are thus consistent with
learning gains during the first year, both in direction and in size.
6 Conclusion
This paper investigates the effect of grade retention and course repetition in higher educa-
tion on dropout decisions, academic performance, and major choice. Using administrative
data for 8 cohorts of first-year undergraduate students at the University of St. Gallen
(2001-2008), this paper studies a system where students have to repeat their first year
when they fall short of a predefined performance requirement. We account for the endo-
30
geneity of the retention status in a regression discontinuity framework. We thus obtain
local estimates for the causal effects of grade retention and course repetition on dropout
probabilities, major choice, credits per semester, and grade point averages up to four
semesters after repeating. Retention increases dropout probabilities immediately after
the first year, and decreases the probability of enrolment into the second year. Course
repetition persistently increases students’ GPAs by about 0.5 standard deviations, while
study pace and major choice remain unaffected. Hence, the findings suggest significant
and persistent improvements in student GPA as a result of repeating.
The findings contribute to the research on the determinants of student success. First,
the study emphasises that course repetition is a policy that is highly effective in improving
the grades of initially low-performing students. This finding is important as repetition poli-
cies in higher education are both widespread and costly. Second, the empirical evidence
contributes to the grade retention literature, which shows that the effects of grade retention
are strongly age-dependent. While retention can have detrimental effects for high school
students, our results suggest that retention can be an effective policy in higher education.
This study leaves several questions unanswered that can be addressed in future re-
search. First, our findings are based on data from one university and, because of the
research design, provide local estimates. It is therefore important to assess the general-
isability of the results by looking at other countries and settings. Second, this study only
investigates the partial effects of the retention system on individual students. It would
be worthwhile to study retention versus other counterfactual policies in a general equilib-
rium setting, also taking into account how the retention policy affects the overall quality of
the student pool. For example, a university that implements a retention policy may deter
students who do not want to expose themselves to such a policy. Third, the pathways
through which retention and course repetition affect dropout decisions and subsequent
student performance are under-researched. Fourth, to assess the scope of the policy, it
will be important to study transitions from school to work and in particular how retention
policies affect individual labour market success.
31
References
ALTONJI, J. G., E. BLOM, AND C. MEGHIR (2012): “Heterogeneity in Human Capital
Investments: High School Curriculum, College Major, and Careers,” Annual Review of
Economics, 4, 185–223.
ARCIDIACONO, P. (2004): “Ability sorting and the returns to college major,” Journal of
Econometrics, 121, 343–375.
BETTINGER, E. P. AND B. T. LONG (2009): “Addressing the Needs of Underprepared
Students in Higher Education: Does College Remediation Work?” Journal of Human
Resources, 44, 736–771.
GREENE, J. AND M. WINTERS (2007): “Revisiting grade retention: An evaluation of
Florida’s test-based promotion policy,” Education Finance and Policy, 2, 319–340.
IMBENS, G. AND K. KALYANARAMAN (2012): “Optimal Bandwidth Choice for the Regres-
sion Discontinuity Estimator,” The Review of Economic Studies, 79, 933–959.
IMBENS, G. AND T. LEMIEUX (2008): “Regression discontinuity designs: A guide to prac-
tice,” Journal of Econometrics, 142, 615–635.
JACOB, B. AND L. LEFGREN (2004): “Remedial education and student achievement: A
regression-discontinuity analysis,” Review of Economics and Statistics, 86, 226–244.
JACOB, B. A. AND L. LEFGREN (2009): “The Effect of Grade Retention on High School
Completion,” American Economic Journal: Applied Economics, 1, 33–58.
LEE, D. S. (2009): “Training, Wages, and Sample Selection: Estimating Sharp Bounds on
Treatment Effects,” The Review of Economic Studies, 76, 1071–1102.
LORENCE, J. AND A. DWORKIN (2006): “Elementary grade retention in Texas and reading
achievement among racial groups: 1994–2002,” Review of Policy Research, 23, 999–
1033.
32
MANACORDA, M. (2012): “The Cost of Grade Retention,” Review of Economics and Statis-
tics, 94, 596–606.
MANSKI, C. F. (1989): “Schooling as experimentation: a reappraisal of the postsecondary
dropout phenomenon,” Economics of Education Review, 8, 305–312.
MCCRARY, J. (2008): “Manipulation of the running variable in the regression discontinuity
design: A density test,” Journal of Econometrics, 142, 698–714.
NICHOLS, A. (2011): “rd 2.0: Revised Stata module for regression discontinuity estima-
tion,” Statistical Software Components, Boston College Department of Economics.
OECD (2014): Education at a Glance 2014: OECD Indicators, Paris: OECD Publishing.
OU, D. (2010): “To leave or not to leave? A regression discontinuity analysis of the impact
of failing the high school exit exam,” Economics of Education Review, 29, 171–186.
RODERICK, M. AND J. NAGAOKA (2005): “Retention under Chicagos high-stakes testing
program: Helpful, harmful, or harmless?” Educational Evaluation and Policy Analysis,
27, 309.
TAUCHMANN, H. (2012): “LEEBOUNDS: Stata module for estimating Lee (2009) treat-
ment effect bounds,” Statistical Software Components, Boston College Department of
Economics.
33
Appendix
Figures
Figure A.1: Time line: Institutional setup
First year
(first attempt)
First year
(second attempt)
pass fail
On-time graduation
dropoutdropout
semester 4
semester 3
semester 10
semester 9
semester 8
semester 7
semester 6
semester 5
semester 2
semester 1
Major-specific
level
exit
pass fail
On-time graduation
Major-specific
level
Note: The figure displays the time line of undergraduate education at the University of St. Gallen. Studentscan decide to drop out at any time, but dropout rates are particularly high during and after the first year.On-time graduation takes place after 6 semesters for non-repeaters and after 8 semesters for repeaters.The majority of students does not graduate on time. Students who fail the second attempt of the first yearare exmatriculated and not allowed to repeat again.
34
Figure A.2: RDD estimates: Student background characteristics
19.5
2020
.521
Age
(ye
ars)
−10 −5 0 5 10Number of minus credits (adjusted)
discontinuity coeff. (se): .02 (.19)
(1) Age (years)
.65
.7.7
5.8
.85
Fra
ctio
n: M
ale
−10 −5 0 5 10Number of minus credits (adjusted)
discontinuity coeff. (se): 0 (.05)
(2) Fraction: Male
.05
.1.1
5.2
.25
.3F
ract
ion:
Non
−S
wis
s na
tiona
lity
−10 −5 0 5 10Number of minus credits (adjusted)
discontinuity coeff. (se): −.04 (.04)
(3) Fraction: Non−Swiss nationality
0.0
5.1
.15
.2F
ract
ion:
Ent
ranc
e te
st
−10 −5 0 5 10Number of minus credits (adjusted)
discontinuity coeff. (se): −.04 (.04)
(4) Fraction: Entrance test
.1.1
5.2
.25
.3F
ract
ion:
Hig
h sc
hool
St.
Gal
len
−10 −5 0 5 10Number of minus credits (adjusted)
discontinuity coeff. (se): −.03 (.04)
(5) Fraction: High school St. Gallen
Note: The figure investigates the smoothness of background characteristics at the cutoff. The black dotsindicate means of characteristics in bins of a width of 2 minus credits. The solid lines present a linear fit toboth sides of the threshold. The number of minus credits is re-centred around the cutoff of 12.25 minuscredits. The dashed lines indicate 95% confidence intervals. The discontinuity coefficient indicates thedistance of the two regression lines exactly at the cutoff. Standard errors are in parentheses. The sampleconsists of all individuals in the estimation sample within a range of 10 minus credits to either side of thecutoff (n = 1,967).
35
Figure A.3: RDD estimates: Major choice
0.2
.4.6
.8M
ajor
cho
ice:
Bus
ines
s
−10 −5 0 5 10Number of minus credits (adjusted)
discontinuity coeff. (se): −.039 (.078)
(1) Business
0.2
.4.6
.8M
ajor
cho
ice:
Eco
nom
ics
−10 −5 0 5 10Number of minus credits (adjusted)
discontinuity coeff. (se): −.007 (.053)
(2) Economics
0.2
.4.6
.8M
ajor
cho
ice:
Inte
rnat
iona
l Affa
irs
−10 −5 0 5 10Number of minus credits (adjusted)
discontinuity coeff. (se): .077 (.063)
(3) International Affairs
0.2
.4.6
.8M
ajor
cho
ice:
Law
and
Eco
nom
ics
−10 −5 0 5 10Number of minus credits (adjusted)
discontinuity coeff. (se): −.067 (.037)
(4) Law and Economics
Note: The panels above provide a graphical illustration of effects of retention on major choice after the firstyear. The dots represent the mean outcomes in bins of 2 minus credits. The solid lines display a linear fit toeither side of the cutoff (dashed lines: 95% confidence intervals). The discontinuity coefficient indicates thedistance of the two regression lines exactly at the cutoff. Standard errors are in parentheses. The sampleconsists of all individuals in the estimation sample within a range of 10 minus credits to either side of thecutoff (n = 1,857).
36
Tables
Table A.1: Graduation statistics for Switzerland
Graduation Average 2007 2008 2009 2010Total (N=82233) 20558 17797 20205 21230 23001Business Administration or Economics (N=12258) 3065 2904 2963 3009 3382in % of Total 15.0 16.3 14.7 14.2 14.7at University of St. Gallen (N=3640) 910 903 903 881 953in % of Total 4.5 5.1 4.5 4.1 4.1in % of Business Administration or Economics 29.8 31.1 30.5 29.3 28.2
Note: Graduation consists of Licentiate, Bachelor or Master in Switzerland. All Percentages arerounded to one decimal place. Source: Federal Statistical Office of Switzerland (2014) and owncalculations. The data can be retrievedfrom http://www.bfs.admin.ch/bfs/portal/de/index/themen/15/06/data/blank/01.html.
Table A.2: Time trend in student numbers at the University of St. Gallen, 1990-2013.
Academic year Total number of Number ofstudents first-year
undergraduates(1) (2) (3)
1990 3908 582... ... ...
2000 4701 8432001 4938 9712002 4917 9532003 4852 9002004 4569 7892005 4508 9542006 4915 10222007 5367 10602008 5928 13152009 6418 13142010 6726 11912011 7126 13322012 7325 12932013 7666 1328
Note: The table shows the time trend for student numbers at the University of St. Gallen. Column(2) shows the total number of enrolled students at the beginning of the academic year, includingall degrees. Column (3) shows the corresponding number of enrolled first-year undergraduates.Source: University of St. Gallen, HSG Annual Reports and own calculations. The annual reportscan be retrieved fromhttp://www.unisg.ch/en/hsgservices/hsgmediacorner/publikationen/hsgjahresbericht.
37
Tabl
eA
.3:
Typi
calc
urric
ulum
ofa
stud
ent
Cal
enda
rWee
k38
45-4
646
5051
3-7
101s
tSem
este
rS
tart
regi
ster
toex
ams
Ass
ignm
ents
End
1stc
entra
lexa
min
atio
npe
riod
getg
rade
sE
xam
LSC
TE
con
BA
Law
Mat
hC
redi
ts3
25.
55
5.5
3.5
Cal
enda
rWee
k8
15-1
650
-15
1616
-21
2125
-29
352n
dS
emes
ter
Sta
rtre
gist
erto
exam
sA
ssig
nmen
tsE
nd2n
dce
ntra
lexa
min
atio
npe
riod
getg
rade
sE
xam
Ess
ayB
A*
LSE
con
BA
Law
Mat
hC
TFL
Cre
dits
52
35.
55
5.5
3.5
24
The
tabl
esh
ows
aty
pica
lcur
ricul
umof
ast
uden
tin
the
first
year
.A
bbre
viat
ions
ofco
urse
s:LS
–Le
ader
ship
skill
s;C
T–
Crit
ical
Thin
king
;BA
–B
usin
ess
Adm
inis
tratio
n:B
A*
–B
usin
ess
Adm
inis
tratio
n:C
ase
stud
y;FL
–Fo
reig
nLa
ngua
ge;A
ssig
nmen
ts–
Ess
ays
oror
alas
sign
men
tsdu
ring
the
sem
este
r;E
xam
s–
Writ
ten
exam
sdu
ring
the
exam
perio
d.S
ourc
e:C
ours
ecu
rric
ula
ofth
eU
nive
rsity
ofS
t.G
alle
n,m
ade
avai
labl
eby
the
univ
ersi
tyad
min
istra
tion.
38
Table A.4: Sample construction, cohorts 2001-2008
% dropped
Entering first-year students 6,706Dropped: Non-German mother tongue 397 5.9%Dropped: Not Business/Economics track 582 8.7%Dropped: Inconsistent or missing data 85 1.3%
Final sample 5,392
The table documents the exclusion of subgroups from the analysis. We exclude students with non-Germanmother tongue who follow different exam procedures. We furthermore exclude students who do not enterthe Business/Economics track of the first year. A minor fraction of students is included due to missing dataentries for some of the variables used in the analysis.
Table A.5: Statistics on retention and repetition, cohorts 2001-2008
Sample of first-year students 5,392Passed 1st attempt 3,817 % of sample 71%Failed 1st attempt 1,575 % of sample 29%Repeater 942 % of sample 17%
% of failed 60%Repeater: Completes all courses (2nd attempt) 741 % of repeaters 79%Passed 2nd attempt 584 % of repeaters 62%
% of repeaters – all courses 79%Passed: 1st and 2nd attempt combined 4,401 % of sample 82%
The table shows statistics on retention, repetition, and passing rates for cohorts 2001-2008. The initialsample is based on all students with non-German mothertongue who enter the Business/Economics track(n = 5,392, see Table A.4).
39
Table A.6: Student background characteristics and outcomes, by type
Type 1: Type 2: Type 3: Type 4:Drops out Not admitted Drops out after Completes
during to 1st/during 2nd full1st semester 2nd semester semester 1st year
Background characteristicsAge 21.0 20.4 20.6 20.3Male 73% 65% 67% 73%Non-Swiss nationality 19% 16% 22% 24%Entrance test 9% 6% 14% 20%High school St. Gallen 17% 19% 13% 15%
Minus credits# MC in 1st semester - 19.4 7.2 1.8# MC in first year - - - 4.7Failed - - - 13%
Course performance 1st semesterBusiness 1 taken 56% 100% 100% 100%Grade Business 1 3.0 3.1 3.7 4.2Economics 1 taken 51% 100% 100% 100%Grade Economics 1 3.1 3.2 4.0 4.6Mathematics 1 taken 44% 100% 100% 100%Grade Mathematics 1 2.8 2.8 3.7 4.5Legal Studies 1 taken 58% 100% 100% 100%Grade Legal Studies 1 3.2 3.1 3.8 4.5
Course performance 2nd semesterBusiness 2 taken - - 40% 100%Grade Business 2 3.6 4.3Economics 2 taken - - 39% 100%Grade Economics 2 3.8 4.5Mathematics 2 taken - - 34% 100%Grade Mathematics 2 3.8 4.5Legal Studies 2 taken - - 40% 100%Grade Legal Studies 2 3.7 4.3Term paper taken - - 67% 100%Grade Term paper 4.6 5.0
Major-specific levelMajor-specific level started 13% 25% 43% 96%
# obs 392 437 181 4,382
The table displays differences in mean background characteristics and outcomes across different types ofstudents. Students are characterised according to their behaviour and performance during the first attemptof the first year. See Section 3.2 for details on the type definition. Course grades are reported as absolutegrades, with 1 being the lowest and 6 being the highest grade. The sample is based on all students withGerman mother tongue who enter the Business/Economics track (5,392 students).
40
Tabl
eA
.7:
Stu
dent
back
grou
ndch
arac
teris
tics
and
perfo
rman
ce,b
yco
hort
Full
sam
ple
2001
2002
2003
2004
2005
2006
2007
2008
#O
bs.
4,38
261
852
445
538
552
256
561
070
3B
ackg
roun
dch
arac
teris
tics
Age
20.2
620
.47
20.5
120
.16
20.1
420
.24
20.2
720
.03
20.2
2M
ale
73%
74%
73%
75%
75%
74%
72%
73%
73%
Non
-Sw
iss
natio
nalit
y24
%19
%25
%26
%27
%24
%21
%26
%26
%E
ntra
nce
test
20%
16%
20%
23%
22%
19%
18%
21%
23%
Hig
hsc
hool
St.
Gal
len
15%
13%
14%
17%
18%
14%
16%
13%
16%
Min
uscr
edits
Frac
tion:
MC>
057
%64
%61
%56
%51
%56
%51
%53
%60
%#
MC
infir
stye
ar4.
674.
425.
594.
244.
225.
403.
844.
515.
02Fa
iled
13%
13%
15%
10%
11%
16%
10%
13%
14%
Cou
rse
perfo
rman
ce1s
tyea
rG
rade
Bus
ines
s1
4.22
4.23
4.22
4.19
4.22
4.25
4.28
4.21
4.15
Gra
deE
cono
mic
s1
4.58
4.58
4.47
4.56
4.80
4.61
4.55
4.63
4.53
Gra
deM
athe
mat
ics
14.
534.
424.
584.
484.
634.
534.
554.
654.
43G
rade
Lega
lStu
dies
14.
474.
494.
464.
544.
644.
474.
594.
344.
33G
rade
Bus
ines
s2
4.26
4.30
4.10
4.46
4.77
4.02
4.42
4.22
4.04
Gra
deE
cono
mic
s2
4.45
4.39
4.26
4.26
4.42
4.52
4.76
4.62
4.36
Gra
deM
athe
mat
ics
24.
524.
514.
434.
604.
854.
544.
384.
474.
51G
rade
Lega
lStu
dies
24.
334.
054.
194.
494.
294.
424.
384.
384.
45G
rade
Term
pape
r4.
984.
904.
984.
924.
984.
995.
025.
015.
00O
utco
mes
:D
ropo
utD
ropo
utaf
ter1
stye
ar2%
3%3%
2%2%
3%2%
2%3%
Rep
eate
r11
%11
%14
%9%
9%14
%8%
11%
12%
Out
com
es:
Maj
or-s
peci
ficle
vel
Maj
or-s
peci
ficle
vels
tart
ed96
%94
%94
%97
%98
%95
%96
%96
%95
%B
ache
lor’s
Deg
ree
com
plet
edW
ithin
4m
ajor
-spe
cific
sem
este
rs(o
ntim
e)(1
)45
%47
%49
%40
%33
%26
%23
%23
%-
With
in5
maj
or-s
peci
ficse
mes
ters
(1)
66%
69%
71%
64%
60%
55%
50%
50%
-W
ithin
6m
ajor
-spe
cific
sem
este
rs(1
)87
%87
%88
%88
%85
%80
%80
%83
%-
The
tabl
esh
ows
desc
riptiv
est
atis
tics
fort
hees
timat
ion
sam
ple,
that
is,f
oral
lstu
dent
sw
hoha
veco
mpl
eted
allfi
rst-y
earc
ours
es,s
epar
atel
yfo
reac
hco
hort
(4,3
82st
uden
ts).
Eac
hco
lum
ndi
spla
ysei
ther
mea
nsor
perc
enta
ges.
Info
rmat
ion
onco
mpl
etio
nof
the
Bac
helo
r’sde
gree
ism
issi
ngfo
rthe
coho
rtof
2008
due
toce
nsor
ing.
(1)Va
lues
nota
vaila
ble
forc
ohor
t200
8.
41
Table A.8: Test for randomness of missing outcomes: Retained students
Dependent variable: Non-missing grades at the major-specific level (binary)
(1) (2) (3) (4)Estimation window 5 MC 5 MC 5 MC 10 MC(max. distance from cutoff)
coef (se) coef (se) coef (se) coef (se)Grade (std) in Business 1 -0.006 -0.014 -0.011
(0.057) (0.057) (0.049)Grade (std) in Economics 1 -0.022 -0.013 -0.005
(0.054) (0.057) (0.048)Grade (std) in Mathematics 1 0.054 0.042 0.032
(0.046) (0.047) (0.039)Grade (std) in Legal Studies 1 0.045 0.060 0.065
(0.062) (0.062) (0.052)Grade (std) in Business 2 0.102* 0.117** 0.118***
(0.054) (0.055) (0.042)Grade (std) in Economics 2 0.076 0.090* 0.083**
(0.048) (0.049) (0.041)Grade (std) in Mathematics 2 -0.029 -0.029 0.020
(0.046) (0.046) (0.038)Grade (std) in Legal Studies 2 0.084 0.070 0.094**
(0.051) (0.052) (0.045)Grade (std) in Term paper 0.027 0.022 0.027
(0.033) (0.034) (0.028)Age (years) -0.018* -0.022** -0.024***
(0.009) (0.010) (0.009)Male -0.042 -0.043 -0.036
(0.057) (0.061) (0.051)Non-Swiss nationality 0.141 0.158 0.093
(0.116) (0.115) (0.077)Entrance test -0.187 -0.186 -0.060
(0.143) (0.141) (0.106)High school St. Gallen 0.017 0.031 -0.005
(0.072) (0.071) (0.056)
Minus credits (third order polynomial) Yes Yes YesCohort dummies Yes Yes Yes
Pseudo R-squared 0.052 0.092 0.078Number of observations 267 267 267 428
The table shows average marginal effects from a probit regression of a binary indicator – whether a studenthas non-missing grades at the major-specific level – on standardised course grades during the first yearand individual student characteristics. The sample includes only students who completed the first year, butfailed. The different specifications allow for different distances from the threshold, i.e., a distance of up to 5minus credits from the threshold in specifications (1)-(3), and a distance of up to 10 minus credits from thethreshold in specification (4). All specifications include cohort dummies and control for the exact number ofminus credits.* Significant at 10%-level, ** Significant at 5%-level, *** Significant at 1%-level. Standard errors are inparentheses.
42
Table A.9: Test for randomness of missing outcomes: Non-retained students
Dependent variable: Non-missing grades at the major-specific level (binary)
(1) (2) (3) (4)Estimation window 5 MC 5 MC 5 MC 10 MC(max. distance from cutoff)
coef (se) coef (se) coef (se) coef (se)Grade (std) in Business 1 -0.007 -0.006 0.015
(0.025) (0.025) (0.012)Grade (std) in Economics 1 0.000 0.015 -0.010
(0.022) (0.023) (0.011)Grade (std) in Mathematics 1 0.009 0.005 -0.011
(0.021) (0.022) (0.011)Grade (std) in Legal Studies 1 0.057** 0.053** 0.019
(0.025) (0.025) (0.012)Grade (std) in Business 2 0.018 0.015 -0.004
(0.019) (0.019) (0.009)Grade (std) in Economics 2 -0.011 -0.006 -0.005
(0.017) (0.017) (0.009)Grade (std) in Mathematics 2 0.023 0.021 0.018*
(0.020) (0.020) (0.010)Grade (std) in Legal Studies 2 -0.019 -0.022 0.002
(0.018) (0.018) (0.010)Grade (std) in Term paper 0.024* 0.019 0.011*
(0.013) (0.013) (0.006)Age (years) 0.017* 0.017* -0.000
(0.009) (0.009) (0.003)Male -0.080*** -0.079** -0.004
(0.030) (0.031) (0.013)Non-Swiss nationality -0.040 -0.012 -0.003
(0.038) (0.040) (0.023)Entrance test 0.070 0.038 0.042
(0.049) (0.050) (0.029)High school St. Gallen 0.023 0.018 0.011
(0.031) (0.031) (0.016)
Minus credits (third order polynomial) Yes Yes Yes YesCohort dummies Yes Yes Yes Yes
Pseudo R-squared 0.125 0.130 0.169 0.084Number of observations 523 523 523 1,539
The table shows average marginal effects from a probit regression of a binary indicator – whether a studenthas non-missing grades at the major-specific level – on standardised course grades during the first yearand individual student characteristics. The sample includes only students who passed their first year intheir first attempt. The different specifications allow for different distances from the threshold, i.e., adistance of up to 5 minus credits from the threshold in specifications (1)-(3), and a distance of up to 10minus credits from the threshold in specification (4). All specifications include cohort dummies and controlfor the exact number of minus credits.* Significant at 10%-level, ** Significant at 5%-level, *** Significant at 1%-level. Standard errors are inparentheses.
43