A Review of the Impact of School Disciplinary Practices on Student
Outcomes2020
A Review of the Impact of School Disciplinary Practices on A Review
of the Impact of School Disciplinary Practices on
Student Outcomes Student Outcomes
Stephanie Thun Bethel University
Part of the Special Education and Teaching Commons
Recommended Citation Recommended Citation Thun, Stephanie, "A
Review of the Impact of School Disciplinary Practices on Student
Outcomes" (2020). All Electronic Theses and Dissertations. 626.
https://spark.bethel.edu/etd/626
This Thesis is brought to you for free and open access by Spark. It
has been accepted for inclusion in All Electronic Theses and
Dissertations by an authorized administrator of Spark. For more
information, please contact kent-
[email protected].
A MASTER’S THESIS
SUBMITTED TO THE FACULTY
FOR THE DEGREE OF
2
Abstract
The literature review sought to determine the impact of traditional
discipline and alternative
behavioral interventions on student outcomes. The research showed
that certain student
populations including Blacks, males, students qualifying for free
or reduced lunch, and students
with special education status were suspended most often.
Exclusionary discipline was
connected to lower academic outcomes, higher drop-out rates, and
increased likelihood of
suspension. Successful use of alternative interventions suggested
that students need to be
taught new behavior skills in order to learn them. Finding show
that traditional discipline is not
able to improve student outcomes. They also show that alternative
interventions need to be in
place in all educational settings to support students so they may
learn the behavior skills
needed to successfully engage in their education.
3
Research Questions
Collaborative and Proactive Solutions
Summary
CHAPTER I: INTRODUCTION
Have you ever heard of a student being suspended from school
because they could not
read or solve a math equation? The answer is likely no. Students
who struggle with the
academic areas often receive interventions and extra supports to
learn the skills they lack. Why
do we treat social-emotional and behavioral skills differently?
Instead of providing
interventions and instruction in these areas, students who struggle
with social-emotional and
behavioral skills are often sent home. In addition to being sent
home, the student is also
expected to learn the skills they need to avoid repeating the
offense on their own. School
systems do not expect students to teach themselves academic skills,
so why do these same
systems expect students to teach themselves emotional-regulation
and coping strategies? The
expectations are inequitable. The following review will address the
impacts of current
traditional disciplinary practices, as well as those of alternative
response practices, on school-
age student outcomes.
History of School Discipline Practice in the United States
Traditional discipline, also referred to as exclusionary
discipline, is a common practice
across the United States. Traditional discipline is the practice of
excluding students through in-
school suspension, out-of-school suspension, and expulsion as a
consequence to rule
infractions. In-school suspension is when a student is removed from
more than half of their
scheduled school day, but remains on school grounds. Out-of-school
suspension is when
students are removed from school for fewer than ten school days and
are not allowed on
school grounds, and expulsion is when a student is removed from
school for ten or more days
and is unenrolled from the school.
5
According to the National Center for Education Statistics,
2,635,742 students were
suspended or expelled from a public school during the 2013-2014
school year. This represents
about 5.3% of all students in the public education system that
year. Traditional discipline
practice has been criticized for being inequitable with specific
populations, such as Black
students, students who qualify for free or reduced lunch, and
students who are eligible for
special education services, being over-represented (Arcia, 2006;
Gorgan & Gage, 2011;
Jacobsen et al., 2019; Raffaele Mendez, 2003). The effectiveness of
exclusionary discipline has
also been questioned due to the lack of an apparent instructional
component to help students
learn the skills they need to avoid future suspensions (Costenbader
& Markson, 1998; Theriot et
al., 2010). Evidence suggests that students do require instruction
in the areas of behavior and
social-emotional in order to learn new skills in those areas (Choi
& HeckenLaible-Gotto, 1998;
Crosby et al., 2018; Greene & Ollendick, 1993; Henderson &
Guy, 2017; Johnson et al., 1997).
Student involvement in decision making around their behavior,
consequences, and
interventions has also proven to be a more effective practice than
unilateral adult decisions
made through traditional discipline (Brasof & Peterson, 2017;
Greene et al., 2004; Martin et al.,
2008; Ollendick et al., 2016).
The United States Department of Education has compiled the
Compendium of School
Discipline Laws and Regulations for the 50 States, Washington,
D.C., and Puerto Rico, which was
last updated in May 2013. The compendium is a collection of the
states’ discipline profiles.
Each state has different laws and regulations, so there is variance
amongst what is permissible
for school discipline across the nation. One component of each
state’s plan is how they will
implement evidence-based multi-tiered supports to all students and
those at-risk for exclusion.
6
This means all schools should have at minimum one form of
evidence-based behavior or social-
emotional intervention in place in order to legally move forward
with disciplinary practices
within the school building. One common tiered intervention is
School-wide Positive Behavior
Interventions and Supports (PBIS). PBIS is a tiered system that,
when implemented with
fidelity, can have positive impacts on student outcomes and
reducing behavior incidents
(Bradshaw et al., 2017; Eraldi et al., 2019; Freeman et al., 2015;
Gage et al. 2018; Gage et al.,
2020; Norcera et al. 2014; Ward & Gersten, 2013).
Research Questions
The following literature review will focus on the impacts of
suspension and alternative
interventions that may be used in-place-of or in-addition to
traditional discipline. The author
aimed to answer the following research questions: What are the
impacts of traditional
discipline on student outcomes? What alternatives options to
traditional discipline exist? What
are the impacts of the alternative options on student
outcomes?
7
CHAPTER II: LITERATURE REVIEW
The literature for this thesis was located through searches of
Academic Search Premiere
and PsychINFO. The list was limited by only reviewing peer reviewed
articles that had an
available full text. The key words used to search were, “positive
school behavior intervention,”
“school discipline impact,” “collaborative problem solving,” and
“school behavior
interventions.”
Costenbader and Markson (1998) researched variables associated with
students who
have been suspended and student perceptions of those events. The
study compared suspended
to non-suspended students, and the students evaluated the
effectiveness of suspension on
addressing their behavior problems.
The study participants were in grades seven through ten from four
schools: urban
middle school, urban high school, small-town middle school,
small-town high school. Seven-
hundred fifty participants completed surveys, and one-hundred
thirty were eliminated because
they were missing more than fifty percent of the question
responses. Of the 620 included
participants, 41 percent reported ISS or OSS in their school
history; 112 reported ISS, 35
reported OSS, and 105 reported both ISS and OSS. The participants
were 48 percent male, 55
percent high school students, 67 percent attended small-town
schools, 50 percent White, 23
percent Black, 8 percent Hispanic, and 13 percent other. The urban
middle and high schools
were 56% Black, and the small-town middle and high schools were 28%
Black.
The survey included 15 items and the student rating scale had 48
items. Participants
completed sections on demographics, extracurricular activities,
involvement with the legal
8
system, and history of suspension during the current school year.
Participants who reported
OSS or ISS completed an additional six-question survey to share the
perception of the event.
The questions addressed the class the behavior occurred in, the
behavior that occurred, their
feelings at the time of suspension, the degree suspension helped
them solve their problem and
ideas of other interventions that might help them solve their
problems more effectively in the
future.
The data showed that 56 percent of ISS and 64 percent of OSS were
assigned to male
participants, which is higher than the overall percentage of males
for the group, which was 48
percent. Also, the non-suspended group was 60% female. Within the
OSS group, 45 percent
were Black, 18 percent were Hispanic, and 12 percent were White,
which also does not align
with the whole group demographics. Another pattern was that
participants who were
suspended were more likely to be involved with the legal system.
Six percent of the non-
suspended group had been involved with the legal system, in
comparison to 14 percent of the
ISS group and 32 percent of the OSS group. Also, participants from
urban schools were more
likely to have been suspended outside of school than small-town
participants.
The most common suspendable offense was physical aggression,
followed by talking
back to staff and using profanity. The OSS group had higher reports
of physical violence and
weapon possession than the ISS group. Although OSS was more common
in urban schools,
small-town schools had more participants who received OSS for
non-violent offenses.
Participants who were suspended reported two common feelings: angry
or happy. Of all
suspended participants, 19 percent felt their suspension helped
them learn a lesson so they will
no longer be suspended again. Twelve percent felt they learned a
lot from the suspension, 36
9
percent felt they learned a little, and 67 percent felt they did
not learn anything from their
suspension that will help them solve their problem and avoid future
suspensions. When asked
to provide reasons for the suspension, participants reported: lack
of self-control, failure to
understand school rules, frustration with schoolwork, lack of
interest in schoolwork, problems
with substance abuse. Over 50 percent of participants reported
taking more interesting or
relevant courses would help them solve their problem and avoid
future suspensions. Over 25
percent of participants shared that learning ways to respond to
situations that led to
suspension would help them solve their problem. The study draws
attention to the need to
include students in the problem-solving process so that students
can feel like they are able to
address their problems to avoid future school exclusion.
Raffaele Mendez (2003) conducted a study to discover predictors of
student suspension
rates and the effect of suspension on students’ educational
achievement and graduation. The
researcher wanted to determine if different groups of students get
suspended more often,
what variables predict the rate of suspension in white and black
6th-grade students, how does
the number of suspensions experienced by a 6th grader relate to
school achievement, and does
the effect of suspensions on achievement vary by race.
The study included students who entered Kindergarten in 1989 in
Pinellas County, FL
with projected graduation in 2002. The study followed 8,268
students from Kindergarten
through 12th grade. The population was 79% white, 19% black, and
1.5% Hispanic.
The longitudinal study collected student surveys, teacher surveys,
demographic
information, standardized test scores, and suspension records from
2nd to 12th grade. The data
10
was compiled and analyzed to identify if any patterns were present
in student exclusion and
academic achievement.
The data showed Black males who received special education services
and free and
reduced lunch represented less than 5% of participants, but they
were 24% of all students
suspended 3-5 times in the 1995-96 school year, 48% of students
suspended 6-8 times, and
56% suspended 12-14 times. In total, 66.37% of Black males who
received free or reduced
lunch and special education services were suspended at least once
in 6th grade, while only
13.6% of Black males in sped who paid for lunch were suspended.
Another finding was 44.12%
of White males receiving free or reduced lunch, and special
education services were suspended
at least once, and 54.29% of White males who paid for lunch and
were in sped were suspended.
Black girls with free or reduced lunch, regardless of special
education status, were suspended
more disproportionately than any other demographic. The most
suspensions for this student
group occurred in 10th grade. The increase in suspension in the
high school years may be due
to the adoption of zero-tolerance policies that occurred when the
participant pool was entering
the 9th grade.
Raffaele Mendez stated her data showed the best predictor for
future suspensions was
past suspensions. Also, suspensions for this population correlated
with poor academic
performance and failure to graduate on time. It should be noted
that the cohort decreased
from 8,673 to 5,830 from 9th to 12th grade, which may be why a
decrease in suspensions was
seen following 10th grade because students dropped-out. The study
does provide patterns in
the data set that should be looked for in other populations in
order to generalize the results.
11
Arcia (2006) conducted a longitudinal retrospective analysis on
suspensions,
achievement, and long-term enrollment status of students in a
large, urban school district. The
study aimed to answer the question, how does suspension impact
student achievement? The
analyses included a large urban school district with the following
student demographics: 58%
Hispanic, 29% Black, 10% White, 3% Other, and almost 3/4ths
free-reduced lunch. Within the
district, the study focused on 49,327 suspended students and a
matched group of 42.809 non-
suspended students, counter-balanced to reflect similar make-up of
races and free/reduced
lunch status.
The study occurred over three school years from 2001 to 2004. The
suspension
percentage was determined by comparing the number of suspended
students to the number of
students enrolled in October. Developmental scores for academic
achievement were
determined by looking at that increase in standardized test scores;
academic gains were
analyzes across years within each participant.
The initial analysis reviewed student suspension rates, suspensions
per student per year,
suspension length, and the maximum days of suspension for one
student. Over the three
school years, 15%, 15.3%, and 16.2% of students were suspended at
least one time. The
average number of suspensions per student per year was 2.2, 2.2,
and 2.3 suspensions per
school year. The average length of suspensions per year was 7.2,
7.2, and 7.3 days per
suspension. The maximum number of days missed in one school year
due to suspension by one
student was 92, 92, and 102. Twenty-six percent of suspended
students were suspended for
more than ten days, and one percent were suspended more than 40
days in one school year.
The analyses also revealed that student suspensions increased with
age. Four times as many 6th
12
graders were suspended than 5th graders and the suspensions were
more than four days longer
than a 5th-grade suspension on average. The most common grade level
suspended was 9th
grade, representing 37% to 38% of the suspended population each
year.
Academic analyses showed students without suspensions gained on
average 198 points
over three years, students with suspension in 1 school year gained
176 points, suspensions in 2
school years gained 168 points, and suspension in 3 school years
gained 159 points. In addition
to lesser gains, there was also a trend that students who received
suspensions had lower
achievement scores than students who did not receive suspensions
prior to the suspension
occurring, so the students who were suspended typically started
with a lower score and then
subsequently made fewer gains.
The final analyses looked at drop-out rates for students who were
suspended in 9th
grade. Of the students who were suspended 1-10 days during their
9th-grade year, 21% had
dropped off three years later. Ninth grades suspended 11-20 days
had a 32% drop out rate
three years later, and those suspended 21 or more days had an a 43%
drop-out rate three years
later. Although the study only represents one school district, the
correlation of suspension to
lesser academic gains and increased school drop-out rate suggests
that other school districts
should review their data and consider alternative options to
exclusionary discipline.
Achilles et al. (2007) studied predictors for exclusion for
elementary students receiving
special education services using the Special Education Elementary
Longitudinal Study (SEELS)
database. The SEELS database followed children from 2000 to 2006.
The current study
included 1,824 participants, age seven to 14-years, from the
database. All participants had a
special education label of EBD, LD, or OHD with a diagnosis of
ADHD. The researchers
13
hypothesized that African American, male, and older participants
with a disability label of EBD
or OHD would have higher exclusion rates than participants with
similar characteristics and a LD
label. Participants who receive free or reduced lunch or lived in
non-two-parent-homes were
predicted to have higher rates of exclusion, as were participants
with multiple school changes,
low parent involvement, lack of extracurriculars, and those who
attend urban schools. Finally,
Achilles et al. (2007) predicted that participants who had later
onset of disability services and
interventions would have higher exclusion rates.
The data used from the SEELS database was collected from three
parent interviews from
2000 to 2006 and participating school districts. The parents
reported disability, history of
exclusion, demographics, family structure, SES information, school
setting (urban, suburban, or
rural), number of school changes since Kindergarten, parent
involvement, their child's school
experience, parent satisfaction with their child's schooling,
extracurricular involvement, age of
disability onset, and if their child received early interventions.
The schools provided additional
demographic information. Of the 1,824 SEELS participants, only 723
participants were used for
this study. Participants were excluded because they were missing
data from their files.
The data showed that students with EBD and ADHD labels were
suspended more than
students with the LD label. It also showed African American
students were excluded more than
Caucasian students and males were excluded more than females. Other
predictors for
exclusion were receiving free or reduced lunch, multiple school
changes, urban setting schools,
participating in extracurriculars, and low parent satisfaction with
schools. Facets of this data
that were not predictors were non-two-parent homes, disability
onset at a later age, no early
14
intervention, and lack of parent involvement. The data also showed
that religious involvement
was a predictor for a lower likelihood of exclusion.
The researchers also looked at how participants with specific
labels differed. The results
showed that older students were excluded more than younger for ADHD
and LD labels, but this
difference was not seen for participants with the EBD label. Also,
in addition to African
American, which was higher for all three labels, Hispanic
participants with the LD label were
more likely to be excluded. The best predictor for exclusion was
low SES status.
This study revealed that there are many predicting factors within
the category of special
education. Students with certain special educations labels in this
sample were excluded more
from school than other special education labels, as we students who
were of low SES status and
non-White.
Jacobsen et al. (2019) examined the rate of suspension and
expulsion of children in
urban elementary students. The data was pulled from the Fragile
Families and Child Wellbeing
Study. The study collected data on 5,000 participants born in urban
hospitals between 1998 to
2000. The hospitals were randomly sampled within 16 cities, which
were selected randomly
from a list of 200,000 urban cities.
Parents and teachers reported on physical aggression (i.e.,
destroying things belonging
to others and fighting) and exclusionary discipline. This data was
then analyzed to assess the
risk of exclusionary discipline by age nine, as well as variability
by race. The analysis found that
participants who were suspended and expelled were more likely to
exhibit physically aggressive
behavior. The suspended participants were also more likely to have
a diagnosis of Attention
Deficit Hyperactivity Disorder (ADHD).
15
The schools attended by the participants were in urban communities.
The schools had
fewer economic resources, a higher percentage of Black students,
and relied heavily on
suspensions and expulsions as a disciplinary strategy. Eleven
percent of the participants had
been removed from school by age 9. When looking at the whole
student population, one out of
25 non-black girls were suspended or expelled compared to four of
25 Black girls who were
suspended. For boys, less than three of 25 non-Black boys were
suspended compared to 10 out
of 25 Black boys who were suspended. Based on this data, Black boys
were three times more
likely to be excluded from school than Hispanic boys and five times
more than white and other
race boys. Black girls were four times more likely to be suspended
or expelled than Hispanic
girls and six times more likely than white or other race
girls.
The primary limitation of the study is that the data came from
interviews. Parents and
teachers may have been biased in their responses to student
behavior. There may also be a
misrepresentation of suspension data, as some parents may have only
included out-of-school
suspension days, omitting in-school suspension. Although the
conclusions of this study are not
conclusive, it does identify the pattern of race being a predictive
factor for exclusionary
discipline.
Theriot et al. (2010) conducted a study that looked at the
likelihood of school exclusion
in middle and high school students based on their last infraction
of the previous school year.
The study was done in a medium-sized school district in the
southeastern US. The participants
were all middle and high school students with one or more
discipline incidents during the 2004-
2005 school year. The district had 14 middle and 14 high schools in
urban, suburban, and a few
rural settings. About 13,000 middle school students and 16,000 high
school students were
16
enrolled in the schools. The student breakdown by race was 81%
Caucasian, 15% African
American, and 2% Hispanic. Forty-one percent of the student
population received free or
reduced lunch and 13% qualified for special education
services.
The study used data from school discipline records of the 2004-2005
school year. The
student's last infraction of the year coded a 1 for exclusion or 0
for not excluded and other
discipline. Exclusion included OSS and expulsion. OSS was defined
as removal from school for
ten days or less and remaining enrolled. Expulsion was defined as
removal for more than ten
days, and the student was unenrolled from the school. Other
variables reviewed were: gender,
ethnicity, special education, ELL, and previous OSS and ISS. The
last offense severity was coded
as zero-tolerance or violent offenses. Zero-tolerance offenses
include alcohol and drug,
weapons, assaulting a teacher or principal. Violent offenses were
either fighting or violence.
Data from the state’s Dept of Education was used to determine:
school type, total enrollment,
percent of the population that was non-white, percent free or
reduced lunch, ethnicity, and
gender of principal.
The data showed 9,706 students had a recorded discipline incident
during 2004-2005,
which is 15% of all middle and high school students in the
district. Students with at least one
infraction were 62% male, 43% free or reduced lunch, and 73% White.
Of the remaining 27%
ethnicity, 86.6% were African American. The analysis showed the
school community type, and
principal ethnicity and gender were non-significant predictors for
exclusion. The predictors that
were statistically significant were African-American race, free or
reduced lunch status, ELL
status, and special education status. Theriot et al. suggested that
student intervention and
school-level intervention is required to meet student and school
needs, specifically aligned with
17
these groups. These predictors cannot be generalized to greater
populations outside of this
sample without replication of these results.
Goran and Gage (2011) studied the relationship between language,
behavior, cognitive
ability, and academic performance for school-age students with
disabilities. The study
compared and contrasted students with Emotional Behavior Disorders
(EBD) and Learning
Disabilities (LD). The researchers hypothesized the following:
students with greater language
deficits will have more suspensions, student cognitive ability will
directly correlate with
academic performance, and students with the EBD label will contrast
in the areas of language,
academics, and past suspensions to students with the LD
label.
The study reviewed data from a medium-sized Midwestern school
district on students
ages 5 to 14 during the 2008-2009 school year. The district served
17,419 students, with 2,316
receiving special education services. The study only included 142
students with EBD and LD
labels. The students with the LD label represented 82.4% of the
study population. Notable
demographic variability was present in the EBD group; forty percent
were female, 60% were
Black, and 80% qualified for free and reduced lunch.
The students included in the study were administered the Clinical
Evaluation of
Language Fundamentals, Fourth Edition (CELF-4) and Cognitive
Ability Test (CogAT) as part of
their special education testing for the district. The expressive
and receptive composite score
from the CELF-4 was used to determine language skills, and the
CogAT determined cognitive
ability for the study. The district also administered the Missouri
Assessment Program (MAP) to
all students. The participants' MAP scores were pulled to determine
academic performance.
Additionally, discipline history was pulled from the school
database. The data were coded as 0,
18
no suspension, 1, in-school suspension (ISS), and 2 for
out-of-school suspension (OSS). Each
score was then multiplied by the number of days that the
participant was suspended. For
example, a student with 3 ISS and 2 OSS days would receive a score
of 7.
The analysis of the data showed no significant differences between
LD and EBD
cognitive ability, academic performance, or language ability. There
were significant differences
in the history of suspension. Language skills were not a predictor
for suspension, but they did
directly correlate with cognitive and academic abilities for both
EBD and LD groups. Cognitive
ability also positively correlated with the academic performance
for both groups. The
difference in the history of suspension showed EBD students were
suspended at a higher rate
than LD students. This was expected because students with the label
of EBD have identified
needs in the area of behavior and emotional processing. What was
unexpected was that EBD
students had similar deficits in the area of language skills,
cognitive ability, and academic
performance to the LD group. These patterns cannot be generalized
to other school districts. It
does draw attention to the need to consider language, cognitive,
and academic deficits as
possible functions of behavior. Future research should also review
how suspension impacts
EBD students who may be at risk for failure in these areas
long-term since this study suggests
that students with the EBD label may have higher suspension rates
than other groups.
Hemphill et al. (2013) conducted a study that compared students in
Australia and
students in Washington state in the United States. The question the
study aimed to answer
was, is school suspension a predictor of youth non-violent
antisocial behavior. For this study,
antisocial behavior was defined as "Behavior that violates personal
rights and societal rules and
conventions for maintaining public order… physical violence,
property offences, and illicit drug
19
use, to socially unacceptable behaviors such as truancy (Hemphill
et al., 2013)." The
researchers hypothesized that suspension is a risk factor for
delinquency and future
imprisonment.
The study included grades seven and nine students in Victoria,
Australia, and
Washington State, USA. There were 3,677 participants who
participated with a 99% retention
over 12-months. One thousand eight-hundred thirty-nine participants
were from Victoria. The
study, conducted in 2002, included 60 public and private schools.
The included classroom
within each school was randomly selected. The demographics of the
participants from Victoria
were: 90% Caucasian, 5% Asian, 5% of other origins. The
participants from Washington State
were 65% White, most common other ethnicities were Hispanic, Asian,
and Native American.
Participants living in low-income housing represented 22% of the
sample.
The study administered surveys from February to June in Washington
and May to
November in Victoria 2002 and 2003. Surveys were completed by
participants in class—absent
students completed under the supervision of trained school
personnel or via telephone with
study staff. The survey measured attitudes towards antisocial
behavior, poor family
management, current alcohol, tobacco, and cannabis use, association
with antisocial friends,
and school suspension.
The analysis showed Victoria scored higher on attitudes towards
antisocial behavior,
poor family management, current alcohol use, and current tobacco
use. Washington State
scored higher on opportunities for pro-social school involvement,
current cannabis use,
association with antisocial friends, and school suspension. The
most common reported
antisocial behavior was stealing small items and being drunk or
high at school. The hypothesis
20
of school suspension being a predictor of non-violent antisocial
behavior was not supported by
the data. Instead, it suggested that peer groups, family
relationships, and school relationships
were more influential. Although suspension is not directly
associated with non-violent
antisocial behavior, it does suggest that positive student outcomes
are influenced by a positive
peer, family, and school relationships.
School exclusion typically increases in grades six through 10.
Students who are older are
typically suspended at higher rates than younger students, but is
there more going on in the
middle school years? Greene and Ollendick (1993) studied if
students with poor academic
transition to middle school are impacted by other social-emotional
difficulties. Poor academic
transition was defined as dropping one grade point or more in
social studies, science, math,
English, and reading. The study conducted provided interventions to
two different groups.
The first intervention was provided through the teacher and only
addressed academics
and classroom support.
the student's social-emotional needs through the school counselor.
Green and Ollendick
wanted to know what intervention was most effective for academics,
and if the interventions
impacted other social-emotional areas.
The study included 66 participants, 37 boys 29 girls, in 6th grade
at four different middle
schools in a southwestern Virginia county. Over 95% of the student
population was White, mid-
to lower-class.
Forty-two participants had a decrease in GPA from the end of 5th to
the second grading
period in 6th grade; the remaining 24 participants did not. The
students with the decrease in
21
GPA were randomly assigned to one of two treatment intervention
groups. The 24 students
whose GPA remained stable were the control group.
Pre-treatment, student participants completed the Children’s
Depression Inventory
(CDI), Revised Children’s Manifest Anxiety Scale (RCMAS), Survey of
Middle School Stressors,
and Piers-Harris Self-Concept Scale. Teachers completed the Revised
Behavior Problem
Checklist (RBPC).
Archival data provided by the school included grades and
standardized test scores.
Throughout the intervention, student participants completed
biweekly surveys to assess their
perception of treatment, teachers completed weekly activity
checklists to track student support
and relationship, and guidance counselors completed a weekly report
of student contact.
All treatment students were placed in two block courses where they
had the same class
with the same teacher for two hours. Block provided students
informal counseling, monitoring,
and encouragement about academics and personal struggles, including
work completion,
building relationships, behavior, and school expectations. Teachers
met twice before
treatment started with the project manager to learn how to conduct
supportive activities that
were done with treatment students at least twice a week during the
fifth grading period (6
weeks). Teachers did not know if their students were in the full or
partial treatment group.
Full treatment students were split into six groups of three to five
students who met
weekly during the 3rd and 4th grading period and biweekly during
the fifth grading period. All
six groups had the same leader, an advanced-level clinician with a
graduate degree in clinical
psychology. The groups discussed differences from elementary to
middle, accepting great
personal responsibility for academics, the importance of work
completion, and how to interact
22
positively with peers and teachers. Additional topics discussed
were: problem-solving training,
strategies for asking for help, and organizational skills. Students
set weekly goals with aligned
incentives for meeting their goals. Parents of full-treatment
students were contacted weekly
and were also encouraged to conduct supportive activities at home.
At the end of the
treatment, student and teacher participants completed all the
reports they completed pre-
treatment.
Prior to the start of treatment, the control group had a higher GPA
and self-esteem.
They also had lower depression scale scores and academic pressure
and stress scores. Teachers
also reported fewer behavior problems for control group students
when compared to students
in the treatment groups. Participants in the full-treatment group
improved .44 GPA points from
pretreatment to posttreatment, and .32 points from pre-treatment to
the 4-month follow-up.
Participants in the partial treatment group showed .31 points
growth from pretreatment to
posttreatment, and a .23 GPA point increase from pretreatment to
the 4-month follow-up. The
full-treatment and partial-treatment had similar GPAs pretreatment
and at the 4-month follow-
up. The only significant difference between the full and partial
treatment groups was the full-
treatment group scores significantly lower scores on the depression
scale from pretreatment to
posttreatment and pretreatment to 4-month follow-up, while the
partial treatment group
showed no significant change in depression score scales. The
study’s results showed
participants who received the academic intervention improved
academically, and
improvements in social-emotional areas required a separate
intervention that specifically
addressed that area in order to show improvement. The academic
intervention did not
improve the social-emotional areas. It cannot be concluded in the
academic intervention or
23
social-emotional intervention for full-treatment participants
impacted the academic
improvement or if the combination of support was needed to achieve
that result. It should be
noted that in this study, students who were not previously
identified as at-risk academically
were labeled as at-risk after completing two of six grading periods
in their first year of middle
school. Future longitudinal research should focus on how targeting
social-emotional and
academic areas during the middle school transition year impacts
school exclusion and office
discipline referrals.
Positive Behavior Interventions and Supports
Bradshaw et al. (2007) researched the effects of PBIS on school
climate and how the
fidelity of implementation impacted the outcomes of PBIS in
schools. The study was a
randomized trial conducted from 2002 to 2007. Thirty-seven public
elementary schools from
five districts in Maryland were included. The majority of staff in
each school displayed a
willingness and commitment to the PBIS program adoption. Twenty-one
of the schools were in
the PBIS intervention condition, and sixteen were in the comparison
condition and committed
to not implementing PBIS for the duration of the study. Half of the
participating schools
received Title I support. The community location of the schools was
48% suburban, 41% urban,
and 11% rural—no significant demographic differences between
treatment and non-treatment
groups.
Data was collected from 2,596 staff members; 1,437 were general
education teachers,
and 1,159 were support staff (school psychologists, counselors,
teaching assistants, office staff,
and resource teachers). The staff were 91.29% female, 86.56%
Caucasian, and 13.44% African
24
American. The staff age breakdown was 32.05% in their twenties,
23.92% in their thirties,
23.15% in their forties, 18.10% in their fifties, and 2.77% were 60
or older. The average number
of staff per school was 68.49. Staff characteristics were collected
from a staff questionnaire,
and school characteristics were provided by the Maryland Department
of Education.
The study was a randomized control effectiveness trial of
school-wide PBIS in
elementary schools. Each school in the PBIS group received training
on how to implement the
intervention program. PBIS teams of five to six teachers and
administration attended a 2-day
summer training. PBIS behavior support coaches also provided
ongoing support and assistance
to schools throughout the study. Participants in the study
completed the Organization Health
Inventory for Elementary Schools (OHI) to collect data on
institutional integrity, staff affiliation,
academic emphasis, collegial leadership, and resource influence.
Participants responded on a
4-point scale, and the OHI score was the average of the individual
responses. The School-Wide
Evaluation Tool (SET) was used to assess PBIS implementation
quality. The SET was completed
by an external observer. The OHI and SET were completed in July
before the initial training and
annually at the beginning of May. Over the five year study, 80 to
86 percent of eligible staff
participated annually.
The OHI data showed significant positive intervention effects on
resource influence,
staff affiliation, and academic emphasis. There was no significant
effect on collegial leadership
or institutional integrity for the intervention group. The SET data
showed that no schools met
the implementation fidelity of PBIS at the start of the study. The
comparison group did have
6.3 percent of schools implement PBIS with fidelity years one,
three, and four, and 18.8 percent
25
in year two. The intervention group had 66.7 percent fidelity after
year one, 85.7% fidelity after
year 2, 95.2% fidelity after year 3, and 100% fidelity after year
four.
The data suggests that formal training and coaching for PBIS does
allow schools to adopt
and implement the PBIS intervention with fidelity, but the amount
of time it takes a school to
develop the tools and skills vary. Also, schools that had no formal
PBIS training were able to
implement PBIS with fidelity, but at a significantly lower rate
than the schools with training.
The OHI data also showed that for these schools, school climate did
improve, according to
school staff. Staff reported that they felt the necessary resources
were available for them to
implement PBIS and reach academic goals. Staff also felt that the
academic emphasis set
realistic goals that were high and attainable. Staff also reported
that PBIS improved the sense
of trust and confidence within their school teams. This study did
rely on teacher report, which
is subject to bias. The data also only represents the participant
schools and cannot be
generalized without replication of results in the various school
contexts.
Ward and Gersten (2013) studied the Safe and Civil Schools (SCS)
approach to SWPBIS.
The SCS program focused on establishing settings, structures, and
systems to facilitate positive
behavior change. The researchers specifically wanted to determine
if SCS training improved the
development and enforcement of school discipline policy and
improved student behavior and
academic achievement.
The study included 32 elementary schools in a large urban school
district. The schools
were randomly assigned to treatment and control groups. The
district’s student population
was 90% free or reduced lunch and 87% minority students. There were
negligible differences
between treatment and control schools.
26
The study used a randomized experimental design. First, school
leadership teams were
formed, including one administration staff, three general education
teachers, one special
education teacher, and two other (i.e., specialists, paras, bus
drivers). The leadership team
attended seven days of training by the SCS consultant. The team
learned how to collect data
through observation of common areas (i.e., playground, hallways,
etc.), ODRs, and results from
safety and climate surveys. The team was also taught how to train
the rest of the building staff
in PBIS. The training sessions occurred in May, June, October, and
February. Treatment
schools received training from May 2008 to 2010, and control
schools started to receive
training in May 2009. The student behavior and achievement data
were pulled for both schools
from 2007 to 2010. Other data was collected on PBIS implementation
fidelity via the BoQ, and
student behavior was measured through staff reports via the
California Healthy Kids Survey
(CHKS).
The results showed that PBIS systems were consistently being used
in both treatment
and control schools by the end of the year with two with 68%
fidelity based on BoQ scores.
Also, twice as many staff reported that they felt a consistent
school-wide plan with clear
expectations was in place from before implementing SCS to after
implementing SCS for two
years. Additional teacher report through the CHKS showed the
following reductions in problem
behavior: 32% decrease in bullying, 67% decrease in classroom
disorder, and 21% decline in
disrespectfulness and defiance. The only significant difference was
in classroom disorder, such
as disruptive behavior and on-task behavior. Additionally, students
were 22% less likely to
report being hit or pushed at school. Suspension data showed a 17%
decrease in suspensions
at the end of year one and a 22% decrease at the end of year two
when compared to the year
27
prior to implementing SCS. The data also showed student academic
performance on statewide
tests improved 14 points in math and 9 points in ELA; these changes
occurred district wide so
likely not due to SCS. The implementation of PBIS in the schools in
the study did positively
impact staff perception of expectations and behavior management
systems. It also decreased
the number of suspensions and classroom behaviors. These results
cannot be generalized
without further replication.
Gage et al. (2018) researched the effect of PBIS implementation in
schools on the
number of discipline incidents, ISS, and OSS and variation in
effectiveness compared to
implementation fidelity. The study included all elementary and
middle-level public schools in
the state of Georgia during the 2015-2016 school year. The school
demographics, discipline,
and PBIS implementation data were provided by the Georgia
Department of Education.
The study included 1,755 schools with an average student enrollment
of 548.5 students.
The student population was 40.3% White, 39% Black, and 13.8%
Hispanic. Sixty-eight and a half
percent of students qualified for free or reduced lunch and twelve
percent received special
education services. The Georgia Department of Education provides
district-level planning,
school team training, tech assistance, and ongoing coaching on
PBIS. Benchmarks of Quality
(BoQ) are used to assess the PBIS implementation fidelity. Schools
are given one of the
following statuses based on their implementation fidelity
percentage: installing below 70%,
emerging 70-85%, operational above 85%. The study only included
emerging and operational
to ensure PBIS was being implemented with fidelity. The comparison
group was data for all
public elementary and middle school data from the Georgia
department of education.
28
The BoQ was a 53-item scale completed to determine the percent of
implementation
fidelity. This was completed by the PBIS school team and external
coach. Academics were
measured in the areas of ELA, math, science, and social studies
using the Georgia Milestone
Assessment System. Behavior incidents, student demographics, ISS
for at least half of a school
day, and OSS were recorded by the Georgia Department of
Education.
The data analysis showed that PBIS schools had statistically fewer
ISS and OSS exclusions
from schools than schools that were implementing PBIS with 70% or
less fidelity. The PBIS
schools had 55.1% fewer OSS, 79.8% fewer ISS, 58.9% fewer
disciplinary incidents than the
comparison schools. There were no statistical differences between
installing and operational
PBIS schools. In this data-set implementation fidelity had a large
effect on student behavior
incidents and exclusion in elementary and middle schools.
Replication of these findings in
different contexts should be done before these findings can be
generalized.
Gage et al. (2020) completed a replication study of their Georgia
study from 2018. The
study looked at the effects of PBIS on OSS, expulsions, and effects
of implementation fidelity.
The study included all California public schools from the 2016-2017
school year. The data was
provided by the California Department of Education. PBIS
implementation fidelity was rated by
the CA PBIS Coalition, which used the following category labels:
bronze, silver, gold, platinum.
Bronze schools completed the Tiered Fidelity Inventory (TFI) and
the online application. Silver
schools implemented PBIS with 70% or greater fidelity on one or
more tiers of the TFI, which
was completed with an external coach, and they submitted the online
application. Gold schools
implemented Tier 1 PBIS with 70% or greater fidelity and 70% or
greater on Tier 2 or 3
according to their completed TFI. They also submitted the online
application and 100 words or
29
less about their school's PBIS effort. Platinum schools implemented
all three tiers with 70% or
better fidelity, submitted rates of OSS and office discipline
referrals (ODRs), sustained or
improved positive academic trends, completed the online
application, and submitted a 100 or
less word statement about their school's PBIS effort.
The schools initially reviewed for inclusion in the study were 60%
elementary schools
with an average enrollment of 622.6 students. The student
demographic averages were 25.7%
white, 5.9% African American, and 53.4% Hispanic. Twenty-three and
two-tenths percent of the
student population qualified as ELL, 12.2% received special
education services, and 62.6%
qualified for free or reduced lunch. The schools had the following
PBIS implementation fidelity
classification: 17 platinum schools, 91 gold schools, 559 silver
schools, and 222 bronze schools.
The final analysis included 98 gold and platinum schools with
available OSS and office discipline
referral data. These schools were selected to ensure PBIS was
implemented with fidelity.
The TFI measured fidelity based on three tiers. Tier 1 was
universal interventions for all
students, Tier 2 was targeted interventions for 10 to 15 percent of
the student population, and
Tier 3 was an intensive intervention for five to 10 percent of the
population. The TFI was
completed with an external PBIS coach. Other categories that were
considered during the
analysis in addition to PBIS implementation fidelity were school
type (regular or alternative),
school-age level, urbanicity, and Title I status. It should be
noted that the California Assessment
of Student Performance scores, which were used to review student
academics, were pulled
from the 2015-2016 school year since the 2016-2017 scores were not
yet available.
In California, students can be excluded for: cause, attempt, or
threaten physical harm,
violence, firearm or weapon possession or use, unlawful possession
or sale of controlled
30
substances, robbery or extortion, damage to school property, theft,
tobacco possession,
committing an obscene act, disrupting school activities (excludes
K-3), sexual assault,
threatening a witness in a disciplinary proceeding, hazing, and
bullying. The suspension data
collected gave a total of suspensions per school and did not
specify if multiple suspensions
were given to one student. The data did identify if students had
been suspended or expelled,
but it did not separate ISS and OSS.
The data showed schools in the treatment group had fewer
suspensions than the
comparison schools, and no difference in expulsion. The difference
in suspensions was found
to be statistically significant. When the Gold and Platinum schools
were separated, there were
no statistically significant differences between Gold schools and
comparison schools; there was
a significant difference between Platinum and comparison school
suspensions.
The results of this study in California were consistent with the
findings in Georgia.
Schools that were implementing PBIS with high fidelity had fewer
suspensions than schools that
were not implementing PBIS with fidelity.
Nocera et al. (2014) studied the effect of positive behavior
supports as part of a school
improvement process on student behavior and academic achievement in
comparison to the
traditional exclusionary reactive approach. The study included one
middle school with 750
students in 7th and 8th grade. More than 50% of the student
population qualified for free-
reduced lunch, and 40% of the student population represented
minority groups.
The study occurred during the 2007-2008 and 2008-2009 school years.
Seventy-five 7th
grade and 75 8th grade students participated in the student during
each school year, totaling
300 students. The study used a mixed methods approach, compilating
quantitative and
31
qualitative data. The qualitative data was collected from
transcripts of administrative and
teacher interviews. Quantitative data was collected from the school
climate survey completed
by students. The survey in 2007 included 47 questions. The data
from this survey informed the
school improvement plan and was used to develop the goals for
academic achievement and
school climate. The created positive behavior plan looked to
increase parent involvement at
school. The plan utilized a tiered support system. Tier I supports
and interventions were
provided to all students. Tier II was received by 10-15% of the
students who were identified as
at-risk for problem behavior, so they received specialized
supports. Tier III included 5-10% of
the students who received support and intervention to address
chronic behavior problems.
The implementation of the positive behavior improvement plan was
implemented by all
teachers in the school. The principal ran in-service with teachers
on logical and natural
behavior consequences. School staff operated under the belief that
behavior continues
because it serves a function and replacement behavior needs to be
taught. Teachers also
attended a two-day seminar on improving school climate and support
student behavior and
achievement. The Tier I supports utilized in the plan included: all
students were taught specific
behaviors that met each student's expectation, students were shown
how expected behaviors
looked in each school environment (i.e., lunch, classroom, etc.),
and positive rewards were
given to students who performed positive behaviors. The school also
had a streamlined
approach to respond to problem behaviors. First, teachers offered
an opportunity for the
student to correct the unwanted behavior before it became a problem
and conferenced with
the student to review behavior expectations before issuing
detention. If not resolved, the
parent was contacted about detention, and a parent meeting followed
detention to avoid
32
future problems. If the behavior continued, then the student
received a behavior referral to
the office. This plan ensured that the teacher, student, and parent
were collaborating to
resolve the problem behavior before involving administration.
For Tier II and III interventions, teachers were provided
additional PD on conflict cycle,
Functional Behavior Analysis (FBA), and Positive Behavior
Intervention and Supports (PBIS). Tier
II conflict cycle training focused on stressful incidents, student
feelings, student behavior, and
adult reactions that create power struggles between students and
teachers. Staff was given
strategies to avoid negative reactions to end the conflict cycle to
avoid power struggles. Tier III
FBA and PBIS training taught teachers how to define the behavior,
collaborate with others,
collect baseline data, develop a hypothesis, determine
reinforcement, and plan interventions.
Behavior Intervention Plans (BIP) for students with Tier III
interventions were developed from
the FBAs, and included strategies for reducing behaviors and
meeting needs in pro-social ways.
A specific area of focus for all student interventions was on
increasing student academic
success to avoid behavior caused by academic failure.
Norcera et al. (2014) analyzed the number of office discipline
referrals, number of
school suspensions, number of behavioral infractions, pre-post
results of a school climate
survey, and faculty interview data. The results showed referrals
declined by 36% from 2007 to
2010, suspensions declined by 39%, and suspension of special
education and minority students
also reduced. It was noted that suspension of special education and
minority students still runs
at a disproportionate rate to typical Caucasian students. In
addition to decreased referrals and
suspensions, documented classroom problem behaviors decreased by
40% from 2007 to 2009.
The behaviors with the greatest improvements were cutting class and
low-level infractions that
33
are responsive to Tier 1 interventions. Tier I students were the
only students who responded
positively to the reward system that worked for the majority of
students. The reward system
was ineffective for Tier II and Tier III students.
Teachers and administration reported that peer and administrative
leadership was
imperative to the success of the plan. Consistency of discipline
empowered students and
teachers because the 5 step plan was predictable and didn’t lead
with a call home or office
referral. Teacher-student conference was perceived by the teacher
as a key piece to success.
The study only represented one middle school's outcomes after
implementing a positive
behavior plan building-wide over two school years. The sample size
and convenience of the
sample limit its generalizability, but it suggests that schools may
benefit from shifting away
from traditional exclusionary discipline practices towards positive
systems that require
teachers, students, and parents to collaborate and problem-solve
through problem behaviors
before involving administration.
Freeman et al. (2015) studied the direct and indirect effects of
PBIS on high school
dropout rates. The researchers specifically aimed to determine if
PBIS implemented with
fidelity could address dropout risk factors, thus reducing high
school dropout rates.
The study used data from 600 to 800 typical high schools from PBIS
Center’s dataset
from 37 states across seven years, 2005 to 2012. Public data from
the state department’s
website was also included. The high schools had an average
enrollment of 1.080 students. The
demographic and community breakdown averages were: 40% free or
reduced lunch, 33%
minority, 22% urban, 21% suburban, 13% large small-towns, and 13%
rural area.
34
The study was a quasi-experiment interrupted time series design.
Implementation
fidelity of PBIS was determined based on BoQ and SET scores. A BoQ
score of 70% and an 80%
SET score was considered implementing with fidelity. The school
were coded based on their
BoQ and SET scores. Schools that met implementation criteria were
labeled the
implementation fidelity group, partially meeting on the SET or BoQ
resulted in a partially
implementing label, and schools with a SET score below 40 or BoQ
score below 35 were
considered not yet implementing schools. Data was pulled from the
state department of
education on dropout rate, student attendance, and academic
performance on state
standardized tests.
The results showed that higher attendance correlated with fewer
dropouts and higher
standardized test scores. Schools that were implementing PBIS with
fidelity had a statistically
significant positive effect on student attendance, especially
schools with large minority
populations. Freeman et al. also noted that schools that
implemented PBIS with fidelity over
many years continued to see small decreases in high school dropout
rates. There were no
significant effects found for schools that were partially
implementing or not implementing PBIS
with fidelity. These are not causal findings, but it does draw
attention to the impact of student
attendance on dropout rates and academic performance. In this
study, implementation of PBIS
with fidelity aligned with improving student attendance.
Eraldi et al. (2019) conducted a pilot study to assess tier one and
two implementation
outcomes for PBIS in urban elementary schools. The researchers
assessed if tier 1 PBIS
interventions reduced ODRs and if tier 2 interventions decreased
diagnostic severity for
children with, or at-risk for, internalizing and externalizing
emotional and behavioral problems.
35
The study included two Kindergarten through eighth-grade public
schools in large cities
in Northeastern USA. The study was conducted over three years.
School A had 648 students:
75% Latino, 18% AA, 1% White, 1% Asian, 5% Other. School B had
1,134 students: 65% Latino,
16% AA, 11% Asian, 4% White, 4% other. One-hundred percent of both
schools were eligible
for free or reduced lunch. From the two schools, 29 parents, 26
school staff, and 23 students
completed acceptability surveys. All students received tier one
interventions, and 114 students
participated in one of the three tier 2 interventions over the
three years. There were 14 groups
for students in grades four through six, and 15 groups were made
for students in grades seven
and eight. Four masters-level school counselors and four graduate
students in psychology
conducted the tier two intervention groups.
The tier 2 evidence-based interventions included: Coping Power
Program (CPP) for
externalizing behaviors, Friends for Life (FRIENDS) for anxiety
symptoms, and Primary and
Secondary Control Enhancement Training (PASCET) for depressions
symptoms. The
interventions each used demonstration, role plays, exposure, and
relaxation.
Doctoral-level psychologist and masters-level school psychologist
ran two-day training
with school leadership teams (professionals, paraprofessionals, and
a parent) on tier one PBIS
intervention. Consultants attended monthly school leadership
meetings to assist the team in
using data to evaluate tier 1 plan. Similarly, Tier 2 counselors
attended an initial training
workshop and had follow-up consultations. Also, a
one-and-a-half-day workshop was
completed on each program conducted by the research team. The
primary investigator and
two postdoctoral fellows in clinical child or community psychology
did weekly consultation with
the school counselors for 45 minutes to review data and plan the
next group intervention
36
session. The tier 2 groups were run by a counselor and a research
team member. The schools
began implementing tier 1 interventions in September of year one,
and the tier 2 interventions
began in January of year one. Tier two interventions included
twelve 45 minute sessions
conducted during lunch time.
Consultants used the School-wide Evaluation Tool (SET) to the
implementation fidelity
of tier one interventions each year. Total ODRs were recorded for
each year to determine the
change in discipline referrals. Implementation fidelity for tier 2
sessions was done by
independent coders who rated recorded intervention sessions using
the Fidelity Checklist
developed for the study. Data was also collected from tier 2
parents at baseline and
posttreatment. Parents of tier two participants completed the NIMH
Diagnostic Interview
Schedule for Children, Computer Version, 4th Edition (NIMH
C-DISC-IV), which was used to
determine disorder diagnoses. Parents also completed the
Interference Thermometer to
determine the degree each disorder impaired the child’s
functioning. Students who scored
positive or intermediate for externalizing behaviors on the
NIMHC-C-DISC-IV and IT scale were
assigned to CPP, positive and intermediate anxiety disorder were
assigned to FRIENDS, and
positive and intermediate depressive disorder were assigned to
PASCET.
The results of the study showed children, parents, and teachers
rated CPP, FRIENDS, and
PASCET as acceptable for all components on the acceptability
survey. The SET score for tier one
implementation fidelity showed both schools met fidelity criteria
for years two and three.
Tier 2 intervention group fidelity was 88% for CPP, 9 87% for
FRIENDS, and 94% for PASCET.
The ODRs decreased from year 1 to year 3 for both schools.
Diagnostic severity also showed
significant changes for intermediate diagnosis students who are
at-risk. There were no
37
significant improvements for positive diagnosis students. This
makes sense because tier two
interventions are designed for at-risk students and are not
expected to be effective
interventions for high-need students. The research data showed that
PBIS intervention
implementation coincided with a decrease in ODRs, and students who
were identified as at-risk
for externalizing behavior, anxiety, and depression reduced their
level of risk from pre-
intervention to post-intervention based on parent feedback.
Collaborative Proactive Solutions
Greene et al. (2004) conducted a study to determine the
effectiveness of Collaborative
Problem Solving, now referred to as Collaborative Proactive
Solutions (CPS), on the behavior of
children with Oppositional-Defiant Disorder (ODD) in comparison to
Parent Training (PT)
programs. ODD is defined as a "recurrent childhood pattern of
developmentally inappropriate
levels of negativistic, defiant, disobedient, and hostile behavior
toward authority figures (Green
et. al, 2004)." The two programs work with parents and children.
There primary contrast
between PT and CPS is CPS focuses on adult-child problem solving,
while PT teaches parents
how to deliver directives and respond to behavior in ways that
motivate their child to comply.
To further explain, CPS has the parent and child work together to
identify problems that have
occurred and work to identify solutions that are agreeable to both
parties. This planning lays
out a previously agreed-upon framework for the parent and child to
follow if a problem
situation arises. PT focuses on how parents respond to
non-compliance and are more reactive
than CPS.
38
The study included 50 children with ODD, ages 4 to 12 years, who
were randomly
assigned to the CPS and PT groups. For every three participants
assigned to CPS, two were
assigned to PT. By the end of the study, 3 participants dropped, so
28 CPS and 19 PT
participants completed the treatment. Participants included in the
study were clinically
referred and met the following criteria: ODD diagnosis, no Conduct
Disorder diagnosis, partially
met criteria for juvenile bipolar disorder or major depressive
disorder, had an IQ of at least 80,
and were not actively suicidal or homicidal. The sample was
primarily White with four Black
and one Asian American. Eighty percent of the participants who
completed the treatment also
completed the 4-month post assessment, 25 CPS, and 16 PT
participants.
To determine participant eligibility, a phone diagnostic screening,
followed by a full
diagnostic interview, was completed. The Kiddie Schedule for
Affective Disorders and
Schizophrenia for School-Age Children-Epidemiologic version was
used to assess child
participant eligibility. Cognitive ability was determined by Block
Design and Vocabulary
subtests of the Wechsler Intelligence Scale for Children-Revised.
The screening was conducted
by raters with an undergraduate degree in psychology, and they were
trained to high levels of
inter-rater reliability. One hundred seventy-two children were
screened and 50 met all criteria.
The PT group received Barkley's (1997) 10-week behavior management
program. The
treatment discussed and educated parents about causes of their
child's defiant behavior,
instructed parents on giving their child positive attention through
a special time, trained
parents to use attending skills to increase their child's
compliance, increased effective parent
commands, implemented a contingency management program, used
time-out procedures,
taught how to manage child's behavior in public, and used daily
school-home report card. The
39
treatment primarily provided instruction to parents on how to
manage their child's behavior
with the intent of avoiding possible behavior triggers and
responding to the behavior
effectively.
CPS treatment aimed to help adults understand cognitive factors
that contribute to
aggressive outbursts, including emotion regulation, frustration
tolerance, problem-solving, and
adaptability skills, and three basic strategies for handling unmet
expectations: the imposition of
adult will, collaborative problem solving, and removing the
expectation. CPS recognized the
impact of the three strategies on adult-child interactions and
aimed to coach parents and
children to become proficient with collaborative problem solving to
resolve disagreements and
defuse conflicts to reduce aggressive outbursts. CPS treatment was
7-16 weeks, with a mean
length of 11 weeks. The treatment was attended primarily by
parents. CPS treatment included:
identified what contributes to noncompliant behavior, created a
user-friendly environment,
described implementation of the collaborative problem-solving
framework, provided
medication education, improved family communication, and provided
cognitive-skills training.
All clinicians for both treatments were doctoral-level clinical
psychologists. Two clinicians
delivered PT, and four delivered CPS. Blind rater listened to 20%
of tapes to determine
consistencies of two treatments across sessions.
The two treatments varied in their primary focus. PT focused on
behavior, and CPS
focused on cognitive-behavioral therapy. The clinicians had weekly
supervision from the
primary investigator to ensure implementation fidelity of the
treatments. No medications were
prescribed or administered to the child participants during
treatment. Child participants were
permitted to continue medications they were on prior to joining the
study. Parents reported
40
medication used weekly to track the change in dosage or termination
of medication that
occurred during treatment.
Other measures that were completed by the parent included:
parent-child relationship
inventory (PRCI) pre-and post-treatment, Parenting Stress Index
(PSI) completed pre-and post-
treatment, parent domain, child domain, and the ODD rating scale
(ODDRS) completed pre-,
post-, and four months post-treatment. Therapists complete the
Clinical Global Impression
(CGI) post-treatment and by parents at four months
post-treatment.
Pretreatment measures indicated no significant differences between
PT and CPS groups
in demographics, cognitive ability, rates of diagnostic
comorbidity, and medication. The CPS
group reported significantly more medication adjustments during
treatment. The majority of
children in both conditions had two or fewer changes in medication.
More medication changes
in the CPS group may be due to medication-education component.
Medication changes were
included as covariant in the final data analysis.
CPS treatment group showed significant improvement on the ODDRS
from pre to post
and pre to 4t month. Both PT and CPS showed large effect sizes
based on ODDRS pre to post-
treatment. The pre to post-treatment results showed CPS made 46%
clinically significant
improvement, and PT made 37% improvement. The pre to 4-month
post-treatment results
showed 60% improvement for CPS and 37% improvement for PT
clinically significant
improvement. The PSI for the CPS group showed significant
improvement in one parent
domain, competence, and three child domains,
distractibility-hyperactivity, adaptability, and
reinforces parent. Borderline significant improvement for the CPS
group occurred in the mood
domain. The PCRI showed significant improvement in limit-setting
and communication and
41
borderline significant improvement in autonomy for the CPS group.
The variance in results
from pre to post-treatment and pre to 4-months-post-treatment
suggested deterioration in
effectiveness for the PT group and improvement for the CPS group.
The CPS group also showed
greater improvement than PT on CGI pre to post-treatment.
The results suggest that PT and CPS are both effective treatment
methods for
improving child behavior in children with ODD. The data suggest
that CPS is the more effective
method long-term, as it showed to increase in effectiveness over
time, while PT remained
stagnant from post-treatment to 4-months post-treatment.
Ollendick et al. (2016) conducted a similar study that compared the
effectiveness of
Parent Management Training (PMT), Collaborative and Proactive
Solutions (CPS), and waitlist
control condition (WLC) for children with ODD. The researcher's
predicted that PMT and CPS
will be more effective than WLC, PMT and CPS will be comparable,
comorbid ADHD would
produce fewer positive treatment outcomes, but anxiety disorder
(AD) would not have an
effect, and older children, males, and low SES participants will
have fewer positive treatment
outcomes.
The study was conducted in rural southwest Virginia. It included
parents who had kids
with ODD who were referred by a mental health professional,
physician, school personal, or
recruited through ads in local papers and on local tv. Two-hundred
fifty-seven kids were
screened via phone for ODD. One-hundred sixty-four met the criteria
via phone and were then
confirmed through a comprehensive in-person assessment, which also
measured comorbid
disorders. The children who met the criteria were seven- to
14-year-old with ODD. Ninety-nine
percent of the participants had a comorbid disorder, and 83% had a
second comorbid disorder.
42
A majority of the comorbid disorders were ADHD or AD: generalized
anxiety, social anxiety, or
separation anxiety. The final study included 134 total
participants. Of those participants, 25%
were on stimulant medication for ADHD, 8% were on a non-stimulant
ADHD medication, 4%
were on an antipsychotic/bipolar medication, 3% were on an
antidepressant medication, 3%
were on an anti-anxiety medication, and 1% was on an antiseizure
medication. Participants
were excluded if their child had a cognitive disorder, autism
spectrum disorder, psychotic
disorder, intellectual impairment, or current suicidal or homicidal
ideation.
Ollendick et al. (2016) randomly assigned the participants to
groups: PMT, CPS, and WLC
6-weeks. Families of kids in the WLC group who continued to qualify
ODD after six weeks were
randomly assigned to the PMT or CPS group. The children were
assessed at the end of the
treatment and six months post-treatment. Two clinicians were
assigned to one family per
assessment. The clinicians were supervised by research assistants
and graduate students in a
clinical psychology APA approved clinical scientist doctoral
program, or they were a
postdoctoral fellow. All clinicians and supervisors were trained to
ensure the reliability and
validity of data collection. No assessment clinicians served as
therapists for participating
families.
To start the study, there were 60 CPS, 63 PMT, and 11 WLC
participants. At the one-
week follow-up, all WLC's were randomly assigned, with four going
to PMT and seven to CPS.
Of the original groups, 43 CPS participants and 46 PMT participants
remained in the treatment.
At the 6-month follow, 27 CPS and 28 PMT participants
remained.
The study used the Peabody Picture Vocab Test, fourth edition, and
Expressive Vocab
test, second edition to gather expressive language ability at
pretreatment. The Anxiety
43
Disorder Interview Schedule (ADIS) for DSM-IV child and parent
versions were videotaped when
administered at the pretreatment and six-month follow-up. The
Clinical Global Impression-
Severity was completed by the same clinician as the ADIS to rate
overall behavior impairment.
The Disruptive Behavior Disorders Rating Scale was completed by
parents at each assessment,
and the Behavior Assessment System for Children – Second edition:
Aggression scale was also
administered at each assessment.
The PMT treatment group was provided by six therapists for nine
consecutive weekly
sessions with parents and one additional session four weeks after
the last session. The program
was modified to include the children in sessions so the parents
could practice skills they were
learning prior to implementing them at home. Each session started
by stating the goals of the
session, and printed handouts were provided to the parent. PMT aims
to educate parents
about the causes of defiant and noncompliant behavior. PMT
instructs parents on positive
attending through the use of special time and trains parents to use
attending skills to increase
compliant behavior. The goals of PMT are to increase the
effectiveness of parental commands,
teach parents to implement a contingency management program, use
time-out procedures,
manage child behavior in public places, and use daily school and
home report cards to
consistently manage child behavior. The PMT therapists received 4
hours of training prior to
the start of treatment and were supervised in each session.
The CPS treatment group included eight therapists who helped
parents and children use
problem-solving methods independently. The number of sessions was
the same as PMT. CPS
focuses on training parents to view their child's behaviors as
manifestations of lagging skills.
44
The parent and child then work together to identify the problem and
come up with mutually
agreeable solutions.
The study results showed PMT and CPS were more effective than the
WLC treatment.
The effectiveness of PMT and CPS was equivalent, with nearly 50% of
the participants being
diagnosis free at post-treatment. Both PMT and CPS were more
effective for younger than
older children. The participants of this study were primarily
middle-class Caucasian families, so
the generalizability is limited. The effectiveness of PMT and CPS
may only be seen in educated
Caucasian families and limited to families with children who have
an ODD diagnosis.
Martin et al. (2008) examined patterns of restraint and seclusion
before and after
implementing CPS with aggressive children and adolescents. The
participants were 755
patients at an inpatient psychiatric facility from 2003 to 2007.
The participants’ median age
was 11. Sixty-three percent were male, 52% White, 25% Black, 23%
Hispanic, and 52% were
Medicaid covered.
The research team conducted a prospective study on data collected
at a psychiatric
inpatient unit for school-age children at Yale-New Haven Children’s
Hospital. The 15-bed
facility averaged 198 admissions per year that lasted an average of
29 days over the course of
the study.
Staff wide training occurred in 2005 on CPS implementation. The
staff team first went
through a three-hour overview, followed by 90 minutes sessions
twice per week to discuss
specific patients and their challenging behaviors. The staff team
included: 25 nurses, 35
counselors, two psychiatrists, one psychologist, three school
personnel, and six trainees.
45
From 2003 to 2007, the average stay decreased two days per year,
from 35 to 22. The
data indicated 2,230 restrictive interventions occurred during the
study, 559 restraints (25%)
and 1,671 seclusions (75%). Approximately five percent of patients
accounted for 50% of
restrictive events. The data showed boys were more likely to be
restrained and secluded than
girls, older age participants were more likely to be restrained and
secluded than younger, and
patients who were covered by Medicaid were more likely to be
secluded. The largest
difference when comparing data was Black participants were more
than four times as likely to
be restrained or secluded as White participants. From pre-CPS
implementation to post-CPS
implementation, the number of yearly restraints decreased from 263
to seven, and seclusions
decreased from 432 to 133. The duration of restraints decreased
from 16 plus or minus 10
hours per month to .3 plus or minus .5 hours per month. The length
of seclusions dropped
from 15 plus or minus 6 hours to 7 plus or minus 6 hours per month.
Staff injuries also
decreased from 55 in 2003 to 29 in 2007. The data suggests that CPS
treatment decreased the
need for staff to use restriction and seclusion with patients in an
inpatient psychiatric unit.
Epstein and Saltzman-Benaiah (2010) studied how to adapt
Collaborative and Proactive
Solutions (CPS) to work in a group setting for parents and children
with Oppositional Defiant
Disorder (ODD) and Tourette’s Syndrome (TS). The study subjects
were parents of children
under the age of 12-years who meet the criteria for TS, another tic
disorder, or ODD. All of the
children were patients in Tourette Syndrome Neurodevelopmental
Clinic at University Health
Network and were assessed by a clinical psychiatrist. The study
excluded children with conduct
disorder (CD), current suicide/homicide, history of brain
injury/neurological conditions, or a
46
full-scale IQ below 80. Families with previous CPS treatment or
psychosocial interventions were
also excluded. Eighteen families were screen, and fifteen met the
study's criteria.
To start, participants completed diagnostic interviews using ODD
and CD subsections of
Schedule for Affective Disorders and Schizophrenia for School-Aged
Children – Present and
Lifetime Version. The three questionnaires were completed at the
intake procedure (baseline),
pre-intervention, post-intervention, and 2-month follow-up.
Participants also had a telephone
interview at pre-intervention, post-intervention, and 2-month
follow-up. Raters, who were
doctoral-level clinical psychologists, additionally administered
Oppositional Defiant Disorder
Rating Scale and the Clinical Global Impression to the
participants.
The participants were separated into two separate treatment groups
were participated
in the treatment at different times. The first group at 11 parents
and seven children. The
second group had eight parents and five children. Parents were
given manual binders, divided
into seven sections. The binders included session outlines, written
material of new concepts,
practice exercises, and homework sheets. The curriculum was
designed to help parents
understand children's behavioral difficulties and emotional
dysregulation are not intentional,
identify contributors to the development of noncompliant behavior,
environmental changes to
prevent difficulty, understand three basic parenting strategies –
Plan B, become comfortable
with Plan B (empathy, define the problem, invite the child to
problem-solving), and recognize
challenges that interfere with effective parenting. Clinical
psychiatrists worked with
participants through the treatment with the goal of helping the
parent participants develop the
skills necessary to use CPS independently with their children.
Participants completed treatment
sessions, work through materials in the binder, and completed
homework. At the conclusion of
47
Child Behavior Inventory (ECBI) parent rating scale, which assessed
frequency and intensity of
behavior at home, social competence scale, and parenting stress
index.
Twelve families completed the treatment. All 12 children had ODD,
11 had TS, and one
had chronic motor tic disorder and an average age of 9.33.
Ninety-one percent of mothers and
eighty-eight percent of fathers missed no more than one treatment
session and no more than
one homework assignment. Mothers and fathers reported significant
improvement on the ECBI
intensity scale and problem scale. Significant improvement was also
reported on the social
competence scale, ODD rating scale, and parenting stress outcomes.
At the end of treatment,
95 percent of parents felt confident in their ability to use CPS
with their child at home.
Although the study does not take place in schools, it does suggest
that with training, people
who are not clinical psychologists are able to utilize this tool
with children independently.
Miller-Slough et al. (2016) studied the impact of parent-child
synchrony on emotional
lability, aggression, and overall functioning. For this study,
synchrony was defined as active
engagement, shared understanding, and willingness to listen to
others. The study included 75
children age seven to 12 years; 46 participants were male. The
parents included 55 mothers
and 20 fathers. All participants received treatment for ODD, met
ODD criteria, and did not
meet criteria for CD, ASD, or psychosis. The participants’
full-scale IQ was 80 or higher, and
they had no current suicidal/homicidal ideation. The majority of
the participants were White
and from two-parent households. In addition to ODD, 65.4% of
participants met the criteria for
ADHD, and 58.7% met the anxiety criteria.
48
The multi-methods assessment recruited participants from a
university-based clinic.
The participants received weekly treatment sessions up to 14
sessions from clinical therapists.
The clinical therapists and assessment interviewers were
post-masters graduate students in
clinical psychology supervised by a licensed clinical psychologist.
Assessment interviewers were
blind to treatment conditions.
The Anxiety Disorder Interview Schedule, Fourth Edition (ADIS-IV)
was used to measure
ODD symptoms and other behavior problems pre- and post-treatment.
Additionally, the CGAS
measured the overall functioning of the participant. Parent-child
synchrony was observed
during parent-child conversation tasks on positive and negative
emotions that occurred during
treatment sessions. The conversations were coded for agreement on
details of an event,
perceived connectedness, and shared affect. Participants were
scored on a 0-4 scale, four
being most synchronous. The final measure used was the BASC-2.
Parents reported their
child’s internalizing and externalizing symptoms and adaptive
functioning via the BASC-2, pre-
and post-treatment. Only the aggression scale scores were used for
the study.
The results showed participants with higher parent-child synchrony
pretreatment were
associated with lower emotional lability p