K. Schatz Internship Portfolio - CECH€¦ · Internship Portfolio Based on 2013-2014 Training in...

Internship Portfolio

Based on 2013-2014 Training in School Psychology

Katie Schatz, M.Ed. University of Cincinnati

2

Table of Contents

Portfolio Entry Alignment with the National Association of School Psychologists Ten Domains of Practice and Program Requirements……………………………………………………………3 Professional Vitae…………………………………………………………………………………4 Utilizing Video Modeling to Improve the Social Skills of Students with Intense Behavioral Needs: A Systems-Wide Change………………………………………………………………….6 Increasing Class-wide Engagement of Fifth-Grade Students While Decreasing Disruptive Behavior Through the Use of an Interdependent Group Contingency…………………………..46 Improving the Math Computation Skills of Third-, Fourth- and Fifth-Grade Students…………73 Increasing the Duration of Time-In-Class of an Eighth-Grade Student Through the Use of Check-in/Check-out and Self-Monitoring Procedures………………………………………….123 Improving the Reading Skills of Middle School Students with Emotional and Behavioral Disorders Through the Use of Peer Assisted Learning Strategies and Self-Graphing…………154 Utilizing a Response to Intervention Model to Determine Eligibility for Special Education Services for a First-Grade Student with Disruptive Behaviors…………………………………193 Improving the Literacy Skills of a First-Grade Student………………………………………...250

TABLE OF PORTFOLIO ALIGNMENT 3

Table 1: Portfolio Case Entry Alignment with NASP Domains and Program Requirements NASP Domains Portfolio Case Entries

1

2

3

4

5

6

7

8

9

10

Program Requirements

Fulfilled Utilizing Video Modeling to Improve the Social Skills of Students

X X X X X X X X • Systems-level entry

Increasing Class-Wide Engagement of Fifth-Grade Students While Decreasing Disruptive Behavior

X X X X X X X • Tier 1 behavior entry

Improving the Math Computation Skills of Third, Fourth and Fifth Grade Students

X X X X X X X X • Tier 1 academic entry

Increasing the Duration of Time-In-Class of an Eighth- Grade Student

X X X X X X X • Tier 2 behavior entry

Improving the Reading Skills of Middle School Students

X X X X X X X • Tier 2 academic entry

Utilizing a RtI Model to Determine Eligibility

X X X X X X X X • Tier 3 behavior entry

• RtI eligibility entry

Improving the Literacy Skills of a First-Grade Student

X X X X X X X • Tier 3 academic entry

NASP Domains:

1. Data-based decision making and accountability 2. Consultation and collaboration 3. Interventions and instructional support to develop academic skills 4. Interventions and mental health services to develop social and life skills 5. School-wide practices to promote learning 6. Preventative and responsive services 7. Family-school collaboration services 8. Diversity in development and learning 9. Research and program evaluation 10. Legal, ethical, and program evaluation

Reference National Association of School Psychologists. (2012). Model for comprehensive and integrated school

psychological services, NASP practice model overview. [Brochure]. Bethesda, MD.

PROFESSIONAL VITAE 4 7920 Cooper Road Katherine E. Schatz [email protected] Cincinnati, OH 45242 513-277-9826

EDUCATION

Educational Specialist (Ed.S.) of School Psychology, Expected April 2014 University of Cincinnati, Cincinnati, OH (Current GPA: 3.98/4.0) NASP Accredited Masters of Education (M.Ed.), June 2012 University of Cincinnati, Cincinnati, OH (Overall GPA: 3.97/4.0)

Bachelors of Arts in Psychology, Minor in Business, June 2011 The Ohio State University, Columbus, OH (Overall GPA: 3.87/4.0)

PROFESSIONAL EXPERIENCE

Intern School Psychologist August 2013 – Present Learning Center at North Norwood, Cincinnati, OH Pleasant Run Elementary, Cincinnati, OH Welch Elementary, Cincinnati, OH Universal Support

• Developed and implemented school-wide video modeling program to improve 11 social skills of 60 students, ages 8-18

• Constructed training for teachers to identify academic needs of students ages 8-18 through AIMSweb assessments

o Administered and scored 120+ AIMSweb assessments • Consulted and implemented interventions with teachers to improve academic and behavioral skills

of 31 students, ages 8-10 • Prepared and delivered 2 professional development training presentations focusing on “functional

behavior assessment and intervention planning” to 70 staff from various school districts Targeted Support

• Analyzed MAP and AIMSweb data to increase math computational skills of 13 fourth and fifth grade students through small-group intervention

Intensive Support • Completed 2 initial evaluations and 10 re-evaluations (ED, MD, CD, SLD, OHI, Autism) • Constructed intervention plans from functional behavior assessments

o Designed and implemented an individualized intervention for an eighth grade student that resulted in 33% increase in time in class 5 days into the intervention

School Psychology Practicum Student August 2012 – April 2013 Kilgour Elementary, Cincinnati, OH Madeira Elementary, Cincinnati, OH Northern Kentucky Head Start, Newport, KY Universal Support

• Designed and monitored the progress of academic and behavioral interventions for 40 students ages 4-8 using a problem-solving tiered approach, in collaboration with teachers and parents

• Developed and delivered 3 staff and parent presentations associated with Head Start o 1 of 3 students out of 12 peers invited to present results of assessments and interventions

to parents and school officials Targeted Support

• Presented 8 lessons to improve friendship-making social skills (e.g. sharing, offering help, listening) of 6 first grade students

Intensive Support • Conducted a functional behavior assessment and functional analysis for a kindergarten student

PROFESSIONAL VITAE 5

Literacy Tutor December 2011 – May 2012 Winton Woods Primary North School, Cincinnati, OH

• Tutored 12 kindergarten students which resulted in 83% of students reaching goals in literacy skills at conclusion of 6-month intervention through the use of Teacher-Directed KPALS 3 times per week

• Evaluated students’ needs based on analysis of DIBELS data

ACTIVITIES AND AWARDS

Crisis Hotline Volunteer September 2009 – June 2010 Suicide Prevention Services, Columbus, OH

• Assessed callers’ suicide risk to evaluate necessary action and direct individuals to medical attention if needed

• Completed 39 hours of training and role-played scenarios to gain experience in crisis protocol • Average of 4 crisis calls taken per day, 200 hours of volunteering, one time per week

University Graduate Scholarship, September 2011 – April 2013 Graduated with Honors (Magna Cum Laude), June 2011

PROFESSIONAL DEVELOPMENT

• Anxious Kids in School: Rethinking Response Strategies • Bullying Prevention Summit • Crisis Prevention and Intervention (CPI) Training

PROFESSIONAL ASSOCIATIONS

• Association for Behavior Analysis International (ABAI) • National Association of School Psychologists (NASP) • Ohio School Psychologists Association (OSPA)

REFERENCES

Angie Bien, Ed.S. School Psychologist Hamilton County ESC 11765 Hamilton Avenue Cincinnati, OH 45231 859-421-1439 [email protected]

Chris Gilkey, Ph.D. School Psychologist Hamilton County ESC 5017 Marion Avenue Cincinnati, OH 45212 513-324-4855 [email protected]

Julie Morrison, Ph.D. Assistant Professor University of Cincinnati School Psychology Program 550 Teachers College Cincinnati, OH 45221 513-478-3517 [email protected]

UTILIZING VIDEO MODELING 6

Utilizing Video Modeling to Improve the Social Skills of Students with Intense Behavioral

Needs: A System-Wide Change

Co. Authored by Lauren McKinley

For this systems-level consultation, an intervention was implemented in a public separate

facility to increase the percentage of time students displayed appropriate social skills. The school

serves approximately 61 students between the ages of eight and 18 from various school districts.

The percentage of students identified with a disability was 100%. The majority of students who

attended the school met the criteria for Emotional Disturbance and required specially designed

instruction to support appropriate behavior as indicated through their Individualized Education

Plans (IEPs). The building-level team collaboratively decided to target 11 social skill behaviors

for intervention. To collect baseline and progress monitoring data, teachers completed direct

behavior rating forms to indicate the percentage of time each student displayed the targeted

social skills throughout the day on a scale from 0-100%.

A presentation was created to introduce the 11 targeted social skills, discuss the

components of the intervention, describe the data collection procedure, and share the social skills

schedule with the entire school staff. Additionally, three training sessions were implemented to

provide more detailed information regarding the systems-wide intervention to smaller groups of

teachers at a time. Subsequently, the social skills intervention was introduced in the classrooms.

To support the students’ development of the targeted pro-social skills, the intervention involved

direct instruction, modeling, opportunities to practice, reinforcement, and generalization

techniques.

The results of the intervention demonstrated that 62.5% of the classrooms reached the

goal by the time data were reported for this consultation. In addition, social validity results


indicated all of the school team members who completed the questionnaire “agreed” or “strongly

agreed” that they would be willing the implement the intervention next year as well. A longer-

term initiative was also to analyze individual student data to make decisions regarding whether

Tier 2 interventions would benefit a smaller percentage of students who required additional

instruction and practice to learn appropriate social skill behaviors.

Methods

Participants and Roles

Building-level team. A building-level team was developed to review the school’s current

social skills program, and to develop a plan to begin the process of implementing a new social

skills program and data collection procedure. The team was comprised of 10 individuals: a

principal, an assistant principal, a licensed school psychologist, a licensed speech-language

pathologist, three intervention specialists/teachers representing different grade-level teams, a

school psychologist intern, a school psychologist doctoral student, and a speech-language

pathologist intern. The building-level team met to collaboratively develop broad systems-level

change initiatives and action steps, and periodically review data to make decisions and evaluate

the effectiveness of the intervention. During the problem-solving process, all team members

were engaged in discussion, planning and decision-making.

Student panel. Five students were selected to participate in a student panel to provide

feedback on the social skills program the school had been implementing. One to two students

from each grade-level team participated and were asked to provide feedback based on their

opinions and the opinions of their peers. The students provided feedback regarding social skills

they feel are important, the degree to which they enjoy and feel comfortable engaging in the


social skills activities, and rewards that are motivating to them. Their feedback was taken into

consideration and influenced the development of the new social skills program.

School staff and students. All teachers were expected to attend the staff developments

and were responsible for implementing the social skills intervention with the students in their

classrooms as a system-wide support. In total there were nine intervention specialists/teachers

and nine assistants, and each classroom had one lead teacher and one assistant teacher.

Parent/guardian permission was obtained for students who participated in the creation of the

videos. Students who were 18 years of age signed assent forms to participate in the videos.

Teachers and students assisted with progress monitoring throughout the year by completing

direct behavior ratings. Once inter-rater agreement between students and teachers reached 80%,

on four out of five consecutive days, students were able to rate themselves each day. Teachers

also rated each student’s behavior on Review 360, a behavior rating system previously adopted

by the school. The teachers and students were asked to complete the Social Skills Improvement

System (SSIS) rating scale prior to the implementation of the intervention and at the end of the

school year as a supplemental measure of the students’ progress. Teachers were asked to

complete social validity questionnaires as well. The school psychology intern served a

consultative role throughout the intervention process and were responsible for conducting

intervention adherence checks while the interventions were in place. The intern received

supervision from a field supervisor and a university supervisor.

Setting

This systems-level consultation was conducted in a Midwestern public separate facility

for students with disabilities, specializing in intensive supports for students with behavioral

and/or mental-health needs. The school serves approximately 61 students between the ages of


eight and 18 from various school districts. Of the students served, approximately 12% are in

grades three through five, 17% are in grades six through eight, and 70% are in grades nine

through 12. The percentage of students identified with a disability was 100%. The majority of

students who attended the school met the criteria for Emotional Disturbance. The school

administratively supported multi-tiered systems of support (MTSS) procedures, which allowed

the school psychologist intern and doctoral student to work with staff to address students’ needs.

The interventions were implemented during a pre-established block of time, allotted for

practicing social skills.

Target Variables and Measurement

Social skills can be broadly defined as socially acceptable and learned forms of behavior

that enable an individual to interact effectively with others and avoid or escape unacceptable

behavior resulting in negative social interactions with others (Elliot & Gresham, 2008). Gresham

(2010) has explained the social difficulties children with emotional and behavioral disorders

(EBD) experience and the impact these issues have on children’s well-being. For instance, he

indicates children with EBD experience significant difficulties in developing and maintaining

interpersonal relationships, displaying pro-social behavior, and achieving social acceptance of

teachers and peers (p. 337). The presence of such social competence deficits can lead to

difficulties in educational, psychosocial and vocational functioning (p. 337). Children who are

poorly accepted by peers and adjust poorly to schooling are at a greater risk for lifelong

maladaptive outcomes (p. 339). Furthermore, research has evidenced predictive relationships

between social behaviors and long-term academic achievement (p. 340). For instance, Malecki

and Elliot (2002) evidenced that social skills correlated approximately .70 with end-of-year

academic achievement. Clearly, it is important for school professionals to design and implement


research-based intervention strategies to support the acquirement and development of pro-social

skills to remediate social competence deficits.

The building-level team determined social skills were an important variable to target for

intervention school-wide. As previously discussed, children with EBD typically have a wide

range of significant social skill deficits (Quinn, Kavale, Mathur, Rutherford, & Forness, 1999).

The majority of students who attended the school met the criteria for Emotional Disturbance and

required specially designed instruction to support appropriate behavior as indicated through their

Individualized Education Plans (IEPs). After consultation with the building-level team and

student panel, it was determined that the school-wide intervention would target 11 social skills

(Appendix B). To identify specific skills to target, the team considered skills addressed through

the previous social skills program as well as skills identified as meaningful through two research-

based programs, the Social Skills Improvement System and Skillstreaming (Elliot & Gresham,

2007; McGinnis & Goldstein, 2007).

Direct behavior ratings were used to collect baseline and progress monitoring data. Direct

behavior ratings have been conceptualized as capturing the strengths of systematic direct

observation and behavior rating scales, and typically entail a rater quantifying their perception of

a directly observed behavior (Chafouleas, 2011). Behavior ratings were completed on a daily

basis for each student. Teachers indicated the percentage of time each student displayed the

respective social skill (when the opportunity arose) throughout the day on a scale from 0-100%

on a rating form. Elementary and middle school students primarily remained with the same

teacher throughout the day; therefore, teachers were asked to give each student one rating per

day (Appendix B). High school students attended classes with different teachers throughout the

day; therefore, students were rated at the end of each class period (Appendix C). For high school


students, an average behavior rating was calculated at the end of each day by adding the ratings,

dividing the sum by the number of class periods, and multiplying by 100. To further examine

students’ performance on specific social skills, each student’s daily ratings were averaged to

obtain a weekly rating for the students. Additionally, weekly class-wide averages were obtained

by summing each student’s weekly average and dividing by the number of students in the

classroom.

Initially, the teachers completed direct behavior ratings on their student’s social skills

behaviors and each student completed direct behavior ratings on himself/herself as well

(Appendix D). Reliability was assessed by comparing the teacher rating with the student rating

for 100% of measurement occasions until inter-rater agreement reached 80% on four out of five

consecutive days, which the teachers kept track of on a monitoring form. A student rating was

considered “in agreement” with a teacher’s rating if the student rating was the same as the

teacher’s rating, or if the student rating was one rating below or above the teacher’s rating

(ratings progressed in 10% increments). When this level of inter-rater agreement was achieved,

each student completed direct behavior ratings on his/her social skills behaviors, and their

teacher conducted reliability checks for 20% of measurement occasions and provided feedback

accordingly. Inter-rater agreement was calculated by dividing the number of agreements by the

total number of agreements and disagreements and multiplying this number by 100 (Reed &

Azulay, 2010).

Inter-Scorer Agreement

At the end of each week, the intern collected the direct behavior rating forms from the

teachers. Inter-scorer agreement data were collected for 10% of measurement occasions (Table

1). To assess the reliability of measurement, the intern re-scored the average behavior ratings


calculated by the teachers. Percentage of agreement was calculated by dividing the number of

agreements by the total number of agreements and disagreements and multiplying this number

by 100 (Reed & Azulay, 2010).

Table 1

Inter-Scorer Agreement Data

Date Average Inter-Scorer Agreement

3/28/14 100%

Goals and Decision Rules

After consulting with the building-level team, it was decided that the goal of the

intervention would be to increase the percentage of time students displayed appropriate social

skills to 80% by May 30, 2014. Visual analysis of the progress monitoring data determined

whether a change in the intervention needed to take place or if students reached the goal.

Decision rules were set regarding the daily report card ratings for individual student data, as well

as class averages. The decision rule for individual student data was four points that fell below the

aim line when the intervention was being implemented correctly (Hixson, Christ, & Bradley-

Johnson, 2008); at this point, the team agreed the student was in need of additional support. The

decision rule for the class averages was four points that fell below the aim line; at this point, the

team provided teacher feedback and additional training in intervention procedures.

Functional Hypothesis

Based on baseline data, the school psychologist intern and doctoral student hypothesized

students needed to be directly taught the necessary social skills in order to exhibit appropriate

behavior. Therefore, the social skills program involved direct instruction, modeling, and


opportunities to practice and receive feedback and reinforcement on skills. The team

hypothesized that if social skills were explicitly taught and reinforced across all settings, and a

procedure was put in place to monitor the extent to which the school-wide intervention was

implemented, then the percentage of time students displayed appropriate social skills should

increase.

Accountability Plan

A multiple baseline design across classrooms was used to examine the effects of the

intervention on the dependent variable. This design was chosen because the nature of the

intervention allowed for the sequential introduction of the intervention across three classrooms at

a time. The design would allow consultants to analyze whether a functional relationship existed

between the intervention and students’ social skill behavior, as well as identify barriers to

intervention implementation prior to introduction of the intervention systems-wide. After three to

four baseline data points were collected, the intervention was introduced in three classrooms.

The intervention was introduced to the remaining classrooms sequentially after one week of

implementation.

A multiple baseline designed also allowed for the simultaneous analysis of each

classroom’s data. An AB design was used to examine the effects of the intervention on the

dependent variable for each classroom. Three to four baseline points were obtained prior to

intervention implementation (A). Subsequently, the social skills intervention was implemented

(B). The aforementioned target variable was measured daily for each student as well as averaged

for each classroom.

Intervention Procedures


Baseline condition. Prior to introduction of the intervention, teachers collected baseline

data on students’ social skills by completing direct behavior ratings for each of their students;

then, the direct behavior ratings were added and divided by the number of students to yield an

average direct behavior rating. Prior to implementation of the intervention, three teachers

collected baseline data for four weeks, three teachers collected baseline data for five weeks, and

three teachers collected baseline data for six weeks. Baseline data are displayed and discussed in

the results section below.

Staff development. Professional development can lead to improvement in instructional

practices and student learning (Borko, 2004). Teachers’ participation in professional

development is beneficial and necessary, as many teachers are not prepared to implement

appropriate teaching practices (Birman, Desimone, Porter, & Garet, 2000). There are several

evidenced-based factors to consider when planning professional development activities to make

training sessions more effective (Birman, Desimone, Porter, & Garet, 2000). Activities should

focus on a specific area or teaching method rather than general teaching methods, teachers

should be encouraged to become engaged in discussion, planning and practice; and continued

professional communication should be encouraged. In addition, collective participation, such as

participation of teachers from the same grade is one way to foster active learning.

After engaging in discussion and planning regarding the new social skills program with

the building-level team, the school psychologist, intern and doctoral student prepared a

PowerPoint to introduce the 11 new social skills, discuss the components of the intervention,

describe the data collection procedure, and share the social skills schedule with the entire school

staff. During the presentation, the presenters showed a completed video, activities to implement

to practice the skills, daily behavior rating forms, and handouts with the definition and objective


of each skill (Appendix A) and the new social skills schedule for the year (Appendix E). The

presenters encouraged the staff to ask questions throughout the presentation, and provide

feedback on aspects they liked and perceived barriers to implementation. For instance, a high

school teacher brought up a concern regarding the daily behavior rating forms. Since high school

students switch classes throughout the day, the concern was that one teacher could not accurately

rate students; therefore, the team decided the students would receive a rating from each teacher

and the ratings would be averaged at the end of the day. In addition to the school-wide staff

development, the intervention and data collection procedures were discussed more in depth

during subsequent teacher trainings.

Teacher training. The school psychologist, intern, and doctoral student prepared a

PowerPoint on the intervention and data collection procedures and presented the PowerPoint

during three teacher-training sessions. The intervention procedures were implemented in

“waves” with three teachers at a time; therefore, each training was delivered to three of the nine

teachers. During this time, the teachers engaged in discussion and planning regarding how to

show the videos to their students, the materials they would need to show the videos, and when to

show the videos. Then, to enact a social skills session a video was shown to the teachers and they

were encouraged to participate in an activity at the end of the video. The teachers were also

asked to plan how to “catch”, acknowledge and reward their students for demonstrating the

social skills throughout the week. Lastly, the data collection procedure was reviewed with the

teachers. Teachers were encouraged to ask questions and provide feedback throughout the

presentation.

It is important to note that sustained implementation of important initiatives is difficult,

and “best practice” would indicate the need for ongoing support in professional development.


Successful professional development efforts involve several elements, including: ongoing

measurement, valued outcomes, evidenced-based interventions, and supports for accurate

practice implementation (Sugai, Horner, & McIntosh, 2010). As part of this systems-wide

consultation, evidenced-based interventions were implemented to increase students’

demonstration of the targeted social skills, student progress was periodically measured through

the use of daily behavior reports, integrity checks were conducted to monitor adherence to

intervention procedures, and teachers were continually encouraged to communicate with each

other, the school psychologist, intern, and doctoral student for support and to have concerns

addressed.

Social skill intervention. Social skill training has become a primary intervention in use

with children with EBD and usually involves selecting or prioritizing critical social skills that

need to be improved; demonstrating, explaining, or modeling these skills; having the child

practice these skills while being coached; providing feedback and reinforcement during practice;

and identifying a variety of social situations in which the skill might be useful (Rutherford,

Quinn, & Mathur, 1996). Direct instruction is an important technique for teaching pro-social

skills, which involves verbally explaining social behavior (Elliot & Gresham, 1993).

Additionally, the use of models to assist in the development of appropriate social behavior has

been demonstrated repeatedly (Cooper, Heron, & Heward, 2007). Planned models, such as a

videotape of a person emitting specific behaviors, are prearranged antecedent stimuli that help

individuals acquire new skills by showing the learner exactly what to do (Cooper et al., 2007).

Video modeling involves having an individual watch a video of her/himself (or someone similar)

engage in the behavior targeted for improvement (Baker, Lang, & O’Reilly, 2009). Video

modeling has been demonstrated to be an effective intervention for children displaying normal


development, children with developmental disabilities, and children with EBD (Baker, Lang, &

O’Reilly, 2009). In addition to direct instruction and modeling, it is also necessary to provide

students with opportunities to practice and receive reinforcement for demonstrating appropriate

behavior to support the development of skills being learned. The principle of reinforcement

claims that when a type of behavior is followed by reinforcement there will be an increased

future frequency of that type of behavior, and research has supported this principle (Cooper,

Heron, & Heward, 2007). Thus, it is reasonable to say that students should be praised for

demonstrating appropriate social behavior to increase the likelihood those behaviors will occur

more frequently in the future.

For this systems-level consultation, the social skills lessons lasted approximately 15 min,

and were implemented five days per week. The lessons incorporated direct instruction, modeling,

opportunities to practice, reinforcement, and generalization techniques to support the students’

development of the targeted pro-social skills. Each week, one of the 11 social skills was

introduced. Video modeling was utilized to teach the targeted social skills to students. Five

videos were created for each skill per age group (i.e., elementary school, middle school, high

school) to model the skill in various settings, including the classroom, hallway, cafeteria and

gym. Each video began by displaying a definition of the social skill, followed by students

appropriately modeling the skill, teacher praise for demonstrating the skill, and a description of

an activity to provide an opportunity for students to practice the skill. The activities incorporated

into the videos were developed based on procedures from a research-based curriculum, the

Social Skills Improvement System (Elliot & Gresham, 2007). Teachers were encouraged to

reinforce their students for demonstrating the social skills throughout the week. A different social

skill was introduced each week and when the videos for all of the skills had been shown, the


teachers would refer to the beginning of the schedule and re-cycle through each social skill

(Appendix E). See Appendix F for example video modeling scripts.

Feedback conditions. Graphs were shared with the building-level team to demonstrate

student progress and make necessary decisions based on data. If visual analysis of the data

indicated students’ social skills were not improving, the team would discuss modifications to the

intervention. In addition, if intervention adherence was low, intervention procedures would be

reviewed with the team and teachers. For the purpose of this consultation entry, information was

reported through 3/21/14; however, the intern continued to consult with the school team

throughout the rest of the year. A performance feedback meeting was scheduled to occur with

each teacher after Spring Break. Additionally, a meeting to share student progress with the entire

school staff was scheduled for 5/14/14.

Adherence Data

Adherence to the intervention was assessed through direct observation with a procedural

checklist based on the essential components of the intervention script for 23.91% of intervention

days (see Table 2). See Appendix G for the adherence checklist utilized in the classrooms. If

intervention adherence was low, feedback and additional training on intervention procedures

were provided to the teacher. Additionally, teachers were asked to complete a checklist to

indicate whether the intervention was implemented each day, which was included at the bottom

of the direct behavior rating form (Appendix H).

Table 2

Direct Observation Adherence Data

Date Adherence

2/4/14 100%


2/12/14 100%

2/21/14 50%

2/25/14 100%

3/6/14 100%

3/7/14 100%

3/11/14 100%

3/13/14 100%

3/14/14 100%

3/20/14 50%

3/21/14 100%

Social Validity

Social validity was collected throughout the intervention process through frequent

meetings and conversation between the school-based team, staff and students. Teachers and

students provided insight into aspects of the previous social skills program they liked and

disliked, and feedback that contributed to the development of the new program. In addition,

teachers completed social validity questionnaires to indicate their satisfaction with the

intervention and its effectiveness (Appendix I).

Table 4 demonstrates the results from the social validity forms filled out by 11 members

of the school team. The data in each cell represent the number of individuals who chose that

rating. Additionally, individuals also commented, “The students seem to be more engaged in

videos and it’s nice that only one staff member is needed”, “The intervention was easy and

effective in my class”, “I do like the videos and it’s great for the students to do them as well as


watch them”, and “The videos have helped to keep the students’ attention and they are more

engaged with the social skill activity.”

Table 4

Social Validity Results

Strongly Disagree

(1)

Disagree (2)

Not Sure (3)

Agree (4)

Strongly Agree (5)

Mean Rating

Standard Deviation

I had adequate input in developing the intervention

1 1 1 2 6 4 1.41

The intervention was easy to follow

- - - 6 5 4.45 .52

I liked the procedures used in this intervention

- - - 4 7 4.64 .5

The intervention was easy to include in my daily routine

- - - 6 5 4.45 .52

I would be willing to use this intervention next year

- - - 4 7 4.64 .5

Overall, this intervention was beneficial for the student(s)

- - - 5 6 4.55 .52

Cultural and Language Differences


This consultation involved working with a diverse population. The school psychologist

intern and doctoral student were culturally responsive to all school staff and students through

conjoint behavioral consultation. When working with school staff and students, the school

psychology students respected each individual’s opinions, attitudes and values. The students also

encouraged others to appreciate the individual differences of those around them.

Results

Class-wide baseline and progress monitoring data are displayed below in Figure 1. In

support of visual analysis, summary statistics were calculated in Table 5. The mean (M) and

standard deviation (SD) of the target variable were calculated. The effect size, percentage of non-

overlapping data (PND), and goal attainment scaling (GAS) were also calculated. For the purpose

of this consultation entry, data were reported through 3/21/14. However, the intervention

continued to be implemented throughout the rest of the year and data were collected on student

progress and class-wide progress until the end of the year. In addition to the class-wide data

presented below, the building-level team analyzed individual student data to make decision

regarding whether Tier 2 interventions would benefit a smaller percentage of students.

Figure 1 demonstrates the results of the social skills intervention on the weekly average

direct behavior rating for each classroom. The ratings represent the percentage of time students

demonstrated the social skill (when the opportunity arose) throughout the week on average. As

can be seen, the level of classroom one’s average direct behavior rating was high during baseline

and the trend was stable (M=90.4; SD=1.81). During the intervention phase, visual analysis of

the data showed the level of the data remained high and moved in an increasing trend. The goal

was met on seven occasions during the baseline and intervention phases. The intervention mean

was 92 (SD=6.94).


The level of classroom two’s average direct behavior rating was moderate during baseline

and the data moved in a stable decreasing trend (M=59.58; SD=7.32). During the intervention

phase, visual analysis of the data showed an increasing trend with variability. Goal attainment

scaling was +1, but the goal was not met by the time data were reported for this consultation. The

intervention mean was 65.52 (SD=3.83). The intern continued to collaborate with the building-

level team and teachers throughout the year to address the students’ needs.

As can be seen, baseline data were not obtained for classroom three. Therefore, a

comparison cannot be made between the percentage of time students demonstrated the social

skills before and after intervention implementation. However, the level of classroom three’s

average direct behavior rating was high during the intervention phase (M=82.48; SD=4.53). The

goal was met on five occasions during the baseline and intervention phases.

The level of classroom four’s average direct behavior rating was moderate-to-high during

baseline, but the data were highly variable (M=52.5; SD=11.67). During the intervention phase,

visual analysis of the data showed an increasing trend. The PND was 100%, indicating the

intervention was effective in increasing the classroom’s average direct behavior rating above

baseline. The goal was met on four occasions during the intervention phase. The intervention

mean was 87.86 (SD=7.86).

The level of classroom five’s average direct behavior rating was moderate-to-high during

baseline, but the data were variable (M=67.35; SD=14.32). During the intervention phase, one

data point was collected, which was above 100% of the baseline data points. This rating of 82.5

exceeded the goal. After this data point was collected, the teacher of classroom five left the

position; therefore, the students in her classroom were moved to different groups. Each student’s


direct behavior ratings began to be incorporated into the class-wide average direct behavior

ratings of their new respective classrooms.

The level of classroom six’s average direct behavior rating was high during baseline and

the trend was variable (M=87.12; SD=12.42). During the intervention phase, visual analysis of

the data showed a decreasing trend initially, followed by an increase in trend based on the last

two data points collected. It is important to note that the lead teacher was absent during a portion

of the time baseline data were being collected; therefore another individual was responsible for

determining the direct behavior ratings until the teacher returned during the intervention phase.

The goal was met on four occasions during the baseline and intervention phases. The

intervention mean was 73.52 (SD=9.53).

The level of classroom seven’s average direct behavior rating was moderate-to-high

during baseline, but the data were variable (M=43.86; SD=19.29). During the intervention phase,

visual analysis of the data showed an increasing trend with slight variability. Goal attainment

scaling was +1, but the goal was not met by the time data were reported for this consultation. The

intervention mean was 61.72 (SD=3.99). The intern continued to collaborate with the building-

level team and teachers throughout the year to address the students’ needs.

The level of classroom eight’s average direct behavior rating was relatively high during

baseline (M=63.76; SD=5.94). During the intervention phase, visual analysis of the data showed

a decrease in the level of the classroom’s ratings based on the first two data points collected. The

level of last two data points collected during the intervention phase was higher, but the data were

below the aim line. It is important to note that the lead teacher was absent during a portion of the

time baseline data were being collected; therefore another individual was responsible for

determining the direct behavior ratings until the teacher returned during the intervention phase.


The intervention mean was 42.64 (SD=20.31). The intern continued to collaborate with the

building-level team and teachers throughout the year to address the students’ needs.



Figure 1. Class-wide average direct behavior ratings

Summary statistics for classrooms one through eight are shown below in Table 5. Effect

size was calculated by subtracting the baseline mean from the intervention mean then dividing

that number by the baseline standard deviation. PND was calculated by dividing the number of

intervention points that did not overlap with the baseline points by the total number of

intervention points. GAS was also determined for the student, where “0” corresponds to no

progress toward the goal, “1” corresponds to progress made toward the goal, and “2” indicates

the goal was met.

Table 5

Summary Statistics for Weekly Average Direct Behavior Ratings for Classrooms 1 through 8

Note: NC=not calculable

Discussion

On average, the results indicated the intervention was successful in providing social skills

support. Specifically, 62.5% of the classrooms reached the goal by the time data were reported

for this consultation. In addition, social validity results indicated all of the school team members

# BL Data Points

BL Mean

BL SD # Intervention Data Points

Intervention Mean

Intervention SD

Effect Size

PND GAS

Class 1 4 90.39 1.81 3 92 6.94 .89 33.33%

+2

Class 2 3 59.58 7.32 7 65.52 3.83 .81 42.86%

+1

Class 3 0 NA NA 6 82.48 4.54 NC NC +2 Class 4 4 52.5 11.67 5 87.86 7.86 3.03 100

% +2

Class 5 4 67.35 14.31 1 NC NC NC 100%

+2

Class 6 4 87.12 12.42 6 73.52 9.53 -1.1 0% +2 Class 7 4 43.86 19.29 5 61.72 3.99 .93 20% +1 Class 8 4 63.76 5.94 5 42.64 20.31 -3.56 0% 0


who completed the questionnaire “agreed” or “strongly agreed” that they would be willing the

implement the intervention next year as well.

Although this systems-wide consultation was a wonderful opportunity for me to practice

and develop components of my professional model of practice, there were some limitations to

this consultation that may have impacted its effectiveness. Direct behavior ratings were used to

collect baseline and progress monitoring data. There are benefits to using direct behavior ratings,

such as their potential ability to capture the strengths of systematic direct observation and

behavior rating scales. However, ratings do entail a rater quantifying their perception of a

directly observed behavior. Compared to direct observation, rating scales are more subjective

measures of change. Despite this limitation, the team agreed direct behavior ratings were an

appropriate tool to monitor the social skill behavior of students at the system-wide level. If

students were identified for Tier 2 or Tier 3 interventions, direct observations could be used to

measure student progress. As a school psychologist, I will use the most valid and reliable

assessment methods available when possible.

Additionally, inter-scorer agreement should have been collected more often. Ideally inter-

scorer agreement should be collected for 20-25% of measurement occasions. As monitoring this

data were difficult due to the large number of direct behavior reports turned in on a daily basis,

in the future I will anticipate this barrier and attempt to utilize other members of the school team

to help with this task as well. It would have also been beneficial to seek the help of the school

team in conducting intervention adherence checks. The intern and school psychologist attempted

to directly observe several classrooms per week to collect inter-scorer agreement data; however,

additional classrooms could have been observed if a larger number of team members also


assisted with this responsibility. As a school psychologist, I will seek the support of other school

team members to efficiently and effectively supports students.

Although there were nine classrooms within the school building, data were obtained for

eight classes. The intern attempted to obtain direct behavior rating data from the teachers as often

as possible. However, no data had been turned in by classroom nine by the time data were

reported for this consultation entry. Additionally, no baseline data were obtained for classroom

three prior to intervention implementation. The team continued to communicate with the teachers

to provide feedback and collaboratively overcome this issue.

Visual analysis of the progress monitoring data indicates that it would have been

beneficial to schedule a performance feedback meeting with classroom eight sooner than the date

that was scheduled as there were several data points below the aim line. It is important to note

that there was a change in the individual who completed the ratings between the baseline and

intervention phases, which may have impacted the level and trend of the data. The team planned

to conduct performance feedback meetings with all teachers following Spring Break to address

teacher and student needs.

Despite the aforementioned limitations, this consultation was an excellent opportunity to

practice and develop my skills. The consultation provided the valuable experience of being

involved in a systems-change effort. I was able to collaborate with a school-based team to

develop broad systems-level change initiatives. The team collaboratively decided to identify a

shorter list of research-based social skills to target through intervention in order to provide

students with additional instruction and opportunities to practice these skills throughout the year.

As the school did not previously have a data collection procedure in place to montitor students’

responsiveness to intervention, the intern collaborated with the team to develop an appropriate


tool to measure students’ social skill behavior. The team engaged in data-based decision making

to analyze student progress. Furthermore, the team planned to utilize the information collected to

identify students who may benefit from more intensive Tier 2 instruction in certain social skill

behaviors. I was honored to be a part of the development of the aforementioned initiatives and

will reflect on the experience to make decisions in my future practice as a school psychologist.


References

Baker, S.D., Lang, R., O’Reilly (2009). Review of video modeling with students with emotional

and behavioral disorders. Education and Treatment of Children, 32(3), 403-420

Birman, B.F., Desimone, L., Porter, A.C., & Garet, M.S. (2000). Designing professional

development that works. Educational Leadership, 57(8), 28-33.

Borko, H. (2004). Professional development and teacher learning: Mapping the terrain.

Educational Researcher, 33(8), 3-15.

Chafouleas, S.M. (2011). Direct behavior rating: A review of the issues and research in

its development. Education and Treatment of Children, 34 (4), 575-591

Cooper, J.O., Heron, T.E., & Heward, W.L. (2006). Applied Behavior Analysis (2nd

edition). Columbus: Prentice Hall/Merrill.

Elliot, S.N. & Gresham, F.M. (1993). Social skills interventions for children. Behavior

Modification, 17(3), 287-313.

Elliot, S.N., & Gresham, F.M. (2007). Social skills improvement system: Classwide

Intervention Program Teacher’s Guide. Bloomington, MN: Pearson Assessments.

Elliot, S.N., & Gresham, F.M. (2008). Social skills improvement system: Intervention

guide. Bloomington, MN: Pearson Assessments.

Gresham, F.M. (2010). Evidence-based social skills interventions: Empirical foundations

for instructional approaches. In M.R. Shinn & H.M. Walker (Eds.), Interventions for

Achievement and Behavior Problems in a Three-Tier Model Including RTI (pp. 337-362).

Bethesda, MD: The National Association of School Psychologists.

Hixon, M., Christ, T.J., & Bradely-Johnson, S. (2008). Best practices in analysis of

progress monitoring data and decision making. In A. Thomas & J. Grimes (Eds.), Best


Practices in School Psychology V (Vol. 6, pp 2133-2146). Bethesda, MD: The National

Association of School Psychologists.

Malecki, C.M., & Elliot, S.N. (2002). Children’s social behaviors as predictors of

academic achievement: A longitudinal analysis. School Psychology Quarterly, 17, 1-23

McGinnis, E., & Goldstein, A.P. (2007). Skillstreaming the elementary school child: New

strategies and perspectives for teaching prosocial skills. Illinois: Research Press.

Quinn, M., Kavale, K., Mathur, S., Rutherford, R., & Forness, S. (1999). A meta-analysis

of social skill interventions for students with emotional and behavioral disorders. Journal

of Emotional and Behavioral Disorders, 7, 54-64

Reed, D.D., & Azulay, R.L. (2010). A Microsoft Excel 2012 based tool for calculating

interobserver agreement. Behavior Analysis in Practice, 4(2), 45-52.

Sugai, G., Horner, R., & McIntosh, K. (2010). Best practices in developing a broad-scale

system of school-wide positive behavior support. In A. Thomas & J. Grimes (Eds.), Best




Appendices

A. 11 Targeted Social Skills

B. Direct Behavior Rating Form – Version 1

C. Direct Behavior Rating Form – Version 2

D. Student Self Rating Form

E. Social Skills Schedule

F. Example Video Modeling Scripts

G. Adherence Checklist

H. Direct Behavior Rating Form with Adherence Checklist

I. Social Validity Questionnaire


Appendix A: 11 Targeted Social Skills

SOCIAL SKILLS PROGRAM FOR 2014 *Reference: Social Skills Improvement System (SSIS) Each day:

• Show video • Complete activity (at end of video) • Try to “catch” students demonstrating the skill throughout the week • Reward students for demonstrating the skill • Try to remind students what their body language should look like for each skill

Social Skills

STOP AND THINK: GOOD CHOICE? BAD CHOICE? Objective: Student will demonstrate understanding/use of “stopping” mechanisms and generate alternative choices and their consequences. Stop:

• Take a deep breath • Count backwards/smile • Walk away/sing a song • Ask for help/hug myself • Know that I can choose

Think: Think about the best choice(s) to make LISTEN TO OTHERS Objective: Students learn to exhibit the verbal and nonverbal behaviors that indicate active listening and comprehension of the message or lesson. Definition: Listening means to pay attention. To listen to others, it is important to use your eyes to look at the speaker, move your body so the speaker can see you, sit quietly, use your ears to hear the speaker, and respond (make eye contact, nod, answer) to show that you understand the speaker. FOLLOW THE STEPS Objective: The student will follow the steps. Specifically, the student will be able to follow the steps/directions that are provided verbally by an adult. The student will also focus on listening to be able to follow the steps.


Definition: Steps/directions are things that help you or show you how to do something. Following the steps involves listening to the person giving the directions, asking about anything that seems confusing, writing the steps in a list or remembering what must be done, and then doing the task and checking that all parts of the directions were completed. FOLLOW THE RULES Objective: The student will follow classroom rules. Specifically, the student will be able to exhibit the nonverbal behaviors that indicate active listening or attending to written rules, followed by verbal or motor behavior that indicates comprehension of and compliance with the rules. Definition: A rule is an understood procedure. Students who follow the rules know of a rule, check that they understand how to follow the rule, think about how to follow the rule and know where and in what situation the rule applies, show that they want to follow the rule, and do the right thing by following the rule. IGNORE DISTRACTIONS/ PAY ATTENTION TO YOUR WORK Objective: The student will ignore distractions from peers when doing class work. The student will focus on skills to be able to pay attention. Definition: Distractions are things that keep you from concentrating on your class work and cause you not to do your best work. To ignore distractions and pay attention to your work, you need to find what’s causing the distraction, count to five, politely ask the person to stop bothering you, concentrate on the task and ignore or don’t look at what’s bothering you, and use self-talk to help you pay attention to your work. ASK FOR HELP Objective: The student will ask for help from an adult or peer. Specifically, the student will be able to exhibit the verbal and nonverbal behaviors that facilitate getting a person’s attention and then requesting their help or assistance. The student will focus on skills learned in “following the steps” to ask for clarification or help when necessary. Definition: The skill asking for help involves asking yourself whether you need help, knowing what the problem is that you need help solving, finding someone who can help you, asking for help nicely, and saying “thank you” to the person who helped you. TAKE TURNS WHEN YOU TALK Objective: The student will take turns in conversations with peers and adults. Specifically, the student will be able to focus on skills learned in “listen to others,” because students cannot take turns in conversations unless they listen to what others are saying.


Definition: The skill taking turns when you talk involves listening to what the speaker says, showing the speaker you are listening (e.g., make good eye contact), talking when it’s your turn, waiting for the other person to talk or take a turn, and repeating the steps. GET ALONG WITH OTHERS Objective: The student cooperates with peers and gets along with others. Specifically, the student will be able to exhibit verbal and nonverbal behaviors that indicate positive interactions with peers during structured and unstructured classroom activities. The student will focus on skills learned in previous units to aid in getting along with others. Definition: Getting along with others means being nice to the people around us. It involves finding what you can do to get along with others, telling someone you want to get along or help, showing that you want to get along by being nice, and doing things such as smiling. STAY CALM WITH OTHERS Objective: The student will control his or her temper in conflict situations with peers. Specifically, the student will identify persons and situations that make him or her angry and use anger-reduction strategies taught in this unit. The student will focus on skills learned in previous units to be able to stay calm with others. Definition: Staying calm means that even if you get mad or frustrated about something, you try not to lose your temper. It involves identifying how you feel when you are mad, stopping and counting to 10, finding what is making you mad, thinking about things that make you less mad and what your choices are, talking with someone to make you feel better, and doing something that makes you feel better by acting out your best choice. BE RESPONSIBLE FOR YOUR BEHAVIOR/ DO THE RIGHT THING Objective: The student will act responsibly with others, at school, at home, and in the community. The student will focus on skills learned in previous units to be able to do the right thing. Definition: Responsibility means you do things that you should do without someone telling you, and you are able to make important decisions. Being responsible for your behavior involves thinking about what is the right thing for you to do, asking what happens if you don’t do the right thing, finding examples of doing the right thing, talking with others about what the right thing is to do, and doing the right thing. DO NICE THINGS FOR OTHERS Objective: The student will do nice things for others in a variety of situations. The student will focus on skills learned in previous units to aid in doing nice things for others.


Definition: Doing nice things means we do good and helpful things because we care about other people. Doing nice things for others involves thinking about how you feel when you are sad or hurt, finding what is making someone else feel sad or hurt, thinking about things that make you feel better and how it feels when someone does something nice for you, talking with the person who feels sad or hurt, and doing someone nice for the person (e.g., compliment, help, listen, share).


Appendix B: Direct Behavior Rating Form – Version 1


Appendix C: Direct Behavior Rating Form – Version 2


Appendix D: Student Self-Rating Form


Appendix E: Social Skills Schedule


Appendix F: Example Video Modeling Scripts

Get Along With Others: (1) Find (2) Tell (3) Show (4) Do • Classroom #1

o Situation: Student sees peer get awarded bonus points (or notices that they have a lot of bonus point written on the board) and decides to congratulate him/her for doing well.

The above situation happens; student: [looks, self-talk, congratulate peer (say in nice way, smile)]

• “I see he/she just got awarded more bonus points. That means he/she must have been doing the right thing. I know that’s hard to do sometimes, so I’m going to congratulate them.”

Teacher: “You found a great way to get along with your peer. You complimented them for doing the right thing and getting bonus points. I could tell you were being genuine because you smiled and said it in a nice way. Good job!”

• Classroom #2 o Situation: Student sees peer “acting out” and decides to remind them that

“elementary/middle/high school kids don’t act that way” and gives him/her a suggestion of something else to do (e.g., go chill, take a walk, etc.).

The above situation happens; student: [looks, self-talk, walk over to peer and say you want to help, give him/her a suggestion to calm down (say in nice way, smile)]

• “He/she seems really worked up. I would like to help him/her calm down, so I’ll go over and say I want to help. I know a few things that help me calm down, so I’ll give him/her some suggestions.”

Teacher: “You found a great way to get along with your peer. When you noticed he was worked up, you let him know you wanted to help. You gave him a suggestion for how to calm down. You said it in a nice way and I bet your friend appreciated your help! Good job!”

• Hallway o Situation: Student notices a peer left their hoodie behind in the hallway and

decides to approach the peer to let them know. The above situation happens; student: [looks, self-talk, walk over to peer

and let him know he left/dropped his hoodie (say in nice way, smile)] • “He left his hoodie behind! If that happened to me, I would want

someone to help by giving my hoodie back to me. I’ll bring his hoodie over to him.”

Teacher: “You found a great way to get along with your peer. When you saw that he left his jacket behind, you let him know and gave it back to him. That was a very thoughtful thing to do. Great job!”

• Lunch o Situation: Student notices a peer who seems isolated from the others during lunch

and decides to approach the peer to invite him/her to eat with them.


The above situation happens; student: [looks, self-talk, go over to peer and say you want to see if they’re OK, ask him if he wants to come sit by you (say in nice way, smile)]

• “I see that he looks lonely over there. If I was sitting by myself or feeling lonely, I would want someone to come see if I was OK. I’ll go over and see if he wants to sit with me.”

Teacher: “You found a great way to get along with your peer. When you noticed that he was sitting by himself, you let him know you wanted to see if he was OK. You asked him to sit with you. That was a very nice thing for you to do and you probably made him feel better. Good job!”

• Gym o Situation: Students are getting ready to play a game and they need to work

together to be successful (let students pick game). The above situation happens; student: [gets ready to play game, self-talk,

say you want to work together to do well in the game (say in nice way, smile), play game]

• “To play this game, I know we need to get along. I’ll let him know that we need to work together. Then, we will do really well in this game!”

• Teacher: “You did a wonderful job getting along with your peer during the game. You let him know that you wanted to work together. You played the game nicely with each other. Good job!


Appendix G: Adherence Checklist


Appendix H: Direct Behavior Rating Form with Adherence Checklist


Appendix I: Social Validity Questionnaire

INCREASING CLASS-WIDE 46

Increasing Class-wide Engagement of Fifth-Grade Students While Decreasing Disruptive

Behavior Through the Use of an Interdependent Group Contingency

For this consultation, an intervention was developed to increase the engagement and

decrease the disruptive behavior in a fifth-grade classroom consisting of 24 students. The fifth-

grade teacher requested help in modifying her class-wide behavior management plan to improve

overall classroom behavior. To observe class-wide behavior, a zone sampling procedure was

used whereby the intern divided the classroom into three sections with eight students seated in

each section. Baseline observations revealed that, on average, the percentage of intervals

disruptive behavior was observed was 40.61% in zone one, 9.61% in zone two, and 23.83% in

zone three. The intern and teacher collaboratively decided to target engagement and disruptive

behavior for intervention. The intern observed class-wide behaviors using an observation code

she developed (Appendix A) to monitor the target variables during the baseline and intervention

phases. The intervention utilized interdependent group contingency, bonus response cost, and

token reinforcement procedures to support student behavior (Appendix B).

The results of the intervention demonstrated that the level of class-wide engagement

remained high on average. Additionally, the percentage of intervals during which disruptive

behavior was observed decreased on average for zone one, zone two and zone three. Class-wide,

the goal for engagement was met on seven occasions and the goal for disruptive behavior was

met on four occasions. Furthermore, the results of the social validity survey indicated the teacher

“agreed (4)” or “strongly agreed (5)” with all items on the survey, suggesting she felt she had

adequate input in developing the intervention script, she liked the intervention procedures, she

would used the intervention again in the future, and she felt the intervention benefited the

students in the classroom.


Methods


The fifth-grade math teacher reported she was having trouble maintaining student

engagement and managing disruptive behavior in her classroom. The teacher worked with a

different group of students during each class period throughout the day, and expressed she was

experiencing difficulty with one classroom in particular. Although the teacher had been

implementing strategies to manage behavior class-wide, she requested help in modifying the plan

to improve overall classroom behavior. The classroom consisted of 24 fifth-grade students.

Baseline observations showed that, on average, class-wide engagement was 96.22%; however,

on average, disruptive behavior occurred during 24.68% of intervals observed across zones one,

two and three. Permission to work with the classroom was obtained from the teacher and school

psychologist.

The fifth-grade teacher and intern collaboratively identified target behaviors and designed

an intervention to increase class-wide engagement and decrease disruptive behavior. The

University of Cincinnati school psychology intern assumed a consultative role throughout the

intervention process. The intern was responsible for creating an intervention script, collecting

baseline and progress monitoring data, and conducting intervention adherence checks while the

intervention was in place. The fifth-grade teacher was responsible for implementing the

intervention in her classroom. In addition, the school psychologist periodically co-observed with

the intern to collect inter-observer agreement (IOA) data. The intern received supervision from a

field supervisor and university supervisor.

Setting


This consultation was conducted in an elementary school in the Midwest. The school

building consisted of students in fourth and fifth grade. The intervention was implemented in a

general education fifth-grade classroom during math whole group instruction.


Through consultation with the fifth-grade teacher, the target variables of engagement and

disruptive behavior were determined to be priority concerns for intervention. Engagement in

activities has been associated with school and subsequent achievement, high school completion,

and physical and emotional well-being (Blum & Libby, 2004; Christenson et al., 2008).

According to Christenson et al. (2008), the importance of engagement at school is undisputed by

educators, and it is clear that engagement is the “bottom line” in interventions to support school

completion. Disruptive behaviors can have a negative impact on student engagement and

academic achievement (Ling, Hawkins, & Weber, 2011). When students exhibit in disruptive

behavior, teachers may need to take time away from class-wide academic activities to manage

behavior (Popkin & Skinner, 2003). Thus, students who engage in disruptive behavior may be

impeding their own learning as well as that of their peers (Ling, Hawkins, & Weber, 2011). For

the aforementioned reasons, it would be beneficial to implement strategies to increase class-wide

engagement and decrease disruptive behavior to support student success.

Engagement and disruptive behavior were targeted due to the students’ relatively high

levels of disruptive behavior during the math period. Engagement was coded if a student was

demonstrating expected behavior and doing what he or she was expected to be doing at that time.

Engagement included actively attending to assigned work (e.g., raising hand, writing, reading out

loud, talking about assigned work, etc.) and passively attending to assigned work (e.g., looking at

the teacher instructing, reading assigned work, etc.). Disruptive behavior was coded if a student


was not demonstrating expected behavior and engaging in another behavior that was preventing

him or her from attending to instruction or completing activities as expected. Furthermore,

disruptive behaviors were defined as behaviors that could be distracting to others in the

classroom. This included engaging disruptive behaviors that involved off-task movement (e.g.,

tapping, leaning out of chair, getting out of seat, touching others, “making faces” at others, etc.)

or verbalizations (e.g., calling out, talking to peer, etc.).

Prior to intervention implementation, three baseline points were obtained for each target

variable. Subsequently, 25 min observations were conducted on average during the math class

once per week during the intervention phase. The intern observed class-wide behaviors using an

observation code she developed, modified from the Teacher Instructional and Caring Contacts

Research Observation Form (ICC) and the Behavioral Observation of Students in School

(BOSS) (Appendix A). The code incorporated class-wide engagement, disruptive behavior,

teacher positive attention for appropriate behavior and number of teacher prompts. A zone

sampling procedure was used whereby the intern divided the classroom into three sections with

eight students seated in each section. Each interval lasted 15 s. During each interval, the intern

scanned one section of the room and recorded whether disruptive behavior occurred. Partial

interval recording was used for disruptive behavior during each interval and momentary time

sampling was used for class-wide engagement at the beginning of every fourth interval. To

calculate class-wide engagement, the number of students present was multiplied by the number

of intervals observed. This determined the total number of student engagement behaviors that

could have been observed. The total number of students noted as engaged was divided by the

total number of engagement opportunities, and the quotient was multiplied by 100. Additionally,


during each interval, partial interval recording was used for teacher positive attention for

appropriate behavior and a frequency count was used to measure teacher prompts.

Inter-observer Agreement

Co-observations were conducted with the supervising school psychologist to ensure

observation accuracy. IOA data were calculated for 33.33% of the observation sessions during

the baseline phase and 25% of the observation sessions during the intervention phase (see Table

1). To calculate IOA for disruptive behavior, occurrence agreement (𝐴!"") was calculated by

dividing the number of agreements for occurrence by the number of agreements and

disagreements. Nonoccurrence agreement (𝐴!"!) was calculated by dividing the number of

agreements for nonoccurrence by the total number agreements and disagreements. Total

agreement (𝐴!"!) was calculated by dividing the number of agreements by the total number of

observation intervals. To calculate IOA for engagement, the partial agreement-within-intervals

approach was used (Reed & Azulay, 2010). IOA was calculated per interval and divided by the

total number of intervals.

Table 1

Inter-observer Agreement Data

Date Disruptive Behavior Engagement

2/20/14 𝐴!""=100% 100%

𝐴!"!=100% 100%

𝐴!"!=100% 100%

2/27/14 𝐴!""=100% 100%

𝐴!"!=100% 100%

𝐴!"!=100% 100%



The intern consulted with the fifth-grade teacher regarding levels of engagement and

disruptive behavior that would be acceptable as well as realistic for the students. Based on

teacher input, it was decided the goals of the intervention would be to increase class-wide

engagement during math whole group instruction to 90%, and decrease disruptive behavior

during math whole group instruction to 10% of the observed intervals by June 11, 2014.

Although baseline data were variable, the students were able to reach these goals once during

baseline; therefore, the intern and teacher considered these goals to be realistic and meaningful.

Visual analysis of the progress monitoring data determined whether a change in the

intervention needed to take place or if the goals were met. The decision rule for engagement was

three or four points that fell below the aim line when the intervention was being implemented

correctly, and the decision rule for disruptive behavior was three or four points that were above

the aim line when the intervention was being implemented correctly (Hixson, Christ, & Bradley-

Johnson, 2008). At this point, the intern and teacher would meet and discuss modifying the

intervention to address the students’ needs. However, if adherence data indicated the steps of the

intervention were not being implemented as intended, the intern would review the procedures

with the teacher during a performance feedback meeting.


Based on teacher interviews and baseline observations, the intern hypothesized the

students exhibited high levels of disruptive behavior to obtain attention. The teacher often

responded to disruptive behavior by providing attention (e.g., re-directions); however, the

students did not typically receive attention for engaging in appropriate behavior. The teacher and


intern believed the students might benefit from direct instruction in expected behavior and

frequent, consistent positive reinforcement for exhibiting these behaviors.

Accountability Plan

An AB design was used to examine the effects of the intervention on the students’ levels

of engagement and disruptive behavior. Three baseline points were obtained prior to intervention

implementation (A). Subsequently, the class-wide intervention was implemented (B). The

aforementioned target variables were measured once per week until the students consistently met

the goals.


Baseline condition. Prior to introduction of the intervention, the intern collected baseline

data on the students’ engagement and disruptive behaviors during math. The consultant used the

observation code (Appendix A) to observe the target variables on three occasions prior to

implementation of intervention modifications and additional strategies. Baseline data are

displayed and discussed in the results section below.

Prior to the intern’s involvement in this consultation, the teacher utilized strategies to

manage behavior class-wide, which incorporated components of group contingency, bonus

response cost, and token reinforcement procedures. Cooper, Heron and Heward (2007) provide a

description of group contingencies, their benefits, and their effectiveness. Group contingencies

can be an effective approach to changing the behavior of many people simultaneously. There are

several advantages to group contingencies, such as its ability to be applied across a variety of

behaviors and students, and decrease disruptive behavior and improve appropriate behavior. An

interdependent group contingency is defined as “one in which all members of a group must meet

the criterion of the contingency, individually and as a group, before any member earns the


reward” (p. 569). Furthermore, the effectiveness of interdependent group contingencies may be

enhanced by randomly arranging some of the components, such as randomly selecting

reinforcers for the contingency. Group contingencies have demonstrated positive effects on the

behavior of students (Kelshaw-Levering, Sterling-Turner, Henry, & Skinner, 2000; Theodore,

Bray, & Kehle, 2001).

It has been suggested that reinforcing appropriate behavior strengthens that behavior

(Skinner, Pappas, & Davis, 2005). As group contingencies involve earning reinforcement, token

economies can be easily incorporated into these procedures. A token economy is a behavior

change system consisting of a list of target behaviors, tokens or points that participants receive

for emitting the target behaviors, and a menu of backup reinforcer items that participants obtain

by exchanging tokens they have earned (Cooper et al., 2007). Furthermore, most token

economies include a token loss contingency for inappropriate behaviors (Cooper et al. 2007). A

socially acceptable method, bonus response cost, involves making additional reinforcers

available noncontingently to the participant, specifically for removal with a response cost

contingency (Cooper et al., 2007). Research has demonstrated that response cost used in

combination with a token reinforcement strategy can be effective at increasing on-task behavior

and decreasing off-task behavior (Rapport, Murphy, & Bailey, 1980, 1982). Additionally, a study

by Musser, Bray, Kehle, and Jenson (2001) showed that a multicomponent intervention

incorporating antecedent strategies (i.e., public posting of classroom rules), response cost, a

token economy, and mystery motivators was effective at reducing students’ disruptive behaviors.

To manage behavior, the teacher wrote the word “pop” on the board. When students

engaged in disruptive behavior, the teacher removed a letter from “pop.” At the end of the class

period, if at least one letter of “pop” remained, the students earned a letter toward the word


“prize.” When students earned all five letters to spell “prize,” they earned a “mystery” prize. The

students popped a balloon to reveal a slip of paper with the name of the prize written on it. The

students were involved in selecting rewards that would be reinforcing to them (e.g., pizza party).

Despite these strategies, the teacher reported concerns regarding the disruptive behavior the

students exhibited and requested help in modifying the plan to improve overall classroom

behavior.

Performance feedback and intervention modification. The intern consulted with the

teacher to share baseline data and discuss modifications to the class-wide intervention with the

aim of supporting appropriate, engagement behavior and decreasing disruptive behavior.

Observations revealed clear rules regarding behavioral expectations were not posted for the

students to see and praise for appropriate behavior was not frequently provided throughout the

class period. Additionally, at times disruptive behavior was acknowledged by removing a letter

from “pop”; however, at other times disruptive behaviors were ignored. The intern and teacher

collaboratively decided to modify the plan to incorporate clear behavioral expectations and

increased praise for appropriate behavior along with the strategies she had been utilizing

(Appendix B).

Clear expectations. To ensure students understood the behaviors expected during math,

several rules were established and the intern encouraged the teacher to directly teach and

reinforce these expectations throughout the class period. Teaching rules is an antecedent-based

strategy to support appropriate behavior (DuPaul & Stoner, 2010). A study by Johnson, Stoner

and Green (1996) found that active teaching of class rules was effective at increasing the

appropriate behavior and decreasing the inappropriate behavior of students. Several classroom

expectations were identified through consultation with the teacher (e.g., try your best, show your


work, be polite and respectful). The teacher agreed to teach the expectations, create a visual of

the expectations and post the visual on the wall.

Praise for appropriate behavior. The teacher also agreed to reinforce appropriate

behavior throughout the class period. To do so, the teacher set a MotivAider to vibrate four or

five times during the class period. She informed the students she would be checking to see if they

were following the expectations during these times. If students were following expectations, the

teacher provided specific praise and allowed the students to keep a letter of “pop.” If students

were not following the expectations, the teacher prompted the students by telling them the

behaviors that were expected and removing a letter from “pop.” As mentioned previously, if at

least one letter of “pop” remained at the end of the class period the students earned a letter

toward “prize.” When students earned all five letters to spell “prize,” the whole group earned a

prize.

Adherence Data

Adherence to the intervention was assessed by the intern through direct observation with

a procedural checklist based on the essential component of the intervention script for 20% of the

intervention sessions (see Table 2). See Appendix C for the adherence checklist used by the

intern in the classroom. If adherence to the intervention was low, the intern would review the

procedures with the teacher.

Table 2

Adherence Data

Date Adherence

2/27/14 75%

3/6/14 50%


3/11/14 50%

3/20/14 50%

Social Validity

The intern school psychologist designed the intervention procedures in collaboration with

the teacher, and permission to work with the classroom was obtained from the teacher and the

school psychologist. Social validity was collected throughout the intervention process through

frequent conversation and meetings between the intern and teacher. In addition, the teacher

completed a social validity questionnaire to indicate her satisfaction with the intervention and its

effectiveness, which can be seen below.

Table 3 demonstrates the results of the social validity form filled out by the teacher. The

check marks in the cells represent the ratings the teacher chose. The teacher “agreed (4)” or

“strongly agreed (5)” with the majority of the items on the survey, indicating she found the

intervention valuable. The teacher also commented, “Katie did a great job!”

Table 3


Strongly Disagree (1)

Disagree (2) Not Sure (3)

Agree (4) Strongly Agree (5)

I had adequate input in developing the intervention script

- - - - ✓

The intervention script was easy to follow

- - - - ✓

I liked the procedures used in this

- - - ✓ -


intervention


- - - ✓ -

I would be willing to use this intervention in the future

- - - ✓ -


- - - ✓ -

Results

Group baseline and progress monitoring data for engagement and disruptive behavior are

displayed below in Figure 1. Baseline and progress monitoring data for disruptive behavior

separated by zone are displayed in Figures 2 through 4. The mean (M) and standard deviation

(SD) of the target variables were calculated. The effect size, percentage of non-overlapping data

(PND), and goal attainment scaling (GAS) were also calculated. For the purpose of this

consultation entry, data were reported through 3/20/14. However, the intervention continued to

be implemented throughout the rest of the year and data were collected on class-wide student

progress until the group consistently met the goal.

Figure 1 demonstrates the results of the intervention on the percentage of intervals the

class exhibited engagement behavior and disruptive behavior. During the baseline phase, the

level of engagement was high and the data moved in a slightly increasing trend (M=96.22;

SD=1.8). However, the level of disruptive behavior ranged from low-to-moderate and the data

were variable (M=24.68; SD=18.5). During the intervention phase, visual analysis showed the

level of engagement remained high and the data moved in a slightly variable trend (M=96.81;


SD=1.27). Additionally, although the data were slightly variable, the level of disruptive behavior

was lower on average (M=9.03; SD=1.94). Across the baseline and intervention phases, the goal

of 90% engagement was exceeded on seven occasions and the goal of no more than 10%

disruptive behavior was met four times. The intern continued to collaborate with the teacher

throughout the rest of the year to address the students’ needs until the goals for both target

variables were consistently met class-wide.

Figure 1. Class-wide engagement and disruptive behavior

Figure 2 demonstrates the results of the intervention on the percentage of intervals

disruptive behavior was observed in zone one of the classroom. During the baseline phase, the

level of disruptive behavior ranged from low-to-high and the trend was highly variable

(M=40.61; SD=38.27). During the intervention phase, visual analysis of the data showed a

slightly variable trend; however, the level of disruptive behavior was lower on average


(M=12.29; SD=4.16). The goal was met once during the baseline phase on 2/24/14 and again

during the intervention phase on 3/6/14.

Figure 2. Percent of intervals disruptive behavior was observed in zone 1


disruptive behavior was observed in zone two of the classroom. During the baseline phase, the

level of disruptive behavior ranged from low-to-moderate and the data moved in a decreasing

trend (M=9.61; SD=10.51). During the intervention phase, visual analysis of the data showed a

slightly increasing trend; however, the level of disruptive behavior was lower on average

(M=5.63; SD=5.33). The goal was met on five occasions on 2/20/14, 2/24/14, 2/27/14, 3/6/14,

and 3/11/14.




disruptive behavior was observed in zone three of the classroom. Similar to observations of zone

two, during the baseline phase the level of disruptive behavior ranged from low-to-moderate and

the data moved in a decreasing trend (M=23.83; SD=13.75). During the intervention phase,

visual analysis of the data showed a slightly variable trend; however, the level of disruptive

behavior was lower on average (M=9.17; SD=6.31). The goal was met once during the baseline

phase on 2/24/14 and twice during the intervention phase on 2/27/14 and 3/20/14.



In addition to visual analysis, summary statistics for the target variables are shown below

in Tables 4 and 5. Effect size was calculated by subtracting the baseline mean from the

intervention mean then dividing that number by the baseline standard deviation. PND was

calculated by dividing the number of intervention points that did not overlap with the baseline

points by the total number of intervention points. GAS was also determined for the student,

where “0” corresponds to no progress toward the goal, “1” corresponds to progress made toward

the goal, and “2” indicates the goal was met.

Table 4

Summary Statistics for Engagement and Disruptive Behavior Class-Wide

# BL Data Points

BL Mean

BL SD

# Intervention Data Points

Intervention Mean

Intervention SD

Effect Size

PND GAS


Table 5

Summary Statistics for Disruptive Behavior for Zone 1, Zone 2, and Zone 3

Discussion

The data collected indicate the level of disruptive behavior decreased compared to

baseline on average, suggesting the intervention had a positive effect on the students in the

classroom. The goal for engagement was met on seven occasions and the goal for disruptive

behavior was met on four occasions by the time data were reported for this consultation. Social

validity results suggest the teacher strongly felt she had adequate input in developing the

intervention script, she liked the procedures used and she thought the intervention benefited the

students.

There were several limitations to this consultation that are necessary to acknowledge.

Given the teacher taught several math classes throughout the day, but was experiencing difficulty

managing behavior during this class period in particular, it would have been beneficial to

conduct observations during another class period to collect additional peer norm data. However,

the data collected indicated the students in each zone were able to exceed both goals at least once

during baseline; therefore, the goals were realistic for the students. Furthermore, the zone

sampling method provided information regarding the level of behavior different groups of peers

Engagement 3 96.22 1.8 4 96.81 1.27 .33 25% +2 Disruptive Behavior

3 24.68 18.5 4 9.03 1.94 .85 0% +2

# BL Data Points

BL Mean

BL SD


Intervention Mean

Intervention SD

Effect Size

PND GAS

Zone 1 3 40.61 38.27 4 12.29 4.16 .74 0% +2 Zone 2 3 9.61 10.51 4 5.63 5.33 .38 0% +2 Zone 3 3 23.83 13.75 4 9.17 6.31 1.07 25% +2


exhibited, and data were useful peer comparisons when setting goals. Setting goals based on peer

comparison micro-norms is an ideal goal setting method. As a school psychologist, I will utilize

this method as one means to determine realistic and meaningful goals for students.

On average, the level of class-wide engagement was high during baseline. To more

confidently ensure data were representative of the students’ level of engagement, it would have

been beneficial to record engagement at the beginning of each interval for one zone at a time,

rather than every fourth interval by scanning the entire classroom. The intern developed the

observation code based off of the ICC, which monitors engagement at the beginning of every

fifth interval by scanning the entire classroom. For this reason, the intern chose a similar method

to monitor engagement as part of this consultation. However, due to the number of students in

the classroom, it was difficult to quickly scan the students and accurately make determinations

regarding their engagement. In the future, I will consider barriers to collecting accurate data and

ensure these barriers are addressed prior to intervention implementation.

It is also important to acknowledge that the third baseline point collected indicated the

percentage of intervals disruptive behavior occurred was low. Although it is desired that

disruptive behavior occur at low levels, it would have been beneficial to collect additional

baseline data until a stable trend was developed. The decreasing trend of disruptive behavior

during baseline makes it difficult to analyze whether the intervention strategies implemented

were responsible for the decrease in the level of disruptive behavior on average. Additional

baseline data could have potentially been used to confirm the teacher’s concerns, support that

additional intervention strategies were meaningful to implement, and provide more information

to analyze whether the intervention strategies implemented were related to the behavior change


that occurred. As a school psychologist, I will analyze data and consult with key stakeholders to

design and implement interventions that are socially meaningful and effective.

Another limitation is the baseline data cannot truly be considered “baseline.” Prior to the

intern’s involvement in the consultation, the teacher implemented several class-wide strategies to

manage behavior. Therefore, these procedures likely impacted the students’ behavior during the

baseline phase. Despite this limitation, visual analysis and summary statistics indicate the

intervention had a positive impact on the students’ average level of engagement and disruptive

behavior. In the future, it will continue to be necessary to consider factors in the classroom

environment to gain an understanding of features that may be impacting student behavior.

When utilizing an interdependent group contingency, it is important to have clear

behavioral expectations. The intern consulted with the teacher to identify several classroom

expectations. To ensure the students had a clear understanding of the behaviors expected in order

to earn praise and keep the letters of “pop”, it would have been beneficial if the expectations

were more descriptive in nature by indicating exactly what the desired replacement behaviors

were (e.g., raise your hand to speak, sit in your seat, keep hands and feet to yourself, etc.).

However, the teacher reported she felt the expectations developed were appropriate for her

classroom and would be explained to the students. As a school psychologist, I will encourage the

development of appropriate and observable behavioral expectations during consultation to ensure

students have a clear understanding of the expectations. However, it will also be important to

collaborate with teachers and other key stakeholders to develop expectations the team deems

appropriate and important.

For the purpose of this consultation, data were reported through 3/20/14. By this point,

direct observations of adherence revealed the teacher was not consistently implementing all of


the components of the intervention as intended. Therefore, the intern scheduled a performance

feedback meeting with the teacher for after Spring Break. During this time, the intern would

review the intervention procedures with the teacher and discuss modifications to the intervention

if necessary. Although the level of student engagement was high and the level of disruptive

behavior was low during the intervention phase despite lack of adherence, it is possible that

adherence to all of the components of the intervention may have decreased disruptive behavior

further. As a school psychologist, it will be essential to discuss barriers to implementation before

an intervention is put in place and frequently throughout the intervention process to increase the

likelihood the intervention will be utilized, which will ultimately support the needs of the

students involved.

Finally, based on baseline and progress monitoring data, the students in zone one

exhibited higher levels of disruptive behavior across phases compared to zones two and three. As

part of this consultation, the teacher reported the entire class could benefit from the

implementation of strategies to manage behavior. However, this group of students may also

benefit from more targeted support to address their needs if progress monitoring data indicate the

students’ level of disruptive behavior increases. If this occurs, the intern will consult with the

teacher regarding additional intervention strategies that could be implemented to support the

students.

This consultation was an excellent opportunity for me to practice and develop

components of my model of professional practice. I served as a child advocate by consulting

with the fifth-grade teacher to design an intervention with the aim of supporting all 24 students in

her classroom. I was also able to practice my skills in collaboration by working with the teacher

to incorporate her input and current classroom behavior management system with additional


research-based strategies to further support behavior at the Tier 1 level. Adhering to an

ecological-behavioral approach to problem-solving, I provided suggestions regarding how

factors in the environment could be manipulated to support students’ appropriate behavior. I also

functioned as a scientist-practitioner by utilizing valid and reliable assessment methods to

measure levels of student engagement and disruptive behavior across baseline and intervention

phases. I used progress monitoring and adherence checks to analyze intervention effectiveness

and develop suggestions regarding how the intervention plan can be improved, which will be

shared with the teacher to make collaborative decisions.


References

Blum, R.W., & Libby, H.P. (Eds.). (2004). School connectedness: Strengthening health

and education outcomes for teenagers. Special issue, Journal of School Health, 74(7),

229-299

Christenson, S.L., Reschly, A.L., Appleton, J.J., Berman-Young, S., Spanjers, D.M., &

Varro, P. (2008). Best practices in fostering student engagement. In A. Thomas & J.

Grimes (Eds.), Best Practices in School Psychology V (Vol. 4, pp 1099-1119). Bethesda,

MD: The National Association of School Psychologists.

DuPaul, G.J., & Stoner, G. (2010). Interventions for Attention-Deficit Hyperactivity

Disorder. In M.R. Shinn & H.M. Walker (Eds.), Interventions for Achievement and

Behavior Problems in a Three-Tier Model Including RTI (pp 825-848). Bethesda, MD:

The National Association of School Psychologists.





Johnson, T.C., Stoner, G., & Green, S.K. (1996). Demonstrating the experimenting

society model with classwide behavior management interventions. School Psychology

Review, 25(2), 199-214.

Kelshaw-Levering, K., Sterling-Turner, H. E., Henry, J. R., & Skinner, C. (2000). Randomized

interdependent group contingencies: Group reinforcement with a twist. Psychology in the

Schools, 37, 523-533

Ling, S., Hawkins, R. O., & Weber, D. (2011). Effects of a classwide interdependent


group contingency designed to improve the behavior of an at-risk student. Journal of

Behavioral Education, 20(2): 103-116.

Musser, E.H., Bray, M.A., Kehle, T.J., & Jenson, W.R. (2001). Reducing disruptive

behaviors in students with serious emotional disturbance. School Psychology Review,

30(2), 294-304.

Popkin, J. E., Skinner, C. H. (2003). Enhancing academic performance in a classroom

serving students with serious emotional disturbance: Interdependent group contingencies

with randomly selected components. School Psychology Review, 32(2): 282-295.

Rapport, M.D., Murphy, A., & Bailey, J.S. (1982). Ritalin vs. response cost in the control

of hyperactive children: A within subject comparison. Journal of Applied Behavior

Analysis, 15(2), 205-216. doi: 10.1901/jaba.1982.15-205



Skinner, C.H., Pappas, D.N., & Davis, K.A. (2005). Enhancing academic engagement:

Providing opportunities for responding and influencing students to choose to respond.

Psychology in the Schools, 42, 389-403.

Theodore, L. A., Bray, M. A., & Kehle, T. J. (2001). Randomization of group contingencies and

reinforcers to reduce classroom disruptive behavior. School Psychology Review, 39, 267-

277


Appendices

A. Observation Code

B. Script

C. Adherence Checklist


Appendix A: Observation Code


Appendix B: Script

5th Grade Class-‐Wide Behavior Intervention Materials needed: poster with class-‐wide rules, timer, balloons, prizes First time introducing the class-‐wide intervention:

1. Explain to the class that the behavior game will be modified. 2. Explain that it will be similar to the behavior game with “pop”, “prize” and the balloons,

but now you will be checking their behavior at certain points throughout the class period to make sure they are on-‐task.

3. Explain that you will be posting a list of classroom expectations, and they must be following these expectations throughout the whole period.

4. Explain that you will set a timer, and when the timer goes off, you will be checking to see if everyone is following the classroom expectations. If the students are not on-‐task, a letter of “pop” will be removed.

5. Explain that if there are letters of “pop” left by the end of the class period, they will earn a letter toward “prize” and when they spell “prize” they can pop a balloon and earn their mystery prize.

6. Explicitly identify and define the behaviors they are expected to display. 7. Provide examples and non-‐examples of following the rules. 8. Ask if the students have any questions and begin implementing the intervention.

Daily script for the class-‐wide intervention:

1. Remind the students of the classroom expectations and that you will be checking to make sure they are following the rules throughout the class period.

2. Praise them for the letters they have earned toward “prize” on previous days and encourage them to try their best so that they continue to earn more letters.

3. Set timer. 4. When timer goes off, check to see if the students are on-‐task. 5. If the students are on-‐task, provide specific praise to the students for following the rules

and tell them that they get to keep their letter. 6. If the students are not on-‐task, tell them what they should be doing and remove a

letter. 7. Repeat steps 2-‐6 until the end of the class period. 8. At the end of the class period, if there are letters of “pop” left, give the students a letter

toward “prize” and praise them for earning a letter.

When students earn all letters toward “prize”, choose a student to pop a balloon and tell students the reward they earned.


Appendix C: Adherence Checklist

IMPROVING THE MATH 73

Improving the Math Computation Skills of Third-, Fourth- and Fifth-Grade Students

A Tier 1 academic intervention was implemented in an elementary school classroom at a

Midwestern public separate facility with seven third, fourth and fifth grade students. The purpose

of the consultation was prevention to help reduce risk by increasing math computation skills. All

seven students were selected for year-long prevention services due to the low scores they

received on math computation benchmark assessments compared to national grade-level peer

norms. The teacher and University of Cincinnati intern maintained a collaborative relationship

throughout the intervention process. AIMSweb math computation (M-COMP) curriculum-based

measurements (CBMs) were used to establish benchmarks three times per year. In addition,

AIMSweb math fact probes were used to progress monitor computation fluency once per week.

Supplemental academic activities were implemented in the classroom during math, and involved

providing students with additional opportunities to practice and receive feedback on math

computation skills.

The results of the intervention indicated that three students met the addition goal and two

students met the subtraction goal by the time data were reported for this consultation. Although

only one student met the M-COMP goal, summary statistics indicated the students made progress

toward the goal and their scores improved on average compared to baseline. In addition, social

validity results indicated the teacher “strongly agreed (5)” with five out of six items on the

survey and “agreed (4)” with one item, suggesting she found the intervention valuable. The

teacher also commented, “It’s been great having someone prepare materials and organize the

data. The teamwork has made doing this feasible in my room.” The intern continued to

collaborate with the teacher throughout the year to support students’ progress toward basic math

fact goals and grade-level computational objectives.


Methods


The classroom consisted of seven students between the ages of eight and ten. Of the

seven students, one student was in third grade, five students were in fourth grade, and one

student was in fifth grade. The group consisted of six boys and one girl. All students were

identified as a child with a disability under the category of Emotional Disturbance. All seven

students were selected for Tier 1 and progress monitored based on initial benchmark scores.

The lead teacher was responsible for implementing the academic interventions with the

students in her classroom as Tier 1 supports. The classroom had one lead teacher and one

assistant teacher who were both monolingual English speakers. Parent permission was obtained

for every student, and the progress of these students was monitored. The teacher assisted with

progress monitoring throughout the year by administering addition, subtraction, multiplication

and/or division probes once per week. The intern school psychologist assumed a consultative

role throughout the intervention process, and also assisted in progress monitoring and conducting

intervention adherence checks while the interventions were in place. The intern school

psychologist received supervision from a field supervisor and university supervisor.

Setting

This consultation was conducted in an elementary school classroom at a Midwestern

public separate facility for students with disabilities, specializing in intensive supports for

students with behavioral and/or mental-health needs. The school administratively supported

multi-tiered systems of support (MTSS) procedures, which allowed the intern to work with

teachers to address students’ needs. By screening the students in the classroom at the beginning

of the year, the intern was able to identify students who may be at-risk in the area of math


computation in the Spring based on grade-level benchmarks as well as historical placement

criteria for instructional and mastery levels (Shapiro, 2011). The interventions were implemented

in the classroom during the students’ math period and transition times.


Basic math computation fluency. Mathematical deficits are widespread in the preschool

through twelfth grade population, and extend beyond the 5-8% of the school-age population who

have some form of math learning disability (Kelley, 2008, p. 419). Mathematics competency is a

national educational goal (Kelley, 2008). The National Mathematics Advisory Panel (2008)

stressed that education in mathematics is a national interest, and claimed that success in

mathematics is important because it gives students college and career options. According to

Shapiro (2011), computational objectives are the foundation upon which success in other aspects

of mathematics is built; for instance, students who cannot master basic computational skills are

very unlikely to succeed at applications (p. 237). Some students who are able to learn basic math

facts have difficulty reaching levels of fluency, and mathematics is an area in which speed and

accuracy are particularly important for success (p. 237). Research has shown that enhancing

fluency with basic math facts may free up cognitive resources that can be applied to learning

more complex tasks, and increasing students’ accuracy and speed of accurate responding to basic

math facts is crucial for developing and mastering more advanced math skills (Poncy, Skinner, &

Jaspers, 2007). Given that basic math computation fluency seems to be linked to the

development of other important keystone behaviors, there are likely many benefits to targeting

this skill.

Benchmark data in math computation were assessed using AIMSweb M-COMP probes

during the Fall benchmark period. Additional baseline data in M-COMP were collected during


September and October, 2013. Consistent with best practice in tiered service delivery, those

students who were below the Spring benchmark target score were selected for intervention

(AIMSweb Default Cut Scores, 2011; Kelley, 2008). In order to assess student progress toward

grade-level computational objectives, M-COMP probes were also administered to the students

during the Winter and Spring benchmark periods.

To analyze the particular computational skills present in the students’ repertoires,

baseline data in specific areas of math computation were assessed using AIMSweb addition,

subtraction, multiplication and division fact probes in the Fall. Baseline data were collected

during October and November 2013. Students who were below the historical placement criteria

for mastery in addition, subtraction multiplication and/or division were selected for intervention

and received additional progress monitoring. In order to assess student progress toward goals and

evaluate intervention effectiveness, math fact probes were administered to the students during

the intervention phases. Initially, students were progress monitored in addition and subtraction.

When a student met both of the aforementioned goals consistently, interventions would be

implemented to target multiplication if necessary. After meeting all of the aforementioned goals

consistently, interventions would be implemented to target division if necessary.

Once per week the children’s progress was individually tested using the AIMSweb math

fact probes. All probes were based on number families 0-12. The probes were administered for 2

min and were scored by counting the number of correct digits (CD) in the answer; examiners

underline the CD the students write and sum the number of underlines. The examiner calculated

the number of digits correct per minute (DCPM) by dividing the sum of the underlines by two.



The teacher and intern both assisted in administering the M-COMP and math fact probes

to collect baseline and progress monitoring data. To assess reliability, inter-scorer agreement was

collected for 33% of measurement occasions during the baseline phase and 25-50% of

measurement occasions during the intervention phase (see Tables 1 and 2). To do so, permanent

product data were re-scored by a second individual; for example, if the teacher scored the probe,

the probe was re-scored by intern. For two examiners, inter-scorer agreement was calculated by

dividing the number of agreements by the total number of agreements and disagreements (Reed

& Azulay, 2010).

Table 1

Inter-Scorer Agreement for Math Fact Probes

Date Addition Subtraction

10/18/13 100% 100%

11/22/13 100% 100%

1/31/14 100% 100%

2/3/14 100% 100%

3/14/14 100% 100%

Table 2

Inter-Scorer Agreement for M-COMP Probes

Date M-COMP

9/25/13 100%

2/19/14 100%



The primary goal of the intervention was to increase the students’ scores on the addition,

subtraction, multiplication and division fact probes. The goal was to increase scores to 20 DCPM

for the third grade student; 30 DCPM for the fourth grade students; and 35 DCPM for the fifth

grade student by May 30, 2014. These goals were based on a historical set of guidelines for

determining instructional level for computation (Shapiro, 2011, p. 155). According to these

guidelines, the mastery level for third grade is 20+ DCPM, the instructional level for fourth

grade and beyond is 20-39 DCPM, and the mastery level for fourth grade and beyond is 40+

DCPM.

A supplementary goal was also set to increase the students’ scores on their grade-level

M-COMP assessment. The goal was to increase the score on the M-COMP to 53 points for the

third grade student, 55 points for the fourth grade students, and 30 points for the fifth grade

student by May 30, 2014. These goals were based on AIMSweb national peer norms representing

the number of points received by the 45th percentile of students in the Spring (AIMSweb Default

Cut Scores, 2011). For two AIMSweb measures, scores at the 45th percentile have been found to

be predictive of success on state tests; therefore, the rationale for the M-COMP cut scores is that

if the lowest-scoring 45% of the national student population has been consistently found to be at

moderate risk, then it is reasonable to use that percentage as a guide to the number of students

who should be identified as at-risk when using other measures (AIMSweb Default Cut Scores,

2011, p. 1).

Visual analysis of the progress monitoring data was used to help determine whether a

student reached the goal or if a change in intervention was needed for a student. The decision

rule for change in intervention was three to four consecutive points below the aimline when the


intervention was being implemented correctly (Hixson, Christ, & Bradley-Johnson, 2008). At

this point, the intern and teacher agreed that a student’s performance was low and stable, and a

change would need to be made to address the student’s needs. After a student scored at or above

the goal for one of the target variables, the intern and teacher agreed he or she could be

monitored less frequently in that area.


The intern hypothesized the students received low scores on computation assessments

because they had not been given enough opportunities to practice addition, subtraction,

multiplication and division problems, and thus had also not received enough feedback on these

skills. The intervention involved providing the students additional opportunities to practice and

receive immediate feedback and error correction on the aforementioned types of computation

problems.

Accountability Plan

An ABCDE design was used to examine the effects of the academic intervention on the

students’ performance on the math fact probes. Two to three baseline points (A) were obtained

for each student on each target variable prior to intervention implementation. Subsequently, a

cover-copy-compare intervention was implemented (B). Additional conditions were added to

accommodate teacher input and as data-based decisions were made for students. A third phase

involved providing performance feedback, modifying the intervention, and sending math fact

flashcards home to parents to use with their children (C). Performance feedback and intervention

modifications were also implemented during the fourth phase (D). During the fifth phase,

performance feedback was provided and an additional computer activity was implemented with


four students (E). The primary target variables were measured once per week for each student

until he or she reached the goal.


Baseline condition. Prior to introduction of the intervention, the intern collected baseline

data on each student’s performance on the AIMSweb M-COMP assessment and the addition,

subtraction, multiplication and division math fact probes. For each student, one to three data

points were collected for each assessment variable. Baseline data are displayed and discussed in

the results section below.

Tier 1 cover-copy-compare activity. A cover-copy-compare activity was implemented

with all seven students in the classroom to increase their basic math computation fluency skills.

Cover-copy-compare can increase accuracy and automatic responding to basic mathematics facts

by occasioning high rates of active academic responding and encouraging accuracy through the

use of immediate feedback (Poncy, Skinner, & Jaspers, 2007). Skinner et al. (1989) developed a

cover-copy-compare procedure for math facts which involved (1) giving the student a sheet of

target problems, (2) teaching the student to study the problem and answer on the left side of the

page, (3) covering the problem and answer, (4) writing the problem and answer on the right side

of the page, and (5) uncovering the problem to evaluate their response (Poncy, Skinner, &

Jaspers, 2007). A cover-copy-compare activity implemented by Poncy, Skinner, and Jaspers

(2007) also utilized a “sprint/practice” sheet at the end of the activity; this intervention resulted

in an increasing trend in digits correct per minute. For this consultation, the intervention

implemented in the classroom utilized a cover-copy-compare and “sprint test” procedure. The

intervention was aimed at improving the scores students received on AIMSweb math fact probes,


and was introduced in the classroom after one to three baseline points per assessment had been

collected for each student.

For the intervention (Appendix A), worksheets were developed to practice addition and

subtraction with number families 0-12 (Appendix B). During the cover-copy-compare activity,

students were directed to look at the first problem on their list, copy the problem in the middle

column, cover both the printed problem and the problem they wrote, and write the problem again

on the right side of the paper. Students were then instructed to compare what they had written

with the printed problem on the left side of the paper. If a student’s response was correct, he/she

was instructed to continue to the next problem. If a student’s response was incorrect, he/she was

instructed to repeat the procedure for the same problem until it was completed correctly.

Following the cover-copy-compare activity, the students were directed to complete a “sprint

test”, which consisted of the same problems the students practiced during the activity (Appendix

C). Students were told to solve the problems on the page and raise their hand when finished. The

teacher checked the students’ responses and provided praise for correct responses; if a problem

was completed incorrectly, the teacher told the student the correct answer and asked them to

repeat it. The teacher marked the problems the student solved correctly and incorrectly on the

sprint test, and if a student solved a problem correctly for three days in a row, the teacher agreed

to drop the problem from the cover-copy-compare worksheet and add a new problem to the list.

Feedback conditions. Graphs were shared with the teacher after every three or four data

points had been collected for each student to demonstrate student progress and make decisions

based on data. If a student experienced three to four points below the aim line, the intern

reviewed the graph with the teacher and discussed modifications to the intervention. If


intervention adherence was low, intervention procedures were reviewed and/or the intervention

plan was revised to accommodate the input of the teacher.

Tier 1 flashcard drill activity. Data indicated the cover-copy-compare activity was not

being implemented as consistently as intended. During a performance feedback meeting on

12/18/13, the teacher expressed that it was difficult to implement the intervention three times per

week; therefore, the intervention plan was modified to accommodate the input of the teacher.

The teacher agreed to implement a flashcard drill activity with the students in her class to

provide additional opportunities to practice and receive feedback on computation skills.

Research has shown that increasing opportunities to respond and providing error

correction and feedback are important components of effective interventions to increase

academic skills. Skinner, Fletcher, and Henington (1996) evidenced that students’ rate of

responding increased when frequent opportunities to respond were provided. Nelson, Alber, and

Grody (2004) recommended students should be immediately corrected when they make errors,

and Skinner, Pappas, and Davis (2005) claimed that reinforcing appropriate behavior strengthens

that behavior. Furthermore, Nelson et al. (2004) demonstrated that when students are given

numerous opportunities to respond and error correction procedures and provided, students

particularly gain fluency skills. Therefore, for this consultation, the flashcard drill activity

implemented coupled increased opportunities to respond with error correction and reinforcement

procedures to improve the students’ basic math computation fluency skills.

For the intervention (Appendix D), flashcards were used to practice basic math facts with

number families 0-12. During the activity, the students were instructed to sit in a circle. The

teacher began by showing a student a flashcard. The student was instructed to answer the

problem on the flashcard. If the student responded correctly, the teacher provided praise, and if


the student responded incorrectly, the teacher told the student the correct answer and asked the

question again. The aforementioned process was repeated with each student until all of the

flashcards had been practiced. Then, the process was completed for other problem types being

practiced with the students (e.g., subtraction). During the week of 1/27/14, the flashcard drill

activity was modified to provide individual students with increased opportunities to respond and

to enable the teacher to check the response of each student in the group. The students were

instructed to write their responses to problems on whiteboards and display their answers to the

teacher. After presenting a flashcard, the teacher scanned the students’ responses and completed

the feedback and error correction procedures as described above.

Parents’ roles in children’s computation skills. After Winter break, the intern and

teacher provided parents with addition and subtraction flashcards to practice with their children

at home. A letter was sent home to explain the purpose of utilizing the flashcards, as well as the

steps to follow when practicing the flashcards, including: showing the flashcard and prompting

the child to respond, and providing praise and error correction. Home support was not monitored

as part of this consultation.

Tier 2 computer activity. Four children were not making adequate progress toward the

goal, and after sharing the students’ graphs with the teacher on 2/21/14, it was decided that a

computer activity would be implemented with the students through the XtraMath program. In

addition to participating in the flashcard drill activity, each student would independently

complete the computer activity for approximately 10 min three days per week. During the

activity, basic math facts were presented to the student and he was instructed to type his

responses to the problems. If the student responded correctly, a “smiley face” appeared on the

screen if the response occurred within three seconds, and a green check mark appeared on the


screen if the response occurred within ten seconds. If the student responded incorrectly or took

more than ten seconds to provide a response, the correct answer was displayed and the student

was prompted to complete the problem again. Problems the student responded incorrectly to

were presented more often to provide additional opportunities to practice and receive feedback.

Adherence Data

Adherence was assessed by the intern through direct observation with a procedural

checklist based on the essential components of the intervention script for 11% of intervention

sessions during the cover-copy-compare phase, 10.26% of intervention sessions during the

flashcard drill phase, and 6.67% of intervention sessions during the computer activity phase (see

Table 3). The procedural checklist for the pertinent activity was used (Appendix E).

Additionally, implementation adherence for the cover-copy-compare activity was collected

through permanent product data, which indicated the intervention was implemented on 33.33%

of days intended. During the flashcard drill phases, the teacher was asked to indicate days when

the flashcards were utilized with each student on an adherence form (Appendix F).

Implementation adherence for each student is displayed in Table 4. The XtraMath program saved

a record of the occasions each student completed the computer activity. Implementation

adherence for each student is displayed in Table 5. Adherence was not monitored for home

procedures in this support consultation.

Table 3

Direct Observation Adherence

Date Cover-Copy-Compare Flashcard Drills Computer Activity

12/3/13 87.5% N/A N/A

1/15/14 N/A 100% N/A


1/31/14 N/A 100% N/A

2/14/14 N/A 66.67% N/A

2/28/14 N/A 100% 100%

Table 4

Teacher Collected Implementation Adherence for Flashcard Drills

Alex Kevin Chris Michael Kaleb Maria Lerrick

66.67% 51.52% 21.21% 48.48% 51.52% 39.39% 48.48%

Table 5

XtraMath Implementation Adherence

Alex Kevin Michael Lerrick

73.33% 80% 13.33% 26.67%

Social Validity

The intern designed the intervention procedures in collaboration with the teacher and

parent permission was obtained for all seven students. Social validity was collected throughout

the intervention process through meetings and frequent conversation between the intern and

teachers. In addition, the teacher completed a social validity questionnaire to indicate her

satisfaction with the intervention(s) and its effectiveness (below).


check marks in the cells represent the ratings the teacher chose. The teacher “strongly agreed

(5)” with the majority of the items, indicating she found the intervention valuable. In addition,


the teacher commented, “It’s been great having someone prepare materials and organize the data.

The teamwork has made doing this feasible in my room. Katie has great ideas too.”

Table 6






- - - - ✓


- - - - ✓


- - - - ✓


- - - ✓ -


- - - - ✓


- - - - ✓

Results

Individual baseline and progress monitoring data for the math fact probes and M-COMP

are displayed below in Figures 1 through 21. In support of visual analysis, summary statistics

were calculated below in Tables 7 through 9. The means (M) and standard deviations (SD) of the


target variables were calculated for each student. The percentage of non-overlapping data (PND),

goal attainment scaling (GAS), and effect size were also calculated for the target variables. For

the purpose of this consultation entry, data were reported through 3/26/14. However, the teacher

continued to implement the interventions throughout the rest of the year and data were collected

on student progress until the end of the year or until the students met the goals.

Figure 1 shows the results of the intervention on Maria’s addition scores. The level of

Maria’s performance on the addition assessment was high during baseline (M=31.25; SD=1.06).

However, since her scores were below the goal the C-C-C intervention was implemented. During

the first intervention phase, visual analysis of the data showed a slightly increasing trend.

Although the level of the first data point during the second intervention phase was lower, Maria

was then able to meet the goal twice on 1/17/14 and 1/24/14. The goal was met two more times

during the third intervention phase on 2/14/14 and 2/21/14. The intervention mean was 34.06

(SD=5.61).


Figure 1. Addition scores for Maria

Figure 2 shows the results of the intervention on Maria’s subtraction scores. The level of

Maria’s performance on the subtraction assessment was moderate during baseline and the data

moved in a decreasing trend (M=16; SD=4.95). Based on teacher input during a performance

feedback meeting, the intervention was modified and rather than implementing the C-C-C

intervention, the teacher began implementing a flashcard drill intervention with the group. As

addition and subtraction flashcards were presented during group sessions, Maria began to be

monitored in subtraction at this time. During the first intervention phase, visual analysis of the

data showed a slightly increasing trend. The data moved in an increasing trend with variability

during the third intervention phase and the goal was met once on 2/21/14. The intervention mean

was 25.75 (SD=5.41). The intern continued to collaborate with the teacher throughout the rest of

the year to address the student’s needs. When the student consistently met the subtraction goal,

interventions would be implemented to target multiplication if necessary.


Figure 2. Subtraction scores for Maria

Figure 3 shows the results of the intervention on Maria’s M-COMP scores. The level of

Maria’s performance on the M-COMP assessment was low during baseline and the data moved

in a decreasing trend (M=7.5; SD=2.12). During the Winter benchmark period, after

implementation of intervention strategies to target addition and subtraction skills, Maria achieved

a score of 8 on the M-COMP assessment, which was in the low range. As mentioned previously,

the intern continued to collaborate with the teacher throughout the rest of the year to address the

student’s needs. As data indicated interventions were not effective in improving Maria’s progress

toward 5th grade-level objectives, the intern would discuss whether intervention strategies needed

to be targeted to address other skills as well.


Figure 3. M-COMP scores for Maria

Figure 4 shows the results of the intervention on Chris’ addition scores. The level of

Chris’ performance on the addition assessment was high during baseline and the data moved in

an increasing trend with variability (M=34.67; SD=6.05). He met the goal on three occasions on

10/18/14, 10/22/14, and 11/1/14 and was therefore monitored less frequently in this area. During

the intervention phase, Chris met the goal again on 1/10/14.


Figure 4. Addition scores for Chris

Figure 5 shows the results of the intervention on Chris’ subtraction scores. The level of

Chris’ performance on the subtraction assessment was moderate-to-high during baseline (M=22;

SD=2.29). However, since his scores were below the goal the C-C-C intervention was

implemented. During the first intervention phase, visual analysis of the data showed a variable

trend; however, the first two data points were above the scores Chris achieved during baseline.

Based on teacher input during a performance feedback meeting, the intervention was modified

and rather than implementing the C-C-C intervention, the teacher began implementing a

flashcard drill intervention with the group. During the second intervention phase, the level of

Chris’ performance on the subtraction assessment was lower. Upon implementation of the third

intervention phase, the data moved in an increasing trend with variability and the goal was met

on one occasion on 3/14/14. The intervention mean was 23.96 (SD=3.85). The intern continued

to collaborate with the teacher throughout the rest of the year to address the student’s needs.


When the student consistently met the subtraction goal, interventions would be implemented to

target multiplication if necessary.

Figure 5. Subtraction scores for Chris

Figure 6 shows the results of the intervention on Chris’ M-COMP scores. The level of

Chris’ performance on the M-COMP assessment was low-to-moderate during baseline

(M=22.33; SD=3.79). During the Winter benchmark period, after implementation of intervention

strategies to target addition and subtraction skills, Chris achieved a score of 42 on the M-COMP

assessment, which was in the high range. However, during the Spring benchmark period, Chris

achieved a score of 26. As mentioned previously, the intern continued to collaborate with the

teacher throughout the rest of the year to address the student’s needs. The intern would discuss

whether intervention strategies needed to be targeted to address other skills as well as Spring

data indicated Chris’ level of performance was below the aim line.


Figure 6. M-COMP scores for Chris

Figure 7 shows Kaleb’s addition scores. The level of Kaleb’s performance on the addition

assessment was high during baseline (M=29.13; SD=3.75). He met the goal on four occasions on

10/18/14, 10/22/14, and 11/1/14 and 12/20/14 and was therefore monitored less frequently in this

area.


Figure 7. Addition scores for Kaleb

Figure 8 shows the results of the intervention on Kaleb’s subtraction scores. The level of

Kaleb’s performance on the subtraction assessment was low-to-moderate during baseline

(M=9.67; SD=4.16). During the first intervention phase, visual analysis of the data showed a

variable trend; however, the level of Kaleb’s performance increased compared to baseline and

there were no overlapping data points. Based on teacher input during a performance feedback

meeting, the intervention was modified and rather than implementing the C-C-C intervention, the

teacher began implementing a flashcard drill intervention with the group. During the second

intervention phase, the data moved in an increasing trend; however, the level of Kaleb’s

performance remained the same. Upon implementation of the third intervention phase, the data

were highly variable; however, the majority of the data points were above the aim line. The

intervention mean was 20.68 (SD=3.51). The intern continued to collaborate with the teacher


throughout the rest of the year to address the student’s needs. When the student consistently met

the subtraction goal, interventions would be implemented to target multiplication if necessary.

Figure 8. Subtraction scores for Kaleb

Figure 9 shows the results of the intervention on Kaleb’s M-COMP scores. The level of

Kaleb’s performance on the M-COMP assessment was moderate during baseline and the data

moved in an increasing trend (M=30.33; SD=4.16). After implementation of intervention

strategies to target subtraction skills, Kaleb exceeded the goal on two occasions on 1/10/14 and

2/28/14. His scores were in the high range and the intervention mean was 62.5 (SD=3.53).


Figure 9. M-COMP scores for Kaleb

Figure 10 shows the results of the intervention on Lerrick’s addition scores. The level of

Lerrick’s performance on the addition assessment was moderate during baseline and the data

moved in an increasing trend (M=15.33; SD=1.76). During the first intervention phase, visual

analysis of the data showed Lerrick’s level of performance remained the same; however, during

the third intervention phase the level of his performance was lower and the data were variable.

Upon implementation of the fourth intervention phase, his level of performance increased and

during the fifth intervention phase the data moved in an increasing trend initially; however, then

the data began to move in a decreasing trend. The intervention mean was 16.69 (SD=3.64). The

intern continued to collaborate with the teacher throughout the rest of the year to address the

student’s needs and the team planned to implement a folding-in flashcard intervention to provide

more opportunities to practice and receive feedback on addition skills.


Figure 10. Addition scores for Lerrick

Figure 10 shows the results of the intervention on Lerrick’s subtraction scores. The level

of Lerrick’s performance on the subtraction assessment was low-to-moderate during baseline and

the trend of the data was variable (M=11.33; SD=1.44). During the first intervention phase,

visual analysis of the data showed the data moved in an increasing trend, but Lerrick’s level of

performance remained the same. During the second and third intervention phases, the level of his

performance did not increase and the data remained variable. The intervention mean was 11.1

(SD=3.28). The intern continued to collaborate with the teacher throughout the rest of the year to

address the student’s needs. When the addition goal was met, the team would implement

additional strategies to further target subtraction skills, such as through the computer activity and

folding-in flashcards.


Figure 11. Subtraction scores for Lerrick

Figure 12 shows the results of the intervention on Lerrick’s M-COMP scores. During the

baseline phase, Lerrick achieved a score of 29 on the M-COMP assessment, which was in the

moderate range. During the Winter and Spring benchmark periods, after implementation of

intervention strategies to target addition and subtraction skills, Lerrick’s scores remained in the

moderate range and the intervention mean was 30.5 (SD=4.95). As mentioned previously, the

intern continued to collaborate with the teacher throughout the rest of the year to address the

student’s needs and support his progress toward 4th grade level objectives.


Figure 12. M-COMP scores for Lerrick

Figure 13 shows the results of the intervention on Michael’s addition scores. The level of

Michael’s performance on the addition assessment was moderate during baseline and the data

were variable (M=14.83; SD=2.52). During the first intervention phase, the level of Michael’s

performance remained the same, but the second data point collected during the phase was above

the baseline data scores. Visual analysis of the data during intervention phases two through four

showed the level of Michael’s performance was lower, but the data moved an increasing trend

with variability across the three phases. The intervention mean was 14.62 (SD=2.51). The intern

continued to collaborate with the teacher throughout the rest of the year to address the student’s

needs and the team planned to implement a folding-in flashcard intervention to provide more

opportunities to practice and receive feedback on addition skills.


Figure 13. Addition scores for Michael

Figure 14 shows the results of the intervention on Michael’s subtraction scores. The level

of Michael’s performance on the subtraction assessment was low during baseline and the trend of

the data was variable (M=7.33; SD=1.61). Visual analysis of the data during intervention phases

one through three showed a slightly increasing trend with high variability. The intervention mean

was 11.18 (SD=2.83). The intern continued to collaborate with the teacher throughout the rest of

the year to address the student’s needs. When the addition goal was met, the team would

implement additional strategies to further target subtraction skills, such as through the computer

activity and folding-in flashcards.


Figure 14. Subtraction scores for Michael

Figure 15 shows the results of the intervention on Michael’s M-COMP scores. During the

baseline phase, the level of Michael’s performance on the M-COMP assessment was low and the

data moved in an increasing trend (M=9.33; SD=4.93). During the Winter and Spring benchmark

periods, after implementation of intervention strategies to target addition and subtraction skills,

the level of Michael’s performance was moderate and there were no overlapping data points;

however, the data were below the aim line. As mentioned previously, the intern continued to

collaborate with the teacher throughout the rest of the year to address the student’s needs and

support his progress toward 4th grade level objectives.


Figure 15. M-COMP scores for Michael

Figure 16 shows the results of the intervention on Alex’s addition scores. The level of

Alex’s performance on the addition assessment was moderate during baseline and the data

moved in a decreasing trend (M=10; SD=2.65). During the first intervention phase, the data were

variable, but there was one data point above the aim line, suggesting the C-C-C intervention may

have benefited the student. Visual analysis of the data during the second intervention phase

showed the level of Alex’s performance decreased and the trend was variable. The level and

trend of the data remained similar during the third intervention phase. During the fourth

intervention phase, the level of Alex’s performance again decreased. The intervention mean was

9.1 (SD=2.26). The intern continued to collaborate with the teacher throughout the rest of the

year to address the student’s needs and the team planned to implement a folding-in flashcard

intervention to provide more opportunities to practice and receive feedback on addition skills.


Figure 16. Addition scores for Alex

Figure 17 shows the results of the intervention on Alex’s subtraction scores. The level of

Alex’s performance on the subtraction assessment was low during baseline (M=5; SD=2.12).

Visual analysis of the data during the intervention phases showed the level of Alex’s

performance remained similar and the trend was variable. The intervention mean was 5.96

(SD=1.12). The intern continued to collaborate with the teacher throughout the rest of the year to

address the student’s needs. When the addition goal was met, the team would implement




Figure 17. Subtraction scores for Alex

Figure 18 shows the results of the intervention on Alex’s M-COMP scores. During the

baseline phase, the level of Alex’s performance on the M-COMP assessment was low and the

data were slightly variable (M=10.67; SD=6.43). During the Winter and Spring benchmark

periods, after implementation of intervention strategies to target addition and subtraction skills,

the level of Alex’s performance remained low; however, the intervention mean of 18.5

(SD=2.12) was above the baseline mean. As mentioned previously, the intern continued to

collaborate with the teacher throughout the rest of the year to address the student’s needs and

support his progress toward 3rd grade level objectives.


Figure 18. M-COMP scores for Alex

Figure 19 shows the results of the intervention on Kevin’s addition scores. The level of

Kevin’s performance on the addition assessment low-to-high during baseline and the data were

highly variable (M=12; SD=8.32). During the first intervention phase, the data moved in an

increasing trend and all data points were near the aim line. Kevin was absent from school for a

period of time during the second intervention phase after Winter break. Visual analysis of the

data during this phase showed the level of Kevin’s performance decreased and the trend was

variable. During the third intervention phase, although the data were variable, the level of

Kevin’s performance improved. The intervention mean was 12 (SD=8.32). The intern continued

to collaborate with the teacher throughout the rest of the year to address the student’s needs and

the team planned to implement a folding-in flashcard intervention to provide more opportunities

to practice and receive feedback on addition skills.


Figure 19. Addition scores for Kevin

Figure 20 shows the results of the intervention on Kevin’s subtraction scores. The level

of Kevin’s performance on the subtraction assessment was low during baseline and the data were

variable (M=5.83; SD=3.25). Visual analysis of the data during the intervention phases showed

the level of Kevin’s performance remained low; however, during the second intervention phase

the data began to move in a slightly increasing trend with variability. The intervention mean was

7.38 (SD=3.07). The intern continued to collaborate with the teacher throughout the rest of the

year to address the student’s needs. When the addition goal was met, the team would implement




Figure 20. Subtraction scores for Kevin

Figure 21 shows the results of the intervention on Kevin’s M-COMP scores. During the

baseline phase, the level of Kevin’s performance on the M-COMP assessment was low (M=14.5;

SD=.71). During the Winter and Spring benchmark periods, after implementation of intervention

strategies to target addition and subtraction skills, the level of Kevin’s performance moderate.

Although this was an improvement compared to the baseline phase, the data points were low the

aim line. The intervention mean was 26.5 (SD=4.95). As mentioned previously, the intern

continued to collaborate with the teacher throughout the rest of the year to address the student’s

needs and support his progress toward 4th grade level objectives.


Figure 21. M-COMP scores for Kevin

Summary statistics for each individual student are shown in Tables 7 through 9. Effect

size was calculated by subtracting the baseline mean from the intervention mean then dividing

that number by the baseline standard deviation. PND was calculated by dividing the number of


intervention points. GAS was also determined for the students, where “0” corresponds to no


the goal was met.

Table 7

Summary Statistics for M-COMP

# Baseline Data Points

Baseline Mean

Baseline SD


Intervention Mean

Intervention SD

Effect Size

PND GAS

Maria 2 7.5 2.12 1 NC NC NC 0% 0 Chris 3 22.33 3.79 2 34 11.31 3.08 100% +1 Kaleb 3 30.33 4.16 2 62.5 3.54 7.73 100% +2


Lerrick 1 NC NC 2 30.5 4.95 NC 50% +1 Michael 3 9.33 4.93 2 26 1.41 3.38 100% +1 Alex 3 10.67 6.43 2 18.5 2.12 1.22 50% +1 Kevin 2 14.5 .71 2 26.5 4.95 16.9 100% +1 Note: NC=not calculable

Table 8

Summary Statistics for Addition


Baseline Mean

Baseline SD


Intervention Mean

Intervention SD

Effect Size

PND GAS

Maria 2 31.25 1.06 9 34.06 5.61 2.65 66.67% +2 Chris 3 34.67 6.05 1 NC NC NC 0% +2 Kaleb 4 29.13 3.75 0 NC NC NC NC +2 Lerrick 3 15.83 1.76 13 16.69 3.64 .49 76.92% +1 Michael 3 14.83 2.52 13 14.62 2.51 -.08 15.38% +1 Alex 3 10 2.65 15 9.1 2.26 -.34 6.67% 0 Kevin 3 12 8.32 11 11.14 4.31 -.1 0% 0 Note: NC=not calculable

Table 9

Summary Statistics for Subtraction


Baseline Mean

Baseline SD


Intervention Mean

Intervention SD

Effect Size

PND GAS

Maria 2 16 4.95 10 25.75 5.41 1.97 90% +2 Chris 3 22 2.29 12 23.96 3.85 .86 41.67% +2 Kaleb 3 9.67 4.16 14 20.68 3.51 2.65 100% +1 Lerrick 3 11.33 1.44 10 11.1 3.28 -.16 20% +1 Michael 3 7.33 1.61 11 11.18 2.83 2.39 81.82% +1 Alex 2 5 2.12 12 5.96 1.12 .45 25% +1 Kevin 4 5.83 3.25 7 7.38 3.07 .48 28.58 +1

Discussion

The results of the intervention indicated that three students met the addition goal and two

students met the subtraction goal by the time data were reported for this consultation. Although

only one student met the M-COMP goal, summary statistics indicated the students made progress

toward the goal and their scores improved on average compared to baseline. In addition, social


validity results indicated the teacher “strongly agreed (5)” with five out of six items on the

survey and “agreed (4)” with one item, suggesting she found the intervention valuable.

There were several limitations to this consultation that may have impacted its

effectiveness. Visual analysis of the progress monitoring data indicated the students’ level of

performance was highest during the C-C-C phase of the intervention on average. Therefore, it

would have been beneficial to continue implementing this intervention; however, the

intervention was modified based on teacher feedback that the materials required for the

intervention were difficult to manage. The teacher agreed to implement a flashcard drill activity

with the students as a group. However, implementation adherence data indicated the flashcard

activity was not being implemented with students as consistently as intended, which may have

impacted the students’ progress. Progress monitoring data were below the aim line for four

students, therefore a computer activity was implemented to provide students with more

opportunities to practice and receive feedback on skills. The teacher and intern agreed the

activity would be engaging for the students and more feasible for the teacher to implement in her

classroom. After a performance feedback meeting, the teacher and intern also made plans to

implement a folding-in flashcard intervention with the four students to support their progress

toward goals. Implementation adherence varied throughout the year, however the intern

continually consulted with the teacher in an effort to design interventions that were research-

based and judged as acceptable by the teacher. In the future it will be essential to discuss barriers

to implementation before an intervention is put in place and frequently throughout the

intervention process to make it more likely the intervention will be consistently used.

Students initially practiced one skill at a time (i.e., addition or subtraction) through the C-

C-C intervention. When the intervention was modified, addition and subtraction flashcards were


practiced with the group and all students began to be progress monitored on both skills.

Additional baseline data in subtraction were not collected for students who had previously been

practicing addition only as the teacher desired to begin the flashcard activity immediately with

the group. Therefore, there was a delay between when baseline data in subtraction had been

collected and when progress monitoring began for some students. Ideally, the amount of time

between baseline and intervention phases should be short to analyze whether the intervention

was effective at improving students’ skills. In the future, I will attempt to collect stable baseline

data within a short period of time followed by immediate implementation of interventions to be

able to make this determination.

As mentioned previously, the intervention was intensified for four students to further

support their progress toward goals. Initially, the teacher and intern agreed to target addition

through the intervention. Although progress monitoring data indicated the students’ may have

benefited from the implementation of more intensive interventions to target subtraction as well,

the team agreed that when the addition goal was met, additional strategies would be implemented

to further target subtraction skills as well. Based on research which indicated mastery of basic

math computation facts can support the development of more complex math problems, the team

determined it would be beneficial to target basic math facts through the intervention. Although

the students made progress toward grade-level computational objectives, only one student met

the M-COMP goal by the time data were reported for this consultation. This suggests the

students may have benefited from the implementation of additional intervention strategies to

target other skills necessary to meet grade-level computational objectives. At the Tier 1 level,

strong instructional practices are necessary to provide all students with skills they need in areas


such as math. As a school psychologist, I will collaborate with my school team to ensure this

foundation is in place and that interventions are being implemented to match students’ needs.


components of my model of practice. As a scientist-practitioner, I engaged in data-based

decision making to link assessment, intervention and evaluation. Progress monitoring data were

collected frequently to evaluate intervention effectiveness, and several modifications and

additions to the intervention were made based on data as well as teacher input, including the

implementation of the flashcard drill activity and computer activity in an effort to support student

success. I was able to practice my skills in collaboration as through frequent consultation with

the teacher regarding interventions that encompassed the critical components of effective

academic interventions as well as were feasible for her to implement in her classroom. This was

highlighted through the teacher’s comment that “the teamwork has made doing this feasible in

my room.” In addition to collaborating with the teacher through this consultation, I encouraged

parents to become involved in improving their students’ math skills. For example, parents were

provided with addition and subtraction flashcards to practice with their children at home. Parents

are important to include in the process of helping children learn and as a school psychologist I

will encourage collaboration between the school team and families to support students.


References

AIMSweb Default Cut Scores Explained. (2011). Pearson Education Inc.





Kelly, B. (2008). Best Practices in Curriculum-Based Evaluation and Math. In A.

Thomas & J. Grimes (Eds.), Best Practices in School Psychology V (Vol. 6, pp 419-438).

Bethesda, MD: The National Association of School Psychologists.

Nelson, J.S., Alber, S.R., & Grody, A. (2004). Effects of systematic error correction and

repeated readings on reading accuracy and proficiency of second graders with disabilities.

Education and Treatment of Children, 27, 186-198.

Poncy, B.C., Skinner, C.H., & Jaspers, K.E. (2007). Evaluating and comparing

interventions designed to enhance math fact accuracy and fluency: Cover, copy, and

compare versus taped problems. Journal of Behavioral Education, 16, 27-37



Shapiro, E.S. (2011). Academic Skills Problems (4th edition). New York: The Guilford

Press.

Skinner, C.H., Fletcher, P.A., & Henington, C. (1996). Increasing learning trial rates by

increasing student response rates. School Psychology Quarterly, 11, 313-325.

Skinner, C.H., Pappas, D.N., & Davis, K.A. (2005). Enhancing academic engagement:

Providing opportunities for responding and influencing students to choose to respond.

Psychology in the Schools, 42, 389-403.


Skinner, C. H., Turco, T. L., Beatty, K. L., & Rasavage, C. (1989). Cover, copy, and

compare: An intervention for increasing multiplication performance. School Psychology

Review, 18, 212–220.


Appendices

A. C-C-C Script

B. C-C-C Worksheets

C. Sprint Test

D. Flashcard Drill Activity Script

E. Adherence Checklists

F. Teacher Adherence Form


Appendix A: C-C-C Script

Cover-‐Copy-‐Compare (Math) Script Materials Needed: Paper with relevant math problems solved correctly down the left hand side, “sprint” test with the same math problems unsolved, “pool” of math problems to choose from, pencil, notecard or blank sheet of paper Before CCC Session:

1. Choose math problems from the “pool” of problems 2. Prepare paper by printing math problems and correct answers down the left side 3. Prepare “sprint” test by printing the same math problems on a separate page, unsolved

During the CCC Session:

4. Give each student one sheet of paper, one sprint test, a pencil, and a notecard or blank sheet of paper

5. Direct students to look at the first problem on their list 6. Tell the students to copy the problem in the middle column 7. After students are done copying the problem, tell the students to cover both the printed problem

and the problem they wrote, and write the problem again on the right side of the paper 8. Instruct the students to compare what they wrote with the printed problem on the left side of the

paper 9. If the student’s response is correct, instruct them to continue to the next problem and repeat the

above procedure 10. If the student’s response is incorrect, instruct them to repeat the procedure for the same problem

until it is completed correctly 11. After all students have mastered the use of this method, walk around and monitor the procedure

to make sure it is being done correctly

During the Sprint Test: 12. After the students have finished the CCC worksheet, direct the students to turn the page to begin

the sprint test 13. Tell the students to solve the problems on the page, and then raise their hand when they’re

finished 14. Check the student’s responses, and for problems that are correct, provide positive praise (e.g.,

“Good, 12 + 6 = 18”) 15. For problems that are incorrect, tell the student the correct answer and ask them to repeat it (e.g.,

“12 + 6 = 18. What does 12 + 6 equal? [Student gives right answer] Good.”) After the Session:

16. Mark the problems the student solved correctly and the problems that were solved incorrectly on the sprint test

17. If the student solves a problem correctly for 3 days in a row on the sprint test, drop the problem from the CCC list (and replace with a new problem during the next session)

18. If a student solves a problem incorrectly on the sprint test, leave the problem on the CCC list and continue to practice using the CCC procedure


Appendix B: C-C-C Worksheet


Appendix C: Sprint Test

Sprint Test Student: Date:

1.

2. 3. 4. 5.

6.

7. 8. 9. 10.

Items solved correctly: *After a problem is solved correctly 3 days in a row, remove from list and replace with a new problem Items solved incorrectly: *Keep on next list


Appendix D: Flashcard Drill Activity Script


Appendix E: Adherence Checklists



Appendix F: Teacher Adherence Form

INCREASING THE DURATION

123

Increasing the Duration of Time-In-Class of an Eighth-Grade Student Through the Use of

Check-in/Check-out and Self-Monitoring Procedures

For this consultation, an intervention was created to increase the duration of time-in-class

and decrease the duration of time-out-of-class of an eighth-grade student, Shaun. A teacher

referred Shaun to the school psychologist due to concerns regarding his off-task behaviors,

which often resulted in high amounts of missed instructional time. The teachers reported at times

Shaun would leave class and wander around the building, or engage in disruptive behaviors

during class, which resulted in being asked to take a “time-out” in a separate room. Baseline data

confirmed the duration of time Shaun spent out of class was high. The University of Cincinnati

school psychology intern, school psychologist, and Shaun’s teachers collaboratively decided to

target duration of time-in-class and time-out-of-class for intervention. To monitor Shaun’s

duration of time-in and time-out-of class during the baseline and intervention phases, his teachers

recorded the times he left and returned to class each day due to behavior on a monitoring form

(Appendix A).

The intervention involved antecedent- and consequent-based strategies to increase

Shaun’s duration of time-in-class. In particular, the intervention package incorporated check-

in/check-out, self-monitoring, token economy, and break procedures. Shaun was involved in the

process of monitoring his own behavior and determining the rewards he would earn for meeting

goals. The results of the intervention demonstrated that, on average, Shaun’s percentage of the

day in class increased compared to baseline and his percentage of the day out of class decreased

compared to baseline. Shaun’s duration of time-in-class was variable during the intervention

phase, but he met the goal on 14 occasions. Additionally, the results of the social validity surveys

indicated the teacher and school psychologist “agreed (4)” or “strongly agreed (5)” with all items


124

on the survey, suggesting they felt they had adequate input in developing the intervention script,

they liked the intervention procedures, they would use the intervention again in the future, and

they felt the intervention benefited the student.

Methods


The teacher referred Shaun, a 13-year-old student, for further classroom support due to

the off-task behaviors he displayed throughout the day, which at times resulted in spending the

entire school day out of the classroom. A review of Shaun’s educational records revealed he had

been diagnosed with a moderate Traumatic Brain Injury (TBI) after being involved in a car

accident and suffering a head injury. His records also indicated a TBI can result in impairments

in one or more areas, including: cognition, language, memory, attention, reasoning, abstract

thinking, judgment, problem solving, psychosocial behavior, and information processing.

Furthermore, his records indicate his parent reported a noticed change in Shaun’s behavior after

the incident, such as increased anger and frustration. An intervention was implemented to

increase Shaun’s time-in-class and decrease his time-out-of class. The student also received

additional Tier 2 classroom supports to target reading skills (reported separately). The supports

involved the implementation of the Peer-Assisted Learning Strategies (PALS) program and self-

graphing procedures. Permission to work with Shaun was obtained from his parent, his teacher

and the school psychologist.

The school psychologist and intern school psychologist were responsible for checking-in

with Shaun three days per week to review the daily goal, the behavioral expectations, and how to

complete the self-monitoring form. Shaun was responsible for monitoring his behavior

throughout the day on the self-monitoring form (Appendix B). The teachers were responsible for


125

providing Shaun praise for appropriate behavior throughout the day, monitoring his time-in and

time-out of class on a data-collection form, and checking-out with Shaun at the end of the day to

determine the number of bonus points earned. The assistant teacher was asked to periodically

record Shaun’s duration of time-in and time-out of class along with the lead teacher to collect

inter-observer agreement (IOA) data. The intern received supervision from a field supervisor and

university supervisor.

Setting

This consultation was conducted in a Midwestern public separate facility for students

with disabilities, specializing in intensive supports for students with behavioral and/or mental-

health needs. The intervention was implemented in the school psychologist’s office and middle

school classroom throughout each day.


Through consultation with Shaun’s teachers, the target variables of time-in and time-out

of class were determined to be priority concerns for intervention. Baseline data indicated that on

average, Shaun spent 65% of the school day in class and 35% of the school day out of class;

therefore, a substantial amount of instruction was being missed. The teachers reported at times

Shaun would leave class without permission and wander around the building, ask to take

frequent breaks to leave the classroom, or engage in disruptive behaviors during class that

resulted in being directed to take a break in a separate room. To gather baseline and progress

monitoring data, the teachers recorded each time Shaun left or was removed from class due to

the aforementioned reasons. If Shaun left class with an adult to participate in a school-related

activity, such as to “check-in” with the intern school psychologist or engage in therapy with the

speech-language pathologist, these times were not included in the calculation of time-out-of


126

class. Time-out-of class was recorded during instances when it was not appropriate to be out of

class.

The team agreed it was necessary to target time-in-class for intervention, with the reason

being that increased time-in-class would have a positive effect on other important “keystone”

behaviors, such as engagement and academic skill acquisition. Improvement in student

attendance increases the amount of time students are able to participate in classroom instruction

(Christenson et. al, 2008). Engagement in activities has been associated with school and

subsequent achievement, high school dropout and completion, and physical and emotional well-

being (Blum & Libby, 2004; Christenson et al., 2008). According to Christenson et al. (2008),

the importance of engagement at school is undisputed by educators, and it is clear that

engagement is the “bottom line” in interventions to support school completion.

Prior to intervention implementation, 14 baseline points were obtained. Subsequently,

data were collected on a daily basis during the intervention phase. Throughout each day, the lead

teacher used a data collection form to indicate whether Shaun was in each class for the entire

period, and if the answer was “no”, she also indicated the time he was removed from class and

the time he returned to class. The durations were added to determine the total amount of time-

out-of class daily. To calculate the percentage of time-out-of class daily, the total number of

minutes out of class was divided by the number of minutes in the school day (i.e., 300), and the

quotient was multiplied by 100.


To ensure observation accuracy, once per week the intern asked the assistant teacher to

record the times Shaun was removed from class and returned to class to supplement the data

collected by the lead teacher. Ultimately, inter-observer agreement (IOA) data were collected for


127

3% of days data were collected by the lead teacher (see Table 1). Mean duration-per-occurrence

IOA was calculated for the duration of time out of class (Cooper, Heron, & Heward, 2007). The

two observers recorded the following durations in minutes for time-in-class across seven class

periods on 3/26/14:

P1 P2 P3 P4 P5 P6 P7

Observer 1 45 45 45 45 30 45 45

Observer 2 45 45 45 45 30 45 45

Then, IOA was calculated by (1) calculating the duration per occurrence IOA for each response

(i.e., P1, 45/45=1, P2, 45/45=1, etc.), (2) adding the individual IOA percentages for each

occurrence (i.e., 1+1+…=7) (3) dividing the sum of the individual IOAs per occurrence by the

total number of responses for which two observers measured duration (i.e., 7/7=1), and (4)

multiplying by 100.

Table 1

Inter-observer Agreement Data

Date Percentage of IOA

3/26/14 100%


After consulting with Shaun’s teachers and the school psychologist, it was decided that

the long-term goal of the intervention would be to increase Shaun’s duration of time-in-class to

90 min per day and decrease his time-out-off class to 10 min per day by May 30, 2014. This

would amount to increasing his percentage of time-in-class to 96% per day and decreasing his

percentage of time-out-of class to 4% per day. This goal was derived based upon input from


128

Shaun’s teachers and the school psychologist regarding goals that would be ambitious yet

reasonable.


intervention needed to take place or if the student reached the goal. The decision rule for time-

out-of class was four points above the aimline when the intervention was being implemented

correctly (Hixson, Christ, & Bradley-Johnson, 2008). At this point, the teachers, intern and

school psychologist agreed a change or modification would need to be made to address Shaun’s

needs. If the long-term goal was consistently met, the intervention would be faded, such as by

increasing the criteria to earn bonus points, providing rewards less often and offering more

natural incentives for meeting the goal.


Based on teacher interview, the team hypothesized the functions of Shaun’s behavior

were to obtain attention and/or escape the situation. When Shaun received a redirection or

reprimand, he often responded by engaging in off-task behavior, walking out of the classroom, or

asking for a break and consequently received additional adult attention and was able to

avoid/escape the situation. Specifically, Shaun typically received additional redirections and was

able to spend time in a separate classroom where he received one-on-one attention from a

teacher. The competing behavior pathway can be used to develop strategies to make undesired

behavior irrelevant, inefficient and ineffective (Crone & Horner, 2003). For this consultation, the

competing behavior pathway form was completed to aid in generating strategies to increase

Shaun’s time-in-class (Appendix C).

Accountability Plan


129

An AB design was used to examine the effects of the intervention on Shaun’s level of

time-in and time-out of class. Fourteen baseline points were obtained prior to intervention

implementation (A). Subsequently, additional supports were implemented; in particular, the

intervention incorporated check-in/check-out, self-monitoring, token economy, and break

procedures (B). The target variable was measured daily until Shaun consistently reached the

goal.


Baseline condition. Prior to introduction of the intervention, the intern asked the teachers

to collect baseline data on Shaun’s time-in and time-out of class throughout the day. The teachers

used a data collection form (Appendix A) to record his duration of time-out-of class on 14

occasions prior to intervention implementation. Baseline data are displayed and discussed in the

results section below.

Check-in/check-out. Check-in/check-out (CICO) is an intervention designed to increase

prosocial behavior and reduce incidences of problem behavior (Campbell & Anderson, 2008;

Ennis, Jolivette, Swoszowski, & Johnson, 2012). CICO consists of (a) a short, positively focused

meeting with an adult at the beginning and end of the day to set goals for the day and review how

the day went, (b) a point card on which, at predetermined times, teachers allot points for meeting

defined behavior goals and provide feedback to the student, and (c) tangible and intangible

rewards for earning a predetermined number of points (Campbell & Anderson, 2008; Ennis et

al., 2012). CICO likely will be most successful for students whose behavior is maintained by

adult attention since the procedure focuses on providing adult attention for appropriate behavior

(Campbell & Anderson, 2008; Ennis et al., 2012). Research has demonstrated that CICO has the

ability to increase desired behavior and decrease problem behavior; furthermore, positive


130

outcomes were evidenced when the intervention took into account outcomes of functional

behavior assessments (Campbell & Anderson, 2008). A study that evaluated the efficacy of

CICO in an alternative residential setting revealed the disruptive behavior of attention-motivated

students decreased during the intervention phase (Ennis et al., 2012).

A CICO procedure was implemented as one component of the intervention package to

increase Shaun’s duration of time-in-class (Appendix D). Three days per week, Shaun would

check-in with the intern school psychologist and school psychologist for approximately 10 min.

During this time, several topics were reviewed with Shaun, including: behavioral expectations,

the daily goal, the number of points earned for meeting the daily goal, and the number of points

needed to earn a reward. Baseline data indicated that on average, Shaun’s time-out-of class was

103 min daily; Shaun’s time-out-of class ranged from 5 min per day to 300 min per day. To

ensure Shaun would come in contact with reinforcement, a daily goal was set for each class

whereby if Shaun missed no more than 15 min of each class, he would be awarded 200 bonus

points. Given there were seven class periods, missing 15 min of each class amounted to 105 min

per day. If Shaun exceeded the goal by remaining in class for the entire period, he was awarded

an additional 100 bonus points. Shaun provided feedback regarding rewards he would like to

earn and the intern, school psychologist, and Shaun discussed the number of bonus points needed

to earn particular rewards. Shaun monitored whether he met the goal for each class period on a

self-monitoring form (Appendix B). Throughout the day, the lead teacher indicated the times

Shaun left and returned from class on a data collection form (Appendix A). Shaun would check-

out with his teachers at the end of each day, and during this time the teacher would compare the

data she collected with Shaun’s self-monitoring form, discuss any discrepancies, and sign

Shaun’s form to indicate she reviewed his progress with him.


131

Self-monitoring. According to Cooper et al. (2007), “self-monitoring is a procedure

whereby a person observes his behavior systematically and records the occurrence or

nonoccurrence of a target behavior” (p. 590). Throughout the intervention, the student and

teacher complete separate rating forms during a class and at the conclusion ratings are compared

for agreement, and students are allowed to earn points for behavioral improvement and accurate

recording (Denzer, 2013). Self-monitoring produces reactive effects and often changes behavior

in the desired direction (Cooper et al., 2007). Research has evidenced self-monitoring

interventions to be effective among students with off-task behavior. For instance, a study by

Denzer (2013) utilized a self-monitoring intervention with an older student whereby at the

beginning of each class the student read his behavioral goals, and at the end of the class he

completed his self-monitoring checklist by indicating whether he followed each goal. The

student then asked his teacher to review his self-assigned points and sign off on whether they

agreed. The student was rewarded with a highly preferred tangible. Results of the study revealed

a positive trend in overall time on-task per class period.

Self-monitoring was utilized as part of the intervention to increase Shaun’s time-in-class.

The daily goal was indicated at the top of Shaun’s self-monitoring form (Appendix B). At the

end of each class period, Shaun marked whether the goal was met, exceeded or not met. Each

check mark was worth a certain number of bonus points, and this value was indicated at the

bottom of self-monitoring form. As mentioned previously, at the end of the day the teacher

would compare Shaun’s self-monitoring form with the data she collected and sign Shaun’s form

to indicate she reviewed his progress with him. During the next check-in time, the intern would

review the number of bonus points Shaun earned on previous days, provide praise, and determine


132

whether he met the criteria to earn his reward. Self-monitoring is often part of an intervention

package that includes reinforcement for meeting goals (Cooper et al., 2007).

Token economy. As evidenced from through the discussion above, a token economy is

typically incorporated into check-in/check-out and self-monitoring procedures. The intervention

involves contingencies in which tokens or points are given following the emission of targeted

behaviors, and tokens are redeemed for reinforcing objects or activities at a later point in time

(Zlomke & Zlomke, 2003). A study which implemented token economy and self-monitoring

components with a 13-year-old male demonstrated that a token economy alone, as well as a

token economy combined with self-monitoring significantly reduced the student’s disruptive and

aggressive classroom behaviors (Zlomke & Zlomke, 2003). Shaun received bonus points for

meeting his goal each class period, and when he earned a certain number of points, he earned a

reward. Shaun was involved in determining the number of bonus points needed to earn rewards

as well as rewards he would like to work for. For example, Shaun used bonus points to earn a

basketball and lunch from his favorite restaurant.

Break procedure. Teaching rules is an antecedent-based strategy to support appropriate

behavior (DuPaul & Stoner, 2010). A study by Johnson, Stoner and Green (1996) found that

active teaching of class rules was effective at increasing the appropriate behavior and decreasing

the inappropriate behavior of students. Teacher interview revealed Shaun often left the classroom

without permission or engaged in disruptive behaviors during class, both of which permitted him

to escape the activity and/or situation. A script was developed to provide direct instruction in the

skills necessary to ask for a break and re-enter the classroom appropriately, as well as the

expectations while in the break room (Appendix E). The intern reviewed these expectations with

Shaun during the check-in time. Shaun was also provided with a visual reminder of these


133

expectations, which he carried with him in a folder throughout the day. If Shaun took the

appropriate steps to ask for a break and followed the expectations while in the break room for at

least 5 min, he was permitted to return to class if he was ready. On the other hand, if Shaun left

the classroom without permission he was permitted to return to class after following the

expectations in the break room for 20 min. The team hypothesized the opportunity to return to

class sooner by demonstrating appropriate behavior would be rewarding for Shaun, as being out

of the classroom for 5 min still allowed him to meet his goal for the class period.

Feedback conditions. Graphs were shared with the team to demonstrate Shaun’s

progress and make decisions based on data. If Shaun experienced four points above the aim line,

the intern reviewed the graph with the teacher and discussed whether the intervention needed to

be modified. Upon intervention implementation, progress monitoring data revealed Shaun was

able to meet the goal on several occasions; however, data collected on 3/11/14, 3/12/14, 3/13/14

and 3/14/14 were above the aim line, therefore a performance feedback meeting was scheduled

to occur with the teacher after Spring Break.

Adherence Data

Adherence to the intervention was assessed by the intern by using a procedural checklist

based on the essential components of the intervention script and checking off tasks as they were

completed for 44.44% of the intervention sessions (see Table 2). See Appendix F for the

adherence checklist used by the intern during Shaun’s check-in time. To monitor adherence to

the self-monitoring component of the intervention package, the intern collected the self-

monitoring forms Shaun completed during the check-in time. The intern used this information to

calculate the percentage of days Shaun monitored his own behavior, which amounted to 63.16%


134

of days. If adherence was low, the intern reviewed the procedures with Shaun and discussed

other strategies to increase adherence (e.g., placed a visual reminder on Shaun’s desk).

Table 2

Adherence Data

Date Adherence

1/31/14 100%

2/4/14 0% (R)

2/12/14 100%

2/14/14 0% (R)

2/21/14 75%

2/25/14 75%

2/28/14 75%

3/5/14 75%

3/12/14 75%

3/24/14 75%

3/25/14 75%

3/26/14 75%

*Note: R=Student Refused

Social Validity

The intern designed the intervention procedures in collaboration with Shaun, his teacher,

and the school psychologist, and involved parent permission to work with Shaun. Social validity

was collected throughout the intervention process through frequent conversation between intern,

teachers, Shaun, and other individuals in the school building (e.g., school psychologist, speech


135

language pathologist, principal). In addition, the teacher and school psychologist completed

social validity questionnaires to indicate their satisfaction with the intervention and its

effectiveness, which can be seen below.

Table 3 demonstrates the results of the social validity forms filled out by the teacher and

school psychologist. The numbers in the cells represent the number of individuals who chose that

rating. The teacher and school psychologist “agreed (4)” or “strongly agreed (5)” with all the

items on the survey, indicating they found the intervention valuable.

Table 3


Strongly Disagree

(1)

Disagree (2)

Not Sure (3)

Agree (4)

Strongly Agree (5)

Mean Rating

Standard Deviation


- - - - 2 5 0


- - - 1 1 4.5 .5


- - - - 2 5 0


- - - 1 1 4.5 .5

I would be willing to use this

- - - - 2 5 0


136

intervention in the future


- - - - 2 5 0

Results

Individual baseline and progress monitoring data for duration of time-in and time-out of

class, and percentage of the day in and out of class, are displayed below in Figures 1 through 4.

In support of visual analysis, summary statistics were calculated in Table 4. The mean (M) and

standard deviation (SD) of the target variable were calculated. The effect size, percentage of non-

overlapping data (PND), and goal attainment scaling (GAS) were also calculated. For the purpose

of this consultation entry, data were reported through 3/26/14. However, the intervention

continued to be implemented throughout the rest of the year and data were collected on student

progress until the end of the year or until the student consistently met the goal.

Figure 1 demonstrates the results of the intervention on the duration of time-out-of class

daily in minutes. Shaun’s duration of time-out-of class was highly variable during baseline. The

level of his duration of time-out-of class ranged from low-to-high (M=103.57; SD=93.83).

During the intervention phase, visual analysis showed the data were still variable. However, the

level of Shaun’s duration of time-out-of class was lower on average (M=37.17; SD=50.51).

Shaun met the goal on 14 occasions from 1/29/14 through 3/26/14. The variability in Shaun’s

duration of time-out-of class suggests a modification to the intervention may be beneficial. The

intern continued to collaborate with the teacher throughout the rest of the year to address Shaun’s

needs until he consistently met the goal.


137

Figure 1. Duration of time out of class (minutes)

Figure 2 demonstrates the results of the intervention on the duration of time-in-class daily

in minutes. Shaun’s duration of time-in-class was highly variable during baseline. The level of

his duration of time-in-class ranged from low-to-high (M=187.86; SD=92.06). During the

intervention phase, visual analysis showed the data were still variable. However, the level of

Shaun’s duration of time-in-class was higher on average (M=258.69; SD=59.53). Shaun met the

goal on 14 occasions from 1/29/14 through 3/26/14.


138

Figure 2. Duration of time in class (minutes)

Figure 3 demonstrates the results of the intervention on the percentage of the day out of

class. Typically, each day consisted of 300 min of class time. However, the school was on a two-

hour delay on 1/23/14 and 2/11/14; therefore, these days consisted of 180 min of class time. The

percentage of the day Shaun was out of class was highly variable during baseline. The level of

his percentage of the day out of class ranged from low-to-high (M=35; SD=30.92). During the


Shaun’s percentage of the day out of class was lower on average (M=12.93; SD=17.70). Shaun

met the goal on 14 occasions from 1/29/14 through 3/26/14. As mentioned previously, the

variability in Shaun’s percentage of the day out of class suggests a modification to the

intervention may be beneficial and the intern continued to collaborate with the teacher

throughout the rest of the year to address Shaun’s needs until he consistently met the goal.


139

Figure 3. Percentage of day out of class

Figure 4 demonstrates the results of the intervention on the percentage of the day in class.

The percentage of the day Shaun was in class was highly variable during baseline. The level of

his percentage of the day in class ranged from low-to-high (M=65; SD=30.92). During the


Shaun’s percentage of the day in class was higher on average (M=87.07; SD=17.70). Shaun met

the goal on 14 occasions from 1/29/14 through 3/26/14.


140

Figure 4. Percentage of day in class

Summary statistics for the target variables are shown below in Table 4. Effect size was

calculated by subtracting the baseline mean from the intervention mean then dividing that

number by the baseline standard deviation. PND was calculated by dividing the number of




the goal was met.

Table 4

Summary Statistics for Time-In and Time-Out of Class, and Percent of Day In and Out of Class


Baseline Mean

Baseline SD


Intervention Mean

Intervention SD

Effect Size

PND GAS


141

Discussion

The progress monitoring data indicate Shaun’s time-in-class increased compared to

baseline and his time-out-of class decreased compared to baseline on average, suggesting the

intervention procedures had a positive effect on the student. Although data were variable during

the intervention phase, Shaun met the goal on 14 occasions by the time data were reported for

this consultation. Social validity results suggest the team liked the intervention procedures and

felt they benefited the student. Additionally, key stakeholders (e.g., principal, speech and

language pathologist, school psychologist) reported they noticed an overall positive difference in

Shaun’s behavior upon implementation of the intervention. For instance, members of the team

reported he engaged in more appropriate interactions with adults and peers in the classroom and

cafeteria; therefore, it is possible the intervention had a positive impact on other important

behaviors as well.


effectiveness. One such limitation is related to the variable used to monitor progress. The amount

of time-in-class is a meaningful target as increased time-in-class can have a positive effect on

Time-in Class

14 187.85 92.06 29 258.69 59.53 .77 34.48%

+2

Time-out-of Class

14 103.57 93.83 29 37.17 50.51 .71 34.48%

+2

% Day in Class

14 65 30.92 29 87.07 17.70 .71 34.48%

+2

% Day Out of Class

14 35 30.92 29 12.93 17.70 .71 34.48%

+2


142

other important “keystone” behaviors, such as engagement and academic skill acquisition. The

team collaboratively decided time-in-class was necessary to target through intervention.

However, it is important to acknowledge that the measure has questionable reliability. The

teacher was asked to record each time the student left class and returned to class throughout the

day, but it is possible the teacher did not accurately record every instance the student was out of

class, which could have affected the level and trend of the data during the baseline and

intervention phases. Systematic direct observation of behavior is a valid and reliable method of

measuring student responsiveness to intervention, and in the future I will utilize this method

whenever possible to analyze student behavior. Despite this limitation, the data collected by the

teacher in combination with the reports provided by several key stakeholders suggest the

intervention benefited the student.

To more confidently ensure observation accuracy, IOA data should have been collected

for 20-25% of days data were collected. The assistant teacher was asked to record the times

Shaun was removed from class and returned to class to supplement the data collected by the lead

teacher. Although the intern continued to re-iterate this request, ultimately IOA data were only

collected for 3% of days data were collected. Due to health reasons, the lead teacher was absent

from school for several weeks and the assistant teacher was the only adult who was with Shaun

throughout the entire school day. Therefore, this impacted the amount of IOA data that were

collected. As a school psychologist, I will attempt to collect IOA data as often as possible to

ensure observation accuracy, which is necessary when analyzing student response to

intervention.

The information used to set goals could have been strengthened. “Best practice” claims

choosing goals based on peer-comparison micro-norms is an ideal goal setting method. Goals


143

were set based through consultation with Shaun’s teachers and the school psychologist based

upon the amount of time-in-class the team deemed to be meaningful and reasonable. However,

upon reflection, it would have been beneficial for the teacher to consistently record peers’ time-

out-of class as well to aid in setting goals. In the future I will ensure information regarding peer

norms is collected to develop appropriate goals and analyze student progress compared to peers.

It is important to note that data collected on 3/11/14, 3/12/14, 3/13/14 and 3/14/14 were

above the aim line; therefore, a performance feedback meeting was scheduled to occur with the

teacher after Spring Break. Although it would have been idea for a performance feedback to

occur during the week of 3/17/14, the lead teacher was absent from school until the following

week and the team felt it would be beneficial for her to be able to attend the meeting as well. The

team planned to implement a video modeling procedure to provide Shaun with additional

modeling and practice regarding the expected behaviors while in class and how to leave the

classroom appropriately.

Another limitation was that the intern was the individual responsible for checking-in with

Shaun three days per day. School psychologists typically have other obligations, which could

interfere with the quality or intensity of intervention implementation. Therefore, it is ideal for

school psychologists to serve consultative roles rather than serve as interventionists. In the future

as a school psychologist, I will attempt to find more appropriate individuals to implement

interventions.

Finally, adherence was not monitored for the break procedure as part of this consultation

as the times Shaun requested breaks were unpredictable. Upon reflection, in the future it would

be beneficial to ask a teacher to collect adherence during similar situations. Implementing

interventions consistently and as intended are essential in order for plans to be effective and to


144

accurately analyze intervention effectiveness. Therefore, in the future I will do my best to

conduct adherence checks through direct observation for 20-25% of intervention sessions.

However, I will also ask teachers to collect adherence to support this information when

necessary.

This consultation was a great opportunity for me to demonstrate, practice and develop

components of my model of practice. I served as an advocate for the child’s needs by consulting

with teachers and the school psychologist to design an intervention to support the student.

Additionally, the child was included in the process of designing his self-monitoring form,

monitoring his own behavior, selecting the criteria needed to earn rewards, and identifying

rewards that would be reinforcing for him. By collaborating with the student, the team was able

to incorporate his input into the intervention process and ensure the targets and goals were

meaningful to him as well.

I utilized an ecological-behavioral approach to providing services by referring to the

competing behavior pathway to analyze factors in the environment affecting Shaun’s behavior.

Furthermore, this information was utilized with the aim of developing strategies to make

undesired behavior irrelevant, inefficient and ineffective and increase Shaun’s time-in-class. As a

scientist-practitioner, I researched evidenced-based strategies to support Shaun based on the

hypothesized function of his behavior. Data were used to monitor Shan’s progress and analyze

intervention effectiveness. Future plans were in place to increase the intensity of the intervention

by providing additional modeling and practice regarding the expected behaviors while in class

and how to leave the classroom appropriately.


145

References

Blum, R.W., & Libby, H.P. (Eds.). (2004). School connectedness: Strengthening health and

education outcomes for teenagers. Special issue, Journal of School Health, 74(7), 229-

299

Campbell, A., & Anderson, C.M. (2008). Enhancing effects of check-in/check-out with function

based support. Behavioral Disorders, 33(4), 233-245

Christenson, S.L., Reschly, A.L., Appleton, J.J., Berman-Young, S., Spanjers, D.M., & Varro, P.

(2008). Best practices in fostering student engagement. In A. Thomas & J. Grimes (Eds.),

Best Practices in School Psychology V (Vol. 4, pp 1099-1119). Bethesda, MD: The

National Association of School Psychologists.

Cooper, J.O., Heron, T.E., & Heward, W.L. (2007) Applied Behavior Analysis (2nd Ed.). Upper

Saddle River, NJ: Pearson.

Crone, D.A., & Horner, R.H. (2003). Building Positive Behavior Supports in Schools:

Functional Behavior Assessments. New York: The Guilford Press.

Denzer, A. Q. (2013). Utility of a self-monitoring intervention in reducing off-task behavior. The

Ohio School Psychologist, 59(1), 4-6

DuPaul, G.J., & Stoner, G. (2010). Interventions for Attention-Deficit Hyperactivity Disorder. In

M.R. Shinn & H.M. Walker (Eds.), Interventions for Achievement and Behavior

Problems in a Three-Tier Model Including RTI (pp 825-848). Bethesda, MD: The


Ennis, R.P., Jolivette, K., Swoszowski, N.C., & Johnson, M.L. (2012). Secondary prevention

efforts at a residential facility for students with emotional and behavioral disorders:

Function-based check-in, check-out. Residential Treatment for Children & Youth, 29, 79-

102


146

Hixon, M., Christ, T.J., & Bradely-Johnson, S. (2008). Best practices in analysis of progress

monitoring data and decision making. In A. Thomas & J. Grimes (Eds.), Best Practices in

School Psychology V (Vol. 6, pp 2133-2146). Bethesda, MD: The National Association

of School Psychologists.

Johnson, T.C., Stoner, G., & Green, S.K. (1996). Demonstrating the experimenting society

model with classwide behavior management interventions. School Psychology Review,

25(2), 199-214.

Zlomke, K., & Zlomke, L. (2003). Token economy plus self-monitoring to reduce disruptive

classroom behaviors. The Behavior Analyst Today, 4(2), 177-182


147

Appendices

A. Data-collection form

B. Self-monitoring form

C. Competing behavior pathway form

D. Intervention script

E. Script for leaving/entering the classroom

F. Adherence checklist


148

Appendix A: Data Collection Form

Time Out of Class

Please record when student is removed from class due to his behavior (by either walking out, asking for a phase 2, being asked to take a phase 2, etc.) Date: Period/Subject In Class

Entire Period? (Y/N)

Time Removed

Time Returned

Notes Participated in Social Skills? (Y/N)

1. (8:30-9:15)

Y

N

Y

N

2. (9:15-10:00)

Y

N

3. (10:00-10:45)

Y

N

4. (10:45-11:30)

Y

N

5. (11:30-12:00)

Y

N

6. (1:00-1:45)

Y

N

7. (1:45-2:30)

Y

N


149

Appendix B: Self-Monitoring Form

DAILY GOAL: Stay in each class. Miss no more than 15 minutes each class.

GOAL MET? (✓ one)

Class Period

Yes

No

In class the Whole Time?

1 (8:30-9:15)

____________________

2 (9:15-10:00)

____________________

3 (10:00-10:45)

____________________

4 (10:45-11:30)

____________________

5 (11:30-12:00)

____________________

6 (1:00-1:45)

____________________

7 (1:45-2:30)

____________________

Bonus points earned today:

____________________

10,000 bonus pts=reward

Yes ✓= 200 bonus pts.

*Try to earn 10,000

5 days in a row

Stayed in class the whole time = 100 bonus pts.


150

Appendix C: Competing Behavior Pathway Form


151

Appendix D: Intervention Script


152

Appendix E: Script for Leaving/Entering Classroom


153

Appendix F: Adherence Checklist

IMPROVING THE READING

154

Improving the Reading Skills of Middle School Students with Emotional and Behavioral

Disorders Through the Use of Peer Assisted Learning Strategies and Self-Graphing

Co-Authored by Lauren McKinley

A Tier 2 academic intervention was implemented in a middle school classroom with nine

students. The classroom was in Midwestern public separate facility for students with disabilities,

specializing in intensive supports for students with behavioral and/or mental-health needs. The

purpose of the intervention was to increase students’ scores on oral reading fluency (ORF) and

Maze reading comprehension probes. The students were targeted for intervention due to the low

scores they received compared to Spring benchmark goals for their grade-level. AIMSweb

curriculum-based measurements (CBMs) were used to measure the targeted students’

responsiveness to intervention two times per week. During the first phase of the intervention, a

self-graphing procedure was implemented with five students to support progress on ORF and

Maze assessments. The team also intended to implement the Peer-Assisted Learning Strategies

(PALS) intervention with all students to further support student progress toward ORF and Maze

goals after students’ scores during the self-graphing phase of the intervention were stable.

Although there were nine students targeted for intervention in the middle school

classroom, the self-graphing intervention had only been implemented with five students by the

time data were reported for this consultation. The intervention had not been implemented with

the remaining four students by 3/28/14 because the students had either been absent or had not yet

achieved a “stable” baseline. Therefore, individual data are only presented for the students who

had begun the intervention. The results of the intervention indicated that 40% of students (2 out

of 5) made progress toward the ORF and Maze goals by the time data were reported for this


155

consultation. Based on teacher report, students enjoyed the self-graphing procedures

implemented and were excited about the ability to visually see their progress.

Methods


There was one classroom of middle school students in the school building. The classroom

consisted of nine students between the ages of 12 and 15. Of the nine students, two were in sixth

grade, six were in seventh grade, and one was in eighth grade. The group consisted of five boys

and four girls. All students were identified as a child with a disability. Eight students were

identified under the category of Emotional Disturbance and one student was identified under the

category of Traumatic Brain Injury. All nine students were targeted for Tier 2 and progress

monitored based on baseline scores. Parent permission to work with the students and include

data in the intern’s report was obtained from all nine students.

Teachers were responsible for implementing the intervention with the students in their

classroom as Tier 2 supports. The classroom had one lead teacher, one assistant teacher and one

reading specialist. University of Cincinnati school psychology practicum students assisted with

the implementation of the self-graphing component of the intervention. The graduate students

also assisted in conducting intervention adherence checks, progress monitoring, and collecting

inter-scorer agreement (ISA) data. The school psychology advanced doctoral student and school

psychology intern assumed consultative roles throughout the intervention process, and also

assisted in progress monitoring and conducting intervention adherence checks while the

intervention was in place. The intern school psychologist received supervision from a field

supervisor and university supervisor.

Setting


156

This consultation was conducted in a middle school classroom at a Midwestern public

separate facility for students with disabilities, specializing in intensive supports for students with

behavioral and/or mental-health needs. The school administratively supported multi-tiered

systems of support (MTSS) procedures, which allowed the graduate students to work with the

teachers to address the students’ needs. The intervention was implemented in the classroom

during the Language Arts period. The self-graphing component of the intervention was

implemented in the school psychologist’s office.


According to Joseph (2008), reading is one of the most fundamental skills that permit a

person to survive and thrive in society, and one of the main goals of education is to help students

move from the learning-to-read stage to a reading-to-learn stage. Yet, some children have

difficulty acquiring the fundamental reading skills necessary to read-to-learn. The critical

component skills of reading are phonemic awareness, alphabetic principle, fluency, vocabulary,

and comprehension.

Oral reading fluency and reading comprehension. As fluency and comprehension are

two of the five critical component skills of reading, these skills can be used to predict reading

proficiency. According to Joseph (2008), fluent readers can read words accurately, effortlessly,

and quickly. Reading fluency is related to reading comprehension performance, which refers to

the ability to understand and derive meaning from text. Reading text fluently is a skill critical to

comprehending text (Sutherland & Snyder, 2007). Therefore, it is likely that improving reading

fluency skills will support reading comprehension. AIMSweb is a valid and reliable curriculum-

based measure and was therefore used to measure reading fluency and comprehension skills as

part of this consultation (Joseph, 2008).


157

AIMSweb oral reading fluency (ORF) and Maze reading comprehension probes were

used to measure the targeted students’ responsiveness to intervention. Baseline data in ORF and

Maze were collected beginning in February 2014. All students were below the Spring benchmark

goal for their grade-level and were therefore targeted for intervention and received additional

progress monitoring (AIMSweb, 2013). At least three baseline data points were collected for

each student; however, baseline data continued to be collected for each student until he or she

achieved a “stable” baseline. In order to assess student progress toward goals and evaluate

intervention effectiveness, ORF and Maze progress monitoring probes were administered to the

students during the intervention phases.

AIMSweb Survey Level Assessment (SLA) procedures were used to identify each

student’s instructional level. In addition progress monitoring students using ORF probes at their

grade-levels, students were progress monitored using ORF probes at their respective instructional

levels as well, if applicable. To determine a student’s instructional level, the student was first

administered a probe at his or her grade level. If the student’s score was below the 25th percentile

when compared to Winter norms, the child was administered a probe at the grade level below

their actual grade level. This process was continued until the student achieved a score at or above

the 25th percentile. The grade level passage at which this occurred was considered to be the

student’s instructional level. This information was also used to assess student progress toward

goals and evaluate intervention effectiveness.

The students’ progress was individually tested using the AIMSweb ORF progress

monitoring probes two times per week. This occurred using progress monitoring probes at the

students’ grade-levels and instructional levels. ORF referred to the number of words read

correctly on each 1 min assessment (AIMSweb, 2013). A response was marked correct if the


158

student read the word correctly, pronounced the word correctly, and read the words in the correct

order; if the student initially mispronounced or omitted a word, but self-corrected within 3 s; and

if the student pronounced a word, but had a common speech problem such as a lisp. A word was

marked as an error if the student mispronounced or substituted a word, omitted or skipped the

word, hesitated or struggled to correctly produce the word for more than 3 s, or transposed the

order of two words. The examiner calculated the number of words read correctly within 1 min.

The students’ progress was also individually tested using the AIMSweb Maze progress

monitoring probes two times per week. The Maze measure is used to examine students’

understanding of content and vocabulary (AIMSweb, 2013). The student is presented with a

grade-level passage with every seventh word replaced with three words in parenthesis. The

student reads for 3 min and selects the one correct word in parenthesis as they progress through

the passage. After 3 min, the number of correctly selected words in the passage is calculated.


The intern, doctoral student and practicum students assisted in administering ORF and

Maze probes to collect baseline and progress monitoring data. Inter-scorer agreement data for

were collected for 20% of measurement occasions for ORF and 11% of measurement occasions

for the Maze (Tables 1 and 2). One graduate student assessed the reliability of measurement by

monitoring the administration of probes and recording correct responses along with another

graduate student to collect inter-scorer agreement. Inter-scorer agreement was calculated by

dividing the number of agreements by the total number of agreements and disagreements and

multiplying by 100 (Reed & Azulay, 2010).

Table 1

Inter-Scorer Agreement Data for ORF


159

Date Inter-Scorer Agreement (ORF)

2/20/14 Student 1: 100%

Student 2: 99.13

Student 3: 99.44%

Student 4: 100%

3/6/14 Student 1: 100%

Student 2: 100%

Student 3: 100%

Student 4: 100%

Student 5: 100%

3/13/14 Student 1: 99.2%

Student 2: 100%

Student 3: 100%

Student 4: 99.2%

3/20/14 Student 1: 99%

Student 2: 99%

3/27/14 Student 1: 100%

Student 2: 100%

Student 3: 100%

Student 4: 100%

Student 5: 100%

Table 2


160

Inter-Scorer Agreement for Maze

Date Inter-Scorer Agreement (Maze)

3/10/14 Student 1: 100%

Student 2: 100%

Student 3: 91.67%

Student 4: 100%

3/24/14 Student 1: 100%

Student 2: 100%

Student 3: 100%


The goal of the academic intervention was to increase the score on the ORF assessment

to 195 words per minute for sixth grade students, 190 words per minute for seventh grade

students, and 185 words per minute for the eighth grade student. Goals were also set to increase

the score on the Maze assessment to 35 responses correct for sixth grade students, 39 responses

correct for seventh grade students, and 36 responses correct for eighth grade students.

Additionally, goals were set to increase students’ scores on the ORF assessment for their

respective instructional levels. Goals were set based on AIMSweb national norms representing

the scores received by the 75th percentile of students during the Spring benchmark period for

grades six through eight. For two AIMSweb measures (including the R-CBM), scores at the 45th

percentile have been found to be predictive of success on state tests (AIMSweb Default Cut

Scores, 2011). The intervention goals were set higher to ensure students acquired the skills

necessary to become successful readers.


161

Visual analysis of the progress monitoring data was used to determine whether a student

reached the goal or if a change in intervention was needed for a student. The decision rule for

change in intervention was three to four consecutive points below the aim line when the


this point, the team agreed a change would need to be made to address the student’s needs. If

data indicated adherence to the intervention was low, teacher feedback and additional training in

intervention procedures were provided.


It was hypothesized that the targeted students received low scores on ORF and Maze

assessments because they had not been given enough opportunities to practice reading fluency

and reading comprehension skills. Thus, the students had also not received enough feedback,

error correction, or reinforcement to support skill development. The intervention involved

providing students with additional opportunities to practice and receive feedback on reading

fluency and reading comprehension skills. Additionally, a self-management component was

implemented to support reading skill improvement.

Accountability Plan

An AB design was used to examine the effects of the interventions on the students’

performance on ORF and Maze assessments. At least three baseline data points were collected

for each student on each target variable prior to intervention implementation; however, baseline

data continued to be collected for each student until he or she achieved a “stable” baseline (A).

Subsequently the self-graphing component of the intervention was implemented (B). A third

phase would involve the implementation of Peer Assisted Learning Strategies (PALS) (C). The


162

third phase of the intervention was implemented beginning during the week of 4/7/14; however,

for the purpose of this consultation entry data were only reported through 3/28/14.


Baseline condition. Prior to introduction of the intervention, baseline data were collected

on each student’s performance on AIMSweb ORF and Maze assessments. As mentioned

previously, at least three data points were collected for each assessment variable. Baseline data

were collected for each student until he or she achieved a “stable” baseline. Baseline data are

displayed and discussed in the results section below.

Self-graphing. The self-graphing component of the intervention was implemented for

students two days per week for 5 to 10 min after a “stable” baseline was achieved. This

component of the intervention was aimed at supporting students’ progress on ORF and Maze

assessments. Self-graphing involves students graphing their own data, which allows individuals

to see a visual representation of their performance over time in comparison to established criteria

(Sutherland & Snyder, 2007). Research has suggested reading performance improves when self-

monitoring methods are used (Joseph & Eveleign, 2011). Furthermore, based on research that

self-graphing resulted in additional increases in academic performance, self-graphing may

enhance the effectiveness of academic interventions such as PALS (DiGangi, Maag, &

Rutherford, 1991). As part of this consultation, self-graphing procedures were implemented

during the first intervention phase prior to the introduction of PALS. Students were trained to

enter their weekly ORF and Maze data into an Excel file with a graph for ORF and Maze

measures to view their progress (Appendix A). Graduate students instructed the students to enter

their data through the use of a self-graphing intervention script (Appendix B).


163

PALS and self-graphing. The PALS component of the intervention was implemented

with students five days per week for 30 min after data moved in a “stable” trend during the self-

graphing phase (Appendix C). The intervention was aimed at further improving the scores

students received on ORF and Maze assessments. PALS, a research-based reading program,

involves pairing students in dyads with a higher-performing reader and lower-performing reader

and materials are read based on the lower-performing student’s instructional level (Fuchs et al.,

2001). The PALS intervention is made up of three main activities, including partner reading,

paragraph shrinking, and prediction relay (Joseph, 2008). Partner reading involves the higher

performing student reading to the lower performing student for 5 min, followed by the lower

performing student re-reading the text. Students provide error correction when their partner

struggles with a word. Paragraph shrinking is aimed at fostering comprehension skills and

involves the higher performing student reading one paragraph at time, followed by the lower

performing reader asking the student to identify the main idea of each paragraph. After one

student reads for several minutes, the partners switch roles. Prediction relay involves the higher

performing student making a prediction about what will happen next in the story, reading further

in the text, and then confirming their prediction and summarizing the main idea of the text.

Again, after the higher performing reader finishes the activity, the lower performing reader also

completes the activity. Research has demonstrated PALS is an effective intervention for

promoting reading achievement for students across grade levels (Fuchs, Fuchs, & Burish, 2000;

Fuchs, Fuchs, & Kazdan, 1999).

As part of this consultation, the above PALS procedures were implemented during the

students’ Language Arts period. Higher performing readers were paired with lower performing

readers based on the instructional levels determined from AIMSweb ORF data. Partners


164

completed the partner reading, paragraph shrinking, and prediction relay activities described

above during intervention sessions. Students followed partner reading, paragraph shrinking and

prediction relay directions during this time (Appendix D) and also completed a student checklist

to ensure all essential steps were completed (Appendix E).

Adherence Data

Adherence to the self-graphing component of the intervention was assessed by the

graduate students through the use of a procedural checklist based on the essential components of

the intervention script (Appendix B). This was accomplished by checking off tasks as they were

completed for 100% of the intervention sessions implemented with each student (see Table 3). If

adherence was low, the intern and doctoral student would review the procedures with the

graduate students. As mentioned previously, the PALS intervention was implemented beginning

during the week of 4/7/14; however, for the purpose of this consultation entry data were only

reported through 3/28/14. Therefore, adherence to the PALS intervention was not included as

part of this consultation entry.

Date Adherence

3/6/14 J: 100%

T: 100%

3/10/14 J: 100%

T: 100%

SH: 100%

3/13/14 J: Absent

T: Absent

SH: 100%


165

Social Validity

Teachers were not asked to complete a social validity form as part of this consultation as

the PALS intervention had only been implemented with two students by the time data were

reported for this consultation. However, social validity was collected throughout the intervention

process through frequent meetings and conversation between the intern, doctoral student,

teachers and students. The teachers expressed the students enjoyed the self-graphing procedures

implemented and were excited about the ability to visually see their progress. Additionally,

3/17/14 J: Absent

T: Absent

SH: 100%

3/20/14 J: 100%

T: 100%

SH: 100%

3/24/14 J: 100%

T: Absent

SH: 100%

K: 100%

3/27/14 J: Absent

T: 100%

SH: 100%

K: 100%

SA: 100%


166

during the first week of PALS implementation, the two students who participated were highly

engaged during the session and appeared to enjoy working with a partner to practice reading

skills.

Results

Individual baseline and progress monitoring data for ORF and Maze are displayed below

in Figures 1 through 13. Although there were nine students targeted for intervention in the

middle school classroom, the self-graphing intervention had only been implemented with five

students by the time data were reported for this consultation. The intervention had not been

implemented with the remaining four students by 3/28/14 because the students had either been

absent or had not yet achieved a “stable” baseline. Therefore, individual data are only presented

for the students who had begun the intervention by 3/28/14. In support of visual analysis,

summary statistics were calculated in Tables 4 and 5. The means (M) and standard deviations

(SD) of the target variables were calculated for each student. The percentage of non-overlapping

data (PND), goal attainment scaling (GAS), and effect size were also calculated for the target

variables.

Figure 1 shows the results of the intervention on J’s ORF scores. The level of J’s

performance on the ORF assessment was moderate during baseline and the trend was variable

(M=114.67; SD=22.89). Visual analysis of the ORF data during the intervention phase showed a

decreasing trend. The intervention mean was 116.67 (M=40.45). The goal was not met by the

time data were reported for this consultation; therefore, there were plans in place to implement

the PALS intervention to provide J with additional opportunities to practice and receive feedback

on reading skills and support progress toward the goal.


167

Figure 1. ORF scores for J at the 6th grade level

Figure 2 shows the results of the intervention on J’s Maze scores. The level of J’s

performance on the Maze assessment was moderate during baseline and the trend was variable

(M=14.6; SD=2.61). Visual analysis of the Maze data during the intervention phase showed an

increasing trend based on the two data points collected. Goal attainment scaling was +1 as the

second intervention data point was higher than the data collected during the baseline phase. The

intervention mean was 17 (M=5.66). However, the goal was not met by the time data were

reported for this consultation. As mentioned previously, there were plans in place to implement

the PALS intervention with J to address his needs.


168

Figure 2. Maze scores for J at the 6th grade level

Figure 3 shows the results of the intervention on T’s ORF scores at the 7th grade level.

The level of T’s performance on the ORF assessment was moderate during baseline and the trend

was variable (M=109; SD=8.97). Visual analysis of the ORF data during the intervention phase

continued to show a variable trend. However, the PND was 50%. Thus, goal attainment scaling

was +1 as T made some progress toward the goal. The intervention mean was 110.5 (M=20.37).

The goal was not met by the time data were reported for this consultation; therefore, there were

plans in place to implement the PALS intervention to provide T with additional opportunities to

practice and receive feedback on reading skills and support progress toward the goal.


169

Figure 3. ORF scores for T at the 7th grade level

Figure 4 shows the results of the intervention on T’s ORF scores at the 6th grade level.

The level of T’s performance on the ORF assessment was moderate during baseline and the trend

was variable (M=126; SD=16.35). Visual analysis of the ORF data during the intervention phase

showed an increasing trend with slight variability. Unlike on the 7th grade level assessment, there

were more data points that remained near the aim line. The intervention mean was 143.75

(SD=3.3), indicating a greater improvement between the baseline and intervention phases on

average than on the 7th grade level assessment. However, the goal was not met by the time data

were reported for this consultation. As mentioned previously, there were plans in place to

implement the PALS intervention with T to further support her progress.


170

Figure 4. ORF scores for T at the 6th grade level

Figure 5 shows the results of the intervention on T’s Maze scores. The level of T’s performance

on the Maze assessment was moderate during baseline on average and the trend was highly

variable (M=23; SD=5.15). Visual analysis of the Maze data during the intervention phase

showed a variable trend; however, the most recent data point collected was above the aim line.

Goal attainment scaling was +1 as the fourth intervention data point was higher than the data

collected during the baseline phase. The intervention mean was 26.25 (M=3.4). However, the

goal was not met by the time data were reported for this consultation. As mentioned previously,

there were plans in place to implement the PALS intervention with T.


171

Figure 5. Maze scores for T at the 7th grade level

Figure 6 shows the results of the intervention on SH’s ORF scores at the 8th grade level.

The level of SH’s performance on the ORF assessment was moderate-to-high during baseline

and the trend was highly variable (M=109.56; SD=22.2). Visual analysis of the ORF data during

the intervention phase also showed a variable trend, but the first three data points collected upon

intervention implementation were above the aim line. The intervention mean was 119.25

(M=19.65). The goal was not met by the time data were reported for this consultation; therefore,

there were plans in place to implement the PALS intervention to provide SH with additional

opportunities to practice and receive feedback on reading skills and support progress toward the

goal.


172

Figure 6. ORF scores for SH at the 8th grade level

Figure 7 shows the results of the intervention on SH’s Maze scores. The level of SH’s

performance on the Maze assessment was moderate-to-high during baseline and the data moved

in an increasing trend with variability (M=22.29; SD=6.02). Visual analysis of the Maze data

during the intervention phase showed the level of SH’s performance decreased compared to

baseline. The intervention mean was 19.75 (M=2.88). The goal was not met by the time data

were reported for this consultation. As mentioned previously, there were plans in place to

implement the PALS intervention with SH.


173

Figure 7. Maze scores for SH at the 8th grade level

Figure 8 shows the results of the intervention on K’s ORF scores at the 7th grade level.

The level of K’s performance on the ORF assessment was moderate during baseline and the

trend was variable (M=93.63; SD=4.14). There was one data point collected during the

intervention phase by the time data were reported for this consultation, which indicated K

achieved a score of 81 words read correctly. The goal was not met by the time data were

reported; therefore, there were plans in place to implement the PALS intervention to provide K

with additional opportunities to practice and receive feedback on reading skills and support

progress toward the goal.


174

Figure 8. ORF scores for K at the 7th grade level

Figure 9 shows the results of the intervention on K’s ORF scores at the 5th grade level.

The level of K’s performance on the ORF assessment was moderate during baseline and the

trend was highly variable (M=112.44; SD=13.79). Again, there was only one data point

collected during the intervention phase by the time data were reported for this consultation,

which indicated K achieved a score of 53 words read correctly. The goal was not met by the time

data were reported for this consultation. As mentioned previously, there were plans in place to

implement the PALS intervention with K to further support his progress.


175

Figure 9. ORF scores for K at the 5th grade level

Figure 10 shows the results of the intervention on K’s Maze scores. The level of K’s

performance on the Maze assessment was low-to-moderate during baseline and the data were

highly variable (M=14; SD=4.39). There was only one data point collected during the

intervention phase by the time data were reported for this consultation, which indicated K

achieved a score of 12 responses correct. The goal was not met by the time data were reported

for this consultation. As mentioned previously, there were plans in place to implement the PALS

intervention with K.


176

Figure 10. Maze scores for K at the 7th grade level

Figure 11 shows the results of the intervention on SA’s ORF scores at the 7th grade level.

The level of SA’s performance on the ORF assessment was moderate during baseline and the

trend was variable (M=94.17; SD=12.14). There was only one data point collected during the

intervention phase by the time data were reported for this consultation, which indicated SA

achieved a score of 103 words read correctly. This data point was on the aim line. The goal was

not met by the time data were reported; therefore, there were plans in place to implement the

PALS intervention to provide SA with additional opportunities to practice and receive feedback

on reading skills and support progress toward the goal.


177

Figure 11. SA ORF scores at the 7th grade level

Figure 12 shows the results of the intervention on SA’s ORF scores at the 6th grade level.

The level of SA’s performance on the ORF assessment was moderate during baseline and the

data moved in an increasing trend with variability (M=109.83; SD=16.27). There was only one

data point collected during the intervention phase by the time data were reported for this

consultation, which indicated SA achieved a score of 106 words read correctly. This data point

was slightly below the aim line. As mentioned previously, there were plans in place to

implement the PALS intervention with SA to further support her progress.


178

Figure 12. SA ORF scores at the 6th grade level

Figure 13 shows the results of the intervention on SA’s Maze scores. The level of SA’s

performance on the Maze assessment was low during baseline (M=10.83; SD=1.94). There was

only one data point collected during the intervention phase by the time data were reported for

this consultation, which indicated SA achieved a score of 14 responses correct. The goal was not

met by the time data were reported for this consultation. As mentioned previously, there were

plans in place to implement the PALS intervention with SA.


179

Figure 13. SA Maze scores at the 7th grade level

Summary statistics for each individual student are shown in Tables 4 and 5. Effect size

was calculated by subtracting the baseline mean from the intervention mean then dividing that



intervention points. GAS was also determined for the students, where “0” corresponds to no


the goal was met.

Table 5

Summary Statistics for ORF


Baseline Mean

Baseline SD


Intervention Mean

Intervention SD

Effect Size

PND GAS


180

J 6 114.67 22.89 3 116.67 40.45 .09 33.33% +1 T (Gr.7)

5 109 8.97 4 110.5 20.37 .17 50% +1

T (Gr.6)

5 126 16.36 4 143.75 3.3 1.08 0% 0

SH 8 109.56 22.2 5 119.25 19.65 .43 0% 0 K (Gr.7)

8 93.63 4.14 1 NC NC NC 0% 0

K (Gr.5)

9 112.44 13.79 1 NC NC NC 0% 0

SA (Gr.7)

6 94.17 12.14 1 NC NC NC 0% 0

SA (Gr. 6)

6 109.83 16.27 1 NC NC NC 0% 0

Note: NC=not calculable

Table 6

Summary Statistics for Maze


Baseline Mean

Baseline SD


Intervention Mean

Intervention SD

Effect Size

PND GAS

J 5 14.6 2.61 2 17 5.66 .92 50% +1 T (Gr.7)

5 23 5.15 4 26.25 3.4 .63 25% +1

SH 7 22.29 6.02 5 19.75 2.87 -.42 0% 0 K (Gr.7)

7 14 4.39 1 NC NC NC 0% 0

SA (Gr.7)

6 10.83 1.94 1 NC NC NC 0% 0

Discussion

The results of the intervention indicated that 40% of students (2 out of 5) made progress

toward the ORF and Maze goals by the time data were reported for this consultation. Based on

teacher report, students enjoyed the self-graphing procedures implemented and were excited

about the ability to visually see their progress. The team planned to implement PALS with all of

the students to address their needs. The first PALS session was during the week of 4/7/14.


effectiveness. Inter-scorer agreement was collected for at least 20% of ORF measurement


181

occasions; however; inter-scorer agreement data should have been collected more on the Maze

measurement. When inter-scorer agreement was calculated, the percentage of agreement was

high, which indicated the raters were accurate while scoring. Inter-scorer agreement is an

important measure when analyzing a student’s progress while receiving intervention support and

in the future I will attempt to collect inter-scorer agreement data for at least 20-25% of

measurement occasions.

It is important to note that adherence data indicated the self-graphing component of the

intervention was not implemented as consistently as intended for some students. Although

graduate students attempted to implement the intervention two times per week for all students, at

times students were absent and did not have the opportunity to receive the intervention. This fact

may have impacted their progress during the first intervention phase.

Upon reflection, it would have been beneficial to provide social validity forms to the

students to obtain feedback regarding their perception of the intervention. Although the PALS

component had not been implemented by the time data were reported for this consultation, the

students could have provided valuable feedback. As a child advocate, in the future I will seek out

student feedback whenever possible to ensure the intervention is meaningful to them as well.

Although the students did not meet the ORF or MAZE goals by the time data were

reported for this consultation, the team planned to implement the PALS intervention with each

student once their data during the self-graphing phase were stable. This decision was made in

order to ultimately be able to make comparisons between the self-graphing and PALS plus self-

graphing phases to determine whether the self-graphing component had an impact on student

progress. The PALS intervention will be implemented throughout the rest of the year until the


182

end of the year or until students meet the goals. As a school psychologist, it will be necessary to

ensure students’ needs are being met through the use of research-based intervention strategies.

This consultation provided meaningful opportunities to practice and develop components

of my personal model of practice. As a scientist-practitioner, I assisted the team in collecting

frequent baseline and progress monitoring data for the students. Data were used to make

determinations regarding when it would be appropriate to implement additional supports for

students. I utilized my understanding of research to help determine evidenced-based strategies to

support students’ reading skills. I was an advocate for the students by being a partner in striving

to find a strategy to support their progress.


183

References

AIMSweb. (2013). Reading Curriculum-Based Measurement Administration and Scoring

Guide. [Data file]. Retrieved from https://aimsweb.pearson.com

AIMSweb Default Cut Scores Explained. (2011). Pearson Education Inc.

DiGangi, S. A., Maag, J. W., & Rutherford, R. B. (1991). Self-graphing of on-task behavior:

Enhancing the reactive effects of self-monitoring on on-task behavior and academic

performance. Learning Disability Quarterly, 14(3), 221-230.

Fuchs, D., Fuchs, L., & Burish, P. (2000). Peer-Assisted Learning Strategies: An evidence-based

practice to promote reading achievement. Learning Disabilities Research & Practice, 15,

85-91.

Fuchs, L.S., Fuchs, D., & Kazdan, S. (1999). Effects of Peer-Assisted Learning Strategies on

high school students with serious reading problems. Remedial and Special Education, 20,

309-318.

Fuchs, D., Fuchs, L. S., Thompson, A., Svenson E., et al. (2001). Peer-assisted learning

strategies in reading: Extensions for kindergarten, first grade, and high school. Remedial

and Special Education, 22(1), 15-21.





Joseph, L.M. (2008). Best practices on interventions for students with reading problems.


184

In A.Thomas & J. Grimes (Eds.), Best Practices in School Psychology V (Vol. 4, pp

1163-1180). Bethesda, MD: The National Association of School Psychologists.

Joseph, L. & Eveleign, E. (2011). A review of the effects of self-monitoring on reading

performance of students with disabilities. The Journal of Special Education, 45(1), 43-53.



Sutherland, K. S., & Snyder A. (2007). Effect of reciprocal peer tutoring and self-

graphing on reading fluency and classroom behavior of middle school students with

emotional or behavioral disorders. Journal of Emotional and Behavioral Disorders, 15

(2), 103-118.


185

Appendices

A. Self-Graph Example

B. Self-Graphing Script

C. PALS Script

D. Partner reading, paragraph shrinking and prediction relay directions

E. PALS Student Checklist


186

Appendix A: Self-Graph Example


187

Appendix B: Self-Graphing Script


188

Appendix C: PALS Script


189

Appendix D: Partner reading, paragraph shrinking and prediction relay directions


190


191


192

Appendix E: PALS Student Checklist

UTILIZING A RESPONSE

193

Utilizing a Response to Intervention Model to Determine Eligibility for Special Education

Services for a First-Grade Student with Disruptive Behaviors

A teacher referred Karen, a first-grade student, for further classroom support due to

concerns regarding the “tantrum” behaviors she displayed throughout the day. The teacher

reported the behaviors of concern included crying, screaming, falling on the floor, leaving the

assigned area and kicking objects. Formal and informal observations confirmed Karen displayed

higher levels of off-task behavior than her peers, and that at times her behaviors escalated and

resulted in missed instructional time. The problem-solving team collaboratively decided to target

off-task behavior for intervention. To monitor Karen’s progress, her teacher recorded frequency

of removals from the classroom and the duration of time spent out of class daily due to

exhibiting off-task and disruptive behavior. Interventions involved strategies to define expected

behavior and reinforce these behaviors in the classroom. Karen’s teacher began utilizing

strategies to manage Karen’s behavior at the beginning of the school year.

Karen’s mother requested that a school-based evaluation be completed due to concerns

that her behaviors were significantly negatively impacting her educational performance. The

school team agreed to complete an evaluation to determine if Karen met eligibility for special

education services. The suspected disability categories were Emotional Disturbance and Other

Health Impairment. The intern school psychologist completed a functional behavior assessment

and provided additional intervention recommendations to the problem-solving team based on the

hypothesized function of Karen’s behavior. The team utilized progress-monitoring data to

analyze intervention effectiveness and the information compiled by the intern school

psychologist, in combination with the information provided by the other members of the

problem-solving team, were used to complete the evaluation and determine eligibility for special


194

education services. The initial evaluation was completed on 12/2/13 and the team determined the

student required specially designed instruction to meet her needs and that she met the criteria to

be eligible for special education services under the category of Emotional Disturbance.

The results of the intervention demonstrated that, on average, the number of minutes of

instruction missed daily decreased compared to baseline. Karen met the goal on 15 occasions

from 9/18/13 through 12/12/13; however, the amount of instruction missed was variable during

the intervention phases. The results of the social validity survey indicated the teacher “strongly

agreed (5)” with all items on the survey, suggesting she felt she had adequate input in developing

the intervention script, the intervention script was easy to follow, she liked the intervention

procedures, the intervention was easy to include in her daily routine, she would use the

intervention again in the future, and she felt the intervention benefited the student.

Methods


Target student. Karen, a six-year-old student, was referred by her first-grade teacher for

further classroom support due to the disruptive behaviors she displayed throughout the day.

Examples of disruptive behaviors included: crying, screaming, falling on the floor, leaving the

assigned area and kicking objects (e.g., doors, chairs). A formal classroom observation using the

Teacher Instructional and Caring Contacts Research Observation Form (ICC) (Appendix A)

revealed class-wide student engagement was 98.86%. After obtaining parent permission to

observe Karen, two initial formal observations were conducted in the classroom using a modified

version of the Behavioral Observation of Students in School (BOSS) (Appendix B). The

observations revealed Karen displayed lower levels of on-task behavior and higher levels of off-

task behavior than her peers. Specifically, average peer engagement was 93.75% and percentage


195

of time off-task was 8.34%, but Karen’s average engagement was 84.9% and percentage of time

off-task was 40.29%. Observations also confirmed that at times her behaviors escalated, became

disruptive and dangerous to herself and others, and resulted in removals from the classroom and

missed instructional time. Additionally, case notes and narrative recording observations

conducted by the school psychologist and intern (dated 9/9/13 and 9/10/13) documented

behavior after being removed from the classroom (Appendix C).

In addition to concerns that off-task behaviors were impacting Karen’s time-in-class and

therefore the amount of instruction she was receiving, the team was concerned about Karen’s

academic performance as well. Karen’s records from Kindergarten indicated her performance in

the areas of reading and math were at or above the level expected; however, in first grade Karen

received low scores on reading assessments compared to Dynamic Indicators of Basic Early

Literacy Skills (DIBELS) – 6th edition benchmark goals (Good & Kaminski, 2002). To provide

additional opportunities to practice reading skills, Karen participated in two intervention small

groups for approximately 55 min, four days per week from 9/30/13 through 12/20/13. The Title 1

tutor who worked with Karen reported that when she was on-task she demonstrated the ability to

learn new skills. Although academic performance was a concern, the team felt it was necessary

to implement interventions to support Karen’s behavior as academic interventions were being

implemented throughout the week. Additionally, improvement in student attendance increases

the amount of time students are able to participate in classroom instruction (Christenson et al.,

2008); therefore, the team believed reinforcing Karen for being on-task with the goal of

increasing time-in-class could support academic skill acquisition.

Interventions were implemented to manage Karen’s behavior and increase her time in

class. Due to concerns that her behaviors were significantly negatively impacting her educational


196

performance, Karen’s parent requested that a school-based evaluation be completed. The school

team agreed to complete an evaluation to determine if Karen met eligibility for a student with a

disability on 10/17/13. The suspected disability categories were Emotional Disturbance and

Other Health Impairment. Permission to work with Karen was obtained from her parent, her

teacher and the school psychologist. See Appendix D for the evaluation team report (ETR)

completed by the problem-solving team.

Problem-solving team. The school problem-solving team consisted of Karen’s mother,

the general education teacher, the school psychologist, the intern school psychologist, the

building principal, the alternative learning program (ALP) teacher, the speech-language

pathologist, and the occupational therapist. Karen’s first-grade teacher was primarily responsible

for implementing the intervention in her classroom and monitoring the amount of missed

instructional time. The ALP teacher and principal also assisted in managing Karen’s behaviors.

The intern school psychologist assumed a consultative role throughout the intervention process;

the intern was responsible for conducting direct observations, interviews with the teacher, and

intervention adherence checks, and communicating frequently with school team members and

completing the multi-factored evaluation. All team members were responsible for providing

information to complete the evaluation. The intern received supervision from the school

psychologist and a university supervisor.

Setting

This consultation was conducted in a public elementary school in the Midwest. The

intervention was primarily implemented in a first-grade classroom, as well as other areas

throughout the school building, such as the ALP room and office. Karen’s teacher rewarded

Karen throughout the day in the classroom; however, her behaviors were managed in the ALP


197

room or office when they became disruptive and/or dangerous. The reading intervention small

group sessions were implemented in the Title 1 tutor’s office four days per week.

Functional Behavior Assessment (FBA) Procedures

An FBA helps with the process of gathering environmental information in order to help

construct a hypothesis regarding the function of a problem behavior. The FBA is also used to

identify the source of reinforcement for problem behaviors, which will serve as the basis for

intervention procedures that are used to reduce the occurrence of the behaviors (Cooper, Heron,

& Heward, 2007). In order to gather information to help develop a functional hypothesis for

Karen’s problem behaviors, an FBA was conducted. The FBA consisted of a record review,

interviews with Karen’s teacher and parent, direct observations of Karen in the classroom, and

rating scales to gain more information about Karen’s behaviors.

Record review. The FBA procedure began by reviewing Karen’s background history. To

complete the record review, the intern school psychologist utilized information from various

sources, including: Karen’s cumulative file, Children’s Hospital, Progress Book records, and

Karen’s parent. Karen had been evaluated by Children’s Hospital and received diagnoses of

Oppositional Defiant Disorder (ODD) and Attention Deficit Hyperactivity Disorder (ADHD).

Karen’s records indicated she had a history of difficulty in consistently managing her behavior in

the school setting. She attended another elementary school for Kindergarten and was suspended

on one occasion. Her records indicated past concerns regarding attention and compliance and

evidenced that individualized interventions had been implemented to manage her behavior in

kindergarten. Between the months of September and December of her first grade year (while the

initial evaluation was being completed), Karen had been “emergency removed” on five

occasions, suspended on three occasions, and removed from class frequently due to her behavior.


198

If a student was “emergency removed”, this meant he or she was removed from the school

building; however, these removals were not considered suspensions.

Teacher interview. An interview with Karen’s teacher was conducted on 10/7/13 in

order to develop a better understanding of possible setting events, triggers, target behaviors and

consequences (Appendix E). The teacher reported Karen’s behaviors were significantly worse

than other students’ behavior in the classroom. She identified several concerns, including

“tantrum” behaviors, such as: disruptive crying, screaming, falling on the floor, crawling on the

floor, leaving the assigned area and kicking objects (e.g., doors, chairs). She also reported Karen

had been sent to the office approximately 14 times within a two-week period, and remained in

the office for approximately 30 min each time. At the time of the interview, the teacher reported

she felt Karen’s behaviors were worse in the afternoon; however, she had difficulty identifying a

consistent, specific setting and trigger for the “tantrum” behaviors. After the behaviors occurred,

Karen typically received teacher attention (i.e., redirections, reprimands) or was able to

avoid/escape an academic demand (i.e., by refusing to complete work or participate, being sent

to the office, etc.). Although Karen’s DIBELS scores were below the beginning of the year

benchmarks, Karen’s teacher reported she demonstrated the ability to successfully complete

classroom assignments. The teacher reported Karen’s behaviors were less intense during recess,

lunch, centers and tutoring in small group, and on days when she took her medication.

Parent interview. In addition to the teacher interview, an interview with Karen’s parent

was conducted on 10/17/13. She described Karen’s strengths as she works well with others and is

intelligent. She shared that Karen enjoys art, playing on the computer, reading, stickers, and

candy. She reported Karen has trouble “being still” at home and touching her makes her upset.

She also reported Karen was on medication which makes her tired. Through parent interview,


199

Observations. In order to supplement the information gathered from the interviews,

observations of Karen in the classroom and office were conducted. The intern school

psychologist observed and coded the occurrence of off-task, disruptive behaviors using narrative

real time and partial interval recording methods. The observations lasted between 15 and 30 min.

In combination with the interviews, the observations confirmed the teacher’s reports and led to

both attention and escape as possible functions of Karen’s behaviors. For example, Karen was

observed on 9/23/13 during Language Arts whole group instruction on the carpet. During this

time, Karen demonstrated on-task behaviors at a rate lower than her peers and off-task behaviors

at a rate higher than her peers. Karen attempted to participate by raising her hand on several

occasions throughout the observation. When Karen exhibited off-task behaviors, the teacher

redirected her immediately. The absence of individual attention during whole group instruction

may have served as a setting event for Karen’s behavior during the activity. Furthermore, Karen

was observed to seek attention by raising her hand. However, Karen consistently received

attention by engaging in off-task behavior, which supported the hypothesis that the function of

her behavior may have been to obtain attention.

Reward preference survey. The intern school psychologist utilized the Dunn-Rankin

Reward Preference Inventory (Appendix F) to help determine Karen’s preferences and what

most motivates her. Karen’s responses indicated she is most motivated by adult approval (e.g.,

praise for good behaviors, recognition from adults, being assigned jobs from adults, etc.).

Karen’s secondary motivator was tangibles (e.g., candy, soft drinks, etc.). The information

obtained was used to help in planning rewards for the intervention.



200

Engagement behaviors are critical to target with students at school, for they have been

associated with many facets of school and subsequent achievement, as well as physical and

emotional well-being (Blum & Libby, 2004; Christenson et al., 2008; Ponitz & Rimm-Kaufman,

2011). If a student is engaged in high levels of off-task behavior, it can in turn be presumed that

the student is engaged in lower levels of on-task behavior, which likely limits attention to

instruction. Through parent and teacher interviews and assessment of Karen’s behaviors through

observations, the target variable of off-task behavior was determined to be a priority concern for

intervention. Off-task behaviors were targeted due to the high levels of off-task behavior the

student displayed during class compared to her peers, which at times resulted in removals from

the classroom and missed instructional time. Off-task behaviors were defined as behaviors that

prevented the student from attending to instruction or completing activities as expected, such as:

staring off, talking to another student, facing away from instruction, crying, screaming, falling on

the floor, leaving the assigned area, and kicking objects (e.g., doors, chairs).

Prior to intervention implementation, five baseline points were obtained. Baseline data

indicated that on average, Karen missed 49.5 min of instruction per day; therefore, a substantial

amount of instruction was being missed. Subsequently, data were collected on a daily basis

during the intervention phases. To gather baseline and progress monitoring data, Karen’s teacher

recorded the frequency of removals from the classroom and the duration of time spent out of

class daily due to exhibiting off-task and disruptive behavior. The teacher indicated the time

Karen left class and the time she returned to class throughout the day on a data collection form

(Appendix G). These durations were added to determine the total amount of instructional time

missed daily.


201

Several direct observations were conducted to supplement this information; however,

these observations were not used to monitor Karen’s behavior on a consistent basis. In addition

to the two initial observations, three observations lasting 20 min on average were conducted

during whole group and small group instruction times. The observations were conducted using

an observation code modified from the Behavioral Observation of Students in School (BOSS)

(Appendix B). The code incorporated engaged time (active, passive) and off-task behavior

(motor, verbal, passive). A random peer was observed every fifth interval. Each interval lasted

15 s. Momentary time sampling at the beginning of each interval was used for engagement and

partial interval recording was used for off-task behavior.

Inter-Observer Agreement (IOA)

The teacher recorded the frequency of removals from the classroom and the duration of

time spent out of class daily. To ensure observation accuracy, it would have been ideal for

another individual in the classroom to record this information along with the teacher for at least

25% of baseline and intervention sessions to calculate IOA. Inter-observer agreement (IOA) data

were collected for 13.89% of days data were collected (see Table 1). The principal, assistant

principal or secretary collected IOA data on 10/21/13, 10/28/13, 11/7/13, 11/13/13 and 11/18/13

by indicating the time Karen was in the office on Progress Book, the school building’s online

student information system. Mean duration-per-occurrence IOA was calculated for the duration

of time-out-of class (Cooper, Heron, & Heward, 2007). As an example, the observers recorded

the following durations for time-out-of class in minutes across two occasions on 10/21/13:

1 2

Observer 1 68 267

Observer 2 0 267


202

Then, IOA was calculated by (1) calculating the duration per occurrence IOA for each response

(e.g., 1, 0/68=0; 2, 267/267=1), (2) adding the individual IOA percentages for each occurrence

(e.g., 0+1=1), (3) dividing the sum of the individual IOAs per occurrence by the total number of

responses for which two observers measured duration (e.g., 1/2=.5), and (4) multiplying by 100

(.5x100=50).

Table 1


Date IOA

10/21/13 50%

10/28/13 33%

11/7/13 100%

11/13/13 33%

11/18/13 50%


“Best practice” claims choosing goals based on peer-comparison micro-norms is an ideal

goal setting method (Hixon, Christ, & Bradley-Johnson, 2008). After consulting with Karen’s

teacher, it was decided that the goal of the intervention would be to decrease instructional time

missed to no more than 5 min per day. The initial evaluation meeting date was scheduled for

12/2/13; therefore, the team aimed to reach the goal by this date. The goal was derived based

upon input from Karen’s teacher as well as from comparing her behavior with that of her peers.

Karen’s teacher reported the average peer was removed from the classroom zero times per week

and therefore missed zero minutes of instructional time due to disruptive behavior.


203


intervention needed to take place or if Karen reached the goal. The decision rule for instructional

time missed (i.e., duration of time out of class) was four points above the aim line when the


this point, a change would need to be made to address Karen’s needs. Suspensions and

emergency removals were included in the calculation of instructional time missed; however,

suspension data were not counted toward the decision rule criteria since Karen was not at school

during these times and therefore the opportunity to behave appropriately was not available.


Based on interviews, observations and rating scales, the team hypothesized Karen

engaged in high levels of off-task behavior to obtain adult attention and/or escape. The teacher

often provided redirections when Karen engaged in off-task behaviors during class. At times,

Karen’s behaviors escalated and ultimately resulted in being sent to the office or ALP room. By

leaving the classroom, Karen was able to escape academic activities and receive attention from

the ALP teacher, principal and assistant principal. Karen’s behaviors resulted in missed

instructional time and may have interrupted the acquisition of new skills. As mentioned

previously, to provide additional opportunities to practice basic reading skills, Karen participated

in small group tutoring for approximately 55 min, four days per week from 9/30/13 through

12/20/13.

Accountability Plan

An ABCD design was used to examine the effects of the intervention on Karen’s level of

instructional time missed. In addition to the class-wide behavior management strategies utilized,

during the second week of the school year (beginning on 9/10/13), Karen’s teacher implemented


204

a sticker chart to reward appropriate behavior. Five baseline points were obtained prior to

implementation of intervention modifications and additional strategies. The intervention was

modified on 9/17/13 (B), 10/23/13 (C), and 11/1/13 (D). The aforementioned target variable was

measured daily to monitor Karen’s progress while the interventions were in place.


Baseline condition. Karen’s teacher implemented strategies to support appropriate

behavior at the Tier 1 level. In addition to class-wide strategies, Karen’s teacher began utilizing a

sticker chard to reward Karen’s appropriate behavior due to immediate concerns regarding

Karen’s behavior at the beginning of the school year. The teacher was asked to collect baseline

data by recording Karen’s duration of time out of class daily (Appendix G). Five baseline points

were obtained prior to implementation of intervention modifications and additional strategies.

Baseline data are displayed and discussed in the results section below.

To manage behavior class-wide, Karen’s teacher implemented strategies to support

appropriate behavior, including establishing several classroom expectations and reinforcing the

expectations throughout the day. These are key features of Positive Behavior Supports (PBS), a

researched-based model that incorporates proactive strategies for reducing challenging behaviors

(Benedict, Horner, & Squires, 2007; Stormont, Lewis, & Beckner, 2005). In a study by Johnson,

Stoner and Green (1996), researchers found that active teaching of class rules was effective at

increasing the appropriate behavior and decreasing the inappropriate behavior of students.

Furthermore, it has been suggested that reinforcing appropriate behavior strengthens that

behavior (Skinner, Pappas, & Davis, 2005). Thus, students should be praised for rule-following

behavior to increase the likelihood that those behaviors will occur more frequently in the future.


205

The teacher identified several classroom expectations for all students to follow related to

behaving appropriately while in class (i.e., raise your hand to talk or share ideas, sit on your

bottom in your chair and on the carpet, keep your hands and feet to yourself, be respectful). An

acknowledgement system was used to systematically reward students for following the rules.

The acknowledgement system involved the use of a “clip chart” whereby students’ names were

moved up on the chart by the teacher when she noticed them behaving appropriately. In addition

to the class-wide strategies used, an individual sticker chart was used to reward Karen

approximately every 30 min if she demonstrated appropriate behavior and remained in the

classroom. Each day the teacher set a goal for Karen; if she earned a certain number of stickers

by the end of the day, she was given a reward (e.g., scented sticker).

Modified reinforcement criteria. Due to teacher report that additional backup

reinforcement was needed to reinforce Karen, a decision was made modify the sticker chart plan

on 9/17/13. Karen was given the opportunity to earn candy in the morning and afternoon based

on the number of stickers on her chart. For instance, if she earned five stickers in the morning

and six stickers in the afternoon, she was given five pieces of candy before lunch and six pieces

of candy at the end of the day.

Visual schedule with increased reinforcement, and review of rules. Performance

feedback regarding Karen’s progress was provided during a meeting with Karen’s teacher, parent

and the school psychologist on 10/17/13. Progress monitoring data indicated Karen was able to

reach the goal on several occasions; however, the data were variable and concerns regarding

behavior remained. The intern school psychologist presented additional intervention

recommendations to the team. Ultimately, the team decided to implement a visual schedule,


206

more frequent reinforcement for appropriate behavior and a procedure involving review of rules

in the ALP room beginning on 10/23/13 (Appendix H).

In addition to establishing classroom expectations and reinforcing the expectations

throughout the day through the use of an acknowledgement system and specific praise, another

key feature of PBS is a posted schedule including sequenced pictures (Benedict, Horner, &

Squires, 2007). To provide a visual reminder of the classroom schedule and expected behavior

throughout the day, Karen was given a book with a picture of each activity including herself

completing the activity. The team decided to reinforce Karen’s on-task behaviors more often in

an attempt to increase the frequency with which these behaviors occurred. To do so, three Velcro

dots were placed below each picture in the book and Karen carried the book with her throughout

the day. When the teacher noticed Karen remaining on-task in class, she gave her a green colored

dot. If she earned three green colored dots during an activity her name was moved up on the

class-wide clip chart. Karen was given the opportunity to earn time with the ALP teacher if her

name was moved to the top of the chart. Conversely, when Karen exhibited off-task disruptive

behavior the teacher gave her a red colored dot and reviewed the classroom expectations with

her. If she received three red colored dots she was brought to the ALP room due to teacher input

that Karen’s behaviors could not be managed in the classroom at this time. While in the ALP

room, the ALP teacher re-iterated the expectations Karen. When her behaviors were under

control and she was calm, Karen was permitted to sit with the ALP teacher at her table and she

was directed to complete unfinished classroom assignments.

Visual schedule with increased reinforcement, and disruptive behaviors ignored.

The intervention plan was modified on 11/1/13 due to continued concerns regarding Karen’s

frequent removals from the classroom and high levels of missed instructional time. The team


207

hypothesized attention from the ALP teacher may have been rewarding for Karen; therefore, the

plan was modified. Based on the behavioral principle of extinction, the frequency of a previously

reinforced behavior can be reduced by withholding the maintaining reinforcer (Cooper, Heron, &

Heward, 2007). Cooper, Heron and Heward (2007) claim when the procedure is matched to the

behavioral function, the intervention is usually effective. When Karen received three red dots for

exhibiting disruptive behavior, rather than being brought to the ALP teacher, the team decided

Karen would be brought to the office and her disruptive behaviors would be completely ignored.

The team agreed Karen would be returned to class when she was calm.

Adherence Data

Adherence data were obtained for 52.63% of the intervention sessions through permanent

product and teacher-collected data. The teacher provided copies of Karen’s sticker chart to the

intern school psychologist. When the intervention was modified, the teacher indicated the

number of green and red dots Karen received during each class period on an adherence form and

provided copies to the intern. The intern used this information to calculate the percentage of class

periods the intervention was utilized each day. Average adherence was 92.29%, and ranged from

44% to 100%. If intervention adherence was low, the intern would review the procedures with

the teacher. Adherence was not collected for the procedures implemented in the ALP room and

office for this support consultation since the times Karen was in these locations due to behavior

were unpredictable.

Social Validity

The intern school psychologist designed the intervention procedures in collaboration with

the problem-solving team, and permission to work with Karen was obtained from her parent, her

teacher and the school psychologist. Social validity was collected throughout the intervention


208

process through frequent conversation and meetings between the intern and problem-solving

team. In addition, at the end of the intervention, the teacher completed a social validity

questionnaire to indicate her satisfaction with the intervention and its effectiveness, which can be

seen below.


check marks in the cells represent the ratings the teacher chose. The teacher “strongly agreed

(5)” with all of the items on the survey, indicating she found the intervention valuable. The

teacher also commented, “Thank you for all your help this year with my data!”

Table 2






- - - - ✓


- - - - ✓


- - - - ✓


- - - - ✓


- - - - ✓


209


- - - - ✓

Results

Individual baseline and progress monitoring data for the number of minutes of instruction

missed daily are displayed below in Figure 1. In support of visual analysis, summary statistics

were calculated in Table 3. The mean (M) and standard deviation (SD) of the target variable were

calculated. The effect size, percentage of non-overlapping data (PND), and goal attainment

scaling (GAS) were also calculated. For the purpose of this consultation entry, data were reported

through 12/12/13. After Winter break, Karen began attending another elementary school;

however, specialized instruction was planned to continue to support the student’s progress.

Figure 1 demonstrates the results of the intervention on the number of minutes of

instruction missed daily. The level of instruction missed ranged from low-to-high during baseline

and the trend was variable (M=116.6; SD=151.38). During first intervention phase, the level of

instruction missed was lower initially and the goal was met on nine occasions from 9/18/13

through 10/3/13; however, on 10/8/13 the data began to move in an increasing trend. The level of

instruction missed was highly variable and ranged from low-to-high during the second

intervention phase. The goal was met once during this phase on 10/23/13. During the third

intervention phase, the level of instruction missed ranged from low-to-high; however, on

11/18/13 the data began to move in a decreasing trend and the goal was met again on five

occasions including 11/20/13, 11/22/13, 12/2/13, 12/3/13, and 12/4/13. The intervention mean

was 86.79 (SD=125.25). Karen began attending another elementary school in January where

specialized instruction was planned to continue to address the student’s needs.


210

Figure 1. Amount of instruction missed daily (minutes)

Summary statistics for the target variable are shown below in Table 3. Effect size was

calculated by subtracting the baseline mean from the intervention mean then dividing that





the goal was met.

Table 3

Summary Statistics for Amount of Instruction Missed Daily


211

Discussion

Social validity results suggest the teacher liked the intervention procedures and she felt

they benefited the student. The results of the intervention demonstrated that, on average, the

number of minutes of instruction missed daily decreased compared to baseline. Karen met the

goal on 15 occasions from 9/18/13 through 12/12/13, suggesting the intervention had a positive

impact. However, the amount of instruction missed was variable during the intervention phases.

The interventions were not effective in reducing and maintaining the number of minutes of

instruction missed daily to a level that was similar to her peers.

The team utilized progress-monitoring data to analyze intervention effectiveness and the

information compiled by the intern school psychologist, in combination with the information

provided by the other members of the problem-solving team, were used to complete the

evaluation and determine eligibility for special education services. The initial evaluation was

completed on 12/2/13 and the team determined the student required specially designed

instruction to meet her needs. Furthermore, the team agreed Karen met the criteria to be eligible

for special education services under the category of Emotional Disturbance. Karen began

attending another elementary school in January where specialized instruction was planned to

continue to address her needs. See Appendix D for the ETR completed by the problem-solving

team.

Evaluation and Eligibility Determination

# BL Data Points

BL Mean

BL SD


Intervention Mean

Intervention SD

Effect Size

PND GAS

Instruction Missed

5 116.6 151.38

39 86.79 125.25 .2 48.72%

+2


212

Establishing discrepant achievement. The strategies the teacher implemented to

support appropriate behavior at the Tier 1 level were effective for most students in Karen’s

classroom. The strategies involved key features of PBS, a researched-based model that

incorporates proactive strategies for reducing challenging behaviors (Benedict, Horner, &

Squires, 2007; Stormont, Lewis, & Beckner, 2005). A formal classroom observation conducted

at the beginning of the year revealed class-wide student engagement was 98.86%. Four

observations conducted during Language Arts whole group instruction revealed Karen displayed

lower levels of on-task behavior and higher levels of off-task behavior than the three random

peers who were observed. Specifically, average peer engagement was 89.31% and percentage of

time off-task was 13.05%, but Karen’s average engagement was 77.35% and percentage of time

off-task was 28.79%. Additionally, Karen’s teacher reported the average peer was removed from

the classroom zero times per week and therefore missed zero minutes of instructional time due to

disruptive behavior. However, on average, Karen missed 116.6 min (SD=151.38) of instruction

daily during the baseline phase and 86.79 min (SD=125.25) of instruction daily during the

intervention phases for reasons related to behavior. Karen’s records from Kindergarten indicated

her performance in the areas of reading and math were at or above the level expected; yet, her

records indicated she had a history of difficulty in consistently managing her behavior in

Kindergarten and interventions had been implemented to support her behavior. The amount of

missed instructional time in first grade may have interrupted the acquisition of new skills in the

classroom. In addition to teacher-collected data, records from Progress Book indicated Karen

had been “emergency removed” from school on five occasions and suspended on three occasions

from September through December 2013.


213

The team demonstrated that Karen was not achieving adequately relative to state grade-

level standards and age in terms of her academic performance. Karen received core instruction

through the school district’s approved curriculum, Treasures. At the beginning of the first-grade

year, Karen received low scores on reading assessments compared to DIBELS – 6th edition

benchmark goals in the areas of letter naming fluency (LNF), phoneme segmentation fluency

(PSF) and nonsense word fluency (NWF). The first-grade beginning of the year benchmark goals

were as follows: LNF – 37; PSF – 35; and NWF – 24. On the LNF benchmark probe Karen

achieved a score of 24 letter names correct, on the PSF benchmark probe she achieved a score of

21 phonemes correct, and on the NWF benchmark probe she achieved a score of 15 letter sounds

correct. On average, Karen’s first-grade peers achieved an LNF score of 42.95 (SD=15.31), a

PSF score of 41.28 (SD=15.53), and a NWF score of 28.91 (SD=18.23). Karen’s performance

represented scores achieved by the 10th-20th percentile of first-grade students during the

beginning of the year benchmark period. To provide additional opportunities to practice and

receive feedback on reading skills, Karen participated in two reading intervention small groups

for approximately 55 min four days per week.

To supplement the aforementioned information, the intern assessed Karen’s academic

skills using AIMSweb curriculum-based measures in November 2013. Karen demonstrated

relative strengths in her LNF, LSF and written expression skills. Karen’s performance on NWF,

PSF, oral reading fluency, reading comprehension, and math computation assessments were

considered to be at or below the 10th percentile when compared to first-grade Winter national

peer norms. As mentioned previously, academic interventions were being implemented

throughout the week. Additionally, records indicated Karen exhibited behavior concerns in

Kindergarten despite her academic performance being at or above the level expected for her age.


214

Thus, although academic performance was a concern the team felt it was essential to implement

interventions to target appropriate behavior and increase time-in-class to support academic skill

acquisition.

Establishing inadequate progress. Through visual analysis of progress monitoring

data, the team determined Karen was not making sufficient progress with the interventions in

place. The interventions implemented involved key features of PBS. In addition to the strategies

the teacher implemented to support behavior at the Tier 1 level, the teacher implemented

strategies to provide Karen increased reinforcement for appropriate behavior in the classroom.

After the teacher raised concerns regarding Karen’s behavior at the beginning of the year, the

intern conducted a functional behavior assessment to help develop a functional hypothesis for

Karen’s behaviors and inform intervention decisions.

Visual analysis of progress monitoring data and teacher input were used to make

adjustments to the intervention when necessary. For example, performance feedback regarding

Karen’s progress was provided during a meeting with Karen’s parent, teacher and the school

psychologist. Data indicated Karen was able to reach the goal on several occasions; however, the

data were variable and the team decided to implement a visual schedule, more frequent

reinforcement for appropriate behavior and a procedure involving review of rules in the ALP

room.

The team attempted to match the interventions to Karen’s needs by utilizing strategies

linked to the hypothesized function of her behavior. Furthermore, Karen demonstrated the ability

to exhibit appropriate behavior on occasions. Therefore, the team believed interventions to

reinforce those behaviors more frequently was an appropriate approach. Permanent product


215

adherence, collected for 52.63% of intervention sessions, indicated the average percentage of

class periods the intervention was implemented each day was 92.29%.

Establishing educational need. Despite the targeted and intensive level of support in

place, there was evidence to suggest Karen’s behavior was preventing her from receiving a

substantial amount of instruction in the classroom and impacting her ability to learn in the

general education setting. The results of the intervention demonstrated that, on average, the

number of minutes of instruction missed daily decreased compared to baseline. Karen met the

goal on 15 occasions, suggesting the intervention had a positive impact. However, the amount of

instruction missed was variable during the intervention phases and interventions were not

effective in reducing and maintaining the number of minutes of instruction missed daily to a

level similar to peers. On average, Karen missed 86.79 min (SD=125.25) of instruction daily

during the intervention phase. The evidence suggested Karen might need ongoing support and

services that could not be appropriately delivered without special education support.

Putting it all together for entitlement decision. The aforementioned information

regarding data from interventions, in addition to assessments of behavior (i.e., Behavior

Assessment for Children (BASC) – 2nd edition), academic skills, fine motor skills,

communication skills, and observations were utilized to complete the evaluation and determine

eligibility for special education services. The problem-solving team compiled an ETR (Appendix

D) and a meeting was held with Karen’s mother, as well as the special education supervisor,

school psychologist, principal, assistant principal, teacher, intervention specialist, occupational

therapist and speech language pathologist. The initial evaluation was completed on 12/2/13 and

the team determined the student required specially designed instruction to meet her needs and


216

continue progressing in the curriculum. Furthermore, the team agreed Karen met the criteria to

be eligible for special education services under the category of Emotional Disturbance.

Under IDEIA, emotional disturbance means a condition exhibiting one or more of the

following characteristics over a long period of time and to a marked degree that it affects a

child’s educational performance, resulting in: (a) an inability to learn that cannot be explained by

intellectual, sensory, or health factors, (b) an inability to build or maintain satisfactory

interpersonal relationships with peers and teachers, (c) inappropriate types of behavior or

feelings under normal circumstances, (d) a general pervasive mood of unhappiness or depression,

or (e) a tendency to develop physical symptoms or fears associated with personal or school

problems. Furthermore, emotional disturbance does not apply to children who are socially

maladjusted, unless it is determined that they have an emotional disturbance. Based on this

definition, the team determined Karen met the criteria to be eligible under the category of

Emotional Disturbance.

The team determined Karen’s behavior had been occurring over a long period of time

based on records from Kindergarten that documented behavior concerns and the implementation

of interventions to manage behavior, in addition to the interventions implemented in first grade

over a three-month period. Furthermore, the team determined Karen’s behaviors were to a

“marked degree” based on their high level of intensity. For example, Karen had demonstrated

behaviors involving physical aggression toward objects and people, as well as self-injurious

behaviors (e.g., teacher documented Karen scratched at her skin for 45 min). Behaviors that were

disruptive or dangerous resulted in removal from the classroom and missed instructional time.

The team determined the behaviors resulted in an inability to learn that could not be explained by

intellectual, sensory or health factors; an inability to maintain satisfactory interpersonal


217

relationships; and inappropriate types of behavior or feelings under normal circumstances, and

were not the result of a social maladjustment.

Karen began attending another elementary school in January where specialized

instruction was planned to continue to address her needs. An Individualized Education Plan

(IEP) was developed for the student, which included goals in the areas of behavior, social skills,

early literacy skills, and math skills to be reached by the end of the IEP term. To achieve goals,

research-based strategies will be implemented including direct instruction, opportunities to

respond, error correction and feedback. Karen will be provided direct instruction in how to ask

for a break when angry or frustrated as well as how to ask for help appropriately when confused

with a request or assignment. In addition, she will be provided direct instruction in how to

independently initiate and respond to conversations during structured social situations through

the use of a checklist of positive interpersonal behaviors. To support literacy skill acquisition and

development, direct instruction will be provided in identifying letter names, letter sounds, and

phonemes, and in blending sounds to read words. Finally, she will be provided direct instruction

in appropriate strategies to solve basic addition and subtraction problems.

In reflection on this experience with my first eligibility determination, there are several

limitations to this consultation that may have impacted its effectiveness, as well as limitations to

the evaluation process that may have grounds for argument. For instance, the FBA process could

have been strengthened. The intern conducted interviews with the teacher and parent to gain a

better understanding of possible setting events, triggers, target behaviors and consequences. It

would have been beneficial if the interview had been conducted using the Functional Assessment

Checklist for Teachers and Staff (FACTS) form. Although the interview was guided by topics

and questions from the FACTS, the FACTS provides a more in-depth analysis of behavior in


218

different activities and routines. This type of analysis may have provided useful information

regarding whether Karen’s behavior served different functions depending on the context, which

would have helped support the interventions were matched to Karen’s needs. Additional initial

observations in during various activities would have also been beneficial to conduct. The intern

conducted two formal observations during Language Arts whole group instruction before the

visual schedule and increased reinforcement were implemented. The teacher expressed she had

the most concerns regarding Karen’s behavior during this time; however, it would have been

useful if direct observations were also conducted during different subjects and activities (e.g.,

independent seatwork, specials, etc.). This information could have been used to analyze if

behavior differed across settings, if behavior concerns were pervasive across all settings, and if

there were particular times Karen would have benefited from more intensive support. According

to Brock (2013) students with “social maladjustment” demonstrate markedly different responses

in different situations or with different individuals. Thus, additional observations could have

been used to further analyze the notion that behavior was not the result of a social

maladjustment.

Given academic and behavioral success are strongly correlated, when developing a

functional hypothesis in the future I would place more emphasis on Karen’s academic concerns

and how these may have been impacting her behavior as well. Although academic performance

was a concern, the team felt it was necessary to implement interventions to support Karen’s

behavior as academic interventions were being implemented four days per week for 55 min. The

team believed reinforcing Karen for being on-task with the goal of increasing time-in-class may

support academic skill acquisition. Karen’s amount of time-out-of class likely had a profound

impact on her academic performance; however, in the future I would share with the team that


219

students may behave to avoid/escape activities that are too difficult. It would have been

beneficial to monitor the academic interventions being implemented through the use of

adherence checks and visual analysis of progress monitoring data to make modifications to the

interventions when necessary. Doing so may have further supported academic skill development

and behavior.

Another limitation to this consultation was related to the variable used to monitor

progress. The amount of missed instructional time provided important information regarding the

impact Karen’s behavior had on her educational performance and allowed the team to analyze

this impact throughout the school day. However, in the future I would conduct systematic direct

observations of the target behaviors to monitor progress in conjunction with this information, as

this is a more valid and reliable method to analyze behavior. Duration of time-out-of class was

an indirect measure of behavior and has questionable validity and reliability as the variable may

have been influenced by the student’s behavior or the teacher’s tolerance and decisions on any

given day. Furthermore, although the teacher agreed to collect data throughout each day, there

were several days of missing data, which may have impacted the trend of the data during the

intervention phases. By directly observing the behaviors of concern, the team could have

analyzed whether the interventions were effective at increasing the level of Karen’s engagement

and decreasing the level of her off-task behavior during each phase. This would have provided

information regarding the most effective intervention strategies to support Karen. Furthermore,

Brock (2013) claims the behaviors of students who are “socially maladjusted” are significantly

responsive to behavioral intervention. Thus, information regarding the effects of the intervention

on the amount of behavioral change that occurred could have been used to aid in eligibility

decision-making.


220

Additionally, IOA data should have been collected more frequently during the baseline

and intervention phases. Ideally, IOA data should be collected for 20-25% of observation

sessions. Although IOA data were collected on several occasions by individuals in the office, it

was difficult to collect IOA data more frequently. As data were collected throughout the entire

school day, another individual would need to be available to observe Karen throughout the day to

record time-out-of class along with the teacher. As a school psychologist I will attempt to collect

technical adequacy data such as IOA as often as possible to ensure observation accuracy.

It is also important to note that the information used to set goals could have been

strengthened. “Best practice” claims choosing goals based on peer-comparison micro-norms is

an ideal goal setting method. The goal of the intervention was derived based upon input from

Karen’s teacher as well as from comparing her behavior with that of her peers. Karen’s teacher

reported the average peer was removed from the classroom zero times per week and therefore

missed 0 min of instructional time due to disruptive behavior. In addition to recording Karen’s

time-out-of class, it would have been beneficial for the teacher to consistently record peers’ time-

out-of class to validate the report that other students miss 0 min of instruction due to behavior.

Documenting peer-norm data is important in order to analyze whether the behavioral and

academic supports in place are effective for the majority of students. In the future I will ensure

this information is collected to develop appropriate goals and analyze student progress compared

to peers.

The student may have benefited from more intensive interventions based on the

hypothesized function of her behavior. Attention was one of the hypothesized functions of

Karen’s behavior; therefore, the teacher implemented strategies to provide Karen increased

reinforcement for appropriate behavior in the classroom. However, Karen also received attention


221

for inappropriate behavior. Specifically, Karen received red colored dots for exhibiting off-task

behavior and if she received three red colored dots she was brought to the ALP room due to

teacher input that Karen’s behaviors could not be managed in the classroom at this time. It is

possible that Karen could have been reinforced by this teacher attention as well as by leaving the

classroom.

The intern made alternative intervention recommendations to the team based on the

hypothesized function of her behavior. For instance, during a performance feedback meeting the

intern suggested Karen might benefit from an intervention involving a differential reinforcement

of other behavior (DRO) with extinction procedure. It was recommended that the teacher provide

Karen with attention contingent upon the occurrence of appropriate replacement behavior, rather

than inappropriate disruptive behavior. If Karen went without engaging in disruptive behavior

for a designated amount of time, the teacher would provide her with positive attention for

appropriate behavior and a “point.” After earning a certain number of points, Karen would be

taken to the ALP room to receive positive attention for appropriate behavior. If off-task behavior

occurred, these behaviors would be ignored if possible. Upon reflection, the student might have

also benefited from more opportunities to practice appropriate behavior, such as through role-

playing scenarios.

Although the intervention the team agreed to implement involved components of the

aforementioned recommendations, it would have been beneficial for the initial schedule of

reinforcement to be based on the average inter-behavior interval to ensure Karen was

consistently coming into contact with reinforcement. As mentioned previously, it would have

also been beneficial if Karen was provided attention and the opportunity to leave the classroom

contingent upon appropriate behavior only. The team modified the plan on 11/1/13 whereby the


222

team agreed to ignore inappropriate behavior in the office. However, Karen was still able to

leave the classroom, which may have been reinforcing her behavior. Due to the fact that the trend

of the data was variable during the last intervention phase, it would have been worthwhile to re-

iterate the previous recommendations made with the team. In the future, I will provide the team

with research articles to support recommendations made and strive to design interventions that

are research-based, linked to the functional hypothesis, of appropriate intensity, and feasible for

the team to implement.

It would have also been ideal to directly observe the intervention being implemented

throughout the day in the classroom, ALP room and office. The intervention was implemented

throughout the entire school day; therefore, adherence data were collected through permanent

product and teacher collected data. Adherence was not collected for the procedures implemented

in the ALP room and office for this support consultation since the times Karen was in these

locations due to behavior were unpredictable. The intern observed the visual schedule and

reinforcement being implemented as documented through three direct observations (See ETR,

Appendix D), although it would have been useful information to conduct further direct

observations of adherence. Implementing interventions consistently and as intended are essential

in order for plans to be effective and to accurately analyze student behavior. Therefore, in the

future I will do my best to conduct adherence checks through direct observation for 20-25% of

intervention sessions.

Lastly, it is important to summarize the decision-making process, evaluation results, and

recommendations in practical and understandable terms in both oral and written form. Data was

used to guide decision-making as part of this consultation, but the process and results could have

been more adequately articulated in the ETR. For example, in the ETR, data were displayed in a


223

graph as instruction missed weekly in an attempt to present the data an understandable way to the

team. It was explained verbally to the team that the amount of instruction missed per week was

impacted by the number of days data were collected by the teacher that week. In the future, I will

write reports in a language that better reflects the decision-making process and results.

As a school psychologist, I will reflect on the aforementioned limitations and utilize

lessons learned to avoid such issues during future consultations. Despite the limitations

discussed, this consultation was a great opportunity for me to demonstrate, practice and develop

components of my model of practice. For this consultation, I utilized an ecological-behavioral

approach to providing services, researched evidence-based strategies to support behavior,

analyzed data to make decisions regarding intervention effectiveness, and engaged in

collaborative problem-solving with many other key stakeholders to advocate for the needs of the

student. The experience also provided me with the opportunity to utilize a Response to

Intervention model to determine eligibility for special education services.


224

References

Benedict, E.A., Horner, R.H., & Squires, J.K. (2007). Assessment and implementation of

positive behavior support in preschools. Topics in Early Childhood Special Education,

27(3), 174-192.

Blum, R.W., & Libby, H.P. (Eds.). (2004). School connectedness: Strengthening health and

education outcomes for teenagers. Special issue, Journal of School Health, 74(7), 229-

299

Christenson, S.L., Reschly, A.L., Appleton, J.J., Berman-Young, S., Spanjers, D.M., & Varro, P.

(2008). Best practices in fostering student engagement. In A. Thomas & J. Grimes (Eds.),

Best Practices in School Psychology V (Vol. 4, pp 1099-1119). Bethesda, MD: The


Cooper, J.O., Heron, T.E., & Heward, W.L. (2007) Applied Behavior Analysis (2nd Ed.). Upper

Saddle River, NJ: Pearson.

Good, R. H., & Kaminski, R. A. (Eds.). (2002). Dynamic indicators of basic early literacy skills

(6th ed.). Eugene, OR: Institute for the Development of Educational Achievement.

Hixon, M., Christ, T.J., & Bradely-Johnson, S. (2008). Best practices in analysis of progress




Johnson, T.C., Stoner, G., & Green, S.K. (1996). Demonstrating the experimenting society

model with classwide behavior management interventions. School Psychology Review,

25(2), 199-214.

Ponitz, C.C., & Rimm-Kaufman, S.E. (2011). Contexts of reading instruction: Implications for


225

literacy skills and kindergarteners’ behavioral engagement. Early Childhood Research

Quarterly, 26, 157-168.

Skinner, C.H., Pappas, D.N., & Davis, K.A. (2005). Enhancing academic engagement: Providing

opportunities for responding and influencing students to choose to respond. Psychology

in the Schools, 42, 389-403.

Stormont, M., Lewis, T.J., & Beckner, R. (2005). Positive behavior support systems: Applying

key features in preschool settings. Teaching Exceptional Children, 37(6), 42-49.


226

Appendices

A. ICC Observation Code

B. Modified BOSS Observation Code

C. Informal Observation Notes

D. Evaluation Team Report

E. Problem-Solving Interview Questions

F. Dunn-Rankin Reward Preference Survey

G. Data Collection Form

H. Intervention Script


227

Appendix A: ICC Observation Code


228

Appendix B: Modified BOSS Observation Code


229

Appendix C: Informal Observation Notes

Case Notes 9/9/13- 1:00 pm- observed K in the office having an outburst (laying on the floor, screaming, trying to escape the office); L was holding her, trying to calm her down. K continued to observe her (see her notes). 9/10/13- 9:30 am-10:03 am; Mrs. H came into my office and told me to go and observe K in the office. She was screaming and laying on the floor. When an adult direction was given she screamed louder. When given 2 choices, she did not comply. I prompted her to stand up and she sat on her knees and scooted towards the office door. She then went into the hallway and continued to scream. Mrs. S picked her up and started to walk her in the hallway but she would not comply and continued to scream and kick. We brought K into my office, and closed the door. She laid on the floor, screamed, and kicked my door. When we interacted with her or gave her directions she continued to yell. She proceeded to kick Mrs. S. She was given 3 warnings to stop kicking and then her shoes were removed. She continued to kick and scream by my office door. I reviewed what she needed to do in order to go back to the classroom or leave my office. She ignored my requests. I repeated what she needed to do (sit up, quiet voice, calm body). She crawled under the table and I told her “that is a good place to cool down, you can cool down under my table.” She kicked all of the chairs out from under the table, and started to kick a chair against the black cabinet. I turned the chair so she could no longer kick it. We then discussed how she is feeling and what made her upset. She was unable to express what made her upset. I then went through all of the emotions (are you mad, sad, happy) and she finally said “I’m hungry.” We went with this and discussed how to use your words to say what you need. She started to calm down. I praised her for having a calm body and asked her to sit in the chair. She complied. We then talked with her and she remained calm. I gave her a choice of if she wanted to eat graham crackers in my office or with Mrs. S. She decided to go with Mrs. S. We reminded her to use her words to express what she needs. She demonstrated calm and polite behavior (pushed all the chairs back under my table). She left with Mrs. S at 10:03. -Additional notes- the more adults and requests being made, the worse her screaming got; using firm and direct commands did not make her more likely to comply when she was upset; when she was given choices when she was upset she did not comply; behavior appeared to be escape maintained or to obtain what she wanted (she kept screaming leave me alone, I want to go home). When she was screaming in my office and I would look at her or give her a command she would scream louder. 9/10/13- 10:40 am- walked into Mrs. H room and K was seated at her desk doing what was expected; Mrs. S was still with her standing behind her but not directly assisting her


230

Appendix D: ETR

INDIVIDUAL EVALUATOR’S ASSESSMENT Section to be completed by each individual evaluator. Evaluator Name: Position: Intern School Psychologist

AREAS OF ASSESSMENT: Academic Skills, Behavior Assessment, Social Emotional Status, Observations Indicate the area(s) that were assessed by the evaluator in accordance with the evaluation plan. OBSERVATIONS SCIENTIFIC, RESEARCH-‐BASED

INTERVENTIONS

NORM-‐REFERENCED ASSESSMENTS

INTERVIEWS CURRICULUM BASED ASSESSMENTS CLASSROOM BASED ASSESSMENTS

REVIEW OF RECORDS AND RELEVANT TREND DATA (SCHOOL RECORDS, WORK SAMPLES, EDUCATIONAL HISTORY)

OTHER (Specify)

ASSESSMENT INFORMATION Provide a summary of the information obtained from the assessment results per the evaluation plan including the child’s strengths, areas of needed baseline data.

SUMMARY OF ASSESSMENT RESULTS: Background History

Kushana is currently a 1st grade student at Welch Elementary in Northwest Local School District (NWLSD) in Cincinnati, Ohio. Prior to attending Welch Elementary, Kushana attended Taylor Elementary in NWLSD for Kindergarten in 2012-2013. She did not attend preschool. Kushana has diagnoses of Attention-Deficit Hyperactivity Disorder (ADHD) and Oppositional Defiant Disorder (ODD) and takes medication for her diagnosis of ADHD two times per day. She also wears glasses. Kushana has demonstrated difficulty in consistently managing her behavior in the school setting. A request for a school-based evaluation was made during the 2012-2013 school year when Kushana attended Taylor Elementary due to concerns regarding her attention and compliance. The team considered an evaluation, however they proposed not to initiate an evaluation because Kushana’s academic skills in the areas of reading and math were at or above the level of performance expected in Kindergarten. During the 2013-2014 school year, her attendance has been regular, but records from Progress Book indicate that Kushana has been emergency removed on five occasions (10/21/13, 10/28/13, 11/7/13, 11/13/13, and 11/18/13). Her records also indicate that she has been suspended on three occasions (9/11/13, 10/29/13, and 11/14/13). In addition, Kushana has been removed from class on several occasions, and has therefore missed instruction, due to her behavior during the months of September, October and November of the 2013-2014 school year (See tables below). In October of the 2013-2014 school year, Kushana’s parent (Valencia Johnson) requested that a school-based evaluation be completed. The team is completing an evaluation to determine if Kushana meets eligibility for a student with a disability. The evaluation will be used to determine if Kushana’s diagnoses of ADHD, ODD and her behaviors are significantly negatively impacting her educational


231

performance and if she will need the support of specially designed instruction in order to progress in the curriculum.

Instruction Missed (Minutes/Month) Month Instruction Missed

September 2013 711 minutes October 2013 1,545 minutes November 2013 1,597 minutes

INFORMATION PROVIDED BY PARENT: During a meeting on 9/16/13, Mrs. Johnson reported that Kushana is intelligent and works well with others. She reported Kushana enjoys art, going on the computer, and reading. She also reported Kushana is motivated by stickers and treats (e.g., candy).

Intervention History Individualized interventions have been implemented in order to manage Kushana’s behavior since Kindergarten. At Taylor Elementary, notes from a meeting on 12/17/12 indicated that the team planned to implement the following strategies: completing daily behavior reports, providing stickers every 15 minutes, sitting with the teacher for seatwork, preferential seating, allowing the use of a fidget item in order to keep hands to self, the opportunity to earn a choice from a choice wheel after earning two stickers, the use of a visual timer, and the use of a script for compliance behavior (i.e., “You have the choice to ‘work’ or take a time out in the office or room”). Notes from a meeting on 2/6/13 indicate that the team met to review Kushana’s progress after implementing the aforementioned strategies and her behavior was still a concern. However, it was reported that the behavior chart with stickers was working. Furthermore, the notes indicate


232

that the team planned to change the criteria for earning stickers and implement several other strategies. The team at Welch Elementary implemented strategies to manage Kushana’s behavior in 1st grade. On 9/10/13, Kushana’s teacher (Mrs. Hedges) began using a sticker chart to reward appropriate behavior approximately every 30 minutes. On 9/17/13, the intervention plan was modified and Kushana had the opportunity to earn candy in the morning and afternoon based on the number of stickers on her chart. On 10/23/13, the intervention plan was modified again. Kushana was given a chart with a picture of each activity throughout the day. During each activity, she had the opportunity to earn more frequent positive reinforcement for behaving appropriately (i.e., if she earned three green Velcro dots, her name was moved up on the class-wide clip chart). If Kushana did not behave appropriately, she was given a red Velcro dot, and if she earned three red Velcro dots she was sent to the office. On 11/1/13, the team decided Kushana’s behaviors would be ignored when she was brought to the office and she would be brought back to class when she was calm. In addition, Mrs. Hedges gave Kushana the opportunity to help in a Kindergarten classroom contingent upon displaying appropriate behavior in the morning and she earned this reward on two occasions. On 9/30/13 Kushana began participating in an ORE tutoring group from 9:10-9:40 and a Title 1 tutoring group from 10:15-10:40 four days per week. These groups provide additional opportunities to practice reading skills, such as letter sound fluency and nonsense word fluency skills.

General Intelligence Mrs. Johnson reported that evaluations have been completed by Children’s Hospital Medical Center. Several attempts were made to obtain the evaluations from Children’s Hospital and from the parent, but were not obtained by the time the evaluation was written. Mrs. Johnson reported Kushana was given an IQ test at Children’s Hospital but the specific test is unknown, and the school-based team did not administer an additional assessment because IQ tests cannot be repeated. The team agrees that Kushana presents with the ability to learn when her behaviors are under control. During this school year she has learned new concepts and skills when she has been in the classroom and her behaviors are under control.

Academic Skills Curriculum-Based Measures

As part of this evaluation, Kushana’s skills were assessed using AIMSweb curriculum-based measures (CBMs). AIMSweb are a set of procedures and assessments for measuring performance of key foundational skills from Kindergarten through 8th grade. AIMSweb CBMs were developed to measure recognized and empirically validated skills related to early literacy, early numeracy, reading, writing, and math outcomes. All of the measures have been thoroughly researched and demonstrated to be reliable and valid indicators of critical basic skills that are predictive of later reading, writing, and math achievement. READING Letter Naming Fluency Kushana was administered an AIMSweb Letter Naming Fluency (LNF) probe. This is a 1-minute assessment that looks at a student’s LNF skills and compares their performance to national peer norms. When given a 1st grade probe on 11/25/13, Kushana earned a score of 46 letter names correct (LNC) with 2 errors. The peer norm (50th percentile) for the Winter of 1st grade is 57 LNC. Her performance is considered to be slightly above the 25th percentile when compared to


233

Winter norms. Letter Sound Fluency Kushana was administered an AIMSweb Letter Sound Fluency (LSF) probe. This is a 1-minute assessment that looks at a student’s LSF skills and compares their performance to national peer norms. When given a 1st grade probe on 11/18/13, Kushana earned a score of 36 letter sounds correct (LSC) with 4 errors. The peer norm (50th percentile) for the Winter of 1st grade is 48 LSC. Her performance is considered to be at the 25th percentile when compared to Winter norms. Nonsense Word Fluency Kushana was administered an AIMSweb Nonsense Word Fluency (NWF) probe. This is a 1-minute assessment that looks at a student’s NWF skills and compares their performance to national peer norms. When given a 1st grade probe on 11/18/13, Kushana earned a score of 14 sounds correct (SC) with 16 errors. The peer norm (50th percentile) for the Winter of 1st grade is 54 SC. Her performance is considered to be below the 10th percentile when compared to Winter norms. Phoneme Segmentation Fluency Kushana was administered an AIMSweb Phoneme Segmentation Fluency (PSF) probe. This is a 1-minute assessment that looks at a student’s PSF skills and compares their performance to national peer norms. When given a 1st grade probe on 11/18/13, Kushana earned a score of 32 phonemes correct (PC) with 8 errors. The peer norm (50th percentile) for the Winter of 1st grade is 51 PC. Her performance is considered to be slightly above the 10th percentile when compared to Winter norms. Oral Reading Fluency Kushana was administered an AIMSweb Reading CBM (R-CBM) probe. This is a 1-minute assessment that looks at a student’s oral reading fluency skills and compares their performance to national peer norms. When given a 1st grade probe on 11/11/13, Kushana earned a score of 8 words read correctly (WRC) with 12 errors. The peer norm (50th percentile) for the Winter of 1st grade is 36 WRC. Her performance is considered to be below the 10th percentile when compared to Winter norms. Reading Comprehension Kushana was administered an AIMSweb MAZE probe. This is a 3-minute assessment that looks at a student’s reading comprehension skills and compares their performance to national peer norms. When given a 1st grade passage on 11/11/13, Kushana earned a score of 1 responses correct (RC) with 9 errors. The peer norm (50th percentile) for the Winter of 1st grade is 4 RC. Her performance is considered to be at the 10th percentile when compared to Winter norms. Before completing the MAZE assessment, Kushana was given a MAZE Practice Test. Although Kushana demonstrated understanding of the directions based on the Practice Test, it was not clear whether she understood how to complete the MAZE assessment. Rather than circling the most appropriate words in the parentheses, she often circled other words in the sentences. Kushana performs below the level of her peers in her LNF, LSF, NWF, PSF, reading fluency and reading comprehension skills based on this information. She has a relative strength in her LNF and LSF skills. Kushana was cooperative during assessment; she followed directions and appeared to be focused on the tasks. Her performance is likely a valid indicator of her abilities. On 11/25/13, the intern school psychologist attempted to administer Kushana additional assessments to gain more information regarding her academic skills. Kushana was administered an LSF assessment; however, she had difficulty focusing during the task (e.g., “jumped around” on the paper, bent over the side of her chair, put her head under the table). Therefore, the results


234

are not considered a valid indicator of her abilities. The intern school psychologist began to administer an NWF assessment; however, the assessment was discontinued due to Kushana’s behavior (e.g., getting out of her seat and walking around the room). WRITING Kushana was provided a prompt (“Every day after school my friends and I would go to the playground and…”) and was asked to think about what she would write for 1 minute and was given 3 minutes to write. Kushana’s writing was scored on total words written (TWW), words spelled correctly (WSC), and correct writing sequences (CWS). Her performance was compared to national peer norms.

Writing CBM Kushana 1st Grade Peer Norm (Winter)

Percentile

TWW 11 13 Between the 25th and 50th %

WSC 5 10 25th % CWS 0 5 Below the 10th

Kushana’s writing sample indicates she is below the peer norm (50th percentile) for TWW, WSC, and CWS. Her performance on this CBM indicates she demonstrates below average writing skills. The words Kushana produced were not in sentence form and did not include correct grammar or punctuation. It was difficult to determine whether the story she produced was on topic due to the spelling errors made. Kushana was cooperative during the assessment; she followed directions and appeared to be focused on the task. Her performance is likely a valid indicator of her abilities. MATH Computation Kushana was administered an AIMSweb Math Computation (M-COMP) CBM. This is an 8-minute math assessment that looks at a student’s computation skills and compares their performance to national peer norms. When given a 1st grade probe on 11/11/13, Kushana earned a score of 2 points. The peer norm (50th percentile) for the Winter of 1st grade is 28 points. Kushana’s performance is considered to be below the 10th percentile when compared to Winter norms. Kushana demonstrated difficulty with the following types of problems: adding two single-digit numbers, subtracting two single-digit numbers, adding one double-digit number and one single-digit number, adding two double-digit numbers, adding three single-digit numbers, and subtracting two double-digit numbers. The problems Kushana solved correctly were single-digit addition problems. Kushana’s performance on this CBM indicates she demonstrates below average math computation skills. During the assessment, Kushana followed directions and appeared to be focused on the task and therefore her performance is likely a valid indicator of her abilities. However, she finished before the 8-minute time limit and was prompted to attempt to complete the problems she had skipped. After attempting these problems, the assessment ended with 1 minute and 41 seconds remaining.

Functional Behavior Assessment Identified Behavior of Concern: “Tantrum” behaviors, such as disruptive crying, screaming, falling on the floor, leaving the assigned area and kicking objects (e.g., doors, chairs) are behaviors of concern. During an interview with Mrs. Hedges on 10/7/13, it was reported that Kushana had been sent to the office approximately 14 times within a two-week period. It was


235

also reported that Kushana remained in the office for approximately 30 minutes each time. During the interview on 10/7/13, Mrs. Hedges also reported that crawling on the floor was a behavior of concern during Language Arts and Math whole group instruction on the carpet. Formal observations conducted on 9/23/13, 9/30/13, 10/28/13, and 11/4/13 supported the teacher’s report that Kushana at times displays off-task motor behaviors during whole group instruction on the carpet. Antecedent: Based on teacher-collected data, Kushana has been initially removed from class most often from 9:00-10:00 and 12:00-1:00 (See graph below). During the interview on 10/7/13, Mrs. Hedges reported that Kushana’s off-task behaviors during whole group instruction on the carpet typically begin immediately when she gets to the carpet. During a follow-up meeting with Mrs. Hedges and Mrs. Johnson on 10/17/13, Mrs. Hedges also reported that Kushana has difficulty during small group and independent seatwork activities. Possible antecedents include the absence of attention, academic demands, and/or prompts and redirections. Consequence: When Kushana engages in the behaviors of concern, she typically receives teacher attention (i.e., prompts, redirections) and/or is able to avoid/escape (i.e., avoids/escapes engaging in academic activities, avoids/escapes completing work, sent to the office, sent to the Alternative Learning Program room). Hypothesized Function: The hypothesized functions of Kushana’s behavior are to obtain adult attention and/or escape.

At the beginning of the year, Kushana’s schedule included: Morning Work (8:50-9:15), Language Arts (9:15-11:20), Lunch/Recess (11:20-12:05), Language Arts (12:05-12:30), Language Arts/Math (12:30-1:00), Math (1:00-1:30), Specials (1:30-2:15), Math (2:15-2:45) and End of day/dismissal (2:45-3:15). At the end of September, Kushana’s schedule was modified and currently includes: Journal (8:50-9:15), ORE (9:10-9:40), Title 1 Group (9:40-10:15), Language Arts Groups (10:15-11:00), Language Arts Whole Group (11:00-12:45), Math (12:45-1:30), Specials (1:30-2:15), Math Groups (2:15-2:45), and End of day/dismissal (2:45-3:15).


236

The activities that have taken place from 9:00-10:00 and 12:00-1:00 include: Language Arts, ORE tutoring group, Title 1 tutoring group, and Math.

Rating Scales BASC-2

As part of this evaluation, Kushana’s behavior was assessed using the Behavior Assessment for Children – 2nd edition (BASC-2). The BASC-2 is a comprehensive measure of both adaptive and problem behaviors. Kushana’s teacher (Mrs. Hedges) provided information regarding Kushana’s behavior in the academic setting. The form was also given to Mrs. Johnson (parent/guardian) to provide information regarding Kushana’s behavior in the home setting, but this was not returned. The teacher rating scale assesses the broad domains of externalizing problems, internalizing problems, school problems, and adaptive skills. The ratings are also used to provide an overall behavioral symptoms index (BSI), which assesses the overall level of problem behaviors. The results of the rating scale completed are summarized in the table below. Each domain yields a t-score which describes the distance from the mean. They are standard scores with a mean of 50 and a standard deviation of 10. A percentile indicates the percentage of people scoring below a given raw score.

Classification Adaptive Scales Clinical Scales T-score Range

Very High Clinically significant 70 and above High At-risk 60-69

Average Average 41-59 At-risk Low 31-40

Clinically significant Very low 30 and above

Teacher Ratings Clinical Scales t-score Percentile Behavior

Level Hyperactivity 71 96 Clinically

significant Aggression 65 91 At-risk Conduct Problems 58 83 Average Externalizing Problems Composite 67 94 At-risk Anxiety 58 81 Average Depression 59 84 Average Somatization 65 91 At-risk Internalizing Problems Composite 64 91 At-risk Attention Problems 62 84 At-risk Learning Problems 62 86 At-risk School Problems Composite 63 90 At-risk Atypicality 66 92 At-risk Withdrawal 72 96 Clinically

significant BSI 70 96 Clinically

significant Adaptive Behaviors t-score Percentile Behavior


237

Level Adaptability 30 3 Clinically

significant Social Skills 37 11 At-risk Leadership 38 12 At-risk Study Skills 35 8 At-risk Functional Communication 36 10 At-risk

Adaptive Skills Composite 33 5 At-risk Based on combined norms, ages 6-7 Mrs. Hedges’ ratings indicate that Kushana demonstrates behaviors that are considered clinically significant in the areas of: hyperactivity, withdrawal, and adaptability. Kushana’s overall composite scores for externalizing problems, internalizing problems, school problems, and adaptive skills were in the at-risk range. Kushana’s overall composite score for behavioral symptoms index (BSI) was in the clinically significant range. The BASC-2 also identifies critical items that may deserve attention based on the ratings provided. According to the ratings provided by Mrs. Hedges, the following are concerning: eats things that are not food, falls down, has eye problems, has toileting accidents, and eats too little.

Behaviors Considered At-Risk/ Clinically Significant -Hyperactivity -Aggression -Somatization -Attention problems -Learning problems -Atypicality -Withdrawal Adaptability -Social skills -Leadership -Study skills -Functional communication

Composite Area Descriptions The externalizing problems composite is characterized by disruptive behavior problems such as aggression, hyperactivity, and delinquency. The internalizing problems composite includes scales that are not marked by acting out behavior. The school problems composite reflects academic difficulties including problems of motivation, attention, and learning and cognition. The adaptive skills composite summarizes appropriate emotional expression and control, daily living skills, communication skills, pro-social, organizational, study, and other adaptive skills. These behaviors are important for functioning at home and school, with peers and in the community. The Behavior Symptoms Index (BSI) reflects the overall level of problem behavior.

Observations Kushana was observed on 9/23/13, 9/30/13, 10/28/13 and 11/4/13 using a modified version of a systematic observation tool (the Behavioral Observation of Students in Schools-BOSS). This


238

involved recording specific target behaviors in 15-second intervals as they occur. Engagement behaviors are observed on the first second of each interval, and off-task behaviors are observed during the remaining 14 seconds of each interval. A sample of peer behavior is provided by sampling (every fifth interval) typical peers who are engaged in the same task as the target student. In Kushana’s class, three students were used as comparison peers and one was observed every 5th interval. Definitions: -Engagement (active, passive)- the student is demonstrating expected behavior and is doing what he/she is expected to be doing at the time. Active engagement is defined as those times when the student is actively attending to the assigned work. Passive engagement is defined as those times when the student is passively attending to assigned work. -Off-task (passive, motor, verbal)- the student is not demonstrating expected behavior or is engaged in another behavior that prevents them from attending to instruction or completing activities as expected. Off-task passive behavior occurs when the student demonstrates off-task behaviors that are not distracting to others but distracting to themselves. Off-task motor behavior occurs when the student is off-task by engaging in a behavior that requires movement. Off-task verbal behavior occurs when the student provides an off-topic verbalization (calling out, talking to their peer, yelling). Date: 9/23/13 Time: 10:05-10:35 Setting: Language Arts whole group instruction on carpet

Target Student: Peer Comparison:

% of Time Engaged 86.46% 100%

-Total % of Time Off-Task 17.71% 4.17% -% of Time Off-Task Motor 14.58% 4.17% -% of Time Off-Task Verbal 1.04% 0% -% of Time Off-Task Passive 3.13% 0%

Off-task Motor behaviors observed: Head down, Bottom off ground, Bending backwards and to side, Moving out of area on carpet, Stretching out leg on carpet, Moving arms

Off-task Verbal behaviors observed: Talking

Off-task Passive behaviors observed: Looking around room

Other Observation Notes: Kushana was observed from 10:05-10:35 during Language Arts whole group instruction. During the observation, the teacher provided instruction on the Smart Board and the students were seated on the carpet. The instruction included teacher directed instruction, opportunities to respond, feedback, and interactive reading. The content included rhyming, reading with the teacher, identifying sounds in words (e.g., “a” sound), and sounding out words. Based on this observation, Kushana demonstrated on-task behaviors at a rate lower than her peers. She demonstrated off-task behaviors at a rate higher than her peers. Specifically, she demonstrated off-task motor, verbal and passive behaviors at a rate higher than her peers. The majority of her off-task behaviors were motor. During this observation, Kushana received approximately 7 individual prompts to remain on-


239

task. The prompts included asking Kushana to sit on her bottom, sit up, return to her area on the carpet, and remain quiet. During the observation, at times Kushana was attending to instruction but engaging in off-task behaviors simultaneously. She required additional prompting to remain on-task. She seemed to respond to class wide pre-corrections to behave appropriately (e.g., sit up, hands in lap, eyes on board). Kushana attempted to participate, such as by raising her hand, but at times she would raise her hand when the teacher did not ask a question. She demonstrated the ability to read words correctly (e.g., “can”) and identify rhyming words correctly. She was able to sound of some of the sounds in the word “pal”, but struggled to blend the sounds on her own.

Date: 9/30/13 Time: 9:45-10:15

Setting: Language Arts whole group instruction on carpet


% of Time Engaged 83.33%

87.5%

-Total % of Time Off-Task 32.29% 12.51% -% of Time Off-Task Motor 30.21% 0% -% of Time Off-Task Verbal 0% 0% -% of Time Off-Task Passive 4.17% 12.5%

Off-task Motor behaviors observed: Turned around and facing away from instruction, Leaning forward on knees, Playing with glasses, Putting glasses in mouth, Bending over, Spinning around, Putting head down, Laying on stomach, Moving out of area on carpet, Bouncing, Moving arms, Biting shirt, Pulling shirt over eyes Off-task Passive behaviors observed: Looking around room

Other Observation Notes: Kushana was observed from 9:45-10:15 during Language Arts whole group instruction. During the observation, the teacher provided instruction on the Smart Board and the students were seated on the carpet. The instruction included teacher directed instruction, opportunities to respond, feedback, and interactive reading. Examples of the activities taking place include: answering questions about a story read by the teacher, phoneme segmentation, blending sounds, reading a story on the Smart Board along with the teacher, identifying words with certain sounds (e.g., words with the short “i” sound), and identifying rhyming words. Based on this observation, Kushana demonstrated on-task behaviors at a rate slightly lower than her peers, however there was not a significant discrepancy. She demonstrated off-task behaviors at a rate higher than her peers. Specifically, she demonstrated off-task motor behaviors at a rate much higher than her peers and off-task passive behaviors at a rate lower than her peers. Kushana and the peers observed did not demonstrate off-task verbal behaviors during the observation. During this observation, Kushana received approximately 5 individual prompts to remain on-task. The prompts included asking Kushana to turn around, face forward, and sit on her bottom. During this observation, Kushana had difficulty remaining engaged during whole group instruction. At times she was attending to instruction but engaging in off-task motor behaviors


240

simultaneously. She required additional prompting to remain on-task. She typically complied with re-directions but received them more often than her peers. Kushana attempted to participate, such as by raising her hand, but at times she would raise her hand when the teacher did not ask a question. She volunteered to participate by raising her hand on approximately 7 occasions. She demonstrated the ability to identify words in a story correctly (e.g., was asked to identify the word “little” and did so successfully), and sounds in words correctly (e.g., was asked to pick a short “i” word and did so successfully). There was one instance when Kushana was picked to stand up in front of the class and sound out the word “quick”. She attempted to do so on her own but needed help sounding out the word.

Date: 10/28/13 Time: 11:03-11:20

Setting: Language Arts whole group instruction on the carpet followed by transition to independent seatwork, Mrs. Hedges’ class



75%

-Total % of Time Off-Task 48.28% 25% -% of Time Off-Task Motor 37.93% 0% -% of Time Off-Task Verbal 0% 0% -% of Time Off-Task Passive 10.34% 25%

Off-task Motor behaviors observed: Turning in circles on carpet, Leaning forward with face on ground, Laying on floor, Rolling on floor, “Scooting” on floor, Moving arms, Turned around on carpet (facing away from instruction taking place at the front of class), Out of seat during independent seatwork Off-task Passive behaviors observed: Looking at students around her, Looking at wall

Other Observation Notes: Kushana was observed from 11:03-11:20 during Language Arts. For the majority of the observation, the students were seated on the carpet while the student teacher stood at the front of the room and provided instruction on the Smart Board. Kushana was seated in the first row on the carpet. The instruction included teacher directed instruction and opportunities to respond and receive feedback. After approximately 14 minutes, the teacher began passing out worksheets to the students. She asked the students to remain on the carpet until they were given their worksheet. Then, the students transitioned to their desks to complete the worksheets. Kushana stood up and left the carpet area before she received her paper. After the other students began working at their desks, Kushana returned to carpet and laid down, rolled on the floor, and “scooted” across the floor. The observation ended at 11:20 after the teacher removed Kushana from the class. Based on this observation, Kushana demonstrated on-task behaviors at a rate lower than her peers. Overall, she demonstrated off-task behaviors at a rate higher than her peers. She demonstrated off-task motor behaviors at a rate higher than her peers. She demonstrated off-task passive behaviors at a rate lower than her peers. Kushana and the peers observed did not demonstrate off-task verbal behaviors during the observation.


241

During this observation, Kushana received approximately 12 individual prompts to remain on-task and/or follow directions. 8/12 individual prompts occurred during carpet time and included asking Kushana to sit on her bottom and/or turn around. 4/12 individual prompts occurred during the transition period between carpet time and independent seatwork and included asking Kushana to return to the carpet to pick up her worksheet or behave appropriately (i.e., when she was laying, rolling, and “scooting” on the carpet). During each activity throughout the day, Kushana has the opportunity to earn 3 green dots for behaving appropriately and then she can “clip up” on the clip chart. Red dots are given for inappropriate behavior. If 3 red dots are given, Kushana is sent to Carrie Sanker’s room for a time-out. After Kushana left the carpet area, Mrs. Hedges asked her to come to her desk. She asked Kushana to make the choice between earning a green dot or a red dot and Kushana chose the red dot. Based on this observation, Kushana has difficulty remaining engaged during whole group activities and transitioning appropriately to independent seatwork activities. In addition, she requires additional prompting to remain on-task. Her off-task behaviors occur a rate higher than her peers.

Date: 10/28/13 Time: 9:14-9:33

Setting: ORE small group Target Student: Peer Comparison:

% of Time Engaged 100%

93.75%

-Total % of Time Off-Task 0% 6.25% -% of Time Off-Task Motor 0% 0% -% of Time Off-Task Verbal 0% 0% -% of Time Off-Task Passive 0% 6.25%

Off-task Motor, Verbal, and Passive behaviors observed: N/A

Other Observation Notes: Kushana was observed from 9:15-9:33 during Oral Reading small group time. During the observation, Kushana and 4 peers were seated on one side of a table while the teacher sat across from them. Kushana was seated on the left side of the table and her peers were seated to her right. The instruction included teacher directed instruction, opportunities to respond, and feedback. During this time, the students formed letters with manipulatives (i.e., Twizzlers), and practiced identifying letter sounds and blending sounds. Based on this observation, Kushana demonstrated on-task behaviors at a rate higher than her peers. She demonstrated off-task behaviors at a rate lower than her peers. During this observation, Kushana did not need to be given any individual prompts to remain on-task. During each activity throughout the day, Kushana has the opportunity to earn 3 green dots for behaving appropriately and then she can “clip up” on the clip chart. Red dots are given for inappropriate behavior. If 3 red dots are given, Kushana is sent to Carrie Sanker’s room for a time-out. Kushana earned positive attention for behaving appropriately and 3 green dots during this time. During this observation, Kushana demonstrated the ability to remain on-task during a small-


242

group academic activity with minimal support. She also demonstrated the ability to form letters correctly, and identify letter sounds and blend sounds to form words correctly (e.g, “hib”, “wing”, “ring”). In addition, she asked for help when needed (e.g., asked for help with making the letters “m” and “w” and forming the word “is”).

Date: 11/4/13 Time: 11:02-11:28

Setting: Language Arts whole group instruction on the carpet followed by whole group instruction/independent work at desk, Mrs. Hedges’ class



94.74%

-Total % of Time Off-Task 16.88% 10.53% -% of Time Off-Task Motor 11.69% 0% -% of Time Off-Task Verbal 0% 0% -% of Time Off-Task Passive 5.19% 10.53%

Off-task Motor behaviors observed: Turning to side, Leaning head down, Leaning head to side, Playing with toy on desk

Off-task Passive behaviors observed: Looking down, Looking at another student’s paper, Staring at paper

Other Observation Notes: Kushana was observed from 11:02-11:28 during Language Arts. At the beginning of the observation, the students were seated on the carpet while the student teacher stood at the front of the room and provided instruction on the Smart Board. Kushana was seated in the first row on the carpet initially, but the teacher asked her to move to the back row on the carpet after approximately 1 minute. The instruction included teacher directed instruction, opportunities to respond, feedback, and error correction. The content involved rhyming, singing a “short e” song, and identifying short e sounds in words. After approximately 12 minutes, the teacher asked the students to transition to their seats. While at their desks, the students worked on a handwriting/spelling activity. Based on this observation, Kushana demonstrated on-task behaviors at a rate lower than her peers. Overall, she demonstrated off-task behaviors at a rate higher than her peers. She demonstrated off-task motor behaviors at a rate higher than her peers. She demonstrated off-task passive behaviors at a rate lower than her peers. Kushana and the peers observed did not demonstrate off-task verbal behaviors during the observation. During this observation, Kushana received approximately 3 individual prompts to remain on-task. 2/3 individual prompts occurred while on the carpet and 1/3 individual prompts occurred while at her desk. During each activity throughout the day, Kushana has the opportunity to earn 3 green dots for behaving appropriately and then she can “clip up” on the clip chart. Red dots are given for inappropriate behavior. Rather than being sent to Carrie Sanker for a time-out when she exhibits inappropriate behavior, she is sent to the office and her behaviors are ignored until she is ready to return to class. The intern and Mrs. Hedges discussed providing more frequent positive attention during whole group instruction on the carpet and ignoring inappropriate


243

behaviors if they are not disruptive to others. In addition, it was decided that if Kushana is able to demonstrate appropriate behaviors during certain points throughout the day, she could help in the Kindergarten classroom as a reward. Based on this observation, Kushana has the ability to remain engaged during whole group activities and independent seatwork activities.

Dunn-Rankin Reward Preference Inventory

Kushana was given the Dunn-Rankin to assess what most motivates her. The inventory asks 40 questions that assess if a child is most motivated by: adult approval, tangibles, independence, peer approval, or competition. This information can be used to help in planning rewards for interventions. Kushana’s responses indicate that she is most motivated by adult approval. These types of rewards include: positive notes, work shown to the class, positive praise for good behaviors, recognition from adults, being assigned jobs from adults, etc. Kushana’s secondary motivator was tangibles. This includes receiving items such as bubble gum, a candy bar, ice cream, or a soft drink, for example. Adult approval (11) and tangibles (9) received higher ratings than the other categories of rewards: independence (7), peer approval (7), and competition (6). However, it is important to note that on the second page of the inventory, Kushana chose the second option for every question; therefore, it would be beneficial to administer the inventory again to verify the results are an accurate representation of what most motivates her.

DESCRIPTION OF EDUCATIONAL NEEDS: (based on the information above, prioritize areas of need (skills the student needs to develop) that are important to the student’s educational success): Kushana will likely benefit from continued development of the following skills: -Increase early literacy skills to eventually increase reading fluency to support comprehension. -Increase reading comprehension skills to be able to answer questions about a text. -Increase written expression skills to eventually be able to write sentences using correct spelling, capitalization, punctuation, and grammar. -Increase basic math skills to eventually be able to add, subtract, multiply and divide. -Increase behavior skills to be able to be successful in the academic, home, and community settings. -Respond appropriately to adult directions and redirections. -Increase pro-social skills to be able to build and maintain relationships. -Demonstrate appropriate coping strategies and ask for breaks when presented with a difficult or undesired task or activity. -Consistently complete academic tasks and activities as expected. -Increase the amount of time in the classroom to be able to receive instruction and increase academic skills. IMPLICATIONS FOR INSTRUCTION AND PROGRESS MONITORING: (Summarize how the educational needs impact the student’s progress in the general education curriculum. What types of supports, services, interventions, accommodations, modifications, or specially designed instruction are needed to address the needs and enable the child to progress in the curriculum?): Kushana will likely benefit from: -Direct instruction in early literacy skills, reading, reading comprehension, writing and math


244

-Direct instruction in appropriate behavior skills -Repeated opportunities to practice skills -Repeated opportunities to receive feedback on skills -Social skills training -Modeling of appropriate behavior -Social stories -Rewards contingent on demonstrating expected behavior that may incorporate adult approval or attention -Use of positive praise and reinforcement -Breaks -Appropriate coping strategies when frustrated -Small group instruction -Highly structured classroom environment -Frequent prompts and re-directions to demonstrate expected behavior -Continued access to typical peers as positive role models to build social skills

Katie Schatz, M.Ed. 12/2/13 Evaluator’s Signature Date


245

Appendix E: Problem Solving Interview Questions


246


247

Appendix F: Dunn Rankin Reward Preference Inventory


248

Appendix G: Data Collection Form


249

Appendix J: Intervention Script

Behavior Intervention Script

Materials needed: list of classroom expectations, visual schedule with 3 Velcro dots below each picture, 6 green-‐ and red-‐colored dots First Day Intervention Implemented

1. Describe new behavior plan a. Review the classroom expectations b. Explain that student should follow these expectations during each activity

throughout the day c. Show the visual schedule to student and point to the Velcro dots below each

picture d. Explain that during each activity, student will have the opportunity to earn a

green-‐colored dot for being in class and following the expectations; if student earns 3 green-‐colored dots during an activity, her name will be moved up on the class-‐wide clip chart

e. Also explain that if student is engaging in behaviors that are “disruptive”, she will be given a red-‐colored dot; if student receives 3 red-‐colored dots during an activity, she will review the classroom expectations and complete assignments in the ALP room until she is calm

Throughout Each Day

1. Review the behavior plan with student at the beginning of the day if needed 2. If student is caught behaving appropriately in class, give her a green-‐colored dot and

positive praise a. If student earns 3 green-‐colored dots, move her name up on the class-‐wide clip

chart 3. If student exhibits “disruptive behavior”, give her a red-‐colored dot and review the

classroom expectations with her a. If student receives 3 red-‐colored dots, take her to ALP room to complete

classroom assignments until she is calm i. While in the ALP room, review the classroom expectations with student

and instruct her to complete classroom assignments Progress Monitoring

• Every day, monitor student’s time-‐in and time-‐out of class by recording the time she was removed from class and the time she returned

IMPROVING THE LITERACY

250

Improving the Literacy Skills of a First-Grade Student

A Tier 3 academic intervention was implemented at a Midwestern elementary school

with a first-grade student, Sarah. The purpose of the intervention was prevention to help reduce

risk by increasing nonsense word fluency (NWF) and oral reading fluency (ORF) skills. The

student was targeted for intervention due to the low scores she received on literacy benchmark

assessments compared to benchmark goals and peer micro-norms. The Dynamic Indicators of

Basic Early Literacy Skills (DIBELS) – 6th edition were used to establish benchmarks and

progress monitor NWF and ORF once per week. In October 2013, the student began

participating in a Tier 2 intervention to target literacy skills through the use of the Triumphs

Reading Intervention Program four times per week for 30 min. Visual analysis of progress

monitoring data indicated Sarah was not improving at the rate necessary to meet end-of-year

goals. Therefore, the intervention was intensified by implementing additional strategies to

provide Sarah extra opportunities to practice and receive feedback on identifying letter sounds

and blending sounds, four times per week for 10 min.

Nonsense word fluency benchmark data collected in January indicated Sarah was

performing below the 2nd percentile when compared to all first grade students. Research

demonstrates mastery of NWF skills supports students in becoming more fluent readers;

therefore, the intervention targeted this skill initially (Joseph, 2008). The instructional hierarchy

model was used to match instruction to the student’s needs. According to Daly III, Chafouleas,

and Skinner (2005), the model describes a learning hierarchy through which students progress as

they learn new skills. Initially, a student needs to acquire new responses. After the responses are

part of a student’s repertoire, he or she should respond accurately more often. Then, the student

can become more fluent and respond both accurately and quickly. Providing additional


251

opportunities for students to practice and receive error correction and feedback on skills is an

appropriate approach to increase accurate and fluent responding and help reduce risk. The

intervention involved these essential components, with the goal of helping Sarah acquire and

become fluent in literacy skills.

The results of the intervention indicate that Sarah’s scores on the NWF and ORF

assessments increased compared to baseline on average, suggesting the intervention procedures

had a positive effect. Although Sarah did not reach the goals by the time data were reported for

this consultation, the intern continued to collaborate with the Title 1 tutor throughout the rest of

the year and when Sarah met the NWF goal, the team planned to modify the intervention to more

specifically target ORF skills. The ROI results suggest Sarah will continue to make valuable

gains throughout the rest of the year. The social validity results suggest the Title 1 tutor felt she

had adequate input in developing the intervention script, she liked the procedures used and she

thought the intervention benefited the student.

Methods


The elementary school consisted of 129 first-grade students across six classrooms. At the

beginning of the year, Sarah was targeted for Tier 2 and progress monitored based on initial

DIBELS benchmark scores, which placed her in the “at-risk” category. Her NWF score also

represented the number of sounds identified by the 12th percentile of first-grade students during

the beginning of the year benchmark period. A Title 1 tutor, Ms. Roberts, referred Sarah for

further academic support due to concerns regarding her progress in the Tier 2 reading

intervention group. Visual analysis of progress monitoring data indicated Sarah’s scores were not

improving at the rate necessary to meet end-of-year goals. Furthermore, her middle of the year


252

NWF benchmark score represented the number of sounds identified by the 1st percentile of first-

grade students. The team decided to intensify the intervention by implementing additional

strategies to increase Sarah’s scores on literacy assessments. Permission to work with Sarah and

include her data in the school psychologist intern’s report was obtained from her parent, teacher

and the school psychologist.

The Title 1 tutor was responsible for implementing the intervention with Sarah in her

office and collecting progress monitoring data. The intern school psychologist assumed a

consultative role throughout the intervention process. In addition, the intern occasionally

monitored the administration of probes to collect inter-scorer agreement (ISA) data and

conducted intervention adherence checks while the intervention was in place. The intern received

supervision from a field supervisor and university supervisor.

Setting

This consultation was conducted in a Midwestern elementary school for students in

Kindergarten through third grade. The elementary school supported multi-tiered systems of

support (MTSS) procedures, which allowed the intern to work with teachers to address students’

needs. The intervention sessions were implemented in the Title 1 tutor’s office four days per

week.


According to Joseph (2008), reading is one of the most fundamental skills that permit a

person to survive and thrive in society, and one of the main goals of education is to help students

move from the learning-to-read stage to a reading-to-learn stage (p. 1163). Yet, some children

have difficulty acquiring the fundamental reading skills necessary to read-to-learn (p. 1163). The


253

critical component skills of reading are phonemic awareness, alphabetic principle, fluency,

vocabulary, and comprehension (p. 1168).

Nonsense word fluency. Acquirement of the alphabetic principle, or letter-sound

correspondences, will help students to eventually read words with ease or automatically (Joseph,

2008). NWF is a measure of students’ knowledge of the alphabetic principle. Research has

demonstrated that NWF is a predictor of reading proficiency. Fien, Baker, Smolkowski, Mercier

Smith, Kame’enui, and Thomas Beck (2008) examined the validity of NWF as an index of

reading proficiency for students in kindergarten through second grade, and results revealed that

NWF accounted for moderate to large amounts of variation on two criterion measures (ORF and

the SAT-10). This evidence supports the use of NWF to screen students for reading issues and

predict early reading proficiency.

DIBELS NWF probes were used to measure Sarah’s responsiveness to intervention.

Benchmark data in NWF were assessed during the beginning of the year benchmark period for

first grade. Sarah was selected for intervention and received additional progress monitoring due

to the low scores she received when compared to beginning of the year DIBELS benchmark

goals and peer micro-norms (Good & Kaminski, 2002). When visual analysis of progress

monitoring data indicated Sarah’s scores were not improving at the rate necessary to meet end-

of-year goals, the team decided to implement additional intervention strategies to target NWF

skills. To assess student progress toward goals and evaluate intervention effectiveness, NWF

progress monitoring probes were administered to Sarah during the intervention phases.

Sarah’s progress was tested using DIBELS NWF progress monitoring probes once per

week. NWF referred to the number of letter sounds a student was able to identify on each one-

minute assessment. The student could either read the whole word or say the sound of each letter.


254

The examiner calculated the number of letter sounds correctly identified within one minute. A

letter was marked as an error if the student omitted the letter, struggled with the letter for more

than three seconds, said the sound of a different letter, or said something that was not a letter

sound. If a student read the whole word, the word was marked as an error if the student

pronounced the word incorrectly.

Oral reading fluency. When Sarah reached the goal for NWF, the team would modify

the intervention to more specifically target ORF skills. Fluency is another critical component

skill of reading. According to Joseph (2008), fluent readers can read words accurately,

effortlessly, and quickly. Furthermore, reading fluency is related to reading comprehension

performance (p. 1170). A study that investigated the predictive validity of scores on the DIBELS

evidenced that students’ first grade ORF scores were significant predictors of performance on

two standardized tests of reading proficiency (Goffreda, Diperna, & Pedersen, 2009). Thus,

evidence also supports the use of ORF to predict reading proficiency.

DIBELS ORF probes were also used to measure Sarah’s responsiveness to intervention.

Benchmark data in ORF are not assessed until the middle of the year for first-grade students.

However, when Sarah was selected for intervention at the beginning of the year, Ms. Roberts

began progress monitoring her using ORF probes once per week. When benchmark data in ORF

were assessed during the middle of the year, Sarah received the low scores when compared to

middle of the year DIBELS benchmark goals (Good & Kaminski, 2002). Furthermore, based on

peer micro-norms Sarah’s ORF benchmark score represented the number of words read correctly

by the 17th percentile of first-grade students. To assess student progress toward goals and

evaluate intervention effectiveness, ORF progress monitoring probes were administered to Sarah


255

during the intervention phases. As mentioned previously, when Sarah reached the goal for NWF,

the team would modify the intervention to more specifically target ORF skills.

Sarah’s progress was tested using DIBELS ORF progress monitoring probes once per

week. ORF referred to the number of words read correctly on each one-minute assessment. A

response was marked correct if the student read the word correctly, pronounced the word

correctly, and read the words in the correct order; if the student initially mispronounced or

omitted a word, but self-corrected within 3 sec; and if the student pronounced a word, but had a

common speech problem such as a lisp. The examiner calculated the number of words read

correctly within one minute. A word was marked as an error if the student mispronounced or

substituted a word, omitted or skipped the word, hesitated or struggled to correctly produce the

word for more than 3 sec, or transposed the order of two words.


The Title 1 tutor administered NWF and ORF probes to collect baseline and progress

monitoring data. Ms. Roberts referred Sarah for further academic support on 2/10/14, which was

near the middle of the year benchmark period. Therefore, during the baseline phase and first

intervention phase ISA data were not collected. During the second intervention phase, the intern

assessed the reliability of measurement by monitoring the administration of probes and recording

correct responses along with the Title 1 tutor for 25% of measurement occasions to collect ISA

data (Table 1). Inter-scorer agreement was calculated by dividing the number of agreements by

the total number of agreements and disagreements (Reed & Azulay, 2010).

Table 1

Inter-Scorer Agreement Data

Date NWF ORF


256

3/21/14 97.87% 100%


The goal of the academic intervention was to increase Sarah’s NWF score to 50 correct

letter sounds per minute and ORF score to 40 words read correct per minute by June 11, 2014.

These goals were based on DIBELS – 6th edition end of year benchmark goals for first grade

(Good & Kaminski, 2002). These scores are considered to be in the “established” and “low risk”

categories. The school set these goals for all first grade students to achieve by the end of the

year.

Visual analysis of the progress monitoring data was used to help determine whether

Sarah reached the goals or if a change in intervention was needed. Initially, decision rules

applied to Sarah’s progress based on NWF scores. After Sarah met or exceeded the NWF goal,

the team would modify the intervention to more specifically target ORF. The decision rule for

change in intervention was four consecutive points that fell below the aimline while the


this point, the team met to discuss how the intervention could be changed to address Sarah’s

needs.


The intern hypothesized Sarah received low scores on literacy assessments because she

had not been given enough opportunities to practice identifying letter sounds and blending

sounds, and thus had also not received enough feedback on these skills. The intern and Title 1

tutor agreed the aforementioned skills needed to be specifically targeted for intervention to

support the development of oral reading fluency skills. The academic intervention involved


257

providing Sarah additional opportunities to practice and receive feedback on identifying letter

sounds and blending sounds. These strategies supplemented the support the student was provided

through the Tier 2 small group intervention.

Accountability Plan

An ABC design was used to examine the effects of the academic interventions on Sarah’s

performance on NWF and ORF assessments. Two baseline points were obtained for the NWF

assessment prior to intervention implementation (A). Subsequently, the reading intervention was

implemented (B). ORF data were collected on nine occasions during this phase. A third phase

involved providing the Title 1 tutor with performance feedback and implementing additional

intervention strategies for 10 min per day (C). The aforementioned target variables were

measured once per week for the student until she consistently met the goals.


Baseline condition. Prior to implementation of the intervention, the Title 1 tutor

collected baseline data on Sarah’s performance on DIBELS NWF assessments. Two baseline

data points were collected for NWF. As mentioned previously, Ms. Roberts began monitoring

Sarah’s progress on ORF probes after she began participating in the Triumphs reading

intervention group. Baseline data are displayed and discussed in the results section below.

Triumphs reading intervention program. In October 2013, Sarah began participating

in an intervention utilizing the Triumphs Reading Intervention Program four times per week for

30 min. The intervention was aimed at improving the scores Sarah received on NWF and ORF

assessments. The strategies the Title 1 tutor utilized involved the critical elements of strong

academic interventions, including modeling/demonstration, opportunities to respond, and

immediate feedback and error correction (Johnson, 2008).


258

Modeling can be utilized to teach appropriate reading behavior. A study conducted by

McCurdy, Cundari, and Lentz (1990) evidenced that children improved their oral reading skills

by observing an individual read a passage proficiently. Additionally, it has been demonstrated

that attentively listening and following along as a teacher or capable peer reads can improve

reading skills (Knapp & Winsor, 1998). Listening while reading involves teachers modeling

reading a passage by reading it aloud to students, requiring them to follow along with their

fingers, and instructing them to reread the passage independently (Johnson, 2008). Listening

while reading has been found to improve oral reading accuracy and fluency (Daly & Martens,

1994).

Another critical element of academic interventions is providing opportunities to respond.

According to Joseph (2008), when educators provide plenty of opportunities for students to read,

students are more likely to acquire, maintain, and generalize skills. Furthermore, Sindelar,

Monda, and O’Shea (1990) demonstrated that increasing practice opportunities to respond could

increase reading fluency and improve recall for students regardless of level of functioning. Thus,

increasing opportunities for students to identify letter sounds, blend sounds and read should help

students acquire these skills and increase the fluency with which they demonstrate these skills.

After a student responds to a given prompt, it is also important for the child to receive

immediate feedback on his or her response. Students should be immediately corrected when they

make errors in reading and encouraged to practice reading words they incorrectly identified to

reduce the likelihood of making the same mistakes again (Nelson, Alber, & Grody, 2004;

Wordsdell et al., 2005). Also, students should be reinforced for appropriate reading behaviors to

strengthen those behaviors (Skinner, Pappas, & Davis, 2005). Nelson et al. (2004) demonstrated


259

that when students are given numerous opportunities to respond and error correction procedures

are provided, students particularly gain fluency skills.

During intervention sessions, the Title 1 tutor coupled modeling, opportunities to

respond, and immediate feedback and error correction to improve Sarah’s NWF and ORF skills

(Appendix A). Typically, during intervention sessions the tutor wrote several “sight words” on

the board and reviewed the words with the students. Next, the tutor retrieved a one-paragraph

passage from the Triumphs materials. The tutor read the passage out loud while the students

followed along by looking at the same passage in front of them. Afterward, the tutor read the

passage again, but paused after each sentence and instructed the students to repeat the sentences

after her. For the next activity, the tutor provided the students with a short book from the

Triumphs materials. Similar to the previous activity, the tutor read each sentence, paused, and

then instructed the students to repeat the sentences after her all together. Lastly, the tutor gave

each student an opportunity to read the entire book individually. The tutor provided praise for

correct reading behavior and immediate error correction if a student made a mistake while

reading.

Feedback conditions. In February 2014, the Title 1 tutor referred Sarah for further

academic support due to concerns regarding her progress in the Tier 2 reading intervention

group. Visual analysis of progress monitoring data indicated Sarah’s scores were not improving

at the rate necessary to meet end-of-year goals. Furthermore, her middle of the year NWF

benchmark score represented the number of sounds identified by the 1st percentile of first-grade

students. Therefore, the team decided to intensify the intervention by implementing additional

strategies to target NWF skills.


260

Upon implementation of the NWF activities, graphs were shared with the Title 1 tutor

after every three or four data points had been collected to demonstrate student progress and make

necessary decisions based on data. If Sarah’s NWF scores experienced three or four points below

the aim line, the graph was shared with the tutor. In addition, if intervention adherence was low,

intervention procedures were reviewed. During this time, intervention modifications and

additional intervention strategies were discussed and collaborative decisions were made.

Additionally, when data indicated Sarah met the NWF goal, the intern and tutor would discuss

modifying the intervention to more specifically target ORF skills.

Supplemental nonsense word fluency activities. In February 2014, visual analysis of

progress monitoring data indicated Sarah’s NWF scores were not improving at the rate necessary

to meet end-of-year goals. According to Joseph (2008), when students have difficulty decoding

words or making letter-sound associations, phonics needs to be directly taught by demonstrating

how to make one-to-one correspondences with letters and sounds in words, and having students

complete this task with guided practice and feedback. To increase Sarah’s NWF skills, the tutor

began implementing NWF activities with her four times per week for 10 min. The tutor

implemented one of three activities each day (Appendix B). Each activity involved showing

Sarah nonsense words, instructing her to say the sound of each letter, and asking her to blend the

nonsense words together. If Sarah responded correctly, the tutor provided praise. If she

responded incorrectly, the tutor corrected the error and prompted Sarah to respond again.

Adherence Data

Adherence was assessed by the intern through direct observation with a procedural

checklist based on the essential components of the intervention script (Appendix C). Adherence

was assessed for 1.32% of intervention sessions during the second phase and 8.33% of


261

intervention sessions during the third phase (Table 2). If intervention adherence was low, the

intern and Title 1 tutor would review the procedures and made necessary changes. Additionally,

beginning during the third intervention phase, the Title 1 tutor was asked to fill out a monthly

calendar of whether or not the intervention was implemented each day beginning on 3/17/14

(Appendix D).

Table 2

Adherence

Date Adherence

2/10/14 100%

3/17/14 100%

Social Validity

Social validity was collected throughout the intervention process through frequent

meetings and conversation between the practicum student, Title 1 tutor and first-grade teacher. In

addition, the Title 1 tutor filled out a social validity questionnaire to indicate her satisfaction with

the intervention and its effectiveness, which can be seen below.

Table 3 demonstrates the results of the social validity form filled out by the Title 1 tutor.

The checkmark in the cells represents the ratings that were chosen. The Title 1 tutor “strongly

agreed (5)” with five out of six items on the survey, indicating she found the intervention

valuable. She “agreed (4)” with the item “the intervention was easy to include in my daily

routine.”

Table 3



262





- - - - ✓


- - - - ✓


- - - - ✓


- - - ✓ -


- - - - ✓


- - - - ✓

Results

Individual baseline and progress monitoring data for NWF and ORF are displayed below

in Figures 1 and 2. In support of visual analysis, summary statistics were calculated in Tables 4


263

and 5. The means (M) and standard deviations (SD) of the target variables were calculated. The

percentage of non-overlapping data (PND), goal attainment scaling (GAS), and effect size were

also calculated for the target variables. For the purpose of this consultation entry, data were

reported through 3/21/14. However, the Title 1 tutor continued to implement the interventions

throughout the rest of the year and data were collected on Sarah’s progress until the end of the

year or until she met the goals.

Figure 1 shows the results of the intervention on Sarah’s NWF scores. The level of

Sarah’s performance on the NWF assessment was low-to-moderate during baseline (M=17.5;

SD=9.19). During the first intervention phase, visual analysis of the NWF data showed a slightly

increasing trend; however, the level of Sarah’s performance remained in the low-to-moderate

range (M=20.5; SD=4.12). Although the third phase of the intervention was scheduled to begin

during the week of 2/17/14, Sarah was absent from school from 2/17/14 through 2/28/14.

Therefore, the third phase of the intervention was implemented beginning on 3/3/14. During the

third intervention phase, the NWF data moved in an increasing trend and the level of Sarah’s

performance was in the moderate-to-high range (M=37.75; SD=6.45). Upon implementation of

additional NWF activities during the third phase, it can be seen that 100% of the NWF scores

were above the scores achieved during the second phase. Overall, the intervention mean was

27.4 (SD=10.2). Sarah did not meet the NWF goal by the time data were reported for this

consultation, but the intern continued to collaborate with the Title 1 tutor throughout the rest of

the year to address the student’s needs.


264

Figure 1. NWF scores for Sarah

Figure 2 shows the results of the intervention on Sarah’s ORF scores. The level of

Sarah’s performance on the ORF assessment was low during the first intervention phase and the

trend of the data was slightly variable (M=5.67; SD=1.73). It can be seen that the number of

errors made while reading was high. On average, Sarah’s accuracy was 36.38% during the

Triumphs phase, which suggests Sarah had not yet acquired the skills needed to read fluently.

The implementation of additional NWF activities during the second intervention phase seemed to

have a positive impact on Sarah’s performance on ORF assessments. Although the level of

Sarah’s performance remained in the low range, visual analysis of the ORF data showed an

increasing trend (M=12.33; SD=1.53). On average, Sarah’s accuracy increased to 58.97%. The

data suggest the additional feedback and error correction procedures implemented through the

NWF fluency activities began to build Sarah’s accuracy, which is necessary before being able to


265

read fluently. As mentioned previously, the intern continued to collaborate with the Title 1 tutor

throughout the rest of the year and when Sarah met the NWF goal, the team planned to modify

the intervention to more specifically target ORF skills.

Figure 2. ORF scores for Sarah

Summary statistics for Sarah are shown in Tables 4 and 5. Effect size was calculated by

subtracting the baseline mean from the intervention mean then dividing that number by the

baseline standard deviation. PND was calculated by dividing the number of intervention points

that did not overlap with the baseline points by the total number of intervention points. GAS was

also determined for the students, where “0” corresponds to no progress toward the goal, “1”

corresponds to progress made toward the goal, and “2” indicates the goal was met.

In addition to the above statistics, rate of improvement (ROI) was calculated for each

target variable. Needed ROI was calculated by subtracting the baseline median from the goal,

then dividing that number by the number of weeks from the time the intervention began to the


266

end of the year. Actual ROI was calculated by subtracting the baseline median from the most

recent intervention score, then dividing that number by the number of weeks the intervention has

been in place.

Table 4

Summary Statistics for NWF and ORF


Baseline Mean

Baseline SD


Intervention Mean

Intervention SD

Effect Size

PND GAS

NWF 2 17.5 9.19 10 27.4 10.2 1.07 40% +1 ORF 9 5.67 1.73 3 12.33 1.53 3.85 100% +1

Table 5

NWF and ORF ROI

Needed ROI Actual ROI

NWF 1.12 1.55

ORF 2.69 3.0

Discussion

The progress monitoring data indicate that Sarah’s scores on the NWF and ORF

assessments increased compared to baseline on average, suggesting the intervention procedures

had a positive effect. Although Sarah did not reach the goals by the time data were reported for

this consultation, the intern continued to collaborate with the Title 1 tutor throughout the rest of

the year and when Sarah met the NWF goal, the team planned to modify the intervention to more

specifically target ORF skills. The ROI results suggest Sarah will continue to make valuable

gains throughout the rest of the year. The social validity results suggest the Title 1 tutor felt she


267

had adequate input in developing the intervention script, she liked the procedures used and she

thought the intervention benefited the student.


effectiveness. Sarah was targeted for intervention at the beginning of the year, prior to the

intern’s involvement with this consultation. Sarah was placed into a Title 1 tutoring group to

target literacy skills through the use of the Triumphs Reading Intervention Program four times

per week for 30 min. However, Triumphs may not have been the most appropriate intervention

for Sarah as her data suggested she needed to increase her NWF skills. Sarah required extra

practice and error correction on basic reading skills, whereas the strategies used as part of the

Triumphs intervention were more focused on targeting ORF skills and placed less emphasis on

identifying letter sounds and blending sounds. Therefore, when the Title 1 tutor referred Sarah

for further academic support, the team intensified the intervention by implementing additional

strategies to provide Sarah extra opportunities to practice and receive feedback on identifying

letter sounds and blending sounds, four times per week for 10 min. As a school psychologist, I

will continue to consult with teams to implement interventions that are matched to student needs.

The Triumphs intervention was implemented in October and the Title 1 tutor collected

progress-monitoring data on Sarah’s NWF and ORF skills throughout the year. However, ISA

data were not collected during the first intervention phase as the intern became involved with the

consultation in February. During the second phase of the intervention, the intern collected ISA

data for 25% of measurement occasions. As often as possible, ISA data should be collected for at

least 20-25% of measurement occasions to assess the reliability of measurement.

The elementary school utilized DIBELS – 6th edition assessments to benchmark students

three times per year and progress monitor students throughout the year. Therefore, the school set


268

goals based on DIBELS – 6th edition benchmarks (Good & Kaminski, 2002). However, it is

important to acknowledge that a newer version of DIBELS was recently developed – DIBELS

Next. Although the benchmark goals for DIBELS – 6th edition and DIBELS Next are both

supported by the DIBELS Data System (DDS), the DIBELS Next goals are recommended as

they are based on more recent national peer norms (University of Oregon Center on Teaching

and Learning, 2012). Therefore, DIBELS Next assessments and benchmarks would have been

the most appropriate to utilize; however, the elementary school had not yet adopted this version.

As a school psychologist, I will encourage my school building to adopt the most current version

of assessments available and set goals based on benchmarks developed from national peer norm

data.

Another limitation is that additional baseline data should have been collected for NWF.

As mentioned previously, the intern became involved with the consultation in February and after

baseline data had been collected. It would have been beneficial if at least three baseline data

points had been collected to develop a stable trend to use as comparison with the intervention

phases. Additionally, Ms. Roberts began monitoring Sarah’s progress on ORF probes after she

began participating in the Triumphs reading intervention group. Therefore, “true” baseline data

were not collected for ORF. It would have been beneficial if baseline data in ORF had been

collected on at least three occasions prior to implementation of the Triumphs intervention. In the

future, I will encourage teams to collect baseline data in order to analyze student response to

intervention.

Adherence data should have been collected for at least 20-25% of intervention sessions.

Again, the intern became involved with the consultation in February; therefore, adherence was

only assessed for 1.32% of sessions during the first intervention phase. During the second


269

intervention phase, adherence was assessed for 8.33% of sessions by the time data were reported

on 3/21/14, which amounted to once during the three-week period since the phase began. The

intern was in the school building one day per week, and at times other obligations conflicted with

the time the Title 1 tutor implemented the intervention with Sarah. However, the intern continued

to attempt to conduct intervention adherence checks throughout the rest of the year. In the future,

adherence checks should be conducted for at least 20-25% of intervention sessions to ensure the

intervention is being implemented consistently and as intended.


components of my model of practice. As a scientist-practitioner, I engaged in data-based

decision making to evaluate intervention effectiveness and link assessment results to

interventions matched to Sarah’s needs. Data were analyzed to collaboratively develop a more

intensive intervention plan to provide Sarah additional opportunities to practice and receive

feedback on early literacy skills. Progress monitoring data were continually collected to monitor

Sarah’s responsiveness to research-based intervention and would ultimately be analyzed to

determine when the intervention should be modified to more specifically target ORF skills. This

consultation allowed me to engage in a proactive, problem-solving approach to prevention and

intervention by providing Sarah support early in the year with the aim of increasing her NWF

and ORF skills.


270

References

Daly III, E.J., Chafouleas, S., & Skinner, C.H. (2005). Interventions for reading problems:

Designing and evaluating effective strategies. New York, NY: The Guilford Press.

Daly III, E. J., & Martens, B. (1994). A comparison of three interventions for increasing oral

reading performance: Application of the instructional hierarchy. Journal of Applied

Behavior Analysis, 29, 507–518

Fien, H., Baker, S.K., Smolkowski, K., Mercier Smith, J.L., Kame’enui, E.J., & Thomas Beck,

C. (2008). Using nonsense word fluency to predict reading proficiency in kindergarten

through second grade for English language learners and native English speakers. School

Psychology Review, 37(3), 391-408.

Goffreda, C.T., Diperna, J.C., & Pedersen, J.A. (2009). Preventive screening for early readers:

Predictive validity of the Dynamic Indicators of Basic Early Literacy Skills (DIBELS).

Psychology in the Schools, 46(6), 539-552

Good, R. H., & Kaminski, R. A. (Eds.). (2002). Dynamic indicators of basic early literacy skills

(6th ed.). Eugene, OR: Institute for the Development of Educational Achievement.

Hixon, M., Christ, T.J., & Bradely-Johnson, S. (2008). Best practices in analysis of progress-




Joseph, L.M. (2008). Best practices on interventions for students with reading problems. In A.

Thomas & J. Grimes (Eds.), Best Practices in School Psychology V (Vol. 4, pp 1163-

1180). Bethesda, MD: The National Association of School Psychologists.

Knapp, N. F., & Winsor, A. P. (1998). A reading apprenticeship for delayed primary readers.


271

Reading Research and Instruction, 38, 13–29

McCurdy, B. L., Cundari, L., & Lentz, F. E. (1990). Enhancing instructional efficiency: An

examination of time delay and the opportunity to observe instruction. Education and

Treatment of Children, 13, 226–238

Nelson, J.S., Alber, S.R., & Grody, A. (2004). Effects of systematic error correction and repeated

readings on reading accuracy and proficiency of second graders with disabilities.

Education and Treatment of Children, 27, 186-198.



Sindelar, P.T., Monda, L.E., & O-Shea, L.J. (1990). Effects of repeated readings on

instructional- and mastery-level readers. Journal of Educational Research, 83, 220-226.

University of Oregon Center on Teaching and Learning (2012). 2012-2013 DIBELS Data System

Update Part II: DIBELS Next Benchmark Goals (Technical Brief No. 1203). Eugene,

OR: University of Oregon.

Wordsdell, A.S., Iwata, B.A., Dozier, C.L., Johnson, A.D., Neidert, P.L., & Thomason, J.L.

(2005). Analysis of response repetition as error-correction strategy during sight-word

reading. Journal of Applied Behavior Analysis, 38, 511-527.


272

Appendices

A. Triumphs Script

B. NWF Activities Scripts

C. Adherence Checklists

D. Adherence Calendar


273

Appendix A: Triumphs Script


274

Appendix B: NWF Activities Script


275

Appendix C: Adherence Checklists


276

Appendix D: Adherence Calendar


277

Date post:	31-May-2020
Category:	Documents
Upload:	others
View:	2 times
Download:	0 times

K. Schatz Internship Portfolio - CECH€¦ · Internship Portfolio Based on 2013-2014 Training in...

Documents