How To Do Classwide Intervention within RTI Amanda VanDerHeyden Education Research and Consulting, Inc.
Transcript
Slide 1
How To Do Classwide Intervention within RTI Amanda VanDerHeyden
Education Research and Consulting, Inc.
Slide 2
Objectives Today Overview of RTI, RTI decision making, and
expected outcomes Specific How-To for Classwide Math Intervention
Implementing intervention for sustenance and system change
Slide 3
Slide 4
Slide 5
Disparities in Achievement Substantially lower level of
performance in Reading at first and second grades for
African-American students relative to their Caucasian peers
(approximately 20 wc/min) Slower growth rate (approximately half)
at both grade levels Differences not observed in Math
Slide 6
In Low-Achieving Classrooms (more than 50% of class scored in
the frustrational range on probes) STEEPTeacher Referral
Sensitivity.75.55 Specificity.88.68 Positive Predictive Power.69.35
Negative Predictive Power.91.82 VanDerHeyden & Witt, 2005
Slide 7
In Average to High-Achieving Classrooms (less than 20% of class
scored in frustrational range on probes) STEEPTeacher Referral
Sensitivity.670 Specificity1.0.67 Positive Predictive Power 1.00
Negative Predictive Power.97.95 VanDerHeyden & Witt, 2005
Slide 8
% Accurate ID by Race MinorityCaucasian STEEP9086 Teacher
Referral 7861 Use of RTI with STEEP approximated base rate by race
and gender AA students showed a disproportionate RTI (.50
versus.07) VanDerHeyden & Witt, 2005
Slide 9
Research Finding VanDerHeyden, Broussard, et al. (2004). Prior
to single instructional session, children receiving special ed
services were significantly lower performing on math probes.
Following single instructional session, no significant difference
was observed.
Slide 10
Mixed Mult/Div/Fractions Probe Classroom F
Slide 11
Sore Thumb Test
Slide 12
Response to Classwide Intervention Student A
Slide 13
Individual Math Intervention Cant Do Problem
Slide 14
Successful Math Intervention BaselineIntervention Novel,
grade-level probe
Slide 15
Why do Classwide Intervention Efficiency Accuracy Efficacy
Slide 16
STEEP Model Screening to Enhance Educational Progress
Slide 17
Tier 1: Screening Screening Math Screening 2 minutes. Scored
for Digits Correct Writing Screening 3 Minutes. Scored for Words
Written Correctly Reading Screening 1 Minute. Scored for Words Read
Correctly
Slide 18
Class-wide Screening
Slide 19
Feedback to Teachers
Slide 20
Tier 2: Class-wide Intervention
Slide 21
No Class-wide Problem Detected
Slide 22
Tier 2: Cant Do/Wont Do Assessment Cant Do/Wont Do
Individually-administered Materials Academic material that student
performed poorly during class assessment. Treasure chest: plastic
box filled with tangible items. 3-7 minutes per child
Slide 23
Cant Do/Wont Do Assessment
Slide 24
Decision Rule Following Cant Do/Wont Do Assessment
Slide 25
Tier 3: Individual Intervention
Slide 26
Response to Intervention Before Intervention During
Intervention Avg. for his Class Intervention in Reading #Correct
Intervention Sessions Each Dot is one Day of Intervention
Slide 27
Before Intervention During Intervention #Correct Avg. for his
Class Response to Intervention
Slide 28
Instructional range Frustrational range Vehicle for System
Change: System-wide Math Problem Each bar is a students
performance
Slide 29
Re-screening Indicates No Systemic Problem Fourth Grade
Slide 30
Rest of Grade at Standard ABCDEFClassroom
Slide 31
Spring 2003 Classroom F F
Slide 32
Teacher moved to lower grade in Fall 2003
Slide 33
Class-wide Intervention Teacher F Mult 0-12 0 20 40 60 80 100
120 10/24/200310/31/2003 11/7/2003 11/14/200311/18/2003 Weeks
Digits Correct Two Minutes
Slide 34
Increased Difficulty- Intervention Continues
Slide 35
Mixed Mult/Div/Fractions Probe Classroom F
Slide 36
Growth Obtained aimline actual growth
Slide 37
Effect on High-Stakes Scores VanDerHeyden, in prep
Slide 38
Effect on High-Stakes Scores VanDerHeyden, in prep
Slide 39
District-wide Implementation Data Vail Unified School District
www.vail.k12.az.us Three years, system-wide implementation of STEEP
grades 1-8
Slide 40
System Outcomes Referrals reduced greater than half % who
qualify from 50% stable baseline over three years to nearly 100%
SLD down from 6% of children in district in 2001-2002 (with
baseline upward trend) to 3.5% in 2003-2004 school year
Corresponding gains on high-stakes tests (VanDerHeyden & Burns,
2005) Intervention successful for about 95 to 98% of children
screened VanDerHeyden, Witt, & Gilbertson, 2007
Findings Number of Evaluations dramatically reduced 70% at
highest referral school Diverse settings, psychologists of diverse
backgrounds and no prior experience with CBM or functional academic
assessment Percentage qualify increased at 4 of 5 schools
Disproportionate representation of males positively affected Number
of children placed dramatically reduced VanDerHeyden, Witt, &
Gilbertson, 2007
Slide 43
Team Decision-Making Agreement RTI + and Evaluated RTI- and Did
Not Evaluate 2003-2004 (3 schools) 100%41% 2004-2005 (5 schools)
100%87% VanDerHeyden, Witt, & Gilbertson, 2007
Slide 44
Team Decision-Making VanDerHeyden, Witt, & Gilbertson,
2007
Slide 45
Fall to Spring Reading Growth VanDerHeyden & Witt,
2005
Slide 46
What Proportion of Ethnicity Represented Before and After
Intervention in Risk Category? VanDerHeyden & Witt, 2005
Slide 47
Identification Accuracy High-achieving classrooms (50%)
Procedures paired with RTI criterion were more accurate than other
commonly used screening devices VanDerHeyden & Witt, 2005
Slide 48
Weighing a cow doesnt make it fatter.
Slide 49
Break
Slide 50
Using Screening Data to Identify Class-wide and System-wide
Instructional Problems
Slide 51
Consider The Task Integrity of Administration Reliability of
Scoring Use software to organize the data
Slide 52
Mult 0-9 4 th Grade Fall Screening
Slide 53
Mult/Div/Fractions 4 th Grade Winter
Slide 54
Grade-wide Data
Slide 55
Slide 56
Slide 57
3 rd Grade Mult 0-9 Spring
Slide 58
Guided Practice
Slide 59
Fourth Grade Reading Level: Math Skill 1: Math Skill 2:
Slide 60
Questions Is there a classwide problem? Is there a gradewide
problem? Whats the most efficient way to deliver intervention?
Slide 61
Slide 62
Slide 63
Slide 64
Slide 65
What Data do you Want for Principal?
Slide 66
Slide 67
Slide 68
Slide 69
Questions Is there a classwide problem? Is there a gradewide
problem? Whats the most efficient way to deliver intervention?
Slide 70
Independent Practice
Slide 71
First Grade Reading What do you want to know? Is there a
class-wide problem? Is there a grade-wide or systemic problem?
Whats the most efficient way to deliver intervention? (whole class,
small group, individual) What is the next step for Class 1, 2, 3,
4?
Slide 72
Class 1
Slide 73
Class 2
Slide 74
Class 3
Slide 75
Class 4
Slide 76
Grade-wide Data
Slide 77
Slide 78
Class 4
Slide 79
Screening tells you How is the core instruction working? What
problems might exist that could be addressed? Most
bang-for-the-buck activity Next most high-yield activity is
classwide intervention at Tier 2.
Slide 80
Screening Guidelines Efforts at Tier 1 pay off with fewer
children needing individual intervention 3 times per year, single
probe Use small team of trained coaches Prepare all needed
materials in a packet for each teacher Score and return within 1
week on graph Use data to generate aimlines, can be used to set
benchmarks
Slide 81
Digits Correct Two Minutes Weeks 121 Pass the AIMS
Slide 82
Academic SystemsBehavioral Systems 1-5% 5-10% 80-90% Intensive,
Individual Interventions Individual Students Assessment-based High
Intensity Of longer duration Intensive, Individual Interventions
Individual Students Assessment-based Intense, durable procedures
Targeted Group Interventions Some students (at-risk) High
efficiency Rapid response Targeted Group Interventions Some
students (at-risk) High efficiency Rapid response Universal
Interventions All students Preventive, proactive Universal
Interventions All settings, all students Preventive, proactive Any
Curriculum Area Students Dave Tilly, 2005
Slide 83
Class-wide Intervention Use pair-peered practice (classwide
peer tutoring, PALS) Model, Guided Practice, Independent timed
practice with delayed error correction, reward contingency
Slide 84
Unsupported means. Team recommends intervention. All materials
have to be created from scratch Teachers sent to organize class and
train No one may be an expert to help the teacher No or not enough
frequent objective data to collect or interpret
Slide 85
With teacher support Consider time, resources, materials Remove
skill barriers with classroom training for students classroom
coaching for teachers Remove implementation barriers after use new
steps follow-up supportive meetings to problem solve. frequent
acknowledgment of a teachers efforts
Slide 86
Time consuming Complex not yet fluent Cant Do Poor management
No child change Wont do No reinforcement for teacher behaviors No
teacher change prevents Address Common Reasons for Resistance Lack
of materials prevents Too much work avoidance prevents
Slide 87
Select a Few Good Interventions to Keep it Simple
ClasswideIndividual MathFlash card Practice Cover copy compare Cue
Cards Highlighted errors ReadingListening PreviewRepeated Readings
Error Correction Key Words
Slide 88
Prepare for Training Day Locate probes (e.g., worksheet
factory, intervention central, basic skill builders) Identify
Graphing Program (excel) Locate or develop scripts (gosbr.net;
interventioncentral.org) Develop quick access to materials for
teachers Determine integrity monitoring plan Identify common time
for intervention Set start date
Slide 89
Set a daily routine. Time, location of materials, process for
weekly assessment. Set a date and time for 30-min training Set a
date for a later 15-minute first practice time with teacher
Slide 90
Materials needed Computer and software to organize data Student
data imported. Clerical person to enter data on- site for tier 1
screen only. Color printer to print graphs + extra color cartridges
Probe materials, digital count-down timers Intervention protocols,
intervention materials (e.g., flashcard sets, reading materials)
Access to copier and some assistance with copying Reinforcers for
treasure chest (no more than $500 per school)
Slide 91
Usually the higher-level reader, reads (models) first. Rotating
high level readers helps maintain motivation
Slide 92
How-To Classwide Math
Slide 93
Slide 94
Slide 95
Intervention Plan- 15 Min per Day Protocol-based classwide peer
tutoring, randomized integrity checks by direct observation Model,
Guide Practice, Independent Timed Practice with delayed error
correction Group performance contingency Teachers encouraged to
Scan papers for high error rates Do 5-min re-teach for those with
high-error rates Provide applied practice using mastery-level
computational skill
Slide 96
Measurement Plan Weekly probe of Intervention skill Weekly
probe of Retention of previously mastered computational skills
Monthly probe using GOM approach to monitor progress toward
year-end computational goals To this you might add an application
measure
Slide 97
Sample Sequence
Slide 98
Intervention Plan Class Median reaches mastery range for skill,
next skill is introduced Following promising results at one site in
2002-2003, lead to implementation district- wide grades 1-8 for all
children by 2004- 2005.
Slide 99
Acquisition Fluency Generalization Instructional Hierarchy To
gain the steepest growth, introduction of new skills should happen
here Core Instruction- Not manipulated But fluency building should
happen here with an instructional level skill Intervention Focus
was here Finally, problem-solving/ application practice should
occur here with a mastery level skill Core Instruction- Not
Manipulated but could be
Provided teacher/student a script that tells. what the student
has to do and when what the teacher should do to support student
how the student will know how he/she is doing Treatment
considerations for integrity issues All steps are clearly needed
Includes lots of student response opportunities Disrupts class as
little as possible Requires little teacher time ( < 15 min/day)
Considers resources to decrease teacher effort Used simple language
All the materials are available Rationale
Slide 104
Trainer 1.Observe the teacher using the steps on the
intervention script 2. Check off steps used. 3. Prompt the teacher
to do any missed step. 4. Problem Solve any noted blockers 5.
Continue until accurately implemented without prompts
Slide 105
Lack of practice with feedback Non-specific steps In adequate
classroom management Not enough child assistance for bx change Lack
of reinforcement For teacher behaviors Low implementation Low
frequencies interferes prevents Why verbal and modeling training
alone do not work: Inadequate materials prevents No instructions
when problems arise
Slide 106
Math Partners Progress Chart Count every digit that is not
circled. This is your score! Write your score on your math sheet.
Find todays date on this page and write your score on the line. Put
a Star on the graph to mark todays score.
Slide 107
Weekly Progress Monitoring Administer classwide math worksheet
Target skill once per week Criterion skills periodically to monitor
growth Use incentives to maximize performance Apply decision
rules
Slide 108
Progress Review Review folders to ensure that intervention was
used correctly for at least 4 days that week If this is not the
case, conduct another in- class training day. Graph weekly progress
monitoring assessment data
Slide 109
Decision making Review data to make decisions: DATA OUTCOME 1:
Class median is below mastery range and most students gaining
digits correct per week. ACTION: Consider implementing intervention
for an additional week and then review progress again.
Slide 110
Decision making DATA OUTCOME 2: Class median is below mastery
range and most students are not gaining digits correct per week:
ACTION: Check Integrity first and address with training if needed.
Consider implementing intervention for an additional week with
incentives or easier task and then review progress again.
Slide 111
Decision making DATA OUTCOME 3: If the class median is above
mastery range then consider: ACTION: Increasing task difficulty and
continuing classwide intervention. ACTION: For students performing
in the frustration range, consider Tier 3 assessment and
intervention.
Slide 112
Tell Rational Step by step protocol Show Model Do Train
students Implement with guided practice Implement independently
with support Training Package
Slide 113
> 80% of interventions are not used without support
Slide 114
Troubleshoot Intervention SupportYesNo Was the intervention
developed to ensure that it required minimal classroom time and
resources and fit within daily classroom routines? Are materials
readily available to the teacher? Was a step-by-step coach card
provided? Was the teacher shown how to implement the intervention
by a coach? Did the coach observe implementation of the
intervention to ensure that the teacher could use the intervention
correctly and had all needed materials? Was weekly follow-up
support provided to the teacher after initial training? Are
integrity data graphed to show used correctly? Is an administrator
involved?
Slide 115
Results
Slide 116
Tier 1 Screening Indicates Class- wide Problem
Slide 117
Tier 2: Class-wide Intervention Teacher F Mult 0-12 0 20 40 60
80 100 120 10/24/200310/31/2003 11/7/2003 11/14/200311/18/2003
Weeks Digits Correct Two Minutes
Slide 118
Increased Difficulty- Intervention Continues
Slide 119
Contextually-Relevant Comparisons and Use of Trend Data
Slide 120
5 th Grade Math Intervention
Slide 121
Slide 122
Instructional range Frustrational range Pre-post changes to
performance detected by CBM Each bar is a students performance
Slide 123
Fourth Grade
Slide 124
Effect on SAT-9 Performance
Slide 125
Effect on CBM Scores
Slide 126
Computation Gains Generalized to High Stakes Test Improvements
(Gains within Multiple Baseline shown as pre-post data)
Slide 127
Gains within Multiple Baseline (shown as pre-post data)
Slide 128
Additional Research Questions What level of performance
predicted strongest subsequent growth given intervention? What
level of performance predicted skill would be retained about 3
months after it was taught? Did mastery of foundation skills
shorten the number of trials required to master more complex
related skills?
Slide 129
What level of performance predicted strongest subsequent growth
given intervention? Across 4 weeks of intervention (4 datapoints)
OLS used to estimate slope Children achieving slopes equal to or
greater than the 66 th percentile were identified as strong
responders Starting fluency (prior to intervention) was identified
for the group of strong responders and range was estimated as
average starting fluency +/- 1 standard deviation Tested new
criterion on second set of scores Burns & VanDerHeyden,
2006
General Findings Growth rates and trials to criterion varied
dramatically across skills Retention probe was strongest predictor
of year- end SAT-9 performance Mastery level performance on early
skills predicted fewer trials to criterion on future related
complex skills Fluency scores higher than mastery predicted
retention of skill over time (about +20 dc/min) VanDerHeyden &
Burns, 2008; VanDerHeyden & Burns, in submission
Academic SystemsBehavioral Systems 1-5% 5-10% 80-90% Intensive,
Individual Interventions Individual Students Assessment-based High
Intensity Of longer duration Intensive, Individual Interventions
Individual Students Assessment-based Intense, durable procedures
Targeted Group Interventions Some students (at-risk) High
efficiency Rapid response Targeted Group Interventions Some
students (at-risk) High efficiency Rapid response Universal
Interventions All students Preventive, proactive Universal
Interventions All settings, all students Preventive, proactive Any
Curriculum Area Students Dave Tilly, 2005
Slide 135
To work smart, we must ask What is the purpose of our
assessment? How do we know it serves our purpose? Is this the
cheapest way to do it?
Slide 136
Our Goal Collect the best information in the shortest possible
period of time
Slide 137
Tier 3 Assessment Data Instructional level performance Error
analysis (high errors, low errors, pattern) Effect of incentives,
practice, easier task Verify intervention effect Same
implementation support as Tier 2 Instructional-level materials;
Criterion-level materials
Slide 138
Tier 3 Implement for 5-15 consecutive sessions with 100%
integrity Link to referral decision Weekly graphs to teacher and
weekly generalization probes outside of classroom, supply new
materials Troubleshoot implementation weekly
Slide 139
Strategy in a Nutshell Identify the goal (DVs) Behavior to
increase (fluency, comprehension) Behavior to decrease (errors)
Match the strategy to the goal (Daly et al., 1996) Monitor the DVs
and the IVs (intervention variables)
Slide 140
Find instructional level (sampling back) Identify the root of
the problem (e.g., division is difficult because subtraction is not
fluent or multiplication is not fluent; poor decoding skills v.
dolce words; production v. accuracy in writing) This is the most
important part of the process
Slide 141
Accurate Letter Naming Fluent Letter Naming Association of
Letters with phonemes Accurate Letter Sound Production Fluent
Letter Sound Production Pronounce beginning word sounds Define the
Behaviors/skills
Identify Reinforcers and Logical Consequences Use a treasure
chest Use an activity survey or reinforcer checklist Use incidental
teaching strategy Use logical or natural consequences
Slide 144
Measure Baseline Performance and Set Goals
Slide 145
Acquisition Fluency Generalization This is the Instructional
Hierarchy To gain the steepest growth, introduction of new skills
should happen here But fluency building should happen here with an
instructional level skill Finally, problem-solving/ application
practice should occur here with a mastery level skill
Slide 146
Functional Assessment What is an effective intervention?
Slide 147
Functional Assessment
Slide 148
BLInterventionPerformance Feedback
Slide 149
Troubleshooting Intervention Effects at Tier 3
Slide 150
Teachers must weigh the following What outcomes does not
completing work produce? Escape from assignment, from classroom
setting Peer attention Adult attention (even if it is negative).
Some students are so motivated to obtain adult attention that it
does not matter if the attention is negative or positive. What
outcomes does completing work produce? Positive feedback from the
teacher Positive attention from peers, status Access to fun
activities or reinforcement Avoidance of punitive consequences
Slide 151
Antecedent Variables Task Difficulty, Sequencing of Skills Time
actively engaged in learning (AET) Opportunities to respond Other
lesson variables (pacing, exemplars) Behaviors interfering with
instruction (teacher and child)
Slide 152
Task Difficulty, Sequencing Sample back measuring fluency of
performance on basic skills The idea is to identify the weak point
in the chain Define the target skill for intervention and the
criterion skill (goal)
Slide 153
Academic Engaged Time Impacts opportunities to respond Robust
predictor of achievement Average 2 nd grader (Rosenshine) spent
less than 1 hour AET per day. Check transitions, classroom
management, time allocated for independent practice, active
monitoring/scanning
Slide 154
Other Lesson Variables Presentation of materials and Sequencing
of Lesson Organized Clear, redundant examples Exemplars sufficient
S+ and S- Checking for student understanding Pacing of lesson
Slide 155
Behaviors Interfering with Instruction/Intervention Teacher
behaviors Implementation accuracy and consistency Fuchs &
Fuchs, 1987; Gresham, 1991; Happe, 1982; Wickstrom, Jones, LaFleur,
& Witt, 1998 Teacher understanding/adequately trained Train to
fluency criterion (Chandler, Lubeck, & Fowler, 1992) Teacher
acceptability of intervention (prospective, ongoing, link to
changes) Adequate resources to conduct intervention
Slide 156
Behaviors Interfering with Instruction Child behaviors
Disruptive or inattentive behaviors Concurrent options available
(access to reinforcing outcomes by not completing intervention)
Consider cant do/wont do (although programming for motivation is
important anyway)
Slide 157
Consequences Reinforcing consequences (for correct and
incorrect performance) Escaping task Extra attention (staying in at
recess may be reinforcing) Feedback Frequency Immediacy Accuracy
Correct error immediately, have student repeat response correctly,
match response to instructional situation & learner (Heubusch
& Lloyd, 1998)
Slide 158
Tier 3 Intervention >5% of children screened (total
population) IF solid Tier 1 Possibly as low as 2% IF solid Tier 1
and Tier 2 About 1-2% failed RTI; 10% of most at-risk VanDerHeyden
et al., 2007
Slide 159
Tier 3 Findings Most interventions for reading Math is next
Math is at least two-dimensional--- computational and operational
fluency plus application or conceptual understanding Most
interventions are not implemented well and thats why they fail Tier
3 interventions are likely to occur on below grade level tasks AND
require acquisition-type instruction (discrimination training to
establish accurate responding)
Slide 160
Successful Math Intervention
Slide 161
Unsuccessful Math Intervention
Slide 162
Integrity Matters 59% Integ96% Integrity
Slide 163
Integrity Matters
Slide 164
Slide 165
Integrity Untreated integrity problems become student learning
deficits, schoolwide learning problems, and false positive decision
errors Integ problems affect dose and quality of the treatment (an
intervention implemented with fidelity is a functionally different
intervention than one implemented inconsistently Integ positively
correlated with student learning gains, amount of intervention
covered Even veteran sites require monitoring and follow-up
Slide 166
Tips for Effective Implementation
Slide 167
Our Recipe for Intervention Success PREPARATION Identify and
Use standard protocols for intervention Develop all needed
materials Develop packets or put on a central web site Determine
graphing program
Slide 168
Our Recipe for Intervention Success TRAIN Explain Watch the
teacher do it with the actual child before you leave Call or meet
teacher after first day to problem solve
Slide 169
DATA COLLECTION AND SUPPORT Each week, graph intervention
performance and do a generalization check with the child. Graphed
feedback to teachers with generalization checks for individual
intervention once per week Response-dependent performance feedback
to sustain implementation accuracy Monthly CBM to track growth and
enhance existing Tier 1 Programs or advise new Tier 1 Data to
principal weekly. Summarize effects and integrity of procedures.
Our Recipe for Intervention Success
Slide 170
DATA DECISION -MAKING RTI successful if child performs
criterion-level probe (from screening) in the instructional range.
RTI unsuccessful if 15 consecutive intervention sessions and
criterion probe is not in the instructional range. Increase task
difficulty for intervention if child scores at mastery on task
during intervention sessions
Slide 171
Guidelines for Implementers Use single trial scores for
screening Following screening, grade-wide graphs to principal
Return data to teachers within 48 hours with personal
interpretation at grade-level team meeting Include principal in
critical meetings Involve teachers at all stages
Slide 172
Guidelines for Implementers Learn about curriculum and
instruction. Integrate RTI with ongoing school and system reform
efforts Thoughtfully merge to subtract duplicate activities and to
enhance more comprehensive supplemental and core instructional
support activities that may be in place Use RTI data to evaluate
the value of ALL instructional programs or resource allocation
decisions. Quantify bang for the buck using student performance
data.
Slide 173
Infrastructure for Implementation Grade-level planning periods
can be utilized Special education team at school can be utilized
School Psych must be on-site 1 day/week Developing master schedule
for Tier 1, 2, and 3 intervention times is useful Integrate efforts
with evaluation referral team efforts (consider major reduction in
meeting time and shift to intervention efforts!) Use existing
instructional periods to target student needs more effectively See
NASDSE blueprint for implementation Brown-Chidsey book coming from
Guilford
Slide 174
For More Information [email protected] www.isteep.com Thank
you to the US Dept of Education for providing all film clips shown
in this presentation