EVIDENCED-BASED PRACTICES: CONSIDERATIONS IN SELECTING A PRACTICE THAT FITS YOUR SETTING AND NEEDS

Objectives of Session

Overview of Definitions and Approaches Identified as Evidenced-Based Practices (EBPs)

Considerations in Selecting EBPs that Match Your School Context & Need School Specific Features Practice or Program Specific Features Coordination & Implementation Within the School

Examples & Issues from Schoolwide Literacy Implementation

Examples & Issues from Schoolwide Positive Behavior Supports

The Important of Evidenced-Based Practices

The role and need of EBPs in education will continue in our accountability driven educational context Derived from the evidenced-based medical practices movement “The practice of evidence based medicine means integrating individual

clinical expertise (the proficiency and judgment that individual clinicians acquire through clinical experience and clinical practice) with the best available external clinical evidence from systematic research.” (Sackett et al., 1996)

Definition of EBP in Education, no one widely agreed upon definition …”practices informed by research findings demonstrating a

(statistical or functional) relationship between the characteristics and consequences of a planned or naturally occurring experience or opportunity where the nature of the relationship directly informs what a practitioner or parent can do to produce a desired outcome. (Dunst & Trivette, 2009, p. 41)

The Role of EBP in Special Education

“All interventions are not equal; some are much more likely than others to positively affect

student outcomes (Cook, Tankersley, & Landrum, 2009, p. 366)

Quality Indicators of Research Single Subject (Horner, Carr, Halle, McGee, Odom, Wolery, 2005)

Qualitative (Brantlinger, Jimenez, Klinger, Pugach, & Richardson, 2005)

Quantitative (Gersten, Fucsh, Compton, Coyne, Greenwood, Innocenti, 2005)

Professional Groups Identifying Evidenced-Based Practices What Works Clearinghouse National Association of School Psychologists, American

Speech Hearing Association, Division of Early Childhood

CEC Division of Research

The Role of EBP in Special Education

Challenges in Using EBP Overly focused on what makes it into publication and gets

replicated (Odom, 2007) Ignores the complexity of transferring research to practice

Our field has limited understanding of the “boundary conditions” of which practices are effective in what contexts, student populations, and personnel (Keyworth, States, & Detrich, 2003)

The role of fidelity of implementation is just beginning to be understood within the contexts of schools (Chard & Harn, 2008; Parisi & Harn, in press)

Researcher Perspective: Are all components implemented for the specified time and quality?

School Perspective: What components are necessary, for how much time, and at what quality to improve student outcomes?

“Identifying practices that are evidenced-based for students with disabilities is a necessary but insufficient step in a process that we hope will culminate in the consistent implementation of the

most effective practices with fidelity, ultimately resulting in improved outcomes for students with disabilities” (Cook, Tankersley, &

Landrum, 2009, p. 381)

Now for the Really Hard Part—Selecting & Implementing

No matter how long the list of evidenced-based practices, we still need to think carefully about selecting a practice that works for your situation. Issues to consider from the research-based study:

How does the study demographics match your student population? What was the beginning level of performance of the students in the

study (i.e., how in need?) How does the schools in the study match your school? Linkage between

general and special education? What percent of the students in the study were identified as having

disabilities? What was the level of training of the implementors? How much training

was needed to implement and sustain? Considerations in Selecting EBPs that Match Your School

Context & Need School Specific Features Practice or Program Specific Features Coordination & Implementation Within the School

Using EBPs does not preclude the need for thinking and judgment

Considerations In Selecting EBPs School Specific Features

Level of Need Current student performance/behavior

General DemographicsLarge population of ELLs

Resources AvailablePersonnel: Title 1, Special Education,

ELLPrograms & Money: Quality of practices

available or already implemented in the building

Take stock before determining needs

Taking Stock & Inventorying Practices

Before considering purchasing new materials/programs, take stock of materials in the building Identify areas of need, make a multi-year plan to

improve supports for the building Document range of materials to ensure materials are

available for use across the building Ensure coordination of instructional approach across

materials/programs

Considerations In Selecting EBPs

Practice or Program Specific Features Considering the Delivery Requirements of the Practice

Amount of Time to Implement (30, 45, 60) Grouping expectations (1:1; 1;5) Amount of Professional Development Needed to Implement Grade-level or Instructional/Behavioral Components

Taught/Addressed Alignment of Practices with Established Practices (Core Programs)

Ensuring ease of transition from more intensive supports (Tier 2/3 Intervention Programs) to less intensive supports (Tier 1-Core)

“Catch-up growth is driven primarily by proportional increases in direct instructional time. Catch-up growth is so difficult to

achieve that it can be the product only of quality instruction in great quantity.” from Fielding, L., Kerr, N. & Rosier, P. (2007). Annual

Growth for All Students, Catch-up Growth for Those Who Are Behind. Kennewick, WA: New Foundation Press

You will never "find" time for anything. If you want time, you must make it. - Charles Bruxton

Considerations In Implementing EBPs

Coordination & Implementation Within the School Within and Across Grade Levels Scheduling Interventions Professional Development

Traditional One Time-necessary but not sufficient Ongoing Coaching Approach-essential to ensure quality

implementation Evaluating Response

Student-Level: typical progress monitoring/formative evaluation

Group Level: Examining how groups of student receiving a given intervention are responding

Are all student not making progress? Check quality of implementation

12

School ASchool B Variability in School-Level

Outcomes for At-Risk Students When examining the

alignment of instructional supports in first grade (Year 2 of Project), two schools who did the “same things” (time, programs, groupings) got very different results both: across years

(compared to themselves)

across sites (compared to each other)

Level of Support Who? With What? What More? How Are We Doing?

Tier 1 / Primary

Which Students:Students on-trackAs measured by:ORF > 44

Name of Program / Materials:Open Court Reading ProgramWhen: 8:45-10:15Activities: All activities in the coreGroup Size: Large and some small group

Name of Program / Materials: Not Applicable

Who to Collect: School wide assessment teamHow Often: 3x yearCriteria: See DIBELS goalsDetermining Fidelity of Implementation (Who, With What, How often?): Literacy Coach, district form, quarterly

Tier 2 /

Secondary

Which Students:Students with difficulties in fluency, decoding, & possibly comp.As measured by:ORF 26-44

Name of Program / Materials:Open Court Reading ProgramWhen: 8:45-10:15Activities: All except independent writingGroup Size: Large and some small group (5-8)

Name of Program / Materials:Read Naturally & Phonics for Reading Program CombinationWho to Deliver: Title StaffWhen: In addition to typical time Specify Time (minutes, days of week):10:30-11 M-F Group Size: 4-8

Who to Collect: Title staffHow Often: 2 x monthCriteria: Keeping pace with DIBLES aimlineDetermining Fidelity of Implementation (Who, With What, How often?): Title teacher, project developed form, monthly)

Tier 3 / Tertiary

Which Students:Students with significant difficulties in fluency, decoding, and comp.

As measured by:ORF < 25

Name of Program / Materials:Open Court Reading ProgramWhen: 8:45-9:15Activities: Vocabulary and oral/comprehension activitiesGroup Size: Large group

Name of Program / Materials:Reading Mastery; Read NaturallyWho to Deliver: Special Education (Reading Mastery) and Title (Read Naturally)When: Within typical instructional & In addition to typical time Specify Time (minutes, days of week):Reading Mastery: 9:15-10 M-FRead Naturally: 1-1:45Group Size: 4 or less

Who to Collect: Special education teamHow Often: WeeklyCriteria: Keeping pace with DIBLES aimlineDetermining Fidelity of Implementation (Who, With What, How often?): Special education teacher, project developed form, monthly

Social Behavior&

Putting it all Together

Tim Lewis, Ph.D.University of Missouri

OSEP Center on Positive Behavior Interventions and Supports

pbis.org

Tina WindettColumbia Public Schools

Starting Point….

• We can’t “make” students learn or behave• We can create environments to increase the

likelihood students learn and behave• Environments that increase the likelihood are

guided by a core curriculum and implemented with consistency and fidelity

Serving Students with Challenging Behavior

All facets of programming should include:  Systematic, data based interventions Continuous assessment and monitoring of progress Provision for practice of new skills Treatment matched to problem Multi-component treatment Programming for transfer & maintenance Commitment to sustained intervention

(Peacock Hill Working Group, 1991)

Evidenced-Based

Academic “Effective instruction” (see nwrel.org)

Antecedent / setting modifications Peer tutoring Direct Instruction Self-management targeting academic related skills Opportunities to Respond

Evidenced-Based

Behavior Environmental modifications and supports Contingent positive performance based feedback Self Management Social Skill Instruction (with maintenance and

generalization strategies)

Evidence-Based

Related Supports* Comprehensive case management / wrap around Family supports/ parent training

*limited empirical support

The Bigger Challenge: Systems to Implement

and Sustain Evidenced-based

Practices

SYST

EMS

PRACTICES

DATASupportingStaff Behavior

SupportingDecisionMaking

SupportingStudent Behavior

PositiveBehaviorSupport OUTCOMES

Social Competence &Academic Achievement

Academic Systems Behavioral Systems

1-5% 1-5%

5-10% 5-10%

80-90% 80-90%

Intensive, Individual Interventions•Individual Students•Assessment-based•High Intensity

Intensive, Individual Interventions•Individual Students•Assessment-based•Intense, durable procedures

Targeted Group Interventions•Some students (at-risk)•High efficiency•Rapid response

Targeted Group Interventions•Some students (at-risk)•High efficiency•Rapid response

Universal Interventions•All students•Preventive, proactive

Universal Interventions•All settings, all students•Preventive, proactive

Designing School-Wide Systems for Student Success

Consistency & Fidelity

On-going, sustained, and purposeful training On-going access to technical assistance Periodic checks

Student outcomes Student perceptions Adult perceptions

Working toward a District-Wide PBS initiative that will sustain over time (Scaling up)

Research Findings on Scaling Up(Fixsen, Naoom, Blase, Friedman, & Wallace, 2005, p. 70)

Best evidence documents what doesn’t work: Information dissemination alone Training by itself

Research Findings on Scaling Up(Fixsen, Naoom, Blase, Friedman, & Wallace, 2005, p. 70)

What does work Long term, multi-level approaches Skills-based training Practice-based coaching Practioner performance-feedback Program evaluation Facilitative administrative practices Methods for systems intervention

Meaningful PD Outcomes

StaffDevelopment

Change inTeacherPractice

Change inTeacherPractice

Change in Student

Outcomes

Change inTeacher Beliefs

Change inTeacher Beliefs

A Model of the Process of Teacher Change

Guskey, 1986

Putting it all Together

Derby Ridge Elementary School

Columbia Public Schools

Derby Elementary School

• PS-Grade 5 Elementary School in Columbia, MO• 715 students• 58% of students on free/reduced lunch• 30% minority, mostly African American• Full range of special education services• Involvement with SW-PBS for 6 years

Derby Ridge Elementary

2008-2009 School Year: Created STAT (Student-Teacher Assistance Team)

Core STAT Team consists of administrator, administrative assistant (with special education background), school psychologist, clinical associate, two school counselors.

Full STAT Team consists of Core Team Members and a teacher representative from each grade level K-5.

Derby Ridge Elementary

The Core STAT Team meets for one hour weekly Works to develop process and forms to identify

students who needed additional behavior support Brings feedback from grade level teams, which helps

identify training focus We knew this work would be slow, but we wanted to

be thorough and involve feedback from all representative groups

Derby Ridge Elementary One member from the Core STAT Team is assigned to

each grade level This Core STAT Team member meets with their assigned

grade level 2-4 times per month for 30-40 min. each time Uses data decision rule, SWIS data and teacher

information to identify students who are Tier 2 and Tier 3 students.

Move through the process with identified students to determine function of behavior and implement appropriate targeted interventions.

Grade level group and Core STAT representative monitor the progress of identified students and interventions.

Derby Ridge Elementary

Professional Development/Training 4 five-hour trainings were provided for Full STAT team

throughout the 08-09 school year, based on needs & requests determined from grade level work

two trainings for all building level certificated staff were provided, based on feedback

MU-PBS also met twice with each grade level team to explain the process, goals, and to determine successes and challenges

Derby Ridge Elementary

• Critical to get transparent feedback along the way from faculty and staff, mostly achieved through anonymous surveys and meetings with external supports

• Successes systematic process to identify students and respond education of faculty & staff on process and the importance of

having universals in place and determining function before beginning intervention

building capacity of staff to understand students better and work with their behaviors

Derby Ridge Elementary

Challenges Maintaining focus and increasing buy-in Working systems for both Tier 2 and Tier 3 students Implementing interventions with fidelity Time and personnel resources to ensure the system is in

place and used as designed Regularly evaluating progress and adjusting as needed,

while still maintaining fidelity and using research-based practices

Basic Steps within Process

1. Classroom Universals in place

2. Identifying students who need additional supports

3. Identifying what supports student needs Environment Intervention

4. Monitor & evaluate progress

Conclusions

“evidenced-based education, like evidenced-based medicine, is not a panacea, a quick fix, cookbook practice, or the provider of ready-made solutions to the demands of modern education. It is a set of principles and practices which can alter the way people think about education, the way the go about educational policy and practice, and the basis upon which they make professional judgments and share their expertise” (Davies, 1999, p. 118)

“Be Cautious About What Is Promised, as there is no guarantee that evidence-based practices travel easily or as expected” Steven Graham

“Special education research, because of its complexity, may be the hardest of the hardest-to-do science. One feature of special education research that makes it more complex is the variability of the participants” (Odom, Brantlinger, Horner, Thompson, & Harris, 2005, p. 139)