Selecting, Training and Coaching for Effective and Sustainable RtI

Implementation

A Model for Leadership TeamsMargie McGlinchey; Ph.D.

Kim St. Martin; M.A.

The Science of Implementation

• An “intervention” is one set of activities

• “Implementation” is a very different set of activities

Leadership teams and principals must hold both of these concepts and issues simultaneously

We want to talk about “Implementation” of Response to Intervention

Special Acknowledgments

the content of this presentation is taken from the work of members of National Implementation Research Network (NIRN) and State Implementation and Scale-up of Evidence-Based Practices (SISEP)

Dean L. Fixsen, Ph.D.,

Karen A. Blase Ph.D.,

Rob Horner Ph.D.,

George Sugai Ph.D.,

Sandra Naoom, MSPH,

Melissa Van Dyke, LCSW,

Allison Metz, Ph.D.

Michelle Duda, Ph.D.

The NIRN

Fixsen, D. L., Naoom, S. F., Blase, K. A., Friedman, R. M. & Wallace, F. (2005). Implementation Research: A Synthesis of the Literature. Tampa, FL: University of South Florida, Louis de la Parte Florida Mental Health Institute, The National Implementation Research Network (FMHI Publication #231). 

Implementation Research: A Synthesis of the Literature

© Dean Fixsen, Karen Blase, Robert Horner, George Sugai, 2008

The Science of Implementation

• Students cannot benefit from interventions they do not experience

• Training by itself does not work

• Policy change by itself does not work

• Financial incentives alone do not work

Capacity Development• To scale up RtI we must first scale up

implementation capacity• Building implementation capacity is

essential to maximizing the use of evidenced based practices and programs

• Large scale, real time change will be the end result

• Data based decision making processes should be the driving force behind analyzing the implementation and the infrastructure

“For every increment of performance I demand from you, I have an equal responsibility to provide you with the capacity to meet that expectation”

(R. Elmore, 2002)

The Charge of the Leadership Team:

• To make sure we develop the capacity of all staff members to perform the responsibilities demanded by an RtI model– This capacity will be different for differing

roles and responsibilities within the developing systems

• To create the organizational structures to support continuous improvement

• To lead when we say we will lead

Activity Part I

• At your tables form groups of four members• Silently read the Implementation Brief and

highlight three or four items that have particular meaning

• In turn, members share one of their items• Repeat so that each member has a turn in each

cycle

Activity Part II

Personal Reflection

1. Have you ever been involved in a project that was unsuccessful?

2. How much attention was paid to the critical features of the practice?

3. How much attention was paid to supporting the implementation of the practice?

4. What are the implications for RtI implementation in your district?

Key Elements of Implementation Science

• Implementation Stages: was discussed in the first session during yesterday’s training

• Implementation Drivers: will be discussed in detail during this session

• Improvement Cycles: will be discussed in detail in the next session

Innovation Fluency: the degree to which the team knows the innovation

• Evidence

• Program and Practice Features

• Intervention Readiness for Replication

• Implementation Capacity Requirements

Key Elements of Implementation Science

Implementation Fluency-the degree to which the team understands and pays attention to:

•Usability of the Intervention

•“Gap” ology

•Intentionality

•Need for Processes and Function Change

Key Elements of Implementation Science

PracticePractice

Implementation Fluency: “Gap”ology

ResearchResearch Implementation

Practice Selection

“IS IT THE RIGHT THING TO DO?”

“CAN WE DO IT THE RIGHT WAY?”

IDENTIFYING NEED • Is the need identified at the district level? Or is it a

need identified at the building level?

• Has this need been identified by the school improvement process as a critical improvement effort?

• Is this a social/behavioral need or an academic need?

• Have social and academic needs been correlated?

• Is there community support for addressing this identified need?

• Other questions to consider?

DETERMINING FIT• What other initiatives must be considered when

address the need ?

• What district/school priorities must be considered?

• Are there priorities identified by the school improvement process related to student performance?

• Are there school-wide structures that need to be considered when determining fit?

• Is there a multi-tiered system of supports in place that must be considered?

EXAMINING EVIDENCE• Do school personnel have the necessary skills

required to:

– Examine research or scientific evidence?

– Analyze and use data to make decisions?

– Understand effect size, efficacy and effectiveness information?

• Are these practices of “examining” information/data etc. embedded in the school improvement process?

– Is there a culture of learning in the “agency” that supports these practices?

RESOURCES• What resources will needed?

• Fiscal

• Personnel

• Technology

• Data & Information

• Training & Coaching

• Are there current resource expenditures that require change, adjustment or elimination?

• Are there current practices that require change, adjustment or elimination?

ASSESSING READINESS

• Is there a commitment to the practice ? Has that commitment been assessed and quantified?

• Are there benchmarks developed to assess and or determine readiness?

• What readiness standards are set to insure personnel are ready to implement?

• What system benchmarks are in place to insure readiness for implementation?

CREATING CAPACITY• Are there certain qualifications needed for

implementation? Do staff meet these qualifications?

• Is there a decision making process for selection for staff training?

• Is there political will and commitment to build capacity?

• Could capacity to implement be sustained over time? Would cost to build and sustain capacity escalate or de-escalate?

Activity• With a partner at your table, think of an

evidence based practice that you are considering adopting in the next year as a part of your RtI implementation

• Use the Practice Selection Guide to evaluate the components of selection. What other information do you need to make a good selection? Make notes to help your remember to follow up when you get back to the building

*it’s the what

Innovation Fluency

ImplementationFluency

*it’s thehow

“Many implementation efforts fail because someone underestimated the scope or importance of preparation. Indeed, the organizational hills are full of managers who believe that an innovation’s technical superiority and strategic importance will guarantee acceptance.”

Leonard-Barton & Kraus,Harvard Business Review, 1985

Implementation Drivers

Fidelity of Implementation

We must think carefully about our purpose, players, and position to determine priorities and courses of action

(Harn, 2008)

“I think you should be more explicit here in step two.”

Then a Then a miracle miracle occurs occurs

Selection Selection

• Implementation Drivers Create the Infrastructure

• Implementation Drivers are mechanisms that

– Help to develop, improve, and sustain practitioners’ ability to implement an intervention to benefit children

– Help ensure sustainability and improvement at the organizational level

What are Implementation Drivers?

• Implementation Drivers are mechanisms that help to develop, improve, and sustain one’s ability to implement an intervention to benefit students. (Competency Drivers)

• Implementation Drivers are mechanisms to create and sustain hospitable organizational and systems environments for effective educational services. (Organization Drivers)

Selection

It’s about “choice” and “fit” for:

– Current Staff

• Early Adopters

• Volunteers

– New Staff

Selection

• Purposes:– Select for the “unteachables” – Screen for pre-requisites– Make expectations explicit– Allow for mutual selection– Improve likelihood of retention after

“investment”– Improve likelihood that training, coaching and

supervision will result in implementation

Selection: Best Practices• Job or role description clarity about

accountability and expectations

• Sampling of skills and experience is related to “new practices” and expectations

• Interactive Interview Process:

– Behavioral Vignettes and Behavior Rehearsals

– Assessment of ability to accept feedback

– Assessment of ability to change own behavior

Selection: Best Practices (cont.)

• Interviewers understand the skills needed and can assess applicants accurately.

• Job Descriptions, Interview, and Evaluation practices are aligned

• Using Data for Integration and Compensatory Features– Feed interview information forward to

trainers, coaches, school administrators– Feedback from exit interviews, retention

data, training data, fidelity data and staff satisfaction data

Selection Selection

Example Job Description• Work effectively with district personnel in developing

building teams to deal effectively with behavior and learning support within a school building

• Provide technical support in data collection systems for participating schools

• Conduct in-service training on a county and local level in the areas of assessment, behavior intervention, and learning support

• Work with MIBLSI schools to customize and continually adjust school-level action plans

• Investigate questions from the field and respond to issues

• Provide content for newsletters, websites, and conference presentations

Activity• What is the status of the previous job

description related to critical skills of RtI? How about in your district?

• How well is the Selection Driver functioning in your district?

• Are interviewing processes aligned with RtI?• How can the implementation drivers be used

to improve your selection processes?

Training: Best Practices

• Theory grounded (adult learning)

• Skill-based

– Behavior Rehearsals vs. Role Plays

– Knowledgeable Feedback Providers

– Practice to Criteria

• Feedback to Selection and Feed Forward to Supervision

• Data-based (pre and post testing)

Effective Training Effective Training

Training: Implementation Drivers

• Selection: How will trainers be selected? What will the job description include? Interview procedures? Ongoing development and support?

• Training: How will the trainers be trained?• Coaching: How will support to trainers be

accomplished?• Performance Assessment: How will the

trainer quality be assessed? Fidelity to training content? Delivery?

Training: Implementation Drivers• Decision Support Data System: What outcomes

will be used to evaluate effectiveness? Changes in team processes? Changes in infrastructure? Changes in student outcomes?

• Facilitative Administration: Using ongoing feedback loops to support sustainability of training support, revising policy to support ongoing training

• Systems Intervention: Multiple levels of technical training support, aligning training and PD needs district-wide

Training Rubrics Activity• Take a few minutes to review the trainer

rubrics.• On a scale of 1-5, rate your current fidelity

assessment practices related to effective training.1=no tool or process in place, and this tool will be

helpful

3=our process is very similar to the rubrics provided

5=we have a tool like the trainer rubrics and other measures of fidelity

Leadership Drivers

• Technical Leadership: leadership practices that ensure intervention practitioners have the necessary technical knowledge and practical skills to effectively carry out a specified practice

• Adaptive Leadership: leadership practices that address and support the development of adaptive skills to manage change, feelings of loss, incompetence or disloyalty

Big Idea

• Drivers occur at all levels of the system but look differently

Building Team

Local Education

Agency Team

Intermediate School

District Team

State Education

Agency Team

Practices (Innovation)

Supporting Infrastructure

(Implementation)

Grade Level Team

Framework for Addressing Practice and Supports

Teacher

Con

text

ual F

ocus

Con

text

ual F

ocus

Unit of ImplementationUnit of Implementation

Provides guidance, visibility, funding, political support for MiBLSi

StudentsStudentsStudentsStudents

Building StaffBuilding StaffBuilding StaffBuilding Staff

Building Building Leadership TeamLeadership Team

Building Building Leadership TeamLeadership Team

LEA District LEA District Leadership TeamLeadership Team

LEA District LEA District Leadership TeamLeadership Team

Across State

Multiple District/Building Teams

All staff

All students

Multiple schools w/in local district

Who is supported?

How is support provided?

Provides guidance, visibility, funding, political support

Provides coaching for District Teams and technical assistance for Building Teams

Provides guidance and manages implementation

Provides effective practices to support students

Improved behavior and reading

ISD Leadership ISD Leadership TeamTeam

ISD Leadership ISD Leadership TeamTeam

Regional Regional Technical Technical AssistanceAssistance

Regional Regional Technical Technical AssistanceAssistance

MichiganMichiganDepartment of Department of

Education/MiBLSi Education/MiBLSi LeadershipLeadership

MichiganMichiganDepartment of Department of

Education/MiBLSi Education/MiBLSi LeadershipLeadership

Multiple schools w/in intermediate district

Provides guidance, visibility, funding, political support

MiBLSi Statewide Structure of support

Activity

ISD Team

Building Staff

Building Implementation

Developing Feedback Loops and Continuous Improvement Cycles

Providing supports for effective practices implemented with fidelity

Providing feedback and data on implementation efforts

District Team

Integrated & Compensatory

Compe

tenc

y D

river

s

Compe

tenc

y D

river

s Organization D

rivers

Organization D

rivers

LeadershipLeadership

Leadership Responsibilities

• Selecting, training, and coaching of evidence based practices requires the leader to demonstrate the following responsibilities:

Michigan’s Implementation Network http://min.cenmi.org

Michigan’s Integrated Behavior and Learning Support Initiativehttp://miblsi.cenmi.org

Web Resources