Date post: | 17-Dec-2015 |
Category: |
Documents |
Upload: | catherine-mclaughlin |
View: | 216 times |
Download: | 1 times |
School-wide PBIS:Using Data for Effective Coaching
Rob HornerUniversity of Oregon
www.pbis.org www.uoecs.org
Goals Address questions you face as a PBIS Coach
Building Decision-Systems Basic Foundations Access to core data sources Rubric for using data to make decisions On-going review and improvement using data
Data needed for effective SWPBIS coaching Using Fidelity data to assist teams in SWPBIS implementation
Team problem solving processes Establishing the Foundations Using data for action planning Using data for implementation assessment and adaptation
Using Data for Decision-makingWhat decisions are needed?
Who needs what information at what time in what form to make effective decisions?
ActionPlanning
Improving Decision-Making
Problem
Problem
Solution
From
To
Problem
Solving Informati
on
Solution
School-Team DecisionsAre we implementing SWPBIS?
Are we implementing at all three tiers? What would be the next smallest change that would
make the biggest effect?
If we are implementing, are the procedures benefiting students?
Are there problems? Define problem with precision:
What, Where, When, Who, Why What proven actions (interventions, practices, packages)
would address the problem yet fit with current strengths? Are selected actions being implemented and effective?
How do they need to be adapted? Are selection actions producing desired outcomes?
Fidelity of ImplementationMeasuring fidelity is more
than accountability. Measuring fidelity and regular action
planning are key procedures for “getting it right”
Help teams use their strengths to find the path that best helps them achieve the SWPBIS core features.
Be careful about adoption too many strategies/ practices.
Do a small number of things well.
Level of Support
Research Measures(2-4 hours)
Annual Self-Assessment Measures
(45-60 min)
Progress Monitoring Measures
(15 min)Universal (Tier I) School-wide
Evaluation Tool (SET)
Self-Assessment Survey (SAS)
Benchmarks of Quality (BoQ)
Team Implementation Checklist (TIC)
Secondary and Tertiary (Tier II, Tier III)
Individual Student School-wide Evaluation Tool
(ISSET)
Benchmarks of Advanced Tiers (BAT)
Measure of Advanced Tiers Tool (MATT)
Overall Implementation
Implementation Phases Inventory (IPI)
Phases of Implementation (POI)
Phases of Implementation Checklist (PIC)
SWPBIS Measures of Fidelity( See PBIS Evaluation Blueprint; Measures available at www.pbisassessment.org)
How Knowledgeable/Comfortable are you with ( 1= low; 5 = high):
School-wide Evaluation Tool SETSelf-Assessment Survey SASTeam Implementation Checklist TICBenchmark of Quality BoQ
Individual Student SET ISSETBenchmark of Advanced Tiers BATMeasure of Advanced Tiers Tool MATT
Coaching for Implementation Are we implementing SWPBIS?
Team Implementation Checklist (15 min) Benchmark of Quality (45-60 min) School-wide Evaluation Tool (2-4 hours)
How Often Should Data be Collected? Initial implementation (every 3rd or 4th meeting) On-going (annually)
------------------------------------------------------------------------
Total Score Are we improving?
Sub-scale score What is working where do we focus next?
Item score Action planning
Elementary
Middle
Team Checklist: Subscale ScoresPercentage of Total Points
Comm
itmen
t
Asse
ssm
ent
Team
Define
Exp
Teac
h Ex
p
Rewar
d
Conse
quen
ce
Class
room
Info
rmat
ion
Dist S
uppo
rt0
10
20
30
40
50
60
70
80
90
100
Time 1 Time 2 Time 3 Time 4
Team Checklist: Subscale ScoresPercentage of Total Points
Comm
itmen
t
Asse
ssm
ent
Team
Define
Exp
Teac
h Ex
p
Rewar
d
Conse
quen
ce
Class
room
Info
rmat
ion
Dist S
uppo
rt0
10
20
30
40
50
60
70
80
90
100
Time 1
Team Checklist: Subscale ScoresPercentage of Total Points
Comm
itmen
t
Asse
ssm
ent
Team
Define
Exp
Teac
h Ex
p
Rewar
d
Conse
quen
ce
Class
room
Info
rmat
ion
Dist S
uppo
rt0
10
20
30
40
50
60
70
80
90
100
Time 1 Time 2
Team Checklist: Subscale ScoresPercentage of Total Points
Comm
itmen
t
Asse
ssm
ent
Team
Define
Exp
Teac
h Ex
p
Rewar
d
Conse
quen
ce
Class
room
Info
rmat
ion
Dist S
uppo
rt0
10
20
30
40
50
60
70
80
90
100
Time 1 Time 2 Time 3
Team Checklist: Subscale ScoresPercentage of Total Points
Comm
itmen
t
Asse
ssm
ent
Team
Define
Exp
Teac
h Ex
p
Rewar
d
Conse
quen
ce
Class
room
Info
rmat
ion
Dist S
uppo
rt0
10
20
30
40
50
60
70
80
90
100
Time 1 Time 2 Time 3 Time 4
Your Turn: What should team focus on ?
Comm
itmen
t
Asse
ssm
ent
Team
Define
Exp
Teac
h Ex
p
Rewar
d
Conse
quen
ce
Class
room
Info
rmat
ion
Dist S
uppo
rt0
102030405060708090
100
Team Implementation Checklist
Your Turn: What should team focus on ?
Comm
itmen
t
Asse
ssm
ent
Team
Define
Exp
Teac
h Ex
p
Rewar
d
Conse
quen
ce
Class
room
Info
rmat
ion
Dist S
uppo
rt0
102030405060708090
100
Team Implementation Checklist
Your Turn: What should team focus on ?
Comm
itmen
t
Asse
ssm
ent
Team
Define
Exp
Teac
h Ex
p
Rewar
d
Conse
quen
ce
Class
room
Info
rmat
ion
Dist S
uppo
rt0
10
20
30
40
50
60
70
80
90
100Time 1 versus Time 2
Demonstration School ExemplarNCES ID: Zenith, Winnemac
Demonstration DistrictNCES ID :
School Year Number of Responses
Date Collected Action Who/When
2009-10 1 09/30/2009
Feature Score (0, 1, 2)
Establish Commitment
1. Administrator's Support & Active Involvement. 12. Faculty/Staff Support. 1
Establish & Maintain Team
3. Team Established (Representative). 14. Team has regular meeting schedule, effective operating procedures.
2
5. Audit is completed for efficient integration of team with other teams/initiatives addressing behavior support.
0
Conduct Self-Assessment
6. Team completes the Team Implementation Checklist (TIC).
2
7. Team summarizes existing school discipline data. 18. Team uses self-assessment information to build implementation Action Plan (areas of immediate focus).
0
Demonstration School ExemplarNCES ID: Zenith, Winnemac
Demonstration DistrictNCES ID :
School Year Number of Responses
Date Collected Action Who/ When
2011-12 1 09/15/2011
Feature Score (0, 1, 2)
Establish Commitment
1. Administrator's Support & Active Involvement. 2
2. Faculty/Staff Support. 2
Establish & Maintain Team
3. Team Established (Representative). 1
4. Team has regular meeting schedule, effective operating procedures.
2
5. Audit is completed for efficient integration of team with other teams/initiatives addressing behavior support.
0
Conduct Self-Assessment
6. Team completes the Team Implementation Checklist (TIC).
2
7. Team summarizes existing school discipline data. 2
8. Team uses self-assessment information to build implementation Action Plan (areas of immediate focus).
2
Using Student Impact DataUniversal ScreeningProgress Monitoring
--------------------------------------------------
Team Meetings and Use of Data for Problem Solving
Universal Screening and Progress Monitoring.
Data Sources Decisions
Universal Screening
Behavior : Oct, Feb
Reading: Sept, Nov, Mar
SWPBIS Fidelity Measures
Teacher reportSSBD/ SSRS/ SSISOffice Discipline Referrals
Oral Reading Fluency (etc)
Are Tier I supports in place?
What students need more intensive support?
Initiate early intervention
Progress Monitoring
Fidelity: Match to level of intensity of support
Impact: Match to level of intensity of support
CICO/ ISIS fidelity measures
Office Discipline ReferralsCheck-in Check-out pointsISIS intensive support
Oral Reading FluencyPhonemic SegmentationComprehension
Are Tier II and Tier III supports implemented as planned?
Are supports effective?
Are more intensive/ individualized supports needed?
~80% of Students
~15%
~5%
0-1 office discipline referral
6+ office discipline referrals
2-5 office discipline referrals
Using office discipline referrals as a metric for universal screening of student social behavior
Newton, J. S., Todd, A. W., Algozzine, K., Horner, R. H., & Algozzine, B. (2009). The Team Initiated Problem Solving (TIPS) Training Manual. Educational and Community Supports, University of Oregon, unpublished training manual. 32
Using the Referrals by Student report as a Universal Screening
Tool
33
Aug Sept Oct Nov Dec Jan Feb Mar Apr May Jun0
2
4
6
8
10
12
0-12-56+
Cum
ula
tive M
ean
OD
Rs Cumulative Mean ODRs Per Month
for 325+ Elementary Schools 08-09
Jennifer Frank, Kent McIntosh, Seth May
Helping Teams Use Data at All Three TiersTier I Office discipline
referrals
~80% of Students
~15%
~5%
ESTABLISHING a CONTINUUM of Data Options within SWPBS
SECONDARY PREVENTION• Fidelity:
• BAT, MATT* Student Outcomes:
• CICO-SWIS• ODRs
TERTIARY PREVENTION• Fidelity: ISSET, BAT, MATT
• ISIS-SWIS
• Student Outcomes: • ISIS-SWIS
PRIMARY PREVENTION• Fidelity: TIC, BoQ, SET
• Student Outcomes: • Office Discipline Referrals
SECONDARY PREVENTION
•
TERTIARY PREVENTION•
PRIMARY PREVENTION• • • • • •
Coaching Tier III Data UseNeeds
◦ System to assist coordination of individualized Team
◦ System for managing (a) assessments, (b) plan development/modification, and (c) team meeting minutes
◦ System for building data collection tailored to individualized plan
Collect Fidelity Data: Are we doing plan? Collect Impact Data: Is plan benefiting student?
◦ System for on-going collection, summarizing and reporting of data for team decision-making.
Helping Teams use Data for Decision-Making
Build team foundation Roles, Agenda, Schedule
Use data to define problems with precision
Use data to build “Action Plans” that fit the local context and will work.
Use data to assess if action plans are being implemented with effect
Problem-Solving Meeting Foundations
Structure of meetings lays foundation for efficiency & effectiveness
Meeting Foundations ElementsFour features of effective
meetings Predictability Participation Accountability Communication
Define roles & responsibilities Facilitator, Minute Taker, Data Analyst
Use electronic meeting minutes format
43
Newton, J. S., Todd, A. W., Algozzine, K., Horner, R. H., & Algozzine, B. (2009). The Team Initiated Problem Solving (TIPS) Training Manual. Educational and Community Supports, University of Oregon, unpublished training manual.
A. Predictability1. Defined roles, responsibilities and expectations
for the meeting2. Start & end on time, if meeting needs to be
extended, get agreement from all members3. Agenda is used to guide meeting topics4. Data are reviewed in first 5 minutes of the
meeting5. Next meeting is scheduled
B. Participation5. 75% of team members present & engaged in
topic(s) 6. Decision makers are present when needed
What makes a successful meeting?
What makes a successful meeting?
C. Accountability7. Facilitator, Minute Taker & Data Analyst come prepared for
meeting & complete during the meeting responsibilities8. System is used for monitoring progress of implemented
solutions (review previous meeting minutes, goal setting)9. System is used for documenting decisions10. Efforts are making a difference in the lives of
children/students.
D. Communication11.All regular team members (absent or present) get access
to the meeting minutes within 24 hours of the meeting12.Team member support to practice team meeting
norms/agreements
Define roles for effective meetingsCore roles
◦Facilitator◦Minute taker◦Data analyst◦Active team member◦Administrator
Backup for each role
Can one person serve multiple roles?
Are there other roles needed?
Typically NOT the administrator
46
Newton, J. S., Todd, A. W., Algozzine, K., Horner, R. H., & Algozzine, B. (2009). The Team Initiated Problem Solving (TIPS) Training Manual. Educational and Community Supports, University of Oregon, unpublished training manual.
Who is Responsible?Action Person Responsible
Reserve Room Facilitator
Recruit items for Agenda Facilitator
Review data prior to the meeting
Data Analyst
Reserve projector and computer for meeting
Minute Taker
Keep discussion focused Facilitator
Record Topics and Decisions on agenda/minutes
Minute taker
Ensure that problems are defined with precision
Facilitator
Ensure that solutions have action plans
Facilitator
Provide “drill down” data during discussion
Data Analyst
End on time Facilitator
Prepare minutes and send to all members
Minute taker
Activity # 1 (7 min)Select
◦Facilitator◦Data Analyst◦Minute Taker
Back up for each
50
Role Primary Backup
Facilitator
Data Analyst
Minute Taker
Next role review date:
Newton, J. S., Todd, A. W., Algozzine, K., Horner, R. H., & Algozzine, B. (2009). The Team Initiated Problem Solving (TIPS) Training Manual. Educational and Community Supports, University of Oregon, unpublished training manual.
Organizing for an effective problem solving conversation
Problem
SolutionOut of
Time
Use Data
A key to collective problem solving is to provide a visual
context that allows everyone to follow and contribute
51
Newton, J. S., Todd, A. W., Algozzine, K., Horner, R. H., & Algozzine, B. (2009). The Team Initiated Problem Solving (TIPS) Training Manual. Educational and Community Supports, University of Oregon, unpublished training manual.
Newton, J. S., Todd, A. W., Horner, R. H., Algozzine, B., & Algozzine, K. (2012) (Version 1.2)
Newton, J. S., Todd, A. W., Horner, R. H., Algozzine, B., & Algozzine, K. (2012) (Version 1.2)
Newton, J. S., Todd, A. W., Horner, R. H., Algozzine, B., & Algozzine, K. (2012) (Version 1.2)
Where in the Form would you place:
1.Planning for next PTA meeting?
2.There have been five fights on the playground in the past 3 weeks.
3.Update on CICO implementation
4.Increasing gang recruitment as an agenda topic for today.
5.Next meeting report on lunch-room status.
Summary: Team-based Problem Solving
Team FoundationsRolesMinute form
Defining ProblemsBuilding SolutionsAction Planning/ Follow up / Adaptation