miblsi.org
Building Tier 1 Data Systems within
the SWPBIS Framework
School-wide Information System (SWIS),
SWPBIS Tiered Fidelity Inventory (SWPBIS TFI), &
Student Risk Screening Scale (SRSS)
May 18, 2017 NEPBIS
Leadership Forum
Jennifer Rollenhagen & Susannah Everett
22
• OSEP PBIS Technical Assistance Center
• PBISApps, University of Oregon
Acknowledgements
33
Session Intentions
Goal
Use a problem-solving model focused on guiding questions to develop evaluation plans, analyze within and across data sources, and share results to support Tier 1 action planning
Resources
• Evaluation Blueprint for School-Wide Positive
Behavior Interventions & Support
• PBIS Apps (pbisapps.org)
• Team Initiated Problem Solving (pbis.org)
Organizing and
Analyzing the
Data
• Developing School and District Specific Guiding Questions
• Data Sources
• Action Planning
Putting It Into
Practice• School Examples
44
Foundational Statement
“Evaluation is the process of collecting and
using information for decision making. A
hallmark of School-wide Positive Behavior
Support (SWPBS) is a commitment to formal
evaluation.”
Evaluation Blueprint for School-Wide PBIS
55
Never initiate an intervention
without having a plan for
evaluating the implementation
of that intervention.
Rob Horner
Foundational Action
66
Why are schools asked to collect, submit, and
analyze SWPBIS data at the school level?
1. To use data as part of a continuous improvement
process
2. To evaluate the effectiveness of SWPBIS
implementation efforts and the effectiveness of
supports to improve student outcomes
SWPBIS Data
What is the intention?
• Compliance vs. Continuous Quality
Improvement• Data Collection and
Organization
• Screening
• Progress Monitoring
• Data-based
Decision Making
• Intervention
Selection
• Intervention Plan
• Intervention
Implementation
88
Systems Change
SYSTEMS
Supporting Staff Behavior
Supporting
Decision
Making
Supporting
Student
Behavior
OUTCOMES
99
Tertiary—intensive,
individualized 5% of student
population
Secondary—targeted, small group
15% of student population
Universal—primary prevention
provided to all students, effective
for approximately 80%
Continuum of Decision Making
All specialized interventions are more effective
and more durable with universal, school-wide
behavioral expectations as a foundation.PBISApps, 2016
1010
• Have a shared goal to improve student
outcomes
• To get there we need to:
• Know if what we are doing is impacting our
outcomes
• Know if we are doing what we said we were going
to do
Big Ideas of Evaluation
1111
Data Based Decision Making for Improved
School Climate
Student Outcome
Data
Fidelity Data
Examples:
• Discipline Referrals
• Student Risk Screening Scale
• Credits Earned, GPA
• Attendance
• Suspensions/Expulsions
• Special Education Referrals
Examples:
• SW-PBIS Tiered Fidelity Inventory
1212
Data and Evaluation Planning
1313
Example School A
• What is our baseline, and where have we made progress
through the year?
• Are we improving on our implementation of PBIS?
• How do our individual staff members rate our
implementation?
• Are our initial efforts having an impact on student
outcomes?
Example School A: Newly implementing
Evaluation Questions
1414
School A: Evaluation PlanMeasure Year One Year Two Year Three
Fall Winter Spring Fall Winter Spring Fall Winter Spring
TFI X X X X
TIC X X X X X X X X X
School
Climate -
Student
XX
X
XX
SASX X X X
SWIS x x x x x x x x x
1515
Example School B
• What specific areas in PBIS are we strong in, and
in which areas do we need more training?
• Are we improving on our implementation of PBIS?
How is our implementation of Tier 2?
• How are we identifying students in need of
additional supports?
Evaluation QuestionsImplementing PBIS for three years, has been using the TIC/SET, and has
been consistently over 80%.
1616
School B: Evaluation PlanMeasure Year One Year Two Year Three
Fall Winter Spring Fall Winter Spring Fall Winter Spring
TFI
All 3 TiersX X X
TFI Tier 2
Subscalex x x x x x
School
Climate -
Student
XX
X
XX
SRSSx x X x x X x x X
SWIS x x x x x x x x x
1717
Evaluation QuestionsImplementing PBIS for three years, including Tier 2 for the last year. Invested
in supporting and integrating student mental health supports into the PBIS
framework.
• In what specific PBIS areas are we strong, and in which
areas do we need more training?
• How are we doing with our Tier 2 implementation? Are our
interventions matched to student need? Are we providing
supports to our at-risk students?
• Are we ready to start implementing Tier 3 systems and
practices? How do we match our practices with student and
families’ needs?
Example School C
1818
School C: Evaluation Plan
Measure Year One Year Two Year Three
Fall Winter Spring Fall Winter Spring Fall Winter Spring
TFI
All 3 TiersX X X
TFI Tier 3
Subscalex x x x x x
School
Climate -
Student
XX
X
XX
SRSSx x X x x X x x X
SWIS x x x x x x x x x
1919
Task: Developing Your Plan: Questions to
Consider
• Which surveys and data sources are you
currently using?
• Do they answer your evaluation questions:
• Are we implementing SWPBIS with fidelity?
• Is there more information that we need?
• Pbisapps.org > PBIS Assessment >
Find Out More about Surveys
• Do your surveys assess the Tiers you are
implementing?
2020
• Behaviors are operationally defined
• Standard practice for all
• The data should be very easy to collect
• 1% of staff time
• Office Discipline Referral Form (ODR)
• Data are presented in picture (graph) format
• Data are used for decision making
• The data must be available when decisions need to be made (weekly?)
• Difference between data needs at a school building versus data needs
for a district
• The people who collect the data must see the information used for
decision-making
Key Features of Data Collection Systems
2121
• Which tools answer our school’s (or district’s)
questions?
• Who will coordinate administration? How and when
will surveys and screening be administered?
• Who is responsible for summarizing the data?
• Which teams will have access to data and when?
• What data (and when) will you share with staff,
families and students, and district stakeholders?
Evaluation Planning
2222
SWIS, TFI, and SRSS
DATA SOURCES
2323
What is the School-Wide Information
System (SWIS)?
The School-Wide Information System (SWIS) is a web-based decision system used to improve
behavior support in schools and other educational facilities by providing school
personnel with accurate, timely, and practical information for making decisions about school
environments
• Make decisions based on data
• It’s more than just record keeping, it’s about decision making
2424
2525
Why was SWIS developed?
Need: School teams implementing school-
wide positive behavior interventions and
supports (SWPBIS) needed effective decision
making at the school building or facility level.
Initial Developers: Seth May, William Ard III,
Anne Todd, Rob Horner, George Sugai,
Aaron Glasgow, and Jeff Sprague
2626
• Why focus on behavior?
• Social behavior is the single most common reason
students are excluded from education (U.S. Dept. of
Ed.)
• What’s the goal?
• To make schools more effective learning environments
• How?
• Repeatedly giving people the right information, at the
right time, in the right format is the single most effective
way to improve decision making and achieve valued
outcomes (Gilbert, 1978)
Value & Utility of SWIS
2727
• Decisions are more likely to be effective and
efficient when they are based on data
• The quality of decision making depends most
on the first step
• Defining the problem to be solved with precision
and clarity
Why Use Data for Decision Making
2828
• Data help us to ask the right questions...they
do not provide the answers
• Use data to:
• Identify problems
• Refine problems
• Define the questions that lead to a solution
• Data help place the “problem” in the context
rather than in the students
Using SWIS Data to Ask the Right Questions
2929
• The right data, at the right time, given to the
right people results in decisions that are
better, faster, cheaper
• To make decisions, schools need decision
systems with data that are:
• Relevant (right information in the right format)
• Accurate (consistent across staff, entered correctly)
• Timely (current, quick to find, continuously
available)
Using SWIS Data for Efficiency
3030
Using SWIS Data for Decision Making
• Universal Screening Tool for Externalizing Behaviors
• Proportion of student with:• 0-1 Major Office Discipline Referrals (ODRs)
• 2-5 Major ODRs
• 6+ Major ODRs
• Progress Monitoring Tool
• Summative Evaluation
• Compare data across time to prevent previous problem patterns
3131
Asking the Right Questions
• What are the data we need for a decision?
• Precise problem statements include information
about the following questions:
• What is the problem behavior?
• How often is the problem happening?
• Where is the problem happening?
• Who is engaged in the behavior?
• When is the problem most likely to occur?
• Why is the problem sustaining?
3232
Example Office Discipline Referral
3333
• Have a shared goal to improve student
outcomes
• To get there we need to:
• Know if what we are doing is impacting our
outcomes
• Know if we are doing what we said we were going
to do
Big Ideas of Evaluation
3434
• SWPBIS TFI Reports provide:
• A summary of the school’s SWPBIS implementation
efforts and encourages discussion of strengths and
areas of need
• A level of analysis that goes beyond initial
implementation of SWPBIS, which addresses
factors that impact sustainability including staff and
leadership changes which could impact SWPBIS
implementation efforts
Fidelity Data
3535
3636
3737
3838
3939
4040
4141
4242
Data Based Decision Making for Improved
School Climate
Student Outcome
Data
Fidelity Data
Examples:
• Discipline Referrals
• Student Risk Screening Scale
• Credits Earned, GPA
• Attendance
• Suspensions/Expulsions
• Special Education Referrals
• School Climate Surveys
Examples:
• SW-PBIS Tiered Fidelity Inventory
4343
Universal Screening Definition
Universal screening is the systematic
assessment of all students on academic and
social-emotional indicators for the purpose of
identifying students who may be at-risk, and
may require support that varies in terms of level,
intensity and duration.
4444
Why Systematic Screening?
• If we do not systematically screen:
• Students may slip through the cracks
• There may not be equitable access to interventions and
other supports
• We could not fully examine the possible interaction of
academic and behavioral needs
• There are challenges to using SWIS as the only measure for
behavior
• Discipline referral data are reflective of of the procedural
fidelity of SWIS implementation
4545
• Systematic screening of behavior
• Examines the overall level of risk in a building
• Informs the use of teacher-delivered strategies
• Identifies students who may be at risk
• Measure that is both psychometrically sound and
socially valid
• Measure that is efficient and effective
• Data used to demonstrate need to allocate
resources to support students
Criteria of a Behavior Universal Screener
4646
Student Risk Screening Scale Introduction
Dr. Kathleen Lane
https://www.youtube.com/watch?v=ali98
QoPVKA&feature=youtu.be
4747
• SRSS: Student Risk Screening Scale
• SRSS-E7: Student Risk Screening Scale – Externalizing
7
• SRSS-I5: Student Risk Screening Scale – Internalizing 5
(Elementary)
• SRSS-I6: Student Risk Screening Scale – Internalizing 5
and Peer Rejection on both scales (Secondary)
• SRSS-IE: Student Risk Screening Scale-Internalizing and
Externalizing
Student Risk Screening Scale Acronyms
4848
Secondary & Elementary: Original 7-item screener to
identify students who are at risk for antisocial behavior:
• Steal
• Lie, Cheat, Sneak
• Behavior Problem
• Peer Rejection*
• Low Academic Achievement
• Negative Attitude
• Aggressive Behavior
SRSS - Externalizing 7 (SRSS-E7)
4949
Elementary: 5 new items associated with
internalizing behaviors
• Emotionally Flat
• Shy, Withdrawn
• Sad, Depressed
• Anxious
• Lonely
SRSS – Internalizing 5 (SRSS-I5)
5050
Secondary: 5 new items associated with
internalizing behaviors, along with Peer Rejection
• Emotionally Flat
• Shy, Withdrawn
• Sad, Depressed
• Anxious
• Lonely
• Peer Rejection*
SRSS– Internalizing 6 (SRSS-I6)
5151
• Students are rated on a 4-point Likert-type
scale for each item
• 0 = Never
• 1 = Occasionally
• 2 = Sometimes
• 3 = Frequently
SRSS Scoring
5252
SRSS Cut Scores
Scale Low
Risk
Moderate
Risk
High
Risk
SRSS-E7Elementary &
Secondary
0-3 4-8 9-21
SRSS-I5Elementary
0-1 2-3 4-15
SRSS-I6Secondary
0-3 4-5 6-18
5353
How SRSS Data are Used
SRSS is used:
• To inform instruction
• To determine access to
interventions when
analyzed with other
data
SRSS is not used:
• For special education
eligibility
• To exclude students
from opportunities
• As a personality test
• As a mental health
assessment
5454
Comparison of SRSS and SWIS DataSRSS-IE SWIS
• Universal Screening Tool
• Measures Externalizing &
Internalizing behaviors
• Preventative
• Can be analyzed in
combination with academic
data for all students
• “October Catch”
• Progress monitoring tool
• Measures externalizing
behavior
• Data on problem behaviors
that have already occurred
• Can examine motivation and
trends – Dashboard reports
and Drill Down
• Can be analyzed in
combination with academic
data for some students
5555
SRSS is administered 3x a year
• Fall: October
• 6 weeks after the start of school
• Winter: December
• 2-3 weeks before winter break
• Spring: April/May
• 6-8 weeks before the end of the school year
When the SRSS is Administered
5656
Always check for accuracy of the data, then...
School Leadership Team:
• Analyzes data to determine school-wide level of risk:
• >80% of students at Low Risk
• Analyzes SRSS, SWIS, & other data together
• Analyzes SRSS with behavior fidelity data: SWPBIS
TFI
Got SRSS Data. . . Now What?
5757
The SRSS-IE provides an overall rating of risk
(low, moderate, or high) for externalizing
behaviors and internalizing behaviors but the
measure is not intended to be analyzed at the
individual item level or combined across both
externalizing and internalizing behaviors at this
time
A Word of Caution
5858
Data Based Decision Making for Improved
School Climate
Student Outcome
Data
Fidelity Data
Examples:
• Discipline Referrals
• Student Risk Screening Scale
• Credits Earned, GPA
• Attendance
• Suspensions/Expulsions
• Special Education Referrals
Examples:
• SW-PBIS Tiered Fidelity Inventory
5959
One School’s Data: SWPBIS TFI
6060
One School’s Data: Discipline
6161
One School’s Data:
SRSS
6262
• Do the current tools you use answer your evaluation
questions?
• Are there any measures you would consider adding (i.e., the
SRSS?) to answer your questions?
• How about deleting?
• Do you have a comprehensive plan? Does the timing work
for your school?
• Do you have access to the surveys and data you need? Who
is responsible for summarizing this information? Which teams
need different data?
• What and when are you sharing data with staff and other
stakeholders?
Turn and Talk
miblsi.orgmiblsi.org
Thank You!!
Jennifer Rollenhagen
Susannah Everett
