Tools to Help Your Team Implement SWPBIS Data-Based Decision Making with PBIS Applications: Idaho PBN Conference Boise State University February 7, 2020 Bert Eliason
Transcript
Tools to Help Your Team Implement SWPBIS
Data-Based Decision Makingwith PBIS Applications:
Idaho PBN ConferenceBoise State University February 7, 2020Bert Eliason
Presenter
Presentation Notes
KC
Session Purpose & ELOs
Purpose: To introduce the tools within PBIS Applications to help teams implement SWPBIS
Essential Learner Outcomes (ELOs): Describe generally what PBIS Applications is and what resources are available to schools
Understand the role of PBIS Assessment in monitoring fidelity of PBIS implementation
Understand the role of the SWIS Suite in monitoring student outcomes
Understand the role of PBIS Evaluation for state, region, or district evaluation
Understand basic navigation of the PBISApps website and the SWIS Suite Demo account
Presenter
Presentation Notes
Trainer Notes: Share with participants the purpose of today’s module and what they should know and be able to do by the end of the module.
PBIS Applications
ECS is a research unit within the University of Oregon’s College of Education.
ECS engages in federal and state funded projects focused on PBIS to support research, teaching, dissemination, and technical assistance.
ECS is one of four co-director sites for the PBIS National Technical Assistance Center.
Educational and Community Supports
About Us
PBIS Applications (PBISApps) is a not-for-profit group operated by faculty and staff at the Educational and Community Supports (ECS) research unit within the College of Education at the University of Oregon.
Mission: PBIS Apps supports educators to create effective, equitable learning environments for all students through high-quality data systems and training.
It is the provider of several web-based applications:
• School-Wide Information System (SWIS) Suite
• PBIS Assessment• PBIS Evaluation
Our applications have been implemented in more than 26,000 schools in the US and abroad.
Rationale for Data-Based Decision Making in SWPBIS
Presenter
Presentation Notes
KC Let’s start by with the big ideas… what is the rationale behind data-based decision making in School-Wide Positive Behavioral Interventions and Supports?
Research on Sustainability of SWPBIS
What single factor is most related to high sustainability of SWPBIS?
7
The frequency with which DATA are presented to all school staff
Kent McIntosh, PhD, Jerin Kim, MS, Sterett H. Mercer, PhD, M. Kathleen Strickland-Cohen, PhD, and Robert H. Horner, PhD (2015).
District buy-in?Amount of money allocated to SWPBIS?Coaching?Percentage of higher SES students?Longer team meetings?Number of cool posters and caught-being-good tickets?
Presenter
Presentation Notes
Recently a survey found that one of the factors most closely related to high sustainability of School-Wide Positive Behavioral Interventions and Supports was (click) the frequency with which data were presented to all school staff. This wasn’t a big surprise since many other studies have shown that using data improve decisions, but sometimes, we forget to share data with our stakeholders, a process that is very important.
Outcome/Goal
SWPBIS and Outcomes
8
Desired Outcome: Enhanced social competence& academic achievement for all students.
How do we get to our desiredoutcome?
SystemsStaff Behavior Supports
PracticesStudent Behavior Supports
DataDecision-Making Supports
Presenter
Presentation Notes
KC While PBIS is becoming more clear about the importance of sharing data, the concept of using data has always been embedded into the model. In SWPBIS we start by identifying desired outcomes or goals, usually related to student social competence and academic achievement, then we align and organize our data, systems, and practices to move us efficiently and effectively toward those goals.
SWPBIS Continuum of Supports (Tiers)
9
All specialized interventions are more effective and more durable with universal, school-wide behavioral expectations as a foundation.
Tertiary—intensive, individualized5% of student population
Secondary—targeted, small group15% of student population
Universal—primary prevention provided to all students, effective for approximately 80%
Presenter
Presentation Notes
KC Our outcomes, data, systems, and practices should span the full continuum of supports. What data, systems, and practices will we provide to all staff and students universally? How will we provide a menu of low-cost, highly efficient secondary supports to catch students who are not successful with universal supports alone? Finally, what are the tertiary data, systems, and practices needed for the remaining 5% of students who need individualized or intensive tiers of support?
Inaccurate Data
False Assumptions
Poor Decisions
Accurate Data
Better Assumptions
Good Decisions
What is Data Integrity?
Data Integrity• Maintaining the accuracy and
consistency of data over its entire life cycle
• Critical aspect to the design, implementation, and usage of any system that stores, processes, or retrieves data
For data to be useful, it should reliably match overall perceptions across staff, students, and families.
10
Presenter
Presentation Notes
KC Before we go into data, it’s important that we review two critical concepts in data-based decision making: data integrity and data confidentiality. Data integrity is maintaining the accuracy and consistency of data over time. For data to be useful it should reliably represent the perceptions across staff, students, and familiar. This requires that there be agreement about what is measured. Inaccurate data skews our assumptions and can result in poor decisions (click). Data that is consistent across our school leads to better assumptions and better decisions.
Types of Information (Data)
11
Outcome Data
Fidelity Data
Is the plan resulting in progress toward our goals?
Did we implement the systems and strategies we agreed on?
Presenter
Presentation Notes
Most of the time, we generally think about student data. (click) Outcome data indicates the impact of our systems and practices. How well are the students responding to our interventions or the social environment? However, that’s not enough. (click) We need two types of data: outcome and fidelity data. Fidelity data indicate how well adults are implementing procedures or interventions. Are we implementing the way it was designed and using the methods we said we would? We want to make sure that schools and educational facilities or organizations are able to keep their finger on the pulse of what is going on to make the best programmatic decisions possible for their students and staff.
Connecting Fidelity & Outcome Data
Lucky SustainingPositive outcomes, low understanding of how they were achieved
Replication of success is unlikely
Positive outcomes, high understanding of how they were achieved
Replication of success likely
Losing Ground LearningUndesired outcomes, low understanding of how they were achieved
Replication of failure likely
Undesired outcomes, high understanding of how they were achieved
Replication of mistakes unlikely
Out
com
es
Fidelity13
Presenter
Presentation Notes
So what happens when you DON’T connect fidelity and outcome data? (click) Losing Quadrant Schools in the “Losing Quadrant” have low results and have no idea what to do to improve. They tend to engage in self-defeating behavior by doing the same thing year after year and expecting different results. They rarely study their systems and practices to determine if they are making an impact on students. They tend to blame the poor performance on their situations and circumstances. (click) Lucky Quadrant Schools in the “Lucky Quadrant” have achieved high results before they walk into school in the morning. They are not able to link their professional practices to results because they do not know how their practices influence the outcomes. (click) Learning Quadrant Although schools in the “Learning Quadrant” have not yet achieved their desired results, they do possess keen insight into the antecedents of excellence. These educators dig deeply into data to determine effective systems and practices that are helping students to achieve. They look for examples of excellence and work to replicate these successful practices. (click) Sustaining Quadrant Schools in the sustaining quadrant enjoy the optimal combination of high results and a keen understanding of the antecedents of excellence. Schools in this quadrant, however, do not rest on their laurels. Rather, they are perpetually seeking opportunities to improve. They continue to implement effective practices as they work with students.
Continuous Quality ImprovementIdentify problems with precision
Establish goal(s)
Develop solution(s)
Implement solution(s) with integrity and fidelity
Monitor outcomes and compare to goal(s)
Reassess and revise solution(s) as needed
Presenter
Presentation Notes
I’m going to define data-based decision making as any cycle or model that uses data to evaluate, plan, and implement. Evaluation includes identifying the current status and any potential problems with precision. Planning includes establishing goals and developing the specific solution strategies that will help us to achieve them. Data continue to be used throughout implementation. As we implement solutions with integrity, we evaluate or monitor our fidelity and outcomes, compare to our goals, and then reassess and revise the plan as needed. This cycle repeats itself until one goal is met and we can move on to another.
Different Applications to Meet the Data Needs
15
Outcome Data
Fidelity Data
Presenter
Presentation Notes
PBIS Evaluation is the “sweet spot” that brings information about whether we’re implementing strategies as designed and information about what effect it’s having on the students together across multiple schools. This allows a new level of decision makers to coordinate resources and supports related to PBIS and social climate.
PBIS Assessment – Fidelity Data
OverviewPBIS Assessment provides surveys for PBIS teams to take as they examine their level of SWPBIS implementation fidelity and look for ways to improve systems and practices to benefit students, their families, and the overall school culture.
16
Presenter
Presentation Notes
PBIS Assessment was first released as EBS Surveys in the 2001-2002 school year. Schools were using surveys and tools created to monitor the fidelity of PBIS implementation but did not have an efficient way to store and evaluate the results. There have been major changes and additional tools added to the system since it’s first release but PBIS Assessment continues to provide schools with access to tools and reports that measure PBIS implementation and inform system-level decision making.
PBIS Assessment
A major feature of SWPBIS is the commitment to ongoing assessment of implementation fidelity.
Surveys are completed online with reports immediately available as soon as a survey is submitted.
PBIS Assessment is absolutely and requires only that someone in the district attend a webinar to learn how to coordinate the surveys.
Available Assessments
Annual Assessment Tool Progress Monitoring Tool
All Tiers Tiered Fidelity Inventory (TFI)
Tier 1 (Universal/Primary)
• Tiered Fidelity Inventory (TFI)
• Benchmarks of Quality (BoQ)• Self-Assessment Survey (SAS)• Early Childhood Benchmarks
Outcome Tool: School Safety Survey (SSS) & School Climate Surveys*
*Student, Staff, Family*English & Spanish
Fidelity Across Tiers with the TFI
19
Presenter
Presentation Notes
CRD
TFI Report Options
Overall (Total Score) Scale (Breakdown by Tier)
20
Presenter
Presentation Notes
CRD
TFI Report Options
Subscale (Breakdown by Tier & Subscale)
21
Presenter
Presentation Notes
CRD
TFI Reporting Options
Items (Breakdown by specific items)
Use the items to build an action plan for areas that can be improved.
22
Presenter
Presentation Notes
CRD
School Climate Surveys
23
School Climate Survey: Student
24
Available in English & Spanish
SCS Student Reporting Options
25
School Climate Survey: Family
26
Available in English & Spanish
SCS Family Reporting Options
27
School Climate Survey: School Personnel
28
Available in English & Spanish
SCS School Personnel Reporting Options
29
The SWIS Suite – Outcome Data
The SWIS Suite is a reliable, confidential, web-based information system to collect, summarize, and use student outcome data for decision making.
The three SWIS applications, SWIS, CICO-SWIS, and I-SWIS, align with the PBIS framework and provide the needed data for both universal screening as well as progress monitoring.
SWIS Suite for Decision Making
Social-Emotional/BehaviorI-SWIS is a decision system for students requiring more intensive and individualized academic, social, or mental health supports.
CICO-SWIS is a decision system for targeted or group-based interventions for students needing additional support beyond the Universal or Tier I system.
The School-Wide Information System (SWIS) is a web-based decision system designed to help school/facility personnel use office referral data to monitor progress of school-wide and individual student interventions.
Presenter
Presentation Notes
Describe how each of the SWIS Suite applications fits into the triangle.
32
A school/site needs to work directly with a certified facilitator to get access to the application(s).
Subscription Fees:• One application - $350/year• Two applications - $460/year• Three applications - $570/year
*prorated each month
School-Wide Information System (SWIS)
SWIS allows school staff to enter office discipline referrals online.
The data are summarized to provide information about individual students, groups of students, or the entire student body over any time period. As a result, the user can:o Review school-wide referral patterns
o Define behavior patterns in greater detail
Presenter
Presentation Notes
First released in the 1998-99 school year, SWIS provides a decision system to organize and analyze discipline referral data for efficient collection and analysis of problem behavior patterns across the school. Data can be summarized and disaggregated in many ways to allow precise problem identification.
SWIS Dashboard
34
• Dashboard ReportsOverview of current school yearo Average Referrals Per Day Per
Month o Referrals by Timeo Referrals by Locationo Referrals by Day of Weeko Referrals by Locationo Referrals by Problem Behavioro Referrals by Gradeo Referrals by Student
• Latest Referrals• Data Integrity Summary
Presenter
Presentation Notes
When you log in the dashboard greets you with a set of reports that summarize referrals for the current school year. In your Account Settings you will be able to identify whether this should show All referrals, Majors only, or minors only. These reports are not used directly for decision making during monthly meetings because they don’t reflect the most recent referrals, past October these would mix old and new problems. It is a good way to orient yourself to the overall patterns and possibly start to identify questions. On the right-hand side, the Latest Referrals are listed which is a good indicator to the data entry users where they left off in entering referrals and also an indicator to other users whether the data are current. If the latest referral is more than a few days old then it would be best to hold on analyzing reports until data entry is caught up.
Core SWIS Reports
35
Presenter
Presentation Notes
Average Referrals Per Day Per Month: Point out the levels, trends, and peaks... Then add the national median... Check out the tables and other options available (again use least year’s data if training in summer or early in the school year) Referrals by Location: Identify the two most common locations when behaviors are reported, show the sort options, show Values Referrals by Problem Behavior: Identify the three most commonly reported behaviors, briefly point out that there might be related behaviors (major and minor defiance) and local priorities (Harassment might be a higher priority than Inappropriate Language) Referrals by Time of Day: Encourage the use of time “ranges” rather than one specific time in order to “catch” related issues, also identify that there are likely different grade levels with different schedules reflected Referrals by Student: Focus on the use of this report for screening, not trying to “blame” specific students for problems or trends in the data overall (it is rare that school-wide trends are significantly impacted by an individual student... Remind users about the October catch study and the PBIS logic discussed in the Data-Based Decision Making section. Referrals by Day of Week: This report is typically more useful when a school has different schedules for different days of the week but could also be helpful if Friday is “music day” or other day-specific routines may be contributing to problems. Referrals by Grade: This information allows the team to identify whether certain grade levels or grade ranges may be over-contributing to referral data. If there are active grade-level teams then it may not be appropriate to focus on conversations on a single grade unless there is representation from the grade-level staff. However, it is still important for the school-wide team to understand those patterns.
SWIS Additional Reports
36
Presenter
Presentation Notes
Multi-Year (Avg Referrals, Location, Behavior): Our first three additional reports simply allow us to look at data across years (up to 5 at a time) to look at levels, trends, and peaks that are similar or different across years. You’ll notice that December continues to be a month where we see a spike in referrals but has increased in the last wo years. Staff Report: We will intentionally skip the Staff Report. This report is only available to our SWIS Admin user(s) ***indicate name here***. The PBIS team might want to receive occasional updates on whether we have all or most staff members submitting referrals but most often this information should be used by someone in an administrative role who can offer supports to staff who are not submitting referrals or who seem to be referring at higher levels than other staff which might indicate a need for additional staffing or tips on classroom management. It would be highly inappropriate for the team to discuss colleagues in most cases! Suspension/Expulsion Report: It’s important quarterly or at least twice annually to check on whether we are formally excluding students from instruction as a result of problem behavior. While there are times when this is unavoidable we want to look for ways to reduce the number of students and amount of time away from their peers and scheduled instruction. Ethnicity Reports: A national conversation continued to prioritize disproportionality by gender, ethnicity/race, and disability. Discipline is one area that commonly comes up in disproportionality. We don’t want a student’s ethnicity or race to influence our decisions but we are all vulnerable to what is often called “implicit” or unintentional bias. SWIS Ethnicity reports focus on looking at referral patterns to help us identify whether we have some groups who are more vulnerable to receiving referrals compared to other groups. It’s important to look at all 4 reports and other data to know whether this might be an issue the team should address. There are some practical strategies we can use to address disproportionality if we identify that this is a need but this would be a more advanced/focused conversation so we’ll hold for now. Triangle Report: According to the PBIS/MTSS logic we will know when our Tier I supports are successful when we consistently see at least 80% of our students meeting expectations without additional supports. Over the course of a year we want to see that no more than 20% of our students receive more than 1 referral or maybe one major and 1 minor (adapt if the school has specific decision rules already in place). When we start to focus on Tier 2 supports then we want to catch even more students. One sign that a school is ready to focus on Tier III supports is that we have no more than 5% of students in need of this level of intensity. The triangle report summarized data from across multiple years. Notice that the current year is in process so it skews the information a bit. It’s usually best to include only full years of data. Year-End Reports: Finally, the Year-End Reports take all of the reports and summarize data so that it can be used for external reporting and annual action planning. The calculations in these reports are a bit different so that data aren’t skewed by shifting school days or enrollment numbers which change from year to year.
Student Dashboard
37
The Student Dashboard provides a set of reports for an individual student (parent-teacher conference, consideration for additional supports, school transcript)
Presenter
Presentation Notes
One last report that I want us to look at is the Student Dashboard report which gives a set of reports for an individual student. Remember earlier I indicated that our Tier I discussions should generally not include individual names. However, there are other teams or situations when reports about an individual student will be very important! Think about parent-teacher conferences or when a student has been identified as a candidate for Tier 2 or Tier 3 supports. We don’t want to week through the drill down to gather information about an individual student, we need an easy button. Given that SWIS data is school-specific and cannot be copied or transferred this is also a useful tool when the student transfers to a new school and we believe that referral data would be important to include for the student.
Check-In Check-Out (CICO-SWIS)
CICO-SWIS allows school staff to enter CICO point card data online.
The data are summarized to provide : oOverall integrity and use of CICO across studentso Individual student progress data (average, by period, by single
period)
Presenter
Presentation Notes
As SWPBIS began to scale up across the U.S., schools recognized that referral data was useful for monitoring at the universal level but students who needed secondary supports required more frequent monitoring. Check-In Check-Out is an evidence-based secondary intervention and provided a format for collecting behavioral data throughout the day so in 2008-09 a new application was build to collect and analyze point data for (and across) students receiving CICO.
CICO-SWIS Reports
Presenter
Presentation Notes
Check-In Check-Out – SWIS (CICO-SWIS) is designed to monitor outcomes for student who receive the secondary or group-based behavioral intervention often called Check-In Check-Out.
Individual Student Intervention System (I-SWIS)
I-SWIS allows school staff to enter individual student support data online.
The data are summarized to provide: o Overall integrity and use of Tier III supports across the school
(how many students, overall status, consistent data collection)
o Individual student progress data (fidelity of plan implementation, student behavioral outcomes, student academic outcomes, related notes)
Presenter
Presentation Notes
For students who need the third (or fourth or fifth) layer of support we often find that we can’t separate social behavior from mental health and/or academic needs so in 2012-13, ISIS-SWIS was released to manage fidelity and outcome data, documentation, and stakeholders involved in designing, implementing, and managing individual student’s support plans. Individualized student supports is anything more individualized than a standardized, group-based intervention so students receiving Tier III supports may or may not have formal identification or support plans such as an BSP or IEP. Anytime we need to provide something unique to the student we need to also individualize the fidelity and outcome data systems. ISIS-SWIS is flexible to allow the student support team to determine what the monitoring and decision system should look like for each individual student.
I-SWIS Reports
Presenter
Presentation Notes
Individual Student Interventions or ISIS-SWIS is a little bit different and provides a comprehensive management system as well as data collection system for students receiving intensive or individualized supports.
Overview
PBIS Evaluation is a web-based application that aggregates and summarizes data from the SWIS Suite and PBIS Assessment across all schools within a state, region, or district.
The data provided in PBIS Evaluation address the three evaluation questions:o Which schools are actively engaged in SWPBIS
implementation?
o Are schools implementing SWPBIS with fidelity?
o Is the implementation having an effect on student behavior?
Presenter
Presentation Notes
In 2005-06, more districts and states were beginning to scale up PBIS implementation across schools. District-, state-, and regional-level PBIS teams were starting to look at data from a new perspective, including PBIS Assessment and SWIS Suite data. A new application was released to take data from both systems and aggregate across schools or cohorts of schools that the team supported.
![Weaving Together RAMP and PBIS - School counselor...9/21/2017 14 •RAMP-PBIS interaction • ^The data is amazing • ^Part of the [school] culture •Challenges and benefits RAMP-PBIS:](https://static.documents.pub/doc/80x56/5f01e6227e708231d40194cd/weaving-together-ramp-and-pbis-school-counselor-9212017-14-aramp-pbis.jpg)
![School Counselors’ Role in Multidisciplinary MTSS APBS ... · based on the data provided by PBIS. … It [PBIS data] might put [students] on the radar who possibly wasn’t” •](https://static.documents.pub/doc/80x56/5f652049ee938f4c1745c685/school-counselorsa-role-in-multidisciplinary-mtss-apbs-based-on-the-data-provided.jpg)
