Date post: | 31-Dec-2015 |
Category: |
Documents |
Upload: | harry-hoover |
View: | 215 times |
Download: | 1 times |
Measuring Implementation:
School-Wide Instructional Staff Perspective
Amy Gaumer Erickson, Ph.D.University of KansasEvaluator: Kansas & Missouri SPDGs
SIG Evaluators WebinarMarch 17, 2011
Gaumer Erickson (2011)
What types of measures are implemented? Self-Report Interviews/Focus Groups Observations Retrospective/Reflective Learning Goal Attainment Ratings of Fidelity Outcome Data
Gaumer Erickson (2011)
What perspectives do we need to get a complete picture of implementation in schools?
Measures/observations completed by: Implementation Coaches School Leadership Teams All School Instructional Staff Students Families/Communities
Gaumer Erickson (2011)
Measures completed by Implementation Coaches SW-PBS Benchmarks of Quality (BoQ)
MiBLSI: http://miblsi.cenmi.org/MiBLSiModel/Evaluation/Measures/BenchmarksofQuality.aspx
SW-PBS School-wide Evaluation Tool (SET) OSEP Technical Assistance Center on
PBIS: http://www.pbis.org/evaluation/evaluation_tools.aspx Gaumer Erickson (2011)
Measures completed by School Leadership Teams SW-PBS Team Implementation Checklist (TIC)
PBIS Surveys: http://www.pbssurveys.org/pages/Home.aspx
Effective Behavior Support Team Implementation Checklist (EBS-SAS) PBIS Surveys:
http://www.pbssurveys.org/pages/Home.aspx Planning and Evaluation Tool for Effective Reading
Supports – Revised (PET-R) DIBELS Data System:
http://dibels.uoregon.edu/resources/ Implementation Rubrics
Illinois PBIS Network: http://www.pbisillinois.org/
Gaumer Erickson (2011)
Student Measures DIBELS
DIBELS Data System: https://dibels.uoregon.edu/resources.php
AIMSweb AIMSweb: http://www.aimsweb.com/
School-Wide Information System (SWIS) SWIS: http://www.swis.org/
State Performance Plan APR Data (Indicators 1, 2, 3, 4, 5, 13, 14)
Student Perceptions
Gaumer Erickson (2011)
Family/Community Measures Parent Satisfaction
Missouri Improvement Program Advanced Questionnaire http://dese.mo.gov/divimprove/sia/dar/advance_questionnaire_surveys.html
Family as a Teacher Inventory Parent-Teacher Involvement Questionnaire
Fast Track Project http://www.fasttrackproject.org/data-instruments.php
Interviews/Focus Groups
Gaumer Erickson (2011)
Measures completed by All School Instructional Staff Self-Assessment Survey (SAS)
PBIS Surveys: http://www.pbssurveys.org/pages/Home.aspx
Standards Assessment Inventory National Staff Development Council
http://www.learningforward.org/standards/sai.cfm
Classroom/Teacher ObservationsGaumer Erickson (2011)
Which perspective do you think your grants measure most effectively?
A: Implementation Coaches B: School Leadership Teams C: All School Instructional Staff D: Stakeholders:
Student/Parents/Communities
Gaumer Erickson (2011)
School Staff Survey Design Identified essential features of school reform
initiatives (e.g., RTI, SW-PBS, PLCs, High Schools that Work, Reading First)
Analyzed other measures completed by school instructional staff (e.g. PBIS Self-Assessment Survey, RTI Implementation Tool, National Staff Development Council Standards Assessment Inventory, Missouri School Improvement Process Faculty Advanced Questionnaire)
Wrote over 100 items and then narrowed it down to 33 through pilot testing with schools and analysis by a state evaluation team
Implemented survey in 14 schoolsGaumer Erickson (2011)
School Staff Survey Reliability Overall Alpha: 0.98 Three Factors
School Implementation (17 items): 0.95 Classroom Implementation (10 items): 0.85 Individual Student Implementation (5 items):
0.90 Four Groups
Teachers (N=294): 0.96 Administrators (N=8): 0.90 Other Certified Staff (N=20): 0.97 Noncertified Staff (N=6): 0.85
Gaumer Erickson (2011)
School Implementation1) I can summarize the school's shared vision/mission.2) I have a clear understanding of the phrase: “tiered levels of academic & behavior
support.”3) I feel that my administrators are committed to implementing tiered levels of
academic & behavior supports. 4) I receive coaching/mentoring to implement evidence-based instructional
practices.5) I think my school does a good job of addressing the academic & behavior needs
of students at tier 1 (universal).6) I think my school does a good job of addressing the academic & behavior needs
of students at tier 2 (small group).7) I think my school does a good job of addressing the academic & behavior needs
of students at tier 3 (intensive).8) I regularly see students move between tiers of support as their academic needs
change.9) I regularly see students move between tiers of support as their behavior needs
change.10) I am involved in meetings where data results are discussed & problem solving
occurs.11) I receive school-wide academic & behavior data in usable & understandable
formats.12) I participate in professional development where I learn how to monitor students'
progress & use progress monitoring data.13) I have the time necessary to analyze student data & problem solve with my
colleagues.14) I think my school is a good place to work.15) I think my school has an effective process in place to identify available resources
(e.g., materials, technology, people).16) I evaluate the effectiveness of core instruction based on progress monitoring
data.17) I think that the current behavior/academic programs in my school are improving
education for students in my school.Gaumer Erickson (2011)
Classroom Implementation1) I use assessment data at least three times a year to monitor
students' progress.2) I adapt the environment, curriculum, & instruction based on
each student’s academic & behavior data.3) I participate in professional development where I learn ways
to improve my instructional practices.4) I receive coaching/mentoring to help me implement tiered
levels of academic & behavior support.5) I have the technology & resources that I need to provide
effective instruction.6) I am able to meet the students' diverse needs.7) I consider my students' background when I teach and/or
interact with students.8) I regularly communicate with families regarding student
academic & behavior goals/progress.9) I make informed decisions based on feedback from families.10) I collaborate with my colleagues on a regular basis.
Gaumer Erickson (2011)
Individual Student Implementation1) I am involved in action planning tiered supports with the
other staff & administrators at my school.2) When I am concerned about a student’s academic
success, I collaborate with a team to identify intervention.
3) I feel that the team that addresses academic needs provides valuable feedback & makes informed decisions.
4) When I am concerned about a student’s behavior success, I collaborate with a team to identify intervention.
5) I feel that the team that addresses behavioral needs provides valuable feedback & makes informed decisions.
6) I think my school does a good job in including parents as team members in data-based decision-making.
Gaumer Erickson (2011)
Reporting Results School Summaries (to support data-
based decision-making and action planning)
School Comparisons Across Years Dissemination & Evaluation of Program
Effectiveness SPDG APR
Gaumer Erickson (2011)
Gaumer Erickson (2011)
Gaumer Erickson (2011)
Gaumer Erickson (2011)
Gaumer Erickson (2011)
SPDG APR ReportingPercent of school personnel involved in the MIM who report the use of data-driven decision making; [correlated with SPP Indicator 3; Program Performance Measure 1.2].To identify the level of data-driven decision making, school staff were asked to rank the statement, I adapt the curriculum, instruction, & environment based on each student’s academic & behavior data, on the MIM School Staff Survey. The results show that 79% of MIM school staff report using data to make adaptations for students. In addition, school staff was asked to rank the following statements related to data-driven decision making:I use assessment data at least three times a year to monitor students' progress. Out of 429 school staff who responded, 351 (82%) ranked this statement as a 4 or 5.I am involved in meetings where data results are discussed & problem solving occurs. Out of 444 school staff who responded, 302 (68%) ranked this statement as a 4 or 5.My students move between tiers of support as their academic needs change. Out of 424 school staff who responded, 247 (58%) ranked this statement as a 4 or 5.My students move between tiers of support as their behavior needs change. Out of 410 school staff who responded, 204 (50%) ranked this statement as a 4 or 5.
Gaumer Erickson (2011)
Discussion What other methods/tools are you using
to obtain data from all school instructional staff?
Is a measure like the School Staff Survey something that would be helpful on your projects?
Gaumer Erickson (2011)
For More Information:Amy Gaumer Erickson, Ph.D.University of [email protected]
Gaumer Erickson (2011)