11

Positive Behavioral Positive Behavioral Interventions and Interventions and

SupportsSupports

Maryland’s State And School Maryland’s State And School

System Implementation System Implementation Strategies, Successes, and Strategies, Successes, and

OutcomesOutcomes

www.pbismaryland.orgwww.pbismaryland.org

3rd International Conference on PBS

March 23, 2006

22

Presentation TeamPresentation Team

• Milt McKennaMilt McKenna

• Susan BarrettSusan Barrett

• Jerry BloomJerry Bloom

• Catherine BradshawCatherine Bradshaw

• Tim McCormickTim McCormick

33

ACKNOWLWEDGEMENTSACKNOWLWEDGEMENTS

• Maryland State Department of Maryland State Department of EducationEducation

• Sheppard Pratt Health SystemSheppard Pratt Health System

• Johns Hopkins UniversityJohns Hopkins University

• 24 Local School Systems24 Local School Systems

• University of OregonUniversity of Oregon

• University of ConnecticutUniversity of Connecticut

44

Advanced OrganizerAdvanced Organizer

• Evolution of PBIS in MDEvolution of PBIS in MD

• Organization and Implementation Organization and Implementation ActivitiesActivities

• Evaluation StructureEvaluation Structure

• Lessons Learned Years 1-5Lessons Learned Years 1-5

• Years 5-10Years 5-10

55

Pennsylvania

Delaw

are

D.C.

Virginia

West Virginia

66

Big picture …..How do you change individual behavior in schools?

How do you create school environments that sustain change?

What system-level mechanisms and infrastructure are necessary to sustain and bring to scale interventions that create desired change?

77

Coordination/ Coordination/ CollaborationCollaboration

1999 - 2006

PBISMSDE Sheppard Pratt Johns Hopkins University Local School Systems

88

Participating PBIS Participating PBIS SchoolsSchools

Total Public Schools: 1380Elementary: 829Middle: 233Senior: 236Combined: 40Other 42Nonpublic 1157

PBIS Schools trained Total: 368

ES: 189K-8: 8MS: 101HS: 35Special: 10Alternative:24

Participating: 301

99

0

263

368

3414

11864

178

30

81

183

133

6142

0

50

100

150

200

250

300

350

400

FY 99 FY 00 FY 01 FY 02 FY 03 FY 04 FY 05

Schools Trained Coaches Trained

Number of MD Schools Teams and Behavior Support Coaches Trained By

Year

1010

Schools Trained and ActiveSchools Trained and Active

14

3

20 12

30

17

54 51

60

49

85

67

94 91

11 11

0

10

20

30

40

50

60

70

80

90

100

1999 2000 2001 2002 2003 2004 2005 2006

Trained Active

1111

MD Implementation MD Implementation ModelModel

•State Leadership TeamState Leadership Team

•Local School System (LSS) Local School System (LSS) ContactContact

•Coach Facilitators/ Coach Facilitators/ CoordinatorsCoordinators

•Behavior Support CoachesBehavior Support Coaches

•School TeamsSchool Teams

1212

State Leadership TeamState Leadership Team::DiversifiedDiversified

• MSDE Division of Special Education/ Early MSDE Division of Special Education/ Early Intervention ServicesIntervention Services

• MSDE Division of Student and School MSDE Division of Student and School ServicesServices

• Sheppard Pratt Health SystemSheppard Pratt Health System

• Johns Hopkins UniversityJohns Hopkins University

• LSS Behavior Support CoachesLSS Behavior Support Coaches

• Mental Health PartnersMental Health Partners

1313

State Leadership Team State Leadership Team FunctionsFunctions

• Set policySet policy• Provide support for local leadershipProvide support for local leadership• Influence System of Change at District Influence System of Change at District

LevelLevel• Assess Training NeedsAssess Training Needs• Event CoordinationEvent Coordination• Provide Training and Technical AssistanceProvide Training and Technical Assistance• Monitor OutcomesMonitor Outcomes

• Features of implementationFeatures of implementation• ReferralsReferrals• Other indicatorsOther indicators

1414

Local School System Local School System (LSS)(LSS)

•24 Local School Systems24 Local School Systems

•Designated Point of ContactDesignated Point of Contact– Director of Student ServicesDirector of Student Services– Director of Special EducationDirector of Special Education

1515

Behavior Support CoachesBehavior Support Coaches• 183 Behavior Support Coaches183 Behavior Support Coaches

– Itinerant positions/funded by LSS Itinerant positions/funded by LSS

• Meet 5 times/yearMeet 5 times/year– NetworkingNetworking– Regional MeetingsRegional Meetings– WorkgroupsWorkgroups

• Coaches ReceptionCoaches Reception• Coaches Newsletter/Coaches Coaches Newsletter/Coaches

CalendarCalendar• Coach = RecruiterCoach = Recruiter

5 LSS Coach Facilitators5 LSS Coach Facilitators

1616

Behavior Support Coach Behavior Support Coach ActivitiesActivities

• FTE allocated to complete tasksFTE allocated to complete tasks• Consistently attend team meetingsConsistently attend team meetings• Assist team with data-based decision-Assist team with data-based decision-

making, planning, and implementationmaking, planning, and implementation• Attend Regional/State Coaches Attend Regional/State Coaches

meetings/trainingsmeetings/trainings• Send information to PBIS State/District Send information to PBIS State/District

Coordinator (e.g., checklists, action plans, Coordinator (e.g., checklists, action plans, etc.)etc.)

• Assist with dissemination activities (e.g., Assist with dissemination activities (e.g., presentations, case studies, articles, etc.)presentations, case studies, articles, etc.)

1717

School School Commitments/AssurancesCommitments/Assurances

• One of top SIT GoalsOne of top SIT Goals

• Active Role By Active Role By AdministrationAdministration

• 3 year timeline3 year timeline

• Use DATA for decision Use DATA for decision making - making - – And to INFORMAnd to INFORM

1818

School Team ActivitiesSchool Team Activities

• Administrator is active and present for meetings.Administrator is active and present for meetings.• Team is making progress on PBIS “Getting Started” Team is making progress on PBIS “Getting Started”

checklist (Form A).checklist (Form A).• Team uses school discipline & related DATA to Team uses school discipline & related DATA to

discuss monthly progress.discuss monthly progress.• Team uses annual action plan to discuss monthly Team uses annual action plan to discuss monthly

progressprogress• Team provides monthly updates/data summaries Team provides monthly updates/data summaries

to entire school staffto entire school staff• Team meetings are efficiently run (e.g., clear Team meetings are efficiently run (e.g., clear

objectives, tasks, goals)objectives, tasks, goals)• Team activities are coordinated with other school Team activities are coordinated with other school

initiatives/committeesinitiatives/committees

1919

Maryland Maryland Annual EventsAnnual Events

• Spring Forum (April 6, 2006)Spring Forum (April 6, 2006)• July Institute (July 10, July 17)July Institute (July 10, July 17)• Coaches Meetings (5/year)Coaches Meetings (5/year)• Regional Team Leader/Coach Regional Team Leader/Coach

Meetings (2/year)Meetings (2/year)• Schools serving students with Schools serving students with

special needs - MANSEF (2/year)special needs - MANSEF (2/year)• High Schools – (2/year)High Schools – (2/year)

2020

Start Up ActivitiesStart Up Activities

•Commitment and Commitment and Application Process Application Process

•Planning PhasesPlanning Phases

•Self AssessmentSelf Assessment

•ProfileProfile

2121

Spring ForumSpring Forum• Purpose: Recruitment Purpose: Recruitment • Date: April Date: April (set one year in advance)(set one year in advance)

• Overview & School PresentationsOverview & School Presentations

• Participants: Participants: – Administrators from “potential new” Administrators from “potential new”

schools schools – ““Key” system personnelKey” system personnel– Potential coachesPotential coaches

Planning Phase I Begins

2222

Summer TrainingSummer Training

•New CoachesNew Coaches•New TeamsNew Teams•Returning TeamsReturning Teams•Exemplar SchoolsExemplar Schools•Receptions and Poster Receptions and Poster

SessionsSessions

2323

Other Training EventsOther Training Events• Local CoordinatorsLocal Coordinators

• State Coaches MeetingsState Coaches Meetings

• Regional MeetingsRegional Meetings

• Targeted GroupTargeted Group

• Behavioral BasicsBehavioral Basics

• School Wide Information System (SWIS)School Wide Information System (SWIS)

• SWIS facilitatorSWIS facilitator

• School-wide Evaluation Tool (SET)School-wide Evaluation Tool (SET)

2424

Evaluation CapacityEvaluation Capacity Establishing Measurable OutcomesEstablishing Measurable Outcomes

1.1. What schools have been trained What schools have been trained and are active?and are active?

2.2. How well are schools implementing How well are schools implementing PBIS?PBIS?

3.3. What impact does PBIS have on What impact does PBIS have on student behavior? Achievement?student behavior? Achievement?

2525

MonitoringMonitoring OutcomesOutcomes

• Team Implementation ChecklistTeam Implementation Checklist

• SWISSWIS

• SETSET

• Coaches ChecklistCoaches Checklist

• Staff SurveyStaff Survey

• Satisfaction SurveysSatisfaction Surveys

• Implementation Phases Inventory (IPI)Implementation Phases Inventory (IPI)

2626

Evaluation ToolsEvaluation Tools

Access 2003 DatabaseAccess 2003 Database

• Data entry/storageData entry/storage

• Report GenerationReport Generation

2727

Evaluation ToolsEvaluation Tools

Maryland website Maryland website www.pbismaryland.orgwww.pbismaryland.orgVarious levels:Various levels: Any userAny user Team/coachTeam/coach

LSS Point of ContactLSS Point of ContactState TeamState Team

Maryland Forms MatrixMaryland Forms MatrixAccess DatabaseAccess DatabaseSWIS SWIS PBS surveys (PBS surveys (www.pbssurveys.orgwww.pbssurveys.org))

2828

2929

How Well are Schools How Well are Schools Implementing?Implementing?

• Systems-wide Evaluation Tool (SET)Systems-wide Evaluation Tool (SET)– AnnuallyAnnually– 7 Features of SW Implementation7 Features of SW Implementation

• Implementation Phases Inventory (IPI)Implementation Phases Inventory (IPI)– Semi-annually Semi-annually – Levels of SW: Preparation, Initiation, Levels of SW: Preparation, Initiation,

Implementation, and MaintenanceImplementation, and Maintenance

3030

School-Wide Evaluation ToolSchool-Wide Evaluation Tool35 Coaches trained as SET assessors35 Coaches trained as SET assessors

15 Contractual SET assessors15 Contractual SET assessors

• 97 SETs completed 200497 SETs completed 2004• 154 SETs completed 2005154 SETs completed 2005• 104 schools have at least two SET 104 schools have at least two SET

scoresscores• 80% Total score is considered 80% Total score is considered

Maintenance Phase (IPI)Maintenance Phase (IPI)• All regions met 80% criterion across All regions met 80% criterion across

schoolsschools• 69% increase after one year of 69% increase after one year of

implementationimplementation

3131

SET Scores by RegionSET Scores by Region

43

2529

42

8691

8493 90

0

10

20

30

40

50

60

70

80

90

100

Ce

ntr

al

Ea

ste

rn

So

uth

ern

Sp

eci

al

We

ste

rn

Pre Post

3232

% of schools at phase11.1%

16.0%

19.1%

53.8%

P reparation Initiation Implementation Maintenance

IPI

* IPI: Implementation Phases Inventory

3333

PROJECT TARGETEvaluating PBIS in Maryland

Dr. Catherine Bradshaw Center for the Prevention of Youth Violence

Johns Hopkins University

Project Target

3434

Randomized Trial of PBIS: Randomized Trial of PBIS: Project TargetProject TargetSampleSample• 37 Elementary Schools 37 Elementary Schools

– AlleganyAllegany(7)(7), Anne Arundel, Anne Arundel(4)(4), Baltimore , Baltimore (13)(13), , Charles(11), & Washington(2)Charles(11), & Washington(2)

– 3,057 Staff & 20,246 Students3,057 Staff & 20,246 Students

DesignDesign• Randomly AssignedRandomly Assigned

– 21 PBIS21 PBIS– 16 Comparison16 Comparison

• Baseline plus 4 yearsBaseline plus 4 years

Funding : NIMH & CDC to the Johns Hopkins Center for the Prevention of Youth Violence

Project Target

3535

Data CollectedData Collected• Disruptive behavior Disruptive behavior

– Teacher Observation of Classroom Adaptation Teacher Observation of Classroom Adaptation (TOCA; Werthamer-Larsson et al., 1991) (TOCA; Werthamer-Larsson et al., 1991) – Student Interactions in Specific SettingsStudent Interactions in Specific Settings (SISS; Cushing & Horner, 2002)(SISS; Cushing & Horner, 2002)– Office discipline referrals Office discipline referrals (SWIS; School-Wide Information System)(SWIS; School-Wide Information System)

• Academic informationAcademic information – AttendanceAttendance– Academic performanceAcademic performance

• School climateSchool climate – School Climate Survey School Climate Survey (Haynes, Emmons, & Comer, 1994)(Haynes, Emmons, & Comer, 1994)

• Organizational healthOrganizational health – Organizational Health Inventory Organizational Health Inventory (Hoy et al., 1990)(Hoy et al., 1990)

• Implementation fidelityImplementation fidelity– System-wide Evaluation Tool System-wide Evaluation Tool (SET; Sugai, Lewis-Palmer, Todd, & Horner, 2001)(SET; Sugai, Lewis-Palmer, Todd, & Horner, 2001)– Staff Survey Staff Survey (Sugai, Todd, & Horner, 2000)(Sugai, Todd, & Horner, 2000)

Project Target

3636

Implementation of PBISImplementation of PBIS

3743

84

56

90

62

0

10

20

30

40

50

60

70

80

90

100

Baseline Year 1 Year 2

PBISComparison

Notes. No significant differences between groups at baseline, but differences at all other years, p<.05.

**

3737

Impact of PBIS on Organizational HealthImpact of PBIS on Organizational Health

3.12

2.94

3.013.012

2.9

2.95

3

3.05

3.1

3.15

Baseline Year 3

PBISComparison

Mea

n O

HI

Note. Adjusted means from 3-level model in Mplus. Intervention effect on slope of overall OHI significant at p<.05.

3838

Summary of Preliminary Summary of Preliminary Findings from Project TargetFindings from Project Target

• PBIS schools have high program fidelityPBIS schools have high program fidelity

• PBIS increased organizational health PBIS increased organizational health

• Some positive effects on student outcomesSome positive effects on student outcomes– Behavior problems in classroom Behavior problems in classroom – Reduced office discipline referralsReduced office discipline referrals– Reduced suspensionsReduced suspensions

Project Target

3939

Lessons LearnedLessons Learned• Establish 3-5 Year CommitmentEstablish 3-5 Year Commitment• Designate State Leader(s)Designate State Leader(s)• Establish Diversified State Establish Diversified State

Leadership TeamLeadership Team• Establish Coaching ModelEstablish Coaching Model• Establish Accountability ModelEstablish Accountability Model

– Monthly ReportsMonthly Reports– Site VisitsSite Visits– Data CollectionData Collection– ExemplarsExemplars

4040

Anticipated GrowthAnticipated Growth50% of MD Schools will be trained by 201050% of MD Schools will be trained by 2010

Chart Title

364434

504574

644714

0

200

400

600

800

FY 05 FY 06 FY 07 FY 08 FY 09 FY 10

Anticipated Growth at 5% Linear (Anticipated Growth at 5%)

4141

MARYLANDMARYLANDYears 5-10Years 5-10

•Expanded State Leadership Expanded State Leadership TeamTeam

•State-Level Advisory State-Level Advisory CommitteeCommittee

•Management TeamManagement Team

•Mental Health Integration Mental Health Integration GrantGrant

4242

5 YEAR GOALS5 YEAR GOALS• Goal 1:Goal 1: Increase state coordination by Increase state coordination by

identifying regional coordinators for Southern, identifying regional coordinators for Southern, Central, Eastern and Western Maryland.Central, Eastern and Western Maryland.

• Goal 2:Goal 2: Increase evaluation capacity by Increase evaluation capacity by expanding web site functionality i.e. online expanding web site functionality i.e. online tutorials, data submission and retrieval.tutorials, data submission and retrieval.

• Goal 3:Goal 3: Increase local training capacity by Increase local training capacity by identifying training coordinator(s).identifying training coordinator(s).

• Goal 4:Goal 4: Increase visibility by identifying an Increase visibility by identifying an individual to write grants, develop marketing individual to write grants, develop marketing material and interface with outside agencies.material and interface with outside agencies.

4343

Useful Web Useful Web LinksLinks

• MarylandMaryland– www.pbismaryland.orgwww.pbismaryland.org

• OregonOregon– www.pbis.orgwww.pbis.org

4444

CONTACT INFORMATIONCONTACT INFORMATIONMilt McKenna – Maryland State Department of Education, mmckenna@msde.state.md.us, 410-767-0304

Susan Barrett - Sheppard Pratt Health System, sbarrett@pbismaryland.org, 410-938-3650Jerry Bloom – Sheppard Pratt Health System,jbloom@pbismaryland.org, 410-938-4661

Catherine Bradshaw – Center for the Prevention of Youth Violence, Johns Hopkins University, cbradsha@jhpsh.edu 410 502-2587

Tim McCormick – Maryland State Department of Education, tmccormick@msde.state.md.us, 410-767-0736