11

Maryland’s State Maryland’s State Implementation Strategies, Implementation Strategies, Successes, and OutcomesSuccesses, and Outcomes

www.pbismaryland.orgwww.pbismaryland.orgSusan BarrettSusan Barrett

sbarrett@pbismaryland.orgsbarrett@pbismaryland.org

22

ACKNOWLWEDGEMENTSACKNOWLWEDGEMENTS

• Maryland State Department of Maryland State Department of EducationEducation

• Sheppard Pratt Health SystemSheppard Pratt Health System

• Johns Hopkins UniversityJohns Hopkins University

• 24 Local School Systems24 Local School Systems

• University of OregonUniversity of Oregon

• University of ConnecticutUniversity of Connecticut

33

Pennsylvania

Delaw

are

D.C.

Virginia

West Virginia

44

15

276

369

494

0 3081

133183

283

3512568

190

6142

0

100

200

300

400

500

FY99

FY00

FY01

FY02

FY03

FY04

FY05

FY06

Schools Trained Coaches Trained

55

PBS Systems Implementation LogicPBS Systems Implementation Logic

Leadership Team

FundingVisibility Political

Support

Training Coaching Evaluation

Active Coordination

Local School Teams/Demonstrations

66

Political SupportPolitical SupportPBIS was part of the top ten MSDE state supplemental PBIS was part of the top ten MSDE state supplemental

budget request for FY’ 00-FY’ 07.budget request for FY’ 00-FY’ 07.

Section 7-304.1 Of the Annotated Code of MarylandSection 7-304.1 Of the Annotated Code of Maryland

• In this section, “Positive Behavioral Interventions and Support Program” means the research-based, systems approach method adopted by the State Board to build capacity among school staff to adopt and sustain the use of positive, effective practices to create learning environments where teachers can teach and students can learn.

• Each county board of education and the Board of School Commissioners of Baltimore City shall require an elementary school that has a suspension rate that exceeds 18 percent of the elementary school’s enrollment to implement:

– A Positive Behavioral Interventions and Support Program; or– An alternative behavioral modification program in

collaboration with the Department.• The State Board shall adopt regulations to implement the

provisions of this section.

77

Section 7-304.1 Of the Section 7-304.1 Of the Annotated Code of MarylandAnnotated Code of Maryland

(a) In this section, “Positive Behavioral Interventions and Support Program” means the research-based, systems approach method adopted by the State Board to build capacity among school staff to adopt and sustain the use of positive, effective practices to create learning environments where teachers can teach and students can learn.

(b) Each county board of education and the Board of School Commissioners of Baltimore City shall require an elementary school that has a suspension rate that exceeds 18 percent of the elementary school’s enrollment to implement:

(1) A Positive Behavioral Interventions and Support Program; or

(2) An alternative behavioral modification program in collaboration with the Department.

(c) The State Board shall adopt regulations to implement the provisions of this section.

88

PBS Systems Implementation LogicPBS Systems Implementation Logic

Leadership Team

FundingVisibility Political

Support

Training Coaching Evaluation

Active Coordination

Local School Teams/Demonstrations

99

State Leadership TeamState Leadership Team

Visibility:Visibility:

MSDE Marketing Department is developing a MSDE Marketing Department is developing a tiered marketing/advertising plan targeting key tiered marketing/advertising plan targeting key PBIS stakeholders to generate more awareness PBIS stakeholders to generate more awareness for the initiative.for the initiative.

Target Audiences: Parents and Community Target Audiences: Parents and Community Members/Leaders, District Leaders, Instructional Members/Leaders, District Leaders, Instructional Leaders, State Leaders, Teachers, StudentsLeaders, State Leaders, Teachers, Students

PoliticiansPoliticians

1010

PBS Systems Implementation LogicPBS Systems Implementation Logic

Leadership Team

FundingVisibility Political

Support

Training Coaching Evaluation

Active Coordination

Local School Teams/Demonstrations

1111

State Leadership TeamState Leadership Team

Funding:Funding:

As the PBIS initiative prepares for a As the PBIS initiative prepares for a large-scale expansion, we are in large-scale expansion, we are in progress on a framework to identify and progress on a framework to identify and secure a broad base of funding support secure a broad base of funding support to ensure the infrastructure required to to ensure the infrastructure required to maintain the consistency and quality of maintain the consistency and quality of technical assistance and evaluation technical assistance and evaluation support to districts.support to districts.

1212

FundingFunding

• PBIS has been one of the top ten line PBIS has been one of the top ten line items of the Maryland State items of the Maryland State Department of Education Department of Education supplemental budget request FY’ 00- supplemental budget request FY’ 00- FY’ 07.FY’ 07.

1313

Funding -RealityFunding -Reality

• Title IV-Safe and Drug Free SchoolsTitle IV-Safe and Drug Free Schools

• Special EducationSpecial Education

• HIV preventionHIV prevention

• Drop Out PreventionDrop Out Prevention

• Sheppard PrattSheppard Pratt

1414

PBS Systems Implementation LogicPBS Systems Implementation Logic

Leadership Team

FundingVisibility Political

Support

Training Coaching Evaluation

Active Coordination

Local School Teams/Demonstrations

1515

Schools Trained and ActiveSchools Trained and Active

14

3

20 1230

17

54 5160

49

85

67

94 91

126 126

0

20

40

60

80

100

120

140

1999 2000 2001 2002 2003 2004 2005 2006

Trained Active

1616

PBS Systems Implementation LogicPBS Systems Implementation Logic

Leadership Team

FundingVisibility Political

Support

Training Coaching Evaluation

Active Coordination

Local School Teams/Demonstrations

1717

State Leadership TeamState Leadership Team::DiversifiedDiversified

• MSDE Division of Special Education/ Early Intervention Services

• MSDE Division of Student and School Services

• Sheppard Pratt Health System

• Johns Hopkins University

• LSS Behavior Support Coaches

• Juvenile Justice

• Governor’s Office of Children

• University of Maryland

• Department of Health and Mental Hygiene

Management Team-Core Group

MSDE, JHU, Sheppard Pratt

1818

Mental Health Integration Mental Health Integration GrantGrant• MARYLAND STATE DEPARTMENT OF EDUCATION MARYLAND STATE DEPARTMENT OF EDUCATION • THE UNIVERSITY OF MARYLAND, BALTIMORE, ON THE UNIVERSITY OF MARYLAND, BALTIMORE, ON

BEHALF OF ITS CENTER FOR SCHOOL MENTAL BEHALF OF ITS CENTER FOR SCHOOL MENTAL HEALTH ANALYSIS AND ACTIONHEALTH ANALYSIS AND ACTION

• DEPARTMENT OF JUVENILE SERVICES DEPARTMENT OF JUVENILE SERVICES • GOVERNOR’S OFFICE FOR CHILDRENGOVERNOR’S OFFICE FOR CHILDREN• MENTAL HYGIENE ADMINISTRATIONMENTAL HYGIENE ADMINISTRATION• THE MENTAL HEALTH ASSOCIATION OF MARYLAND THE MENTAL HEALTH ASSOCIATION OF MARYLAND • THE MARYLAND COALITION OF FAMILIES FOR THE MARYLAND COALITION OF FAMILIES FOR

CHILDREN’S MENTAL HEALTHCHILDREN’S MENTAL HEALTH• MARYLAND ASSEMBLY ON SCHOOL-BASED MARYLAND ASSEMBLY ON SCHOOL-BASED

HEALTHCARE HEALTHCARE • SHEPPARD PRATT HEALTH SYSTEMSHEPPARD PRATT HEALTH SYSTEM

1919

State Leadership Team State Leadership Team FunctionsFunctions

• Set policySet policy• Provide support for local leadershipProvide support for local leadership• Influence System of Change at District Influence System of Change at District

LevelLevel• Assess Training NeedsAssess Training Needs• Event CoordinationEvent Coordination• Provide Training and Technical AssistanceProvide Training and Technical Assistance• Monitor OutcomesMonitor Outcomes

• Features of implementationFeatures of implementation• ReferralsReferrals• Other indicatorsOther indicators

2020

Building Local CapacityBuilding Local Capacity

Roles and ResponsibilitiesRoles and Responsibilities

2121

PBS Systems Implementation LogicPBS Systems Implementation Logic

Leadership Team

FundingVisibility Political

Support

Training Coaching Evaluation

Active Coordination

Local School Teams/Demonstrations

2222

Behavior Support CoachesBehavior Support Coaches• 283 Behavior Support Coaches283 Behavior Support Coaches

– Itinerant positions/funded by LSS Itinerant positions/funded by LSS

• Meet 5 times/yearMeet 5 times/year– NetworkingNetworking– Regional MeetingsRegional Meetings– WorkgroupsWorkgroups

• Coaches ReceptionCoaches Reception• Coaches Newsletter/Coaches Coaches Newsletter/Coaches

CalendarCalendar• Coach = RecruiterCoach = Recruiter

8 LSS Coordinators8 LSS Coordinators

2323

Roles of CoachRoles of Coach

• Sustainability & AccountabilitySustainability & Accountability

• Hands-on technical assistanceHands-on technical assistance• Guide problem solvingGuide problem solving• Local trainingLocal training• Team start-up & sustainabilityTeam start-up & sustainability• Public relations/communicationsPublic relations/communications

• Support local leadershipSupport local leadership• Local coordination of resourcesLocal coordination of resources• Provide prompts & reinforcersProvide prompts & reinforcers

2424

Behavior Support Coach Behavior Support Coach ActivitiesActivities

• FTE allocated to complete tasksFTE allocated to complete tasks• Consistently attend team meetingsConsistently attend team meetings• Assist team with data-based decision-Assist team with data-based decision-

making, planning, and implementationmaking, planning, and implementation• Attend Regional/State Coaches Attend Regional/State Coaches

meetings/trainingsmeetings/trainings• Send information to PBIS State/District Send information to PBIS State/District

Coordinator (e.g., checklists, action plans, Coordinator (e.g., checklists, action plans, etc.)etc.)

• Assist with dissemination activities (e.g., Assist with dissemination activities (e.g., presentations, case studies, articles, etc.)presentations, case studies, articles, etc.)

2525

PBS Systems Implementation LogicPBS Systems Implementation Logic

Leadership Team

FundingVisibility Political

Support

Training Coaching Evaluation

Active Coordination

Local School Teams/Demonstrations

2626

Maryland Maryland Annual EventsAnnual Events

• Spring Forum (April 6, 2006)Spring Forum (April 6, 2006)• July Institute (July 10, July 17)July Institute (July 10, July 17)• Coaches Meetings (5/year)Coaches Meetings (5/year)• Regional Team Leader/Coach Regional Team Leader/Coach

Meetings (2/year)Meetings (2/year)• Schools serving students with Schools serving students with

special needs - MANSEF (2/year)special needs - MANSEF (2/year)• High Schools – (2/year)High Schools – (2/year)

2727

Spring ForumSpring Forum• Purpose: Recruitment Purpose: Recruitment • Date: April Date: April (set one year in advance)(set one year in advance)

• Overview & School PresentationsOverview & School Presentations

• Participants: Participants: – Administrators from “potential new” Administrators from “potential new”

schools schools – ““Key” system personnelKey” system personnel– Potential coachesPotential coaches

Planning Phase I Begins

2828

Summer TrainingSummer Training

•New CoachesNew Coaches•New TeamsNew Teams•Returning TeamsReturning Teams•Exemplar SchoolsExemplar Schools•Receptions and Poster Receptions and Poster

SessionsSessions

2929

Other Training EventsOther Training Events• Local Coordinators/TrainersLocal Coordinators/Trainers

• State Coaches MeetingsState Coaches Meetings

• Regional MeetingsRegional Meetings

• Targeted GroupTargeted Group

• Behavioral BasicsBehavioral Basics

• School Wide Information System (SWIS)School Wide Information System (SWIS)

• SWIS facilitatorSWIS facilitator

• School-wide Evaluation Tool (SET)School-wide Evaluation Tool (SET)

3030

PBS Systems Implementation LogicPBS Systems Implementation Logic

Leadership Team

FundingVisibility Political

Support

Training Coaching Evaluation

Active Coordination

Local School Teams/Demonstrations

3131

Evaluation CapacityEvaluation Capacity Establishing Measurable OutcomesEstablishing Measurable Outcomes

1.1. What schools have been trained What schools have been trained and are active?and are active?

2.2. How well are schools implementing How well are schools implementing PBIS?PBIS?

3.3. What impact does PBIS have on What impact does PBIS have on student behavior? Achievement?student behavior? Achievement?

3232

MonitoringMonitoring OutcomesOutcomes

• Team Implementation ChecklistTeam Implementation Checklist

• SWISSWIS

• SETSET

• Coaches ChecklistCoaches Checklist

• Staff SurveyStaff Survey

• Satisfaction SurveysSatisfaction Surveys

• Implementation Phases Inventory (IPI)Implementation Phases Inventory (IPI)

3333

Evaluation ToolsEvaluation Tools

Access 2003 DatabaseAccess 2003 Database

• Data entry/storageData entry/storage

• Report GenerationReport Generation

3434

Evaluation ToolsEvaluation Tools

Maryland website www.pbismaryland.orgMaryland website www.pbismaryland.orgVarious levels:Various levels: Any userAny user Team/coachTeam/coach

LSS Point of ContactLSS Point of ContactState TeamState Team

Maryland Forms MatrixMaryland Forms MatrixAccess DatabaseAccess DatabaseSWIS SWIS PBS surveys (PBS surveys (www.pbssurveys.orgwww.pbssurveys.org))

3535

3636

How Well are Schools How Well are Schools Implementing?Implementing?

• Systems-wide Evaluation Tool (SET)Systems-wide Evaluation Tool (SET)– AnnuallyAnnually– 7 Features of SW Implementation7 Features of SW Implementation

• Implementation Phases Inventory (IPI)Implementation Phases Inventory (IPI)– Semi-annually Semi-annually – Levels of SW: Preparation, Initiation, Levels of SW: Preparation, Initiation,

Implementation, and MaintenanceImplementation, and Maintenance

3737

School-Wide Evaluation ToolSchool-Wide Evaluation Tool35 Coaches trained as SET assessors35 Coaches trained as SET assessors

15 Contractual SET assessors15 Contractual SET assessors

• 97 SETs completed 200497 SETs completed 2004• 154 SETs completed 2005154 SETs completed 2005• 104 schools have at least two SET 104 schools have at least two SET

scoresscores• 80% Total score is considered 80% Total score is considered

Maintenance Phase (IPI)Maintenance Phase (IPI)• All regions met 80% criterion across All regions met 80% criterion across

schoolsschools• 69% increase after one year of 69% increase after one year of

implementationimplementation

3838

SET Scores by RegionSET Scores by Region

43

2529

42

8691

8493 90

0

10

20

30

40

50

60

70

80

90

100

Ce

ntr

al

Ea

ste

rn

So

uth

ern

Sp

eci

al

We

ste

rn

Pre Post

3939

% of schools at phase11.1%

16.0%

19.1%

53.8%

P reparation Initiation Implementation Maintenance

IPI

* IPI: Implementation Phases Inventory

4040

PROJECT TARGETEvaluating PBIS in Maryland

Dr. Catherine Bradshaw Center for the Prevention of Youth Violence

Johns Hopkins University

Project Target

4141

Randomized Trial of PBIS: Randomized Trial of PBIS: Project TargetProject TargetSampleSample• 37 Elementary Schools 37 Elementary Schools

– AlleganyAllegany(7)(7), Anne Arundel, Anne Arundel(4)(4), Baltimore , Baltimore (13)(13), , Charles(11), & Washington(2)Charles(11), & Washington(2)

– 3,057 Staff & 20,246 Students3,057 Staff & 20,246 Students

DesignDesign• Randomly AssignedRandomly Assigned

– 21 PBIS21 PBIS– 16 Comparison16 Comparison

• Baseline plus 4 yearsBaseline plus 4 years

Funding : NIMH & CDC to the Johns Hopkins Center for the Prevention of Youth Violence

Project Target

4242

Data CollectedData Collected• Disruptive behavior Disruptive behavior

– Teacher Observation of Classroom Adaptation Teacher Observation of Classroom Adaptation (TOCA; Werthamer-Larsson et al., 1991) (TOCA; Werthamer-Larsson et al., 1991) – Student Interactions in Specific SettingsStudent Interactions in Specific Settings (SISS; Cushing & Horner, 2002)(SISS; Cushing & Horner, 2002)– Office discipline referrals Office discipline referrals (SWIS; School-Wide Information System)(SWIS; School-Wide Information System)

• Academic informationAcademic information – AttendanceAttendance– Academic performanceAcademic performance

• School climateSchool climate – School Climate Survey School Climate Survey (Haynes, Emmons, & Comer, 1994)(Haynes, Emmons, & Comer, 1994)

• Organizational healthOrganizational health – Organizational Health Inventory Organizational Health Inventory (Hoy et al., 1990)(Hoy et al., 1990)

• Implementation fidelityImplementation fidelity– System-wide Evaluation Tool System-wide Evaluation Tool (SET; Sugai, Lewis-Palmer, Todd, & Horner, 2001)(SET; Sugai, Lewis-Palmer, Todd, & Horner, 2001)– Staff Survey Staff Survey (Sugai, Todd, & Horner, 2000)(Sugai, Todd, & Horner, 2000)

Project Target

4343

Implementation of PBISImplementation of PBIS

3743

84

56

90

62

0

10

20

30

40

50

60

70

80

90

100

Baseline Year 1 Year 2

PBISComparison

Notes. No significant differences between groups at baseline, but differences at all other years, p<.05.

**

4444

Impact of PBIS on Organizational HealthImpact of PBIS on Organizational Health

3.12

2.94

3.013.012

2.9

2.95

3

3.05

3.1

3.15

Baseline Year 3

PBISComparison

Mea

n O

HI

Note. Adjusted means from 3-level model in Mplus. Intervention effect on slope of overall OHI significant at p<.05.

4545

Summary of Preliminary Summary of Preliminary Findings from Project TargetFindings from Project Target

• PBIS schools have high program fidelityPBIS schools have high program fidelity

• PBIS increased organizational health PBIS increased organizational health

• Some positive effects on student outcomesSome positive effects on student outcomes– Behavior problems in classroom Behavior problems in classroom – Reduced office discipline referralsReduced office discipline referrals– Reduced suspensionsReduced suspensions

Project Target

4646

PBS Systems Implementation LogicPBS Systems Implementation Logic

Leadership Team

FundingVisibility Political

Support

Training Coaching Evaluation

Action Plan

Local School Teams/Demonstrations

4747

Anticipated GrowthAnticipated Growth50% of MD Schools will be trained by 201050% of MD Schools will be trained by 2010

Chart Title

364434

504574

644714

0

200

400

600

800

FY 05 FY 06 FY 07 FY 08 FY 09 FY 10

Anticipated Growth at 5% Linear (Anticipated Growth at 5%)

4848

5 YEAR GOALS5 YEAR GOALS• Goal 1:Goal 1: Increase state coordination by Increase state coordination by

identifying regional coordinators for Southern, identifying regional coordinators for Southern, Central, Eastern and Western Maryland.Central, Eastern and Western Maryland.

• Goal 2:Goal 2: Increase evaluation capacity by Increase evaluation capacity by expanding web site functionality i.e. online expanding web site functionality i.e. online tutorials, data submission and retrieval.tutorials, data submission and retrieval.

• Goal 3:Goal 3: Increase local training capacity by Increase local training capacity by identifying training coordinator(s).identifying training coordinator(s).

• Goal 4:Goal 4: Increase visibility by identifying an Increase visibility by identifying an individual to write grants, develop marketing individual to write grants, develop marketing material and interface with outside agencies.material and interface with outside agencies.

4949

Lessons LearnedLessons Learned

• Pacing Pacing • Be PatientBe Patient• ExemplarExemplar• FeedbackFeedback• Social Marketing-target Social Marketing-target

audiencesaudiences• Role of CoachRole of Coach• ProcessProcess