Circe Stumbo, West Wind Education Policy Inc. and CCSSO CCSSO/SCEE National Summit on Educator...

transcript

Circe Stumbo, West Wind Education Policy Inc. and CCSSO

CCSSO/SCEE National Summit on

Educator Effectiveness

April 10, 2013

Implementation Science:

Closing Gaps Between Policy and Practice

Acknowledgements

Special thanks to Dean Fixsen, Karen Blase, and the National Implementation Research Network for their generosity in sharing their findings and their passion for all things implementation. This PowerPoint slide deck is built off their original work, with permission.

Sixth Year of Working with SEAs

State Implementation & Scaling-up of Evidence-based Practices Center

scalingup.org

Implementation Science`

A policy is one thing

Implementation of a policy is a very different thing

Students cannot benefit from a policy they do not experience

An Implementation Failure labeled as an Intervention Failure

Example: $500 million for Homebuilders (Family Support Services) 1993-1998

Funding only for interventions

No fidelity criteria insisted upon by the developers

National evaluation = not effective

Chapin Hall Center for Children, 2002

An Implementation Failure labeled as an Intervention Failure

Example: $500 million for Homebuilders (Family Support Services) 1993-1998

Funding only for interventions

No fidelity criteria insisted upon by the developers

National evaluation = not effective But, over 25% was spent on in-office

interventions with parents or children(< 0 fidelity)

Chapin Hall Center for Children, 2002

Longitudinal Studies of CSR Programs

Source: Aladjem & Borman, 2006; Vernez, Karam, Mariano, & DeMartini, 2006

Evidence-Base For Effectiveness

Every Teacher Trained

Every Teacher Continually Supported

Actual SupportsYears 1-3

Fewer than 50% of the teachers received some training

Fewer than 25% of those teachers received support

Fewer than 10% of the schools used the CSR as intended

Vast majority of students did not benefit

OutcomesYears 4-5

Formula for Success

Socially Significant Outcomes

Effective Policies/

Interventions

Effective Implementation

Methods

Enabling Contexts

Formula for Success

WHY: Socially

Significant Outcomes

Effective Policies/

Interventions

Methods

Enabling Contexts

(Vision!)

Formula for Success

WHY: Socially

WHAT: Effective Policies/

Interventions

Methods

Enabling Contexts

Formula for Success

WHY: Socially

Interventions

Methods

WHERE: Enabling Contexts

Formula for Success

WHY: Socially

Interventions

HOW & WHO: Effective

Implementation Methods

WHERE: Enabling Contexts

Table Discussions

Share a situation you have experienced/ observed where one of the variables in the formula was zero

Implementation Science

Excellent evidence for what does not work

Implementation without changing supporting roles and functions

Implementation by edict

Diffusion/dissemination of information by itself

Implementation by “following the money”

Training alone, no matter how well done

Paul Nutt (2002). Why Decisions Fail

What does work? An Implementation Framework

Implementation Teams

Implementation Drivers

Improvement Cycles

Implementation Stages

http://sisep.fpg.unc.edu/learning-zone/science-of-implementation/implementation-frameworks

An Implementation Framework

Improvement Cycles

IMPLEMENTATION

Implementation Science

Effective NOT Effective

Effective

NOTEffectiveIN

N Student Benefits

(Institute of Medicine, 2000; 2001; New Freedom Commission on Mental Health, 2003; National Commission on Excellence in Education,1983; Department of Health and Human Services, 1999)

Poor outcomes

Inconsistent, not sustainable, poor outcomes

Poor outcomes; sometimes

harmful

IMPLEMENTATION

Effective

80%, 3 Yrs 14%, 17 Yrs

Impl. Team NO Impl. Team

NOTEffective

Balas & Boren, 2000 Green, 2008

Fixsen, Blase, Timbers, & Wolf, 2001

Implementation Team

A group that knows:

the intervention/policy

implementation

improvement cycles

Simultaneous, Multi-Level Interventions

School

Management (leadership, policy)

Administration (HR, structure)

Supervision (nature, content)

Teacher

Region

District

Partners

Iowa’s Collaborating for Kids (C4K)

Students & Families

Building Teachers and Staff

Building Leadership and Implementation

District Leadership and Implementation

Regional Implementation

State Transformation

StateDepartment of

Education Leadership

StateDepartment of

Cascading Logic Model

An Implementation Framework

Improvement Cycles

Practice-Policy Communication Loops

Plan-Do-Study-Act cycles

Usability Testing

Students & Families

StateDepartment of

Practice/Policy Communication

Students & Families

StateDepartment of

Students & Families

StateDepartment of

Students & Families

StateDepartment of

North Carolina Example

Students & Families

StateDepartment of

Authentic Work Informs the

PEP-PIP Cycle

Students & Families

StateDepartment of

Authentic Work Informs the

PEP-PIP Cycle

Table Discussions

How do teams at each level of your system interact?

What work are the teams doing with each other?

What feedback loops exist between teams?

How might you increase interaction among teams?

Have/might you incorporate improvement cycles into pilots and policy innovations?

Growing Implementation Capacity Using

Transformation Zones

Implementation Team

Minimum of 3 people (4-5 preferred)

Tolerate turnover; sustainable

State Transformation Team

First School Implementation Team

N = 10 Schools

Growing Implementation Capacity

(Phase 1)

First Regional Implementation Team

N = 9 Staff

Invest up front to build

capacity

First School Implementation Team

N = 10 Schools

(Phase 2)

N = 3 Staff

Subsequent Regional Implementation Team (3 Staff)

First School Implementation Team, Expanded

N = 20 Schools

(Phase 2)

N = 3 Staff

Subsequent School Implementation Teams

N=20 Schools

N = 20 Schools

(Phase 2)

N = 3 Staff

N=20 Schools

N = 20 Schools

(Phase 2)

N = 3 Staff

N=20 Schools

N = 20 Schools

(Phase 2)

N = 3 Staff

N=20 Schools

Table Discussions

What responsibility will your state take to ensure implementation of your policies?

How can you grow implementation capacity within your state?

How can you rethink staffing in order to invest in the early stages of implementation?

One key take-away

What questions do you have?

Possibilities to consider….

SCEE webinars

SCEE Topical Meetings

SCEE Discussion Groups

Individual state follow-up

Thank you

Circe Stumbo, West Wind Education Policy Inc. and CCSSO CCSSO/SCEE National Summit on Educator...

Documents