Date post: | 01-Jan-2016 |
Category: |
Documents |
Upload: | duane-reed |
View: | 212 times |
Download: | 0 times |
Dean L. Fixsen, Karen A. Blase,
Michelle Duda, Sandra Naoom,
Melissa Van Dyke, Frances WallaceNational Implementation Research Network
Louis de la Parte Florida Mental Health Institute
Effectively Using Innovations in Education
MiBLSi Meeting 2008
Copyright © Dean L. Fixsen and Karen A. Blase, 2008
Evidence-Based Movement
The “evidence-based movement” is an international experiment to make better use of research findings in typical service settings.
The purpose is to produce greater benefits to consumers and society.
Copyright © Dean L. Fixsen and Karen A. Blase, 2008
Education
65 million kids
6 million teachers and staff
100,000+ schools
3,143 counties
60 states & U.S. jurisdictions
Copyright © Dean L. Fixsen and Karen A. Blase, 2008
Science to Service
SCIENCE SERVICEGAPIMPLEMENTATION
Copyright © Dean L. Fixsen and Karen A. Blase, 2008
Science to ServiceScience to Service Gap
What is known is not what is adopted to help children, families, and caregivers
Implementation Gap What is adopted is not used with fidelity
and good outcomes for consumers.
What is used with fidelity is not sustained for a useful period of time.
What is sustained is not used on a scale sufficient to impact social problems.
Copyright © Dean L. Fixsen and Karen A. Blase, 2008
Implementation
Since the 1960s there have been significant advancements in the science and practice of implementation
These advancements have many implications for making changes and improvements in human services
Copyright © Dean L. Fixsen and Karen A. Blase, 2008
“Letting it happen”
Directives (e.g. laws, regulations), diffusion, and dissemination of information (e.g. publications, meetings, manuals)
Recipients are accountable for making use of the information to the extent they can
Based on Greenhalgh, Robert, MacFarlane, Bate, & Kyriakidou, 2004
Copyright © Dean L. Fixsen and Karen A. Blase, 2008
“Helping it happen”
Dissemination and active promotion via meetings, conferences, training to provide more detailed information, and some skill building and consultation on the process
Recipients are accountable for making use of the information and training to the extent they can
Based on Greenhalgh, Robert, MacFarlane, Bate, & Kyriakidou, 2004
Copyright © Dean L. Fixsen and Karen A. Blase, 2008
“Making it happen”
Implementation: Active involvement of implementation teams that work at the intersection of practices, programs, systems, communities, & scientists
Implementation teams are accountable for assuring use of innovations with fidelity and good outcomes
Based on Greenhalgh, Robert, MacFarlane, Bate, & Kyriakidou, 2004
Copyright © Dean L. Fixsen and Karen A. Blase, 2008
0
100
200
300
400
500
600
700
800
900
≤1972 1974 1976 1978 1980 1982
0
50
100
150
200
250
300
CumulativeCouples
CumulativeHomes
Fixsen, Blase, Timbers, & Wolf (2001)
Teaching–Family Model
Copyright © Dean L. Fixsen and Karen A. Blase, 2008
Follow Through Programs
Figure 1: This figure shows the average effects of nine Follow Through models on measures of basic skills (word knowledge, spelling, language, and math computation), cognitive-conceptual skills (reading comprehension, math concepts, and math problem solving) and self-concept. This figure is
adapted from Engelmann, S. and Carnine, D. (1982), Theory of Instruction: Principles and applications. New York: Irvington Press.
Copyright © Dean L. Fixsen and Karen A. Blase, 2008
School Wide PBS
SWPBS Schools
0
1000
2000
3000
4000
5000
6000
7000
Aug 04 Aug 05 Aug 06 Aug 07
Year
Tota
l Num
ber
Sch
ools
4.2% of all schools
Copyright © Dean L. Fixsen and Karen A. Blase, 2008
Implementation Reviews
Human service prevention and treatment programs (e.g. substance abuse, adult / children’s MH, justice, health, education)
Advanced manufacturing technologies
AMA clinical guidelines
Engineering: bridge maintenance
Hotel service management
National franchise operations
Cancer prevention & treatment
Copyright © Dean L. Fixsen and Karen A. Blase, 2008
Ineffective MethodsExcellent experimental evidence for
what does not work
Diffusion/dissemination of information by itself does not lead to successful implementation (research literature, mailings, promulgation of practice guidelines)
Training alone, no matter how well done, does not lead to successful implementation
Copyright © Dean L. Fixsen and Karen A. Blase, 2008
Ineffective MethodsExcellent evidence for what does
not work
Implementation by edict by itself does not work
Implementation by “following the money” by itself does not work
Implementation without changing supporting roles and functions does not work Paul Nutt (2002). Why Decisions Fail
Copyright © Dean L. Fixsen and Karen A. Blase, 2008
What Works
Effective intervention practices
+
Effective implementation practices
=
Good outcomes for consumers
Copyright © Dean L. Fixsen and Karen A. Blase, 2008
Implement Innovations
Effective NOT Effective
Effective
NOT Effective
IMPLEMENTATION
INT
ER
VE
NT
ION Student Benefits
Copyright © Dean L. Fixsen and Karen A. Blase, 2008
Implementation
An effective intervention is one thing
Effective implementation of an effective intervention is a very different thing
Copyright © Dean L. Fixsen and Karen A. Blase, 2008
EBPs & Implementation
From an implementation perspective, what do we need to know about innovations such as evidence-based programs?
Copyright © Dean L. Fixsen and Karen A. Blase, 2008
EBPs & Implementation
The usability of a program has little to do with the quality or weight of the evidence regarding that program
Evidence on intervention effectiveness for specific populations helps us choose what to implement
Evidence on the effectiveness of the intervention does not help implement the program or practice successfully
Copyright © Dean L. Fixsen and Karen A. Blase, 2008
EBPs & Implementation
Core intervention components
Clearly described (who/what)
Practical measure of fidelity (critical)
Fully operationalized (teachable)
Field tested (practicable; complete)
Contextualized (org./systems fit)
Effective (worth the effort)
Copyright © Dean L. Fixsen and Karen A. Blase, 2008
Implementation
What do we need to know about successful implementation methods?
Copyright © Dean L. Fixsen and Karen A. Blase, 2008
Stages of Implementation
Implementation is not an event
A mission-oriented process involving multiple decisions, actions, and corrections
“Pursue to a conclusion” dictionary.com
Copyright © Dean L. Fixsen and Karen A. Blase, 2008
Stages of Implementation
Exploration
Installation
Initial Implementation
Full Implementation
Innovation
Sustainability
Implementation occurs in stages:
Fixsen, Naoom, Blase, Friedman, & Wallace, 2005
2 – 4 Years
Copyright © Dean L. Fixsen and Karen A. Blase, 2008
Stages of Implementation
Exploration
Installation
Initial Implementation
Full Implementation
Innovation
Sustainability
Implementation occurs in stages:
Fixsen, Naoom, Blase, Friedman, & Wallace, 2005
2 – 4 Years
Intervention Outcomes
0% 100%
Implementation Outcomes
Copyright © Dean L. Fixsen and Karen A. Blase, 2008
INTEGRATED & COMPENSATORY
CONSULTATION & COACHING
CONSULTATION & COACHING
STAFF PERFORMANCE EVALUATION
STAFF PERFORMANCE EVALUATION
FACILITATIVE ADMINISTRATIVE
SUPPORTS
FACILITATIVE ADMINISTRATIVE
SUPPORTS
RECRUITMENT AND SELECTION
RECRUITMENT AND SELECTION
PRESERVICE TRAINING
PRESERVICE TRAINING
SYSTEMSINTERVENTIONS
SYSTEMSINTERVENTIONS
Implementation Drivers
DECISION SUPPORT DATA SYSTEMS
DECISION SUPPORT DATA SYSTEMS
Copyright © Dean L. Fixsen and Karen A. Blase, 2008
Teaching Interaction Components
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
N = 7 N = 6 N = 3 N = 7 N = 8 N = 9
A B C D E F
Workshops
Perc
en
t o
f C
om
po
nen
ts
Dem
on
str
ate
d
PRE
POST
OUTCOMES(% of Participants who Demonstrate Knowledge, Demonstrate
new Skills in a Training Setting, and Use new Skills in the Classroom)
TRAININGCOMPONENTS
KnowledgeSkill
DemonstrationUse in the Classroom
Theory and Discussion
10%
5% 0%
..+Demonstration in Training
30%20%
0%
…+ Practice & Feedback in Training
60% 60% 5%
…+ Coaching in Classroom
95% 95% 95%
Joyce and Showers, 2002
Copyright © Dean L. Fixsen and Karen A. Blase, 2008
Keys to CoachingCoach with expertise in the innovation
Coaching skills (how to: observe & describe behavior; give & take feedback; do behavior rehearsal; shape & reinforce competence)
Coaching content integrated with selection, training, and staff performance assessment content
Coaching delivery plan (coach: practitioner ratio; frequency and nature of contact with the practitioner)
Copyright © Dean L. Fixsen and Karen A. Blase, 2008
Keys to CoachingMultiple sources of information about each practitioner (weekly direct observation; data/ information reviews)
Coaching content related to craft knowledge & personal style (innovation – craft – person fit)
Staff development plan (frequently updated)
360 – degree feedback
Copyright © Dean L. Fixsen and Karen A. Blase, 2008
Implement Innovations
Effective NOT Effective
Effective
NOT Effective
IMPLEMENTATION
INT
ER
VE
NT
ION Perf. Assess.
Fidelity
Implementation without fidelity does not work
Copyright © Dean L. Fixsen and Karen A. Blase, 2008
Functional Fidelity Assessment
0%
10%
20%
30%
40%
50%
Homes
1.0
1.2
1.5
1.7
1.9
2.0
1.1
1.4
1.6
1.8
1.3
Delinquency
Teaching
rs = – .94
Pe
rce
nt
Pa
ren
tal-
Tea
ch
ing
Wit
h Y
ou
ths
Me
an
Se
lf Rep
orte
dD
elin
qu
en
cy
Ratin
gs
1 2 3 4 5 6
Bedlington, et al., 1988
Copyright © Dean L. Fixsen and Karen A. Blase, 2008
Functional Fidelity Assessment
0%
10%
20%
30%
40%
50%
Homes
5.0
5.4
6.0
6.4
6.8
7.0
5.2
5.8
6.2
6.6
5.6
Youth Ratings
Teaching
rs = .94
Pe
rce
nt
Pa
ren
tal-
Tea
ch
ing
Wit
h Y
ou
ths
Me
an
Yo
uth
Sa
tisfac
tion
Ra
ting
s
1 2 3 4 5 6 7 8 9 10
Bedlington, et al., 1988
Copyright © Dean L. Fixsen and Karen A. Blase, 2008
INTEGRATED & COMPENSATORY
CONSULTATION & COACHING
CONSULTATION & COACHING
STAFF PERFORMANCE EVALUATION
STAFF PERFORMANCE EVALUATION
FACILITATIVE ADMINISTRATIVE
SUPPORTS
FACILITATIVE ADMINISTRATIVE
SUPPORTS
RECRUITMENT AND SELECTION
RECRUITMENT AND SELECTION
PRESERVICE TRAINING
PRESERVICE TRAINING
SYSTEMSINTERVENTIONS
SYSTEMSINTERVENTIONS
Implementation Drivers
DECISION SUPPORT DATA SYSTEMS
DECISION SUPPORT DATA SYSTEMS
Copyright © Dean L. Fixsen and Karen A. Blase, 2008
Implementation
Who does the work?
Copyright © Dean L. Fixsen and Karen A. Blase, 2008
Implementation Team
A group that knows the innovation very well (formal and craft knowledge)
A group that knows how to implement that innovation with fidelity and good effect
A group that accumulates data & experiential knowledge -- more effective and efficient over time (information economics, K. Arrow)
Copyright © Dean L. Fixsen and Karen A. Blase, 2008
Implementation Team
School
Management (leadership, policy)
Administration (HR, structure)
Supervision (nature, content)
Teacher
State and Community Context
District
Imp
lem
enta
tio
n T
eam
Simultaneous, Multi-Level Interventions
Copyright © Dean L. Fixsen and Karen A. Blase, 2008
Implementation TeamChange the behavior of adult education professionals
“Systems don’t change, people do” (J.W.)
Change organizational structures, cultures, and climates
Change the thinking of system directors and policy makers
Successful and sustainable implementation of evidence-based programs always requires organization and systems change.
0.0
1.0
2.0
3.0
4.0
7/7
7
7/7
8
7/7
9
7/8
0
7/8
1
7/8
2
7/8
3
7/8
4
N = 41 Group Homes (152 Couples)
Tra
inin
g L
eve
l
0.0
1.0
2.0
3.0
4.0
5.0
6.0
7.0
Eff
ect
ive
ne
ss
Training Outcome
Implementation & Outcome
Fixsen, Phillips, et al. (1985)
r = .95
Copyright © Dean L. Fixsen and Karen A. Blase, 2008
Costs and Savings
Implementation Costs & Savings(Inflation Adjusted)
60
70
80
90
100
110
120
1 Yr Pre During PostYear 1
PostYear 2
PostYear 3
Ch
ang
e in
Bu
dg
et (
Per
cen
t)
Copyright © Dean L. Fixsen and Karen A. Blase, 2008
Budget Changes (4 years)
Pre Post
Administration 39% 19%
Treatment Units 59% 75%
Training & Eval. 2% 6%
Reduction in administrative staff, reduction in maintenance staff and repairs, reduction in food/staff costs, nearly 100% staff redeployment/rehiring into newly defined positions
Copyright © Dean L. Fixsen and Karen A. Blase, 2008
“Making it happen”
Implementation: Active involvement of implementation teams that work at the intersection of practices, programs, systems, communities, & scientists
Implementation teams are accountable for assuring use of innovations with fidelity and good outcomes
Based on Greenhalgh, Robert, MacFarlane, Bate, & Kyriakidou, 2004
Copyright © Dean L. Fixsen and Karen A. Blase, 2008
Implementation Team
Implementation Team
Prepare Communities
Prepare schools faculty, staff
Work with Researchers
Assure Implementation
Prepare Districts Assure Student Benefits
Copyright © Dean L. Fixsen and Karen A. Blase, 2008
ORG
S
T
CPE
DSDS
FA
SI
Developer - IT
Developer
IT
IT
IT
IT
IT - Developer
Most Developers
IT = Implementation Team
Copyright © Dean L. Fixsen and Karen A. Blase, 2008
ORG
S
T
C
PE
DSDS
FA
SI
DeveloperDeveloper
Developer
IT
IT
IT
IT
Some Developers
Copyright © Dean L. Fixsen and Karen A. Blase, 2008
ORG
S
T
C
PE
DSDS
FA
SI
IT
Sustainability Stage
Copyright © Dean L. Fixsen and Karen A. Blase, 2008
Systems Change
“Systems trump programs.”Patrick McCarthy, Annie E. Casey Foundation
Copyright © Dean L. Fixsen and Karen A. Blase, 2008
Systems Change
Teachers / staff impact children
It is the job of administrators, managers, and funders to align policies and structures to facilitate effective teacher / staff practices
There is no such thing as an “administrative decision” – they are all education decisions
Copyright © Dean L. Fixsen and Karen A. Blase, 2008
Systems Change
New practices do not fare well in old organizational structures and systems
Develop new position descriptions and job functions in state departments and districts focused on implementation (effective use) of policies and innovations
Copyright © Dean L. Fixsen and Karen A. Blase, 2008
Federal Spending on K-12 Education under the Elementary and Secondary Education Act and NAEP Reading Scores (Age 9)
Copyright © Dean L. Fixsen and Karen A. Blase, 2008
A Sobering Observation
"All organizations [and systems] are designed, intentionally or unwittingly, to achieve precisely the results they get."
R. Spencer DarlingBusiness Expert
Copyright © Dean L. Fixsen and Karen A. Blase, 2008
Systems Change
State Department
District
Schools
Teachers / Staff
Effective Practices
AL
IGN
ME
NT
Federal Departments
Imp
lem
enta
tio
n T
eam
s
Copyright © Dean L. Fixsen and Karen A. Blase, 2008
Systems Change
Policy (Plan)
Practice (Do)
Structure
Procedure
Policy
Practice
Form follows Function
Imp
lem
enta
tio
n
Tea
m
Stu
dy - A
ct
OUTCOMES
Copyright © Dean L. Fixsen and Karen A. Blase, 2008
Creating Capacity for Competent Change
State Implementation and Scaling up of Evidence-based Practices
Capacity = knowledge, skills, and experience; self-correcting, self-sustaining knowledge utilization
State Management
Group
State Transformation
Team
Regional Implementation
Team
N = 50 – 200 Schools
Regional Implementation
Team
N = 50 – 200 Schools
Regional Implementation
Team
N = 50 – 200 Schools
Regional Implementation
Team
N = 50 – 200 Schools
IMPLEMENTATION CAPACITY FOR
SCALING UP EBPs
Intensive Development
Saturation
State Capacity Development
0
2
4
6
8
10
12
14
16
1 2 3 4 5 6 7 8 9 10 11 12
Years
Reg
iona
l Im
pl. T
eam
s RITs
STTs
Intensive Development
Saturation44/51
Copyright © Dean L. Fixsen and Karen A. Blase, 2008
Outcomes for Children, Families,
& Communities
Functional Education
Bureaucracy
Teachers
Policies
Schools
Copyright © Dean L. Fixsen and Karen A. Blase, 2008
SISEP State Implementation and Scaling up
of Evidence-based Practices
www.scalingup.org“Just for States”
Selection criteria/ rationales
“Resources”
Concept paper
Annotated bibliography
Copyright © Dean L. Fixsen and Karen A. Blase, 2008
Thank YouWe thank the following for their support
Annie E. Casey Foundation (EBPs and cultural competence)
William T. Grant Foundation (implementation literature review)
Substance Abuse and Mental Health Services Administration (implementation strategies grants; NREPP reviews; SOC analyses of implementation; national implementation awards)
Centers for Disease Control & Prevention (implementation research contract)
National Institute of Mental Health (research and training grants)
Juvenile Justice and Delinquency Prevention (program development and evaluation grants
Office of Special Education Programs (Capacity Development Center contract)
Agency for Children and Families (Child Welfare Leadership Development contract)
Copyright © Dean L. Fixsen and Karen A. Blase, 2008
For More Information
Dean L. Fixsen
813-974-4446
Karen A. Blase
813-974-4463
National Implementation Research Network
At the Louis de la Parte Florida Mental Health Institute
University of South Florida
http://nirn.fmhi.usf.edu
Copyright © Dean L. Fixsen and Karen A. Blase, 2008
For More Information
Fixsen, D. L., Naoom, S. F., Blase, K. A., Friedman, R. M. & Wallace, F. (2005). Implementation Research: A Synthesis of the Literature. Tampa, FL: University of South Florida, Louis de la Parte Florida Mental Health Institute, The National Implementation Research Network (FMHI Publication #231).
Download all or part of the monograph at:http://nirn.fmhi.usf.edu/resources/publications/Monograph/index.cfm
Implementation Research: A Synthesis of the Literature