Socially Significant Solutions - Helping Families...

Dean Fixsen & Karen Blase, National Implementation Research Network

1

Karen A. Blase and Dean L. Fixsen

National Implementation Research Network

FPG Child Development Institute

University of North Carolina at Chapel Hill

Helping Families Change

Conference

Feb 14, 2013Los Angeles, CA

Implementation Science –Building the Bridge from Good Intentions to Great Outcomes

The “evidence-based movement” is an international experiment to make better use of research findings in typical service settings.

The purpose is to produce greater benefits to children, families, individuals, and societies.

Socially Significant Solutions


2


It is one thing to say with the prophet Amos, “Let justice roll down like mighty waters,” and quite another to work out the irrigation system.

~ William Sloane CoffinUS Social activist and clergyman


3

Science to Service Gap

What is known is not what is used to help children, families, individuals, and communities

Implementation GapWhat is adopted is not used with fidelity and

good outcomes for consumers.

What is used with fidelity is not sustained for a useful period of time.

What is used with fidelity is not used on a scale sufficient to impact social problems.

Science to Service GapImplementation Gap

Experimental Data Show These Methods, When Used Alone, Are Insufficient:

• Diffusion/ Dissemination of information

• Training

• Passing laws/ mandates/ regulations

• Providing funding/ incentives

• Organization change/ reorganization

Data: 5% to 15% Realize Intended Outcomes

Implementation Science

NECESSARY BUT NOT SUFFICIENT


4

Evidence-base Actual SupportsYears 1-3

OutcomesYears 4-5

Every Teacher Trained Fewer than 50% of the teachers received some training

Fewer than 10% of the schools used the CSR as intended

Every Teacher Continually Supported

Fewer than 25% of those teachers received support

Vast majority of students did not benefit

Aladjem & Borman, 2006; Vernez, Karam, Mariano, & DeMartini, 2006

Longitudinal Studies of a Variety of Comprehensive School Reforms


• $100 billion for innovative programs (USDE)

• $63 billion for maternal health programs (USAID)

• $4 billion for homevisiting programs (ACF)

• No funding set aside for implementation supports for these program initiatives



5

National Assessment of Education Progress (NAEP)

40 Years of Variation Around a Mediocre Mean

Invent a new future

• Stop wasting time and money on things that don’t work (and never have!)

– Sugai’s Law: For every new initiative, stop two current ones.

– De-scale; Avoid layering and fragmentation

• Set aside 15% of funds for implementation

• Require quarterly reports of fidelity data

System Reinvention


6

Tra dire e fare

C'e di mezzo il mare

~ Italian Proverb

Between saying and doing is the sea.

� Intervention Criteria

� Implementation Stages

� Implementation Drivers

� Improvement Cycles

� Implementation Teams

APPLIED Implementation Science


7

Making It HappenActive Implementation

• Letting it happen– Recipients are accountable

• Helping it happen– Recipients are accountable

• Making it happen– Purposeful and proactive use of

implementation practice and science

– Implementation Teams are accountable

Based on Hall & Hord (1987); Greenhalgh, Robert, MacFarlane, Bate, & Kyriakidou (2004); Fixsen, Blase, Duda, Naoom, & Van Dyke (2010)

Nĺ Fheicfidh tu plow an reimse ag casadh se os achionni d’intinn

~ Gaelic Proverb

You’ll never plough the field by turning it over in

your mind….


8

WHYSocially

Significant Outcomes

WHATEffective

Interventions

HOW/WHOEffective

Implementation Methods

WHEREEnabling Contexts

Formula for Success

WHYSocially


WHAT:Effective

Interventions

HOW/WHO:Effective


WHEREEnabling Contexts

Formula for Success


9







• About 18% of outcome studies (N= 1,200+) assessed the independent variable (“it”)

– About 7% linked essential components (fidelity) to outcomes (we know what produces desired outcomes)

Moncher & Prinz, 1991;Gresham, et al., 1993; Dane & Schneider, 1998; Durlak & DuPre, 2008

What is “it”?


10

Clear description of the program

Philosophy, values, principles (guidance)

Inclusion – exclusion criteria (beneficiaries)

Clear essential functions that define the program (core components)

Operational definitions of essential functions (practice profiles; do, say)

Practical performance assessment

Highly correlated (0.70+) with desired outcomes

Usable Intervention Criteria

Most evidence-based “interventions” do not meet usability criteria

Practitioners and managers use interventions (not standards for rigor)

Scaling for population benefits requires usable interventions

Usable Intervention Criteria


11

Socially Significant Outcomes

WHAT:Effective

Interventions

HOW/WHO:Effective


Enabling Contexts

Formula for Success

A fundamental truth:

• People cannot benefit from programs they do not experience

Effective Innovations

• If we cannot implement them as designed (with fidelity)

• They will not produce desired outcomes



12

(Institute of Medicine, 2000; 2001; 2009; New Freedom Commission on Mental Health, 2003; National Commission on Excellence in Education,1983; Department of Health and Human Services, 1999)

Effective NOT Effective

Effective

NOT Effective

IMPLEMENTATIONIN

TE

RV

EN

TIO

N Actual Benefits

Inconsistent; Not Sustainable; Poor outcomes

Poor outcomes Poor outcomes; Sometimes harmful


DBT


13

DBT TAUIndividual Psychotherapy $3,885 $2,915

Group Psychotherapy $1,514 $147

Day Treatment $11 $876

Emergency Room Visits $226 $569

Psychiatric Inpatient Days $2,614 $12,008

Medical Inpatient Days $360 $1,094

TOTAL $8,610 $17,609

ONE YEAR HEALTH CARE COSTS PER PATIENT: Dialectical Behavior TherapyLinehan et al., 1991








14

Stages of Implementation



15


� You don’t get to skip any!

� If you make a judgment about “effectiveness” too early you might be making a mistake.

� Activities need to match the stage.

� Different stages for multiple initiatives� “Where are we now with this initiative?”

� Stages will need to be “revisited” – important!• New providers, new communities, new government officials,

new families








16

© Fixsen & Blase, 2008

Fidelity Assessment

Coaching

Training

Selection

Systems Support & Intervention

FacilitativeAdministration

Decision SupportData System

AdaptiveTechnical

Leadership DriverLeadership Driver

Consistent Use of

The Innovation

Reliable Benefits (Outcomes)

Where the “WHAT”Meets the “HOW”

Integrated and Compensatory


Fidelity Assessment

Coaching

Training

Selection

Systems Support & Intervention

FacilitativeAdministration

Decision SupportData System

AdaptiveTechnical

Leadership DriverLeadership Driver

Consistent Use of

The Innovation

Reliable Benefits (Outcomes)

Where the “WHAT”Meets the “HOW”

INFRASTRUCTUREPlan

MaintainImprove


17

OUTCOMES(% of Participants who Demonstrate Knowledge, Demonstrate

new Skills in a Training Setting, and Use new Skills in the Classroom)

TRAININGCOMPONENTS

KnowledgeSkill

DemonstrationUse in the Classroom

Theory and Discussion

10% 5% 0%

..+Demonstration in Training

30%20%

0%

…+ Practice & Feedback in Training

60% 60% 5%

…+ Coaching in Classroom

95% 95% 95%

Joyce and Showers, 2002

Staff Training & Coaching

Monitoring/Coaching and Fidelity, and Practitioner Turnover

N=153

Aarons et al., 2009


18

Results from Child Wellbeing Project

Case Example: Metz et al.

Component T1 T2 T3

Selection 1.44 2.00* 1.89*

Training 1.33 1.50* 1.10

Coaching 1.27 1.73* 1.83*

Perf. Assessment 0.78 1.34 2.00*

DSDS 0.18 1.36 2.00*

Fac. Administration 1.38 2.00* 2.00*

Systems Intervention 1.29 1.86* 2.00*

Average Composite

Score 1.1 1.68* 1.83*

Fidelity (% of

cases)

18% 83% 83%

Success Coach model involved intense program development of core intervention components and accompanying implementation drivers








19

What Do They Have in Common?

�Policy to Practice to Policy Cycles

�Transformation Zones

�Rapid Cycle Teams�Problem-solving�Practice Improvement

�Usability Testing

PDSA Cycles

Shewhart (1924); Deming (1948); Six-Sigma (1990)

– Plan – Develop specific things to do

– Do – Do them (make sure)

– Study – See what happens

– Act – Make adjustments

– Cycle – Do over and over again until the goal is reached or the problem is clear (again)


20

What Do They Have in Common?

�Policy to Practice to Policy Cycles

�Transformation Zones

�Rapid Cycle Teams�Problem-solving�Practice Improvement

�Usability Testing








21

Minimum of three people (four or more preferred) with expertise in:

Innovations

Implementation

Organization change

Tolerate turnover; teams are sustainableeven when the players come and go (Higgins, Weiner, & Young, 2012; Klest & Patras, 2011)

Implementation Team

• Supply side: Effective programs go where they are most welcome

– Let it happen; Help it happen

– Islands of excellence

• Demand side: Effective programs go where they are most needed

– Make it happen; whole populations

– A sea of change

Population Benefits


22

Organization Supports

Management (leadership, policy)

Administration (HR, structure)

Supervision (nature, content)

Practitioner/Staff Competence

State and Community Supports

Regional Supports

Imp

lem

en

tati

on

Te

am

Simultaneous, Multi-Level Interventions

Federal and National Supports

Implementation Team


80%

Implementation Team

Prepare Organizations

Prepare Practitioners and Staff

Work with Researchers

Assure Implementation

Prepare Regions Assure Intended Benefits

Create Readiness

Parents and Stakeholders 20%

Implementation Team


23

Expert Impl. Team NO Impl. Team

Effective

Effective use of Implementation Science & Practice

IMPLEMENTATIONIN

TE

RV

EN

TIO

N

80%, 3 Yrs 14%, 17 Yrs

Balas & Boren, 2000 Green, 2008

Fixsen, Blase, Timbers, & Wolf, 2001

Letting it Happen Helping it Happen

Implementation Team

0

10

20

30

40

50

60

70

80

90

100

Practitioner Development Organization Development

Op

era

tin

g 6

+ Y

rs.

N = 84 N = 219

Fixsen, Blase, Timbers, & Wolf (2001)

Program Sustainability


24


WHAT:Effective

Interventions

HOW/WHO:Effective


WHERE:Enabling Contexts

Formula for Success

You should be more You should be more You should be more You should be more explicit here in Step 2!explicit here in Step 2!explicit here in Step 2!explicit here in Step 2!


25

Why Focus on Context?

– Because the Current System is perfectly designed to get the current results!

– And Systems Can Trump Programs!

Existing System

Effective Innovations

Are Changed to

Fit The System

Existing System Is

Changed To Support

The Effectiveness Of

The Innovation

Effective Innovation

System Reinvention


26

Implementation Team

StateManagement

Team

PractitionersInnovations

Children, Families

Sys

tem

C

ha

ng

e

Adaptive Challenges

• Duplication

• Fragmentation

• Hiring criteria

• Salaries

• Credentialing

• Licensing

• Time/ scheduling

• Union contracts

• RFP methods

• Federal/ State laws

“E

xte

rnal”

Syste

m C

han

ge S

up

po

rt

Pra

cti

ce-P

olicy

Co

mm

un

icati

on

Lo

op

Po

licy E

nab

led

P

ractic

e

Reinvent Systems: Manage Change

“The fault cannot lie in

the part responsible

for the repair.”Ashby (1956)

Bureaucracy

Practitioners

Policies

Provider Organizations

Functional Systems



27

WHY:Socially


WHAT:Effective

Interventions

HOW/WHO:Effective


WHERE:Enabling Contexts

Formula for Success

What Will It Take to Make Socially Significant Progress?

�Science and Data Related to…. � Interventions and Programs� Implementation, Organization, and Systems Change

�Funded Infrastructure (Drivers)�Informed & engaged consumers & communities�Linked Teams

�Skillful Purveyors & Intermediaries �Competent Providers�Supportive Organizations�Engaged communities and stakeholders

�Hospitable & Aligned Systems�Leadership at All Levels


28

What did you expect? What did you expect? What did you expect? What did you expect? e = mce = mce = mce = mc2222


29

www.implementationconference.org

August 19-21, 2013

Dean L. Fixsen, Ph.D.

– 919-966-3892

– [email protected]

Karen A. Blase, Ph.D.

– 919-966-9050

– [email protected]

Frank Porter Graham Child Development Institute

University of North Carolina

Chapel Hill, NC

http://nirn.fpg.unc.edu/www.scalingup.org

www.implementationconference.org

For More Information


30

HTTP://NIRN.FPG.UNC.EDU

Fixsen, D. L., Naoom, S. F., Blase, K. A., Friedman, R. M. & Wallace, F. (2005). Implementation Research: A Synthesis of the Literature. Tampa, FL: University of South Florida, Louis de la Parte Florida Mental Health Institute, The National Implementation Research Network (FMHI Publication #231).

Implementation Research: A Synthesis of the Literature


©Copyright Dean Fixsen and Karen Blase

This content is licensed under Creative Commons license CC BY-NC-ND, Attribution-NonCommercial-NoDerivs. You are free to share, copy, distribute and transmit the work under the following conditions: Attribution — You must attribute the work in the manner specified by the author or licensor (but not in any way that suggests that they endorse you or your use of the work); Noncommercial — You may not use this work for commercial purposes; No Derivative Works — You may not alter or transform this work. Any of the above conditions can be waived if you get permission from the copyright holder.

http://creativecommons.org/licenses/by-nc-nd/3.0


31

• Annie E. Casey Foundation (EBPs and Cultural Competence)

• William T. Grant Foundation (Implementation Literature Review)

• Substance Abuse and Mental Health Services Administration (Implementation Strategies Grants; National Implementation Awards)

• Centers for Disease Control & Prevention (Implementation Research)

• National Institute of Mental Health (Research And Training Grants)

• Juvenile Justice and Delinquency Prevention (Program Development And Evaluation Grants)

• Office of Special Education Programs (Scaling up and Capacity Development Center)

• Administration for Children and Families (Child Welfare Leadership; Capacity Development Center)

• Duke Endowment (Child Welfare Reform)

Thank You for Your Support

Date post:	14-Jun-2020
Category:	Documents
Upload:	others
View:	2 times
Download:	0 times

Socially Significant Solutions - Helping Families...

Documents