Michelle A. Duda, Ph.D., BCBA,Dean L. Fixsen, Ph.D.
& Karen A. Blase Ph.D.,
Melissa Van Dyke, LCSWFrank Porter Graham Child Development Institute
University of North Carolina – Chapel Hill
Working Smarter: The implementation journey to
improve student outcomes
Innovation FluencyPart 1
A Problem
• Too often, the choice of an intervention to implement has had little to do with the quality or weight of the evidence regarding practices
• Decisions are made for a variety of reasons not necessarily related to the data about effectiveness (e.g. philosophy, values, comfort, acceptance, availability, finances)
Follow Through Programs
Figure 1: This figure shows the average effects of nine Follow Through models on measures of basic skills (word knowledge, spelling, language, and math computation), cognitive-conceptual skills (reading comprehension, math concepts, and math problem solving) and self-concept. This figure is
adapted from Engelmann, S. and Carnine, D. (1982), Theory of Instruction: Principles and applications. New York: Irvington Press.
Innovation Fluency
• Definition: Innovation Fluency refers to the degree to which the team knows the innovation with respect to:
1.Evidence2.Program and Practice Features3.Intervention Readiness for Replication4.Implementation Capacity Requirements
• Because all of the above impact the success of implementation efforts
Intervention Science
Research has identified programs and practices that can help students, communities, and education systems
• What Works Clearinghouse (http://ies.ed.gov/ncee/wwc/)
• NREPP (http://www.nrepp.samhsa.gov/)
• Colorado Blueprints (http://www.colorado.edu/cspv/index.html)
• Education research journals/ American Education Research Association (http://www.aera.net/)/ J. of Evidence-Based Practices in Schools
1. Research View of Evidence
• Descriptions of practices • Observational & survey information • Correlational studies• Quasi-experimental studies• Randomized clinical trials (programs)
o in situations of convenienceo in clinical context
• Single-subject experiments (practices, components, programs, systems)
A Different View of Evidence
• From an implementation perspective, what do we need to know about evidence?
1. Evidence Related to the Intervention
o Theoretical Foundations – Intervention What principles make “it” work?
o Empirical Support – Intervention How much research has been done? How much of a difference could it make (e.g.
effect size)?o Fidelity Data (e.g. predictive and discriminative)o Practical to Collect and Analyzeo Cost-effectiveness Datao Populations (e.g. culture, race, language,
ethnicity)o Expected Student Outcomes – Intervention
What outcomes have been achieved in the “real world”?
2. Program and Practice Features
o Start where it matters: At the interface between Teachers and Students
o Build on those Program and Practice features that interface as the foundation, then….
o Identify what needs to happen at the classroom level so the teacher can interact with students as intended
o Identify what needs to happen at the school level to support what happens at the classroom level
o Identify what needs to happen at the Division level to support school and classroom implementation
o ….and the State?
A Different View of Program and Practice Features
• From an implementation perspective, what do we need to know about P & P Features?
2. Program and Practice Features
1.What you can observe happening at classroom, building, and within the division level ?
2. Fit with current priorities and issues
3. Resource Requirements
Program and Practice Features through an Implementation Lens
o Fit with: Current District and School Initiatives & Priorities AYP Priorities RtI Processes Current Organizational Structures at School and District Level Parent and Community Values – Social Validity Current skill set of staff expected to implement the innovation
o Resource Requirements Materials Staffing (recruitment, reallocation) Training costs Coaching and Supervision IT and Data Requirements Administrative time and supports
Do you Recognize The Implementation
Drivers?
Innovation Fluency
• The “it” must be operationalized whether it is:
An Evidence-Based Practice or Program A Best Practice Initiative A Systems Change Initiative
• Operationalize Part of Speech: verb Definition: to define a
concept or variable so that it can be measured or expressed quantitatively
Webster's New Millennium™ Dictionary of English, Preview Edition (v 0.9.7)Copyright © 2003-2008 Lexico Publishing Group, LLC
Practice Profile
• Increased Innovation Fluency Through the Development and Use of Practice Profiles
• Identifies critical components• For each critical component:
o Identifies gold standardo Identifies acceptable variations in practiceo Identifies ineffective practices and
undesirable practices
o Heartland Area Education Agency 11, Iowa
Practice Profiles
• Each critical component is a heading• Each level of implementation becomes a
dimension on the rubric associated with that critical component.
Critical Component Ideal Implementation
Acceptable Variation
Unacceptable Variation
Unacceptable Variation
Critical Component 1: Description
Description of implementer
behavior
DrasticMutation
Adapted from work by Heartland Area Education Agency 11, Iowa
ACTIVITY ONE
Critical Component
Ideal Implementation
Acceptable Variation
Unacceptable Variation
Unacceptable Variation
(Drastic Mutation)
ESD Team Membership
Team Has broad representation
Membership includes invested individuals with knowledge about behavior and ESD implementation.
Team Incl. Behavior Specialist, Principal, 3 lead teachers, parent, school psych
Membership includes invested individuals with knowledge about behavior and ESD implementation.Team Incl.Behavior Specialist, Principal, 2 lead teachers and a parent
Membership includes individuals with minimal knowledge about behavior and ESD implementation.
Team incl.Admin Assistant and 2 teachers
Membership includes individuals with common planning periods.
Team includes 3 teachers, but team members change each semester
Part ONE: Practice using the practice profile format.
EXAMPLE: School Level
ACTIVITY ONE
Critical Component
Ideal Implementation
Acceptable Variation
Unacceptable Variation
Unacceptable Variation
(Drastic Mutation)
Link school teams to supporting resources
Part ONE: Practice using the practice profile format.
EXAMPLE: Coaching/District Level
ACTIVITY ONE
Part Two: Reflection Questions
• How does a schools' implementation of a core feature support the division goals/outcomes?
• How does the division support the school in implementing these core features?
3. Intervention Readiness for Replication
o What are the Qualified “purveyor” and researcher connections?
o Are the Content ‘experts’ and qualified TA available?o Have Replication Success (e.g. % of attempted replications still
implementing) been demonstrated? o Are the Core Intervention Components (program features)
well operationalized so that Practice Profiles can be developed if necessary?
o Have the Implementation Drivers been: Well operationalized (Selection, Training, Coaching, Data
processes, etc)? Demonstrated to be effective (e.g. change & sustain new
teacher behavior)? Practical and affordable (e.g. Drivers have been utilized in
typical settings)?
4. Implementation Capacity Requirements
o Sufficient Staff who meet minimum requirements
o Region/Division prepared to define practice/program features and elements if necessary (e.g. Practice Profiles)
o Region/Division able to sustain Implementation Drivers Fiscally Structurally Functionally
o Buy-in Process Operationalized School District Parents
Innovation Fluency
• Implementation Teams need to understand the innovation so that they can:
o Help Districts/Divisions “choose” wisely Address a prevalent and significant need Worth it to scale-up (e.g. effect size of .50 or greater)
o Evaluate the innovation’s readiness for implementation
o Assess the implementation capacity of the District in relation to a number of factors related to the innovation
For More Information
Dean L. Fixsen, Ph.D.• 919-966-3892• [email protected]
Michelle A. Duda, Ph.D.• 919-636-0843• [email protected]
At the Frank Porter Graham Child Development InstituteUniversity of North Carolina
Chapel Hill, NC
www.scalingup.org http://nirn.fpg.unc.edu/
http://www.fpg.unc.edu/~nirn/resources/publications/Monograph/
For More Information
Fixsen, D. L., Naoom, S. F., Blase, K. A., Friedman, R. M. & Wallace, F. (2005). Implementation Research: A Synthesis of the Literature. Tampa, FL: University of South Florida, Louis de la Parte Florida Mental Health Institute, The National Implementation Research Network (FMHI Publication #231).Download all or part of the monograph at:http://www.fpg.unc.edu/~nirn/resources/publications/Monograph/ To order the monograph go to:https://fmhi.pro-copy.com/