Some Emerging Characteristics of Sustainable Practices

Thanks to Wing Institute, NIRN and

Kent McIntosh

Nature of the Problem

• In education innovations come and go in 18-48 months (Latham, 1988).

• Alderman & Taylor (2003) Optimally, sustainability should be a focus from the day a project is implemented. With most projects, the pressure of just becoming operational often postpones such a focus until well into the 2nd year.

MemoTo: School Administrators

From: District Administrators• In keeping with the new state initiative, this fall we

will be implementing an exciting new district initiative of SNI in place of LYI. All in-service days previously scheduled for LYI will be rescheduled as staff development for SNI. The $500 for release time and materials for LYI will be discontinued and provided instead for SNI. By the way, you will need to create local SNI teams that meet weekly. The former members of your LYI team would be perfect for this new team. Your new SNI binders will be coming next week. Have a great year!!!

Why Such a Short Life Span?

• High EffortInnovation more difficult than expected.Causes too much change.Takes too much time.

Why Such a Short Life Span?

• Poor system designSupporters leave.Personnel lack training.External funds run out. Inadequate supervision.No accountability.No consequences for early termination.

Even Well Tested Programs Often Fail to Sustain

• Elliott & Mihalic (2004) review Blueprint Model Programs (violence prevention and drug prevention programs) replication in community settings.Programs reviewed across 5 dimensions

Site selection Training Technical assistance Fidelity Sustainability.

Even Well Tested Programs Often Fail to Sustain

• Critical elements in site readinessWell connected local championStrong administrative supportFormal organizational commitmentsFormal organizational staffing stabilityUp front commitment of necessary resourcesProgram credibility within the communityProgram sustained by the existing operational budget.

Even Well Tested Programs Often Fail to Sustain

• Critical elements of trainingAdhere to requirements (planning phase) for training,

skills, and education.Natural Implementers identified before scheduling

training.Encourage administrators to attend training- A MUST!Plan and budget for staff turnover. Implement program immediately after training.

Even Well Tested Programs Often Fail to Sustain

• Critical elements of Technical AssistanceProactive plan for technical assistance.

• Critical elements of FidelityMonitor fidelity- TIC, IPI

• Critical elements of SustainabilityFunction of how well other dimensions are

implemented.

Why Such a Short Life Span?

• Those responsible for developing effective interventions do not necessarily have the skills to effect large systems change.Systems change is different level of intervention.

Adults are the target of change rather than student behavior.

Ineffective Methods

• Excellent evidence for what does not work

Implementation by edict does not work Implementation by “following the money” does not

work Implementation without changing supporting roles and

functions does not work

» Paul Nutt (2002) Why Decisions Fail

Emerging Features of Sustainable Programs

• Maintain over time.• Maintain across generations of practitioners.• Operate within existing financial and staffing

resources.

Implications of Current Measures

• Current accountability measures of NCLB may reflect a change in emphasis.Problem is that NCLB specifies outcomes but does not

specify behaviors to accomplish outcome. Consequence may be that test scores improve but student

learning does not.Multiple instances of cheating reported Many schools spend great deal of time “teaching to the test.”

• Remains to be seen if these accountability measures result in more effective practices that sustain.

Do you have a deep understanding of the principles of sustainability?

• Common perception that sustainability is a ethereal, theoretical concept (Vaughn et al, 2000)

• We all have experiences with it• The same principles of individual behavior still

apply to systems…

OngoingChallenge

StudentOutcomes

Select Practices& Implement with Fidelity

Antecedent Behavior Consequence

OngoingChallenge

StudentOutcomes

Select Practices& Implement with Fidelity

Antecedent Behavior Consequence

Barriers to Sustainability: The Three Cs

Changes in Context

- Lack of contextual fit

- New challenges exist

- Competing initiatives

Changes in Capacity

- Loss of funding

- Attrition of key personnel

Changes in Consequences

- Diminished effectiveness due to poor fidelity

- Outcomes no longer perceived as important

A SWPBS Sustainability Study(Doolittle, 2006)

• Sample: 285 schools with SET scores• Differences between schools that implemented and

those that did not:Expectations TaughtMonitoring and Decision Making

• Differences between schools that sustained and those that did not:On-going Behavioral Reward SystemManagement (Administrator)

OngoingChallenge

StudentOutcomes

Select Practices& Implement with Fidelity

Antecedent Behavior Consequence

Identifying& Modifying

Practices

ValuedOutcomes

Practice Implementation

Identifying& Modifying

PracticesContinuous

Measurement

Data-BasedProb.

Solving

Antecedent

Consequence

Behavior

Behavior

ValuedOutcomes

Practice Implementation

Identifying& Modifying

Practices

Efficiency

EffectivenessPriority

Continuous Regeneration

Continuous

Measurement

Data-BasedProb.

Solving

Capacity

Building

ValuedOutcomes

Practice Implementation

Identifying& Modifying

Practices

Efficiency

EffectivenessPriority

Continuous Regeneration

Continuous

Measurement

Data-BasedProb.

Solving

Capacity

Building

ValuedOutcomes

Priority

PRIORITY• Importance in

comparison to other practices

• Connection to other initiatives

• Incorporation into core system components

Behavioral Principle:Competing Schedules of Reinforcement

ValuedOutcomes

Priority

ENHANCING PRIORITY• Braid project into other

initiativesShow how practice can

lead to other outcomes of new initiatives

• Get into written policy• Advocate for improved

visibilityPresent data to people with

resourcesDescribe effects of

abandoning support for the practice

ValuedOutcomes

Practice Implementation

Identifying& Modifying

Practices

Efficiency

EffectivenessPriority

Continuous Regeneration

Continuous

Measurement

Data-BasedProb.

Solving

Capacity

Building

Identifying& Modifying

Practices

Effectiveness

EFFECTIVENESS Extent to which the

practice results in desired outcomes

Choice of practices should be based on proven effectiveness

Effects must be attributed to the practice

Behavioral Principle:Reinforcement

Identifying& Modifying

Practices

Effectiveness

ENHANCINGEFFECTIVENESS Select practices that are

likely to produce the desired outcomes (i.e. Evidence-Based Practices)

Share data that show how adoption is related to effects

ValuedOutcomes

Practice Implementation

Identifying& Modifying

Practices

Efficiency

EffectivenessPriority

Continuous Regeneration

Continuous

Measurement

Data-BasedProb.

Solving

Capacity

Building

Practice Implementation

Efficiency

EFFICIENCY Relationship between continued effort and

continued effectiveness Weighed against other potential practices

Behavioral Principle:Maintenance

Practice Implementation

Efficiency

ENHANCING EFFICIENCY A durable practice should become more efficient

over time Easier on implementers

Repetition builds fluency Easier to modify materials than create them

Easier on resources Fewer visits from external consultants Fewer release days

ValuedOutcomes

Practice Implementation

Identifying& Modifying

Practices

Efficiency

EffectivenessPriority

Continuous Regeneration

Continuous

Measurement

Data-BasedProb.

Solving

Capacity

Building

Continuous Regeneration

Continuous

Measurement

Data-BasedProb.

Solving

Capacity

Building

CONTINUOUS REGENERATION Iterative monitoring of

fidelity and outcomes Adaptation and re-

adaptation over time while keeping critical features intact

Ongoing investment in the practice

Behavioral Principle:Generalization

Continuous Regeneration

Continuous

Measurement

Data-BasedProb.

Solving

Capacity

Building

ENHANCINGCONTINUOUS REGENERATION DATA-BASED

DECISION MAKING Fidelity Outcomes Use in problem-solving

Expand to new areas Adjust practices for a

changing environment Cultivate local expertise Connect with a

community of practice

ValuedOutcomes

Practice Implementation

Identifying& Modifying

Practices

Efficiency

EffectivenessPriority

Continuous Regeneration

Continuous

Measurement

Data-BasedProb.

Solving

Capacity

Building

Create a Plan to Sustain from the Start

• “Train and Hope”Not an effective approach to implement a practice

• “Implement and Hope” Not an effective way to sustain a practice “Implementation are where good ideas go to die”

4 Big Ideas to Plan for Sustainability…

1. Start with the Ending

• Let the outcomes drive the selection of practices• Identify the valued outcomes for everyone

No one has ever been bullied or nagged into long-term sustainability

• Measure and use data in decision making

2. Death, Taxes, and… …Attrition

• If the fidelity drops, the effects stop• Plan for your champions to move on/up• Focus on POSITIONS, not PERSONS

Create positions tied to the practice Titles Job Descriptions FTE

3. If you keep doing what you’re doing, you MAY NOT keep getting

what you’re getting• Environments change – Continuous Improvement

adjust to changes (e.g. OISM, MISI)

• New ideas keep the practice novel• Spread the practice

To new settingsTo new systems

Were there 4 Big Ideas?

How Can We Increase Sustainability of Practices?

• Establish “Communities of Practice” at all levelsExecutive Coaching, Team Leaders, Student Support

Team-e.g. BISCC

• Does it get easier? School level v. LEA level• Tipping Point

How Can We Increase Sustainability of Practices?

• Continue Pro-active technical assistance- CoachesHelp solve the real problems of implementation.

• Continue to Monitor integrity of implementation.Without monitoring, the system likely to drift back to

previous practices.Recognition Program, SWIS

• Anticipate 3-5 years before system is fully operational.Emphasizes the need to plan for multigenerational

support.

Future Research

• We need it• Descriptive

Case studies of successes/failures

• ExperimentalTest the sustainability model

References

Alderman, H. S. & Taylor, L. (2003). On sustainability of project innovations as systemic change. Journal of Education and Psychological Consultation, 14 (1), 1-25.

Baum, W. M. (2000). Being concrete about culture and cultural evolution. In N. Thompson and F. Tonneau (Eds.) Perspectives in Ethology (Vol. 13, pp. 181-212). New York: Kluwer Academic/Plenum

Doolittle (2006)Elliott, D. S. & Mihalic, S. (2004). Issues in disseminating and replicating effective prevention

programs. Prevention Science, 5(1), 47-53.Glenn, S. S. (2003). Operant contingencies and the origin of cultures. In K. A. Lattal and P. N.

Chase (Eds.) Behavior Theory and Philosophy. New York: Kluwer Academic/Plenum.Harris, M. (1979). Cultural Materialism: The struggle for a science of culture. New York:

Simon and Shuster.Latham, G. (1988). The birth and death cycles of educational innovations. Principal, 68(1), p41 -

43.McIntosh, K., Horner, R. H., & Sugai, G. (in preparation). Sustainability of systems-level

evidence-based practices in schools: Current knowledge and future directions.

Rogers, E. M. (2003). Diffusion of Innovations (5th Edition). New York: Free Press.