Implementing Evidence-Based Practices: Strategies in Mental Health and Substance Abuse Child and Adolescent Programs

IADDA Conference

Oakbrook, IL

October 3rd, 2007

Panel Participants:Presenters:

Susan Harrington Godley, Rh.D.—Chestnut Health SystemsDanielle Kirby, MPH, MSEd—Division of Alcoholism and Substance Abuse (DASA)Andrea Kuebbeler, LCSW—Alternatives, Inc.Amy Starin, LCSW—DMH Child & Adolescent NetworkHelen R. Stewart, LCSW—Pillars

Discussant:Stanley G. McCracken, Ph.D., LCSW, RDDP— The University of Chicago, School of Social Service Administration

Overview of Evidence-Based Practices

Stanley McCracken

University of Chicago

School of Social Service Administration

Susan H. Godley

Chestnut Health Systems

Bloomington, IL

Ways of Viewing EBPEBP is a process. EBP is a way of doing practice that integrates the best evidence with clinical expertise and consumer values. (EBP as a verb.) (Sackett et al., 2000)

PractitionerExpertise

BestEvidence

Client Values & Preferences

EBP

Ways of Viewing EBP

EBP is a product. An evidence-based practice is any practice that has been established as effective through scientific research according to some set of explicit criteria. (EBP as a noun.) (Drake, 2001)

EB Interventions. (A-CRA, MET/CBT5)EB Skill sets. (CBT, Behavioral Parent

Training)

Definition of Implementation

“…Specified set of activities designed to put into practice an activity or program of known dimensions…such that independent observers can detect its presence and strength.”

(Fixsen et al, 2004, p. 5)

Fixsen et al., 2004, p. 29

Definition of Fidelity

Strategies used to monitor the faithful delivery of a manual-guided behavioral intervention

Important dimensions include adherence (i.e., extent to which intervention

procedures were delivered as prescribed in the treatment manual)

competence (i.e., qualitative measure of the skillfulness in which intervention procedures are delivered)

Different Types of Manuals

Session Driven

Procedure Driven

Principle Driven

Study of Therapists’ Reactions to Manual-Guided

TherapyQualitative Interview

Questions Compare & contrast doing therapy with &

without a manual.Were there times when you deviated &

why?How was manual-based therapy able to

address individual needs?

Therapists Interviewed

At least 3 from each intervention; total of 16 therapists and 3 CM1 to 18 months experience with manualAge ranges from 24-55 with a M age of 37M of 7 years in drug abuse counseling, services to adolescent, and services to family10 had master’s degrees, 6 had bachelor degrees, and 3 had doctoral degrees5 had previous experience with M-G therapy

Results

Structure, Consistency, Focus

All 19 therapists said that MGT provided structure & consistency30% noted it helped them prepare for a session6 noted it helped them focus during a session4 out of 6 supervisors talked about quality control

Restrictiveness

57% noted some aspect of restrictiveness

42% said it limited their ability to respond to individual needs

Cut across all interventions, but highest percent (70%) were in relation to group

Exceptions

4 therapists discussed how they were able to incorporate their personal style and individualize the treatment. the use of the check-in time at the beginning choosing role-play situations related to

circumstances of the group

74% indicated the manual they used was flexible enough to address individual needs

Division of Mental HealthChild & Adolescent Services:Approach to EBP

Amy StarinDivision of Mental Health

Child & Adolescent Network

Division of MH – C & A Services

Approach to EBPAdvisory CommitteeStarted in November 2005Membership includesCMH AgenciesUniversity ProfessorsParentsAdvocacy Organizations

DMH C & A Provider Survey

303 responses

75% knew what EBP was

Agencies not prepared to assist clinicians in accessing or applying research

44% report having been ‘trained’ in EBP

Mostly through a workshop – ineffective

92% Interested or Very interested in learning in EBP

EBP – The Noun or the Verb?

An extremely diverse client base

Very narrowly defined EBP interventions

DMH decision………..

The “VERB”Evidence Informed

PracticeProcess of Infusing concepts of science into our

C & A system Individual Assessment Define a client specific question What does the research say? Review evidence with client and make a decision

based on client values Implement the intervention Measure the outcomes & evaluate

Evidence Informed Practice Definition

“A collaborative effort by children, families, and practitioners to identify and implement practices that are appropriate to the needs of the child and family, reflective of available research, and measured to ensure the selected practices lead to improved meaningful outcomes.”

5 Pronged Approach

1. Evidence Based Skill Sets

2. Agency Leadership Seminars

3. University Partnerships

4. Consumer Education

5. Policy Implications

EBP Skill Sets - Pilot

CBT & Behavioral Parent Training

22 Pilot agencies (FY 07/08)

8 Didactic days & twice monthly phone supervision over the course of 12 months

University Evaluation

Provider Agency Leadership Training

Quarterly seminars at 3 locations in the state.

University Partnerships

3 State University programs have developed C & A EBP Certification programs for 2nd year masters students. Students are being admitted this year.

Importance of collaboration between Academic and Field training.

Consumer Education

Consumers are powerful change agents

Consumer conferences and speaking to advocacy groups

State Policy Barriers & Supports

Development of Action Steps for each level of the system.Division of Mental HealthUniversityAgencyClinicianConsumer Advocacy GroupConsumer Family

Division of Mental HealthChild & Adolescent Services: A Provider’s Perspective

Helen R. Stewart, LCSWPillars

Parent Behavior Training:Implementing in community-based

settings

Different Perspectives and integration of them into one treatment- common language/common ground: Initial skepticism about EBPPsychodynamic treatment vs. PBT Interpersonal therapy vs. PBT

Parent Behavior Training, cont’d

Engaging parents: “Pre-treatment” PhaseTime FramesConstantly revisiting engagement

Parent Behavior Training, cont’d

Adapting PBT:Trainers adaptationsClinicians adaptations:

Home-based servicesComplex, multi-problem familiesMultiple caretakers, multiple siblingsCrisisFamily level of functioning

Parent Behavior Training, cont’d

Cultural adaptations:LanguageCultural perspectives on parenting in generalHandouts

Division of Alcoholism and Substance Abuse Adolescent Coordination Grant

Danielle KirbyDivision of Alcoholism and Substance Abuse

Chicago, IL

DASA Why Evidence-Based Practices?

Outcomes

NOMs (National Outcome Measures) / State Plan

Performance-Based ContractingIllinois State Adolescent Coordination Grant (IL-SAC)

IL-SAC Illinois Adolescent Substance Abuse

Treatment Coordination Initiative

Funded by SAMHSA/CSAT

(Substance Abuse and Mental Health Services Administration/Center for Substance Abuse Treatment)

3 years: August 1, 2005 – July 31, 2008

Year 3: August 1, 2007 – July 31, 2008

More Information: www.IllinoisTreatmentWorks.org

IL-SAC 13 Required ActivitiesActivity #4

4.Evidence-based treatment: Identify barriers (fiscal, regulatory, and policy) that impede the adoption and provision of accessible evidence-based treatment across the full continuum of care recommended by the American Society of Addiction Medicine (ASAM). Devise and implement strategies, in concert with all other State-agencies that may fund and/or regulate these services, to improve the access to treatment, increase capacity and quality, and expand the available continuum in communities and throughout the State implementing treatment interventions with a scientific evidence base for the population to be served.

Considerations in ImplementingEvidence-Based Practices

Defining SAMHSA’s National Registry of Evidence Based

Programs & Practices (NREPP)7 OptionsNo MISA EBP

Manual-Based Treatment Noun vs. Verb Practice-Based Evidence

Funding: Training and Technology Transfer

IL-SAC “Pilot” Program

DMH’s Approach

Division of Alcoholism and Substance Abuse Adolescent Coordination Grant:EBT Implementation

Susan H. GodleyChestnut Health Systems

Bloomington, IL

Overview

DASA released application for participation in GAIN and EBT training in Jan ’06

8 out of 22 applicant agencies were chosen to participate and began the training process in April ‘06

Coaching callsAs needed

Upload session recordings & data to the

web;Get expert ratings andnarrative feedback

4-day centralized

training session

EBT

Technical Assistance

Certification Requirements are clearly delineated & monitored

Record clinical and

supervision sessions

Treatment Manual

and

Knowledge Test

Performance Indicators

are monitored

After certification monthly fidelity checks

A-CRA Clinician Certification

RequirementsTake a knowledge test

Attend the 3.5 day training

Attend coaching calls

Participate in local supervision sessions

Demonstrate competency on 9 core

A-CRA procedures through DSR reviews

Sample Procedure Rating1 2 3 4

5| | | |

|

poor needs satisfactory very excellent improvement good

Caregiver Overview, Rapport Building, and Motivation:

48. ____ ____ Provided an overview of ACRA 49. ____ ____ Set positive expectations

50. ____ ____ Reviewed research regarding parenting practices51. ____ ____ Identified CG reinforcers for continued work

52. ____ ____ Kept discussion (about adolescent) positive

Each column represents a different session

A 3 or better on all components denotes competency

Narrative Comments Are Also Provided

Progress: MET/CBT5

3 sites sent 10 staff to trainings

Appears to be implemented well at 2 agencies

Third agency still working towards implementation

Progress: A-CRA

Five Sites sent 26 Individuals to trainings

One agency chose not to implement

Being Implemented at 4 agencies

Barriers/Advantages to EBT Implementation

Barriers Lack of

understanding/

commitment to time demands of certification process

Limited clinical supervision time

Advantages Like having an

approach that all are trained on instead of everyone ‘doing their own thing’

Most felt these approaches worked well with their clients

Division of Alcoholism and Substance Abuse Adolescent Coordination Grant:A Provider’s Perspective

Andrea KuebbelerAlternatives, Inc.

Chicago, IL

IL DASA EBT ProjectAlternatives, Inc.

Why we chose to participate.Outcomes for Adolescent substance abusersSkill building for StaffAbility to generate increased fundingParticipation in a learning communityAdvance adolescent substance abuse

treatment within Illinois

Barriers Encountered

Equipment costs

Supervision/Management Time

Staff Turnover

Staff Resistance

Data Collection

Sustainability

Benefits to Agency

Staff Training

Staff Supervision-Audio/Video Taping

Consistent Assessment/Treatment Model

Common Language

Influence Statewide adolescent practice