Identifying and Implementing Evidence-Based Mental Health and Substance Abuse Services

Kevin D. Hennessy, Ph.D.Science to Service CoordinatorSubstance Abuse & Mental Health Services AdministrationU.S. Department of Health & Human Services

September 16, 2005

“A Life in the Community For Everyone”

“Building Resilience & Facilitating Recovery”

A Cross-Cutting Principle: Science to Services/Evidence-Based Practices

How do we translate research into practice?

How do we connect services to science?

Few research innovations are implemented successfully in typical health care settings

Many widely used interventions have little if any research support

Most in treatment don’t receive evidence-based interventions

Many reports exist documenting the problem and recommending actions

Bridging the Gap Between Research and Practice: Forging Partnerships with Community-Based Drug and Alcohol Treatment (Institute of Medicine Report) – 1998Mental Health: A Report of the Surgeon General – 2000Achieving the Promise: Transforming Mental Health Care in America (The President’s New Freedom Mental Health Commission) – 2003

Knowledge to practice gap remains at 15-20 years

An old and well-defined problem

Publication

Bibliographic databases

Submission

Reviews, guidelines, textbook

Negative results

variable

0.3 year

6. 0 - 13.0 years50%

46%

18%

35%

0.6 year

0.5 year

9.3 years

It takes 17 years to turn 14 per cent of original researchIt takes 17 years to turn 14 per cent of original research to the benefit of patient careto the benefit of patient care

Dickersin, 1987

Koren, 1989

Balas, 1995

Poynard, 1985

Kumar, 1992

Kumar, 1992

Poyer, 1982

Antman, 1992

Negative results

Lack of numbers

Expertopinion

Inconsistentindexing

17:14

Original research

Acceptance

Implementation

Identify evidence-based practices (NREPP)

Develop guides, manuals and other materialsEvidence-Based Practice Implementation Resource Kits

Disseminate materials and provide trainings through regional centers

Centers for the Application of Prevention Technology (CAPTs)Addiction Technology Transfer Centers (ATTCs)Center for Mental Health Services Technical Assistance Centers

Create interagency and private-public partnerships

SAMHSA Activities for Reducing the Knowledge – Practice Gap

National Registry of Evidence-based Programs and

Practices (NREPP)

Purpose: A voluntary rating and classification system for mental health and substance abuse prevention and treatment interventions.

Goal: NREPP will become a leading national resource for science-based information on substance abuse and mental health prevention and treatment interventions.

NREPP Timeline

1998…Started with a focus on substance abuse prevention as the National Registry of Effective Prevention Programs

1998-2003…reviewed and rated over 1,100 substance abuse prevention programs

information on over 150 Model, Effective, and Promising Programs on web site (www.modelprograms.samhsa.gov)

2004…initial expansion of system to include substance abuse treatment, mental health promotion, and mental health treatment programs

NREPPCurrent Status

Formal public comment process by SAMHSA this summerNotice in August 26th Federal RegisterAvailable through SAMHSA web site – access through “Quick Picks” on SAMHSA’s home page (www.samhsa.gov)Written comments due no later than October 25th

U.S. Mail – SAMHSA, c/o NREPP Notice, 1 Choke Cherry Road, Rockville, MD 20877Electronic Mail – nreppcomments@samhsa.hhs.gov

Posting of recently completed reviews (using older NREPP system) by Fall 2005

No new reviews until 2006

New NREPP Website will be launched in 2006: www.nationalregistry.samhsa.gov

NREPP: Comprehensive & Transparent

Comprehensive

Scientists independently evaluate and classify outcomes for programs/practices based on statistical significance, replications and 16 evidence quality criteria

Program impact and practical significance will be reported through effect sizes

For status programs, additional information on implementation, training, quality tools, staffing, costs, etc. will be provided

NREPP: Comprehensive and Transparent

Transparent

Review results will be posted on the new NREPP website

By 2006 Web site will feature: Outcomes searchable database Links to educational materials on review criteria Self-assessment tool for candidate programs Links to technical assistance centers and supports

NREPP: Draft Rating Criteria

Two Types of Criteria

Evidence Rating Criteria

“Utility Descriptors”

NREPP: Evidence Rating Criteria16 evidence rating criteria applied to each

program outcome (0 to 4 scale).

Include:Theory-driven selection of measures & analytic methodsReliabilityValidityIntervention and comparison fidelityAssurances to participantsStandardized Data CollectionSelection biasAttrition and missing dataAnalysis meets data assumptions

NREPP: “Utility Descriptors”

Look beyond scientific evidence and ensure that programs are able to be implemented.

Assessments also made of the following:ImplementationQuality MonitoringUnintended or Adverse EventsPopulation CoverageCultural RelevanceStaffing Cost

NREPP: Questions to the Public

From the August 26th Federal Register Notice (FRN)

Is proposed system objective, transparent, efficient, and scientifically defensible?

How might SAMHSA engage stakeholders in determining priority review areas?

How best to use statistical significance and measures of effect size in NREPP?

Beneficial to use multiple categories of effectiveness?

Approach to assessing and presenting cultural relevance?

NREPP: Questions to the Public

From the August 26th Federal Register Notice (FRN) - continued

Approach to re-reviewing existing NREPP programs?

Types of technical assistance needed to promote adoption of NREPP interventions?

How best to involve consumers, families, and other non-scientists in NREPP?

What, if any, guidance on use of NREPP interventions within block grants?

How best to promote other sources – e.g., clinical judgement, consumer values, etc – in decisions regarding selection, delivery and financing of services?

Panel to annually review NREPP operational and technical suggestions?

NREPP’s Reach

Influences SAMHSA discretionary and block grant investments

Serves as a resource for states and communities seeking to implement evidence-based Mental Health & Substance Abuse prevention and treatment services

Provides an important tool for both public and private purchasers in selection of effective services

What type of Products?

• Training Manuals• Evidence-based Implementation Resource Kits (CMHS)• Interactive Web-based Technical Support (Prevention Platform)• Treatment Improvement Protocols (TIPs)• Quick Reference Cards & Pocket-size Guides• Fact Sheets, Brochures, Reports, and Periodicals

Publications and Materials on Best Practices in Mental Health and

Substance Abuse Prevention & Treatment

Who Uses the Products?Program Administrators & StaffCliniciansPolicymakersConsumers

Where to Find the Products:For SA Treatment go to www.kap.samhsa.govFor SA Prevention go to www.prevention.samhsa.govFor Mental Health go to www.mentalhealth.samhsa.gov

Publications and Materials on Best Practices in Mental Health and

Substance Abuse Prevention & Treatment

Focus on six practicesAssertive community treatmentFamily psychoeducationSupported employmentIntegrated dual disorders treatmentIllness management and recovery skillsStandardized pharmacological treatment

Two PhasesI – Development (Fall 2000 – Summer 2002)II – Pilot-Testing (Summer 2002 – Summer 2005)

More information is available through SAMHSA’s web site (www.samhsa.gov) under “Mental Health System Transformation”

Evidence-Based PracticeImplementation Resource Kits

Kits have targeted and distinct components for:ConsumersFamily and Other SupportsPractitioners and Clinical SupervisorsMental Health Program LeadersPublic Mental Health Authorities/Administrators

Kits include:Research summariesInformation sheets for all stakeholdersTip sheets for program leaders and administratorsProgram manuals/workbooks for practitionersIntroductory and training/demonstration videosFidelity assessment tools and cultural competence statementsOutcome measures

Evidence-Based PracticeImplementation Resource Kits

Centers for the Application of Prevention Technology

Materials, Trainings & Technical Assistancefor Prevention

www.captus.org

Purpose: Assist States and Communities in the application

of evidence-based substance abuse prevention programs,

practices, and policies

Goal: To increase the impact of prevention

Materials, Trainings & Technical Assistancefor Treatment

SAMHSA’s Addiction Technology Transfer Centers

www.nattc.org

Purpose: To transmit the latest knowledge, skills and attitudes of professional addiction treatment practice

Goal: To enhance clinical practice

Currently 55 Centers provide a range of services – many are free but charges may apply for some:Technical assistanceInformation and referralsOn-site consultationTrainingLibrary servicesPublicationsOther resources

More information is available through SAMHSA’s web site:www.mentalhealth.samhsa.gov/links/

Center for Mental Health ServicesResearch, Training and TA Centers

Collaborations with National Institutes of Health to jointly fund state planning grants facilitating implementation of evidence-based practices

– RFA-MH-03-007 – State Implementation of Evidence-Based Practices: Bridging Science and Service (with NIMH)

– RFA-MH-05-004 – State Implementation of Evidence-Based Practices II: Bridging Science and Service (with NIMH)

– RFA-DA-05-002 – Enhancing State Capacity to Foster Adoption of Science-Based Practices (with NIDA)

Promoting Interagency Partnerships

Partnership between:SAMHSA’s Center for Substance Abuse Treatment (CSAT)Robert Wood Johnson Foundation (RWJF)

Goals of improving organizational processes to facilitate client access and retention in addictions treatment. Initial successes include: Reduced Waiting Times by 31% (n=24)Reduced No-Shows by 22% (n=13)Increased Admissions by 25% (n=19) Increased Treatment Continuation Rates by 33% (n=8)

More info available through web site – www.niatx.org

Creating Public – Private Partnerships

Focus on elements of health care redesign detailed in Crossing the Quality Chasm (Institute of Medicine, 2001)

– Apply evidence to health care delivery

– Align payment policies with quality improvement

– Prepare the workforce

– Use information technology

Advancing Science to Service Efforts and Reducing the Knowledge – Practice Gap

“The future is here.

It’s just not widely distributed yet.”

In Conclusion