Evidence-Based Recommendations for Participation Station

Marcie TimmermanCapstone PresentationMHA Candidate, 2012

UK College of Public Health

Participation Station

MISSION STATEMENT: To provide a program-based environment that educates, empowers and enhances the lives of individuals moving toward recovery and mental wellness.

The Problem(s)

• Need funding diversification: grant applications

• Need to “prove” their success to date• Need to convince other agencies to

collaborate for advocacy and mission• Pursuing internal continuous quality

improvement

Project Purpose

• Analysis of program evaluation and outcomes data collected Sept-Dec 2011

• Recommendations for improvement based on that analysis

Literature Review

• Gained understanding of models and ideologies which led to PS creation

• Learned vocabulary and theories behind tools questions

• Little to no available measurement tools, peer-reviewed “best practices”, etc. for services like PS in literature

Methods: FACIT• Fidelity Assessment Common Ingredients Tool

(FACIT) from SAMHSA with benchmark data– Structured Observation– Structured item-by-item questionnaireEvaluations could be internal or external.

• Internal only• Interviewed 12 leaders, 75% of whom were peers

Example of FACIT Survey Question

1.1.3 “Hiring Decisions

(1)Consumers are not involved in any hiring decisions

(2) Consumers have some involvement in hiring decisions

(3) Consumers are responsible for making most of the hiring decisions (50% or more).

(4) Consumers are responsible for making all hiring decisions.

Methods: POP• Peer Outcomes Protocol (POP)

– Structured questionnaire with anchored scale responses for each participant

• 1 for “Disagree”• 2 for “Somewhat Disagree”• 3 for “Somewhat Agree”• 4 for “Agree”

– Last 10 questions “Coercion Scale”• Anchored 1-5 with “5” as best

– Some questions worded so that the “4” is bad and “1” is good

Examples of POP Questions

1. Overall, I am satisfied with this peer support program.

2. I am satisfied with the peer program facilities, such as the condition and layout of the rooms and building.

4. Overall, the program services are useful to me.

FACIT Results & Analysis

FACIT Results: Year One

Structure Environment Belief Sys-tems

Peer Support

Education Advocacy0

5

10

15

20

25

30

35

40

45

50Comparison of Domain Scores to National Benchmark

Yr 1 Benchmark

Domain

To

tal

Sc

ore

FACIT Analysis“Red” Analysis: 2+ below benchmark,

Areas of deficiency

“Yellow” Analysis: 1-1.99 below benchmark,Close, but not quite

“Green” Analysis: Proficient

“Purple” Analysis: 1 or more above benchmark,Excellence

1.1.1 Board Participa-tion

1.1.4 Budget Control 1.1.3 Hiring Decisions0

0.5

1

1.5

2

2.5

3

3.5

4

4.5

5"Red" Scored Data Year 1

Benchmark

ITEM

MEA

N

Red Analysis & Recommendations

STRUCTURE DOMAIN:• Board Participation• Hiring Decisions• Budget Control

1.3.2 Linkage to Other Consumer-Operated Services

3.3.1 Personal Empowerment

1.1.2 Consumer Staff

3.5 Recovery 4.4.2 Informal Crisis Prevention

0

1

2

3

4

5

6 “Yellow” Scored Data

Year 1 Benchmark

ITEM

MEA

N

Yellow Analysis & Recommendations

STRUCTURE DOMAIN• Consumer Staff• Linkages with other Consumer-Operated SrvcsBELIEF SYSTEMS DOMAIN• Personal Empowerment• RecoveryPEER SUPPORT DOMAIN• Informal Crisis Prevention

3.4 Choice 3.7 Spiritual Growth

1.3.1 Linkage with TMHS

2.2.1 Lack of Coerciveness

1.3.3 Linkage with Other

Service Agencies

0

1

2

3

4

5

6“Purple” Scored Data

Year 1Benchmark

ITEM

MEA

N

Purple Analysis & RecommendationsSTRUCTURE DOMAIN• Linkage with Other Service Agencies• Linkage with Traditional Mental Health Services

ENVIRONMENT DOMAIN• Lack of coerciveness

BELIEF SYSTEMS• Choice• Spiritual Growth

EDUCATION DOMAIN• Formal curriculums in problem-solving and self-

management

POPResults & Analysis

POP Results• Mean: Goals met in all categories except

meeting employment needs of participants

• Median: “Staff threatens me with loss of housing” is irrelevant to services at PS, had a median indicating “not applicable”

• Mode: Coercion scale uniformly positive with “5” as mode – best score possible.

Project LimitationsFACIT• Internal review only• Performed over several weeks• No internal comparison (1st time)

POP• Interviewer error• Small sample size• No benchmark data available

Recommendations: FACIT• “Red” Recommendations

– Continue separation plan– Training

• “Yellow” Recommendations– Training – Formal crisis prevention process

• “Purple” Recommendations– Monitor environment– Reach out

Recommendations: POP

• Employment assistance referral plan• Community Resources manual• Monitoring

Implementation Plan• Establish dedicated quality improvement team

from peers• Develop key indicators to monitor which are

relevant to mission and applicable to funding requests

• Small cycles of change – path of least resistance to start

• Additional training on CQI• Inform stakeholders of results

Benefits to Participation Station

• Usable data for grant applications• Evidence-based practices• Training needs outlined • Small cycles of change start with the

recommendations

Useful Courses• HA601• HA602• HA603• HA604• HA621• HA623• HA642

• HA660• HA673• HA711• HA785• CPH758

Martha Riddell, MPH, DrPH, ChairAndrew Johnson, PhD, CommitteeSarah Wackerbarth, PhD, Reader

David Riggsby, Practitioner

Questions/Comments