1

National Council on Aging (NCOA)

Empowering Older People to Take More Control of Their Health Through

Evidence-Based Prevention Programs: A Capping Report

Administered: September 2011 – December 2012

Prepared by:

Janet C. Frank and Christy Ann Lau

UCLA Multicampus Program in Geriatric Medicine and Gerontology

Submitted to: National Council on Aging

Revised Submission

March 26, 2013

2

TABLE OF CONTENTS

I. EXECUTIVE SUMMARY ..................................................................................................... 4

II. INTRODUCTION ............................................................................................................... 7

A. Background .................................................................................................................... 7

1. Program Purpose and Goals ........................................................................................ 7

2. Program Evolution Over Time .................................................................................... 7

a. The Early Years ...................................................................................................... 7

b. National Expansion and Systems Integration ........................................................ 8

c. Empowering Older People to Take More Control of Their Health Through

Evidence-Based Prevention Programs .................................................................. 8

d. Empowering Communities to Sustain Evidence-Based Disease and Disability

Prevention Programs (Empowering Communities) ............................................... 9

3. Approach to the Study ................................................................................................ 9

a. Study Limitations ................................................................................................... 10

4. Description of EBHP Programs Supported ................................................................. 10

5. States Implementing Evidence-Based Health Promotion and Disease Management

Programs (EBPs) ......................................................................................................... 12

6. Types of Partners and Their Roles .............................................................................. 12

B. Program Outcomes ....................................................................................................... 14

1. Persons Served ............................................................................................................ 14

2. Outcomes of Interventions on Program Participants .................................................. 16

3. National Program Impacts .......................................................................................... 18

4. Program Fidelity and Quality Assurance .................................................................... 20

5. Best Practices .............................................................................................................. 20

a. Marketing/Outreach ............................................................................................... 20

b. Worker Training ..................................................................................................... 21

c. Infrastructure Development ................................................................................... 22

d. Fidelity and Quality Assurance ............................................................................. 23

e. Evaluation .............................................................................................................. 23

C. Challenges ...................................................................................................................... 24

1. Marketing/Outreach .................................................................................................... 24

a. Rural Issues ............................................................................................................ 24

b. Transportation ....................................................................................................... 24

c. Program Characteristics and Requirements .......................................................... 25

d. Outreach to Minority and Underserved Populations ............................................ 25

2. Worker Training .......................................................................................................... 26

a. Trainer and Leader Recruitment ........................................................................... 26

b. Trainer and Leader Engagement and Retention .................................................... 27

3. Infrastructure Development ........................................................................................ 28

4. Fidelity and Quality Assurance ................................................................................... 29

5. Evaluation ................................................................................................................... 30

D. Sustainability ................................................................................................................. 31

1. Embedding Programs into Systems of Services ......................................................... 31

2. Establishment of New Systems, Positions, Units or Programs ................................... 32

3

3. New Policy Development ........................................................................................... 33

4. Sustainability as a Project Theme ............................................................................... 34

5. Sustainability Plan Documentation ............................................................................. 35

6. New Funding ............................................................................................................... 36

E. Lessons Learned ............................................................................................................ 37

1. Reach ........................................................................................................................... 37

2. Effectiveness ............................................................................................................... 39

3. Adoption ..................................................................................................................... 39

4. Implementation ........................................................................................................... 40

5. Maintenance and Sustainability .................................................................................. 41

F. Products Developed ....................................................................................................... 43

G. Conclusion ..................................................................................................................... 45

APPENDICES .................................................................................................................... 47

Appendix A: Data Extraction Tools and Scoring Rubrics ............................................. 47

Appendix B: Case Studies of Five Grants ....................................................................... 56

Appendix C: Products/Resources Developed .................................................................. 80

Appendix D: Lessons Learned .......................................................................................... 100

4

I. EXECUTIVE SUMMARY

In 2006 and 2007, the Administration on Aging made a major investment in the national

expansion of evidence-based (EB) health promotion and disease management programs in the

Empowering Older People to Take More Control of Their Health Through Evidence-Based

Prevention Programs that provided funding to 24 states. This initiative was followed in 2010

with additional funding from the American Recovery and Reinvestment (ARRA) entitled

Empowering Communities to Sustain Evidence-Based Disease and Disability Prevention

Programs, to expand the capacity and delivery of the EB programs in these states. These grants

began in 2006, 2007 and all state grant projects under these two initiatives were completed in

2012.

Through the National Council on Aging, the Administration on Aging commissioned an

evaluation to document the successes, challenges, accomplishments, lessons learned, and

products produced through these two major grant initiatives. This report is based on data derived

from the states’ Final Reports, and agency (AoA) and resource center (NCOA) administrative

materials. The evaluation employed both quantitative (descriptive) and qualitative (content

analyses) methods. The report is organized to answer the key questions of interest to the agency

as outlined above.

The states, in general, exceeded the grant goals they had set for themselves. All states goals and

activities were consistent with the funding guidance and intent of the funding. The 24 states

reported supporting 21 total evidence-based health promotion and disease management programs

during the grant period. All states were expected to support the expansion of CDSMP program

capacity and offerings, and all states also provided more than just the CDSMP programs. The

most prevalent programs provided, besides the general CDSMP, included A Matter of Balance

(MoB) offered by 14 states and EnhanceFitness (EF) offered by 10 states. Almost all states

exceeded their goals for numbers of participants recruited into programs, and established

important infrastructure protocols and partnerships. All 24 states involved a working partnership

at the state level of aging services and public health departments for project leadership. Ten

states identified their state’s Medicaid program and six included their Aging and Disability

Resource Centers as key project partners. Four states identified their states’ Department of

Corrections as a key partner. Four states engaged Tribal Entities as key partners and eleven

states partnered with universities, primarily to provide evaluation expertise. The states excelled

at building partnerships across many community sectors to impact policy, provide programs,

provide referrals to programs, assure fidelity, and document outcomes.

Program outcomes examined in this report included, but were not limited to, number of program

completers and trainers/leaders, key partnerships developed, geographic coverage, the

development of a quality improvement/fidelity plan and a sustainable infrastructure. Across all

24 state grantees and 21 evidence-based programs offered, a total of 136,441 people were

reached. About 25% of states were “exemplary” in reaching their target population goal, 17%

“exceed” their goals for reaching their target population, 37% “met” their goals, 13% “fell short”

of their goals, and 8% did not provide information or did not specify a goal. The majority of

people reached were over the age of 65, female, about half lived alone and 68% were Caucasian,

13% African-American and 11% Hispanic/Latino. Seventeen states went beyond grant

5

expectations to provide participant level program outcome and health improvements post

programs, although typically follow up response rates were problematic. The most common

program interventions that produced participant outcomes were EnhanceFitness, CDSMP and A

Matter of Balance.

Notable impact was made in creating aging service and health partnerships, as 21% of grantees

proved “exemplary” in their accomplishments, Additionally, many states made significant

progress toward sustainability either by finding new funding, leveraging existing funding, or

developing or maintaining an infrastructure for offering programs and training leaders.

Additionally, 51% of state grantees made substantial progress in creating an infrastructure for

program delivery, referrals and registration. Eighty-eight percent of the States met or exceeded

their goals for maintaining fidelity within the programs they were offering and assuring quality

in all program implementation processes.

All states identified challenges, with the predominance of them in the areas of implementation

and sustainability. Because these programs are so highly relevant to addressing minority

populations’ health needs, states made this a priority focus. States found that their ability to

successfully recruit minority groups required a multifaceted approach: they needed to engage

peer group champions and local community leaders to support program marketing, they needed

to recruit leaders who are members of the participant groups they are trying to recruit into the

programs, they sometimes had to adapt the programs to make them more relevant, and also

needed to work with trusted agencies already serving these populations as partners and referral

agencies. The evaluation report also documents best practices, solutions, lessons learned and

exemplary case study states.

We used the RE-AIM framework to capture and categorize some 120 key lessons learned. From

the lessons learned, we identified a number of key recommendations for states to use in the

future. Within the “Reach” category, states suggested that its best to pick partners already

serving your target audience(s) to reach ethnic and underserved populations and to use GIS

mapping tools for expansion planning to identify current program locations and trainer

availability to indicate uncovered areas for development and expansion. Many states struggled

with rural transportation problems, and recommendations emerged to try ride sharing programs,

scheduling classes after other activities at times when people would already be at the site, and in

some cases, consider whether program leaders can pick up participants on their way to the site.

To assure “Effectiveness”, states recommended to get input and buy in for the scope and

requirements of any planned evaluation with the agencies you expect to participate in advance

and to assure the highest quality of leaders, set up a screening and/or interview protocol for

potential leaders before enrolling them in a leader training program.

In “Adoption”, it is best, when possible, to have a dedicated employee who specifically is in

charge of heath promotion programs within the AAAs; to use an organizational readiness tool

and spend time meeting with potential agencies before partnering; and to make sure staff at all

levels in organization understand the commitment they are making in offering the programs.

6

For “Implementation”, a key role at the state level continues to be the development and oversight

of protocols to foster fidelity monitoring and quality at all levels of program implementation.

Another recommendation is to use the Cost Calculator not only to identify program costs but to

also identify cost variations across programs and delivery sites/regions.

In the “Maintenance and Sustainability” category, key recommendations included the use of

business planning principles to approach sustainability and to diversify funding sources in

sustainability planning and to have a paid program coordinator to manage program logistics –

this is the best investment that can be made according to a number of states. Another innovative

recommendation was to structure partner contracts that are based on the number of completers by

grantee. In that way, there is a measurable outcome while incentivizing providers to engage

participants and yield a high number of completers per workshop. Two final recommendations

were to make sure that EBP is in the state plan for both aging and public health; and that having

good outcome data on participant health improvement and costs provides the basis for state

budget support and makes the business case for proposals for additional funding. The

commitment of state leadership and local coordinators is evident and a critical ingredient to

program success and sustainability.

The evaluation report includes a number of tables with detailed specific information and a series

of Appendices that provide the evaluation rubric tools, and complete lists of recommendations

and products and resources produced by the states.

7

II. INTRODUCTION

A. Background

1. PROGRAM PURPOSE AND GOALS

AoA funding support to states under the Evidence-Based Disease and Disability Prevention

programs has been provided to empower older adults to take control of their health. In these

programs, older adults learn to maintain a healthy lifestyle through increased self-efficacy and

self-management behaviors (www.aoa.gov).

A major expectation of the Empowering Older People initiative was to deliver high quality

evidence-based programs that maintain fidelity to both the original design and to the research

outcomes associated with the evidence-based models that are being deployed and to reach the

maximum number of older adults that are at risk who can benefit from the programs. The AoA

expectation reflected both the design and implementation of efficient and well-managed

programs, and the need to find and commit funds from other public and private sources to these

programs (as has occurred at the national level). By making these programs available in their

communities, older adults were being empowered to take control of their health. Programs

included:

Physical activity programs, such as EnhanceFitness or Healthy Moves for Aging Well, which provide safe and effective low-impact aerobic exercise, strength training, and

stretching.

Falls management programs such as A Matter of Balance, which addresses fear of falling, and Stepping On and Tai Chi: Moving for Better Balance, which build muscle strength

and improve balance to prevent falls.

Nutrition programs, such as Healthy Eating for Successful Living among Older Adults, which teaches older adults the value of choosing and eating healthy foods, and

maintaining an active lifestyle.

Depression and/or Substance Abuse Programs, such as PEARLS and Healthy IDEAS, which teach older adults how to manage their mild to moderate depression.

Medication Management Programs, such as HomeMeds.

Stanford University Chronic Disease Self-Management Programs (CDSMP), which are effective in helping people with chronic conditions change their behaviors, improve their

health status, and reduce their use of hospital services.

2. PROGRAM EVOLUTION OVER TIME

a. The Early Years

The programs funded in 2006-7 that are the focus of this report represent a major expansion of

the building blocks that had been put into place beginning in 2001. The programs began

modestly in 2001 with John A. Hartford Foundation support of four demonstration projects led

by the National Council on Aging to test the ability and interest of aging service organizations to

http://www.aoa.gov/

8

actually lead and sponsor evidence-based health promotion (EBHP) programs. Making EBHP

programs more accessible by placing them into community agencies was a major step, since

previously the programs were based in a small number of research-oriented universities and a

limited number of partnering organizations. In 2003 AoA funded 14 model projects that were

primed by community agencies and included a major evaluation component, typically provided

by a university partner. These model projects carefully documented the planning and

implementation process of offering the programs in aging service community agencies and

organizations while protecting the fidelity of the core components that made the programs

effective.

b. National Expansion and Systems Integration

Since 2003, AoA has supported states as they have developed infrastructure, workforce, and

capacity to deliver EBHPs through the aging services network and local partners (see Table 1).

The Empowering Older People initiative, described below, was by far the largest program

sponsored by AoA in support of the expansion of evidence-based programs and the integration of

them into the fabric of community program delivery to support the health improvements of older

people. From 2003 – 2012, AoA provided $23 million in funding for the Evidence-based

Disease and Disability Prevention Program (EBDDP) to support programs aimed at keeping

older adults healthy and engaged in their communities.

Table 1: Evolution of AoA-Funded EBHP Programs in the United States

• 2003: AoA model projects (14) served 5,000 people

– Programs included CDSMP, falls management, depression, physical activity, medication management,

and nutrition

– Documented fidelity and focus on evaluation

– Produced replication reports

• 2006: AoA “Empowering Older People” funded in 16 states

• 2007: “Empowering Older People” expands to 8 more states

• 2010: AoA ARRA Projects: 47 states/territories

c. Empowering Older People to Take More Control of Their Health through Evidence-Based

Prevention Programs (Empowering Older People)

In FY 2006, the Administration on Aging awarded cooperative agreement funding to 24 states to

support dissemination of evidence-based programs. These grants were designed to mobilize the

9

aging, public health, and non-profit networks at the State and local level to accelerate the

translation of HHS funded research into practice through the deployment of low-cost evidence-

based disease and disability prevention programs at the community level. An AoA goal for the

projects was for state units on aging and state health departments to collaborate on the provision

of policy leadership and on-going support for local partnerships involving non-profit aging

services providers, area agencies on aging, health organizations, the business sector, and other

potential partners from the private and public sectors.

d. Empowering Communities to Sustain Evidence-Based Disease and Disability Prevention

Programs (Empowering Communities)

In 2010, AoA issued a limited competition for one additional year of funding to the 24 states that

had been funded under the 2006 program. The new initiative was to support the continued

growth of partnership activities at both the state and community level. This opportunity allowed

further advancement of collaborations with state units on aging and state health departments on

the provision of health policy leadership, and the on-going strengthening of local partnerships

involving area agencies on aging, local departments of public health, non-profit aging services

providers, health and health insurance organizations, and other partners from the private and

public sectors.

3. APPROACH TO THE STUDY

This report will present information and data provided through the 24 funded states’ Final

Reports and related AoA and NCOA administrative and programmatic data. The state funded

programs were scheduled to end in May 2011, but most states (75%) received no-cost

extensions. The evaluators reviewed the original grant applications, final reports with their

extensive appendices, state profiles, and grant management reports as the primary data sources

for data extraction. Data extraction tools and evaluation rubrics were developed in six general

areas of inquiry listed below. The tools and rubrics are provided in Appendix A.

1. How well were project outcomes achieved? 2. What major challenges were encountered and what solutions for these challenges worked

best?

3. Taken together (across grantees) what was the major impact of the program and what lessons were identified that will assist future efforts by AoA in this program area?

4. What features supported states in their own formal program evaluation efforts? 5. What evidence is there that programs will be sustained or replicated? What program

features and/or partners support embedding the program into systems?

6. What types of resources and products were developed by the projects?

The data was available on a rolling basis as grantees completed their projects and subsequently

provided their final reports. One evaluator reviewed state materials for 10 states (Frank) and the

other evaluator reviewed the remaining 14 states (Lau). The two evaluators worked closely

together in completing the data extraction and met weekly to review the evaluation matrices and

10

discuss any issues that were identified. The two evaluators also completed four state data

extractions from each other’s state assignments to assure inter-rater reliability in score

assignment and use of the tools.

The data was then entered into an Excel database and included both rubric scores (quantitative)

and extensive qualitative notes within the six areas of inquiry. Analyses included descriptive

statistics (frequencies, means, and standard deviations), bivariate analyses and qualitative content

analyses that included both conceptual groupings, frequency counts, and the creation of

inventory lists (e.g. products list).

a. Study Limitations

The data presented in this report is derived from the written materials provided to the

Administration of Aging by the states and other AoA and NCOA administrative materials (e.g.

AoA grant monitoring reports, original state grant proposals). There is the potential that if the

final reports did not include all relevant information sought during the data extraction, this

missing information would result in an incomplete accounting of states’ accomplishments or

incorrect scores assigned within the rubrics. The quality and validity of the evaluation data was

dependent on the completeness and quality of the final reports and appendices provided to AoA

by the states.

4. DESCRIPTION OF EBHP PROGRAMS SUPPORTED

The 24 states reported supporting 21 total evidence-based health promotion and disease

management programs (Table 2 below) during the grant period. These programs can be

organized into several general categories: Stanford University chronic disease self-management

programs (CDSMP) (English and Spanish) both general and specialized (diabetes, pain,

arthritis); falls management programs; physical activity programs; behavioral health, medication

management and lifestyle improvement programs. All states were expected to support the

expansion of CDSMP program capacity and offerings, and all states also provided more than just

the CDSMP programs. The most prevalent programs provided besides the general CDSMP

included A Matter of Balance (MoB) offered by 14 states and EnhanceFitness (EF) offered by 10

states (see Table 2).

11

Table 2: EB Programs Offered by States During Grant Period

EB Program Number of States Offering Number of Participants

Trained by Program

Stanford Chronic Disease Self-Management Programs

Chronic Disease Self-Management Program 24 80,386*

Diabetes Self-Management Program 10 35,278*

Tomando Control de su Salud 5 9,889* **

Programa de Manejo Personal de la

Diabetes 1 0000

Arthritis Self-Management Program 1 3,177*

Chronic Pain Self-Management Program 1 0000

Falls Management

A Matter of Balance (MOB) 14 21,072

Tai Chi: Moving for Better Balance 4 1,937

Asunto de Equilibrio (Spanish MOB) 1 3,585

Step-by-Step 1 172

Stepping On 1 2,755

Physical Activity Programs

EnhanceFitness 10 11,320

Active Living Every Day 2 623

Fit & Strong! 2 94

Active Choices 1 24

Strong for Life 1 483

Healthy Moves for Aging Well 1 345

Behavioral Health Program

Healthy IDEAS 4 5,288

Medication Management

HomeMeds 3 5,672

Life Style Improvement Programs

Healthy Eating for Successful Living among

Older Adults 2 1,754**

EnhanceWellness 1 131

*States reported training numbers across multiple programs (e.g., CDSMP/Tomando/DSMP); exact numbers by

individual program undeterminable/may be duplicative

**Figure includes missing data, as some states omitted training numbers from reported programs

0000 = state(s) did not report training numbers for specified program

12

5. STATES IMPLEMENTING EVIDENCE-BASED HEALTH PROMOTION AND DISEASE

MANAGEMENT PROGRAMS (EBPS)

The 24 states that were funded in this initiative are identified in Figure 1 below. Each state

identified the geographic target areas (by region or county) in their grant proposal that they

planned to focus their expansion efforts within. The planned expansion coverage is depicted by

the blue color designation in the map. At the conclusion of the project, 50% of states (n = 12)

had exceeded planned geographic coverage, 29% (n = 7) met their geographic coverage goals,

and 21% (n = 5) of states fell short of meeting their geographic expansion plans.

Figure 1: State grantee reach by county

NOTE: Twenty-four grantee states’ targeted counties are shown in blue; unmarked states were not covered by this

grant

6. TYPES OF PARTNERS AND THEIR ROLES

Within the grant guidance, states were required to demonstrate that the projects would involve a

partnership between the state level aging services (e.g. State Unit on Aging) and the state level

health department (e.g. State Department of Public Health). There was also encouragement in

the grant guidance to include public health services funders (e.g. Medicaid) as a key partner. All

24 states involved a working partnership at the state level of aging services and public health

13

departments for project leadership. Ten states identified their state’s Medicaid program and six

included their Aging and Disability Resource Centers as key project partners. As shown in

Figure 2, all states relied on local AAA’s as key partners and developed community

organizations and local agencies as partners. The local organizations included health

departments, parks and recreation departments, senior housing, faith based communities, and

county and city service programs. Health care services providers (hospitals, clinics), health

districts, physician groups, and health plans were all included in the “health care” category

shown in Figure 2. Four states (Ohio, Oklahoma, Oregon, and South Carolina) identified their

states’ Department of Corrections as a key partner. Four states engaged Tribal Entities (Arizona,

Oklahoma, Oregon and Minnesota) as key partners and eleven states partnered with universities,

primarily to provide evaluation expertise. Three states (Florida, Massachusetts, and Michigan)

noted local foundations as key partners for their projects.

Figure 2: Key Project Partners

In almost all states, agencies and departments formed statewide collaborative networks for

oversight and to serve as steering committees for the projects. Several states established formal

statewide collaborative organizations. For example, Colorado formed the public-private

Consortium for Older Adult Wellness; Hawaii developed the Hawaii Healthy Aging Partnership;

and Massachusetts established the Massachusetts Disease Management Coalition. Texas

developed the Texas Strategic Health Partnership, whereas Arkansas established the CDSMP

Partners & Stakeholders’ Group. Wisconsin developed the Community-Academic Aging

Research Network and the Evidence-based Coordinating Community. Wisconsin was the only

state to establish a not-for-profit organization during the project, the Wisconsin Institute for

Healthy Aging (WIHA). The WIHA’s steering committee provided project oversight and also

applied for grant funding that was not available to governmental agencies. The states excelled

14

at building partnerships across many community sectors to impact policy, provide programs,

provide referrals to programs, assure fidelity, and document outcomes.

B. Program Outcomes

The evaluation assessed how well state grantees achieved the goals they set forth in their original

grant proposals. It also documents how well the programs retained their original design and

were aligned with AoA’s overall goals for the initiative. Program outcomes examined in this

grant included, but were not limited to, number of program completers and trainers/leaders, key

partnerships developed, geographic coverage, the development of a quality improvement/fidelity

plan and a sustainable infrastructure. For a complete list of the constructs that were assessed, see

Appendix A, Tables 1 and 3.

1. PERSONS SERVED

Across all 24 state grantees and 21 evidence-based programs offered, a total of 136,441 people

were reached. Among the most popular programs offered were the Chronic Disease Self-

Management Program (CDSMP), A Matter of Balance (MOB), the Diabetes Self-Management

Program, and EnhanceFitness, the four of which, combined, served 113,877 people in total. For

each of the programs offered, grantees usually set goals for their targeted number of program

participants. As noted in Table 3, 42% of grants were “exemplary” in achieving their participant

goals, 29% “exceeded” their goals, 4% “met” their goals, 8% “fell short” of their goals, and 17%

did not provide outcome information or did not designate a goal.

Table 3: How Well Did the State Achieve Its Program Outcomes?

Did not provide

information

or no goal (Score = 0)

FELL

SHORT of

achieving

outcome

goals (Score = 1)

MET

outcome

goals (Score = 2)

EXCEEDED

outcome

goals (Score = 3)

EXEMPLARY

in achieving

outcome goals

(Score = 4)

TOTAL (N = 24)

Program Completers 17% (n=4) 8% (n=2) 4% (n=1) 29% (n=7) 42% (n=10) 24

# of Trainers/Leaders 42% (n=10) 4% (n=1) 17% (n=4) 25% (n=6) 13% (n=3) 24

Key Partnerships 0% (n=0) 0% (n=0) 42% (n=10) 42% (n=10) 17% (n=4) 24

Geographic

Coverage/Target

Population 8% (n=2) 8% (n=2) 37% (n=9) 17% (n=4) 25% (n=6) 24

Aging/Public Health

Leadership 4% (n=1) 0% (n=0) 58% (n=14) 21% (n=5) 17% (n=4) 24

Quality

Improvement/Fidelity

Plan 8% (n=2) 4% (n=1) 50% (n=12) 4% (n=1) 34% (n=8) 24

Sustainable

Infrastructure 4% (n=1) 4% (n=1) 29% (n=7) 34% (n=8) 29% (n=7) 24

15

Table 3 also indicates that 25% of states were “exemplary” in reaching their target population,

17% “exceed” their goals for reaching their target population, 37% “met” their goals, 8% “fell

short” of their goals, and 8% did not provide information or did not specify a goal. State

grantees that fell short of reaching their target population were Maryland and Oklahoma. The

Maryland grant noted that delays in contracting processes resulted in no workshops being offered

in one area. Finally, Oklahoma’s report stated a goal of establishing 100+ permanent program

sites. However, their state profile showed that they fell a little short of that goal with only 90

workshop sites (which is still a substantial accomplishment). Those that did not provide

information or did not specify a goal were Arkansas and Oregon.

Table 4 below presents demographic data of program participants across all 24 states through

five grant years. Note that completion of the demographic data form was not mandatory, so data

is only provided for the 80,067 participants who provided this information.

Table 4: Participant Demographics (N = 80,067)

Demographic Construct 5-Year Total % of Known Statistics

Age

Under 60 5,328 9%

16

60-64 3,796 7%

65-69 6,589 12%

70-74 8,405 15%

75-79 9,166 16%

80-84 9,766 17%

85-89 8,122 14%

90 and Over 5,078 9%

Unknown 23,817 30% (% of total)

Gender

Female 46,465 79%

Male 12,097 21%

Unknown 21,505 27% (% of total)

Living Arrangement

Living Alone 26,455 50%

Living With Someone 26,166 50%

Unknown 27,446 34% (% of total)

Race/Ethnicity

Native American 928 2%

Asian 1,747 3%

Black 7,027 13%

Pacific Islander 194 0%

Hispanic/Latino 6,232 11%

White 38,109 68%

Other Race 585 1%

Multi-Racial 1,291 2%

Unknown 24,040 30% (% of total)

Source of data: The National Council on Aging (NCOA)

2. OUTCOMES OF INTERVENTIONS ON PROGRAM PARTICIPANTS

Seventeen grants conducted extensive evaluations of participant level outcomes in the programs

they offered. While the majority of these efforts centered on measurements such as health

outcomes, physical activity, and hospital readmissions, a few states chose to focus on program

satisfaction and fidelity. These states included: Illinois, which focused on participant and leader

satisfaction; Michigan, which conducted an extensive fidelity study in partnership with Michigan

State University; and Texas, which performed an evaluation of program implementation

processes across sites using the RE-AIM framework. This section, however, will focus on those

state grantees that provided participant level health and wellness outcomes as a product of

participating in the evidence-based program interventions. These states included: Arizona,

Connecticut, Hawaii, Idaho, Iowa, Maine, Massachusetts, New Jersey, New York, Ohio,

Oklahoma, Oregon, South Carolina, and Wisconsin. States with exemplar evaluation efforts are

discussed below.

17

The most common program interventions that produced participant outcomes were

EnhanceFitness, CDSMP and A Matter of Balance. States offering EnhanceFitness used tools

such as the chair stand (lower body strength), arm curl (upper body strength), and timed Up and

Go (transfer ability and risk of falls) tests to quantify participants’ health improvements while

enrolled in the program. Both Arizona and Hawaii reported that participants improved at an

average rate of 18% for the number of chair stands they could perform in 30 seconds, 22% for

the number of arm curls they could do in 30 seconds, and 11% for the length of time in seconds

it took to complete the Up and Go assessment.

States that reported evaluation data for participants of CDSMP utilized survey instruments to

measure constructs pre- and post-program such as:

Participant self-rated health

Number of times participants discussed health conditions with their doctors

Health care utilization (e.g., physician visits, emergency department visits, nights of hospitalization)

Fatigue, and

Pain

Iowa, as a key example, reported results at baseline, six months, and one year post-program for

all constructs measured except pain and fatigue, which were reported at baseline and one year

post-program. See Table 5 below.

Table 5: Changes in Health Outcomes for Participants of Iowa’s CDSMP

Baseline 6 Months Post- 1 Year Post-

Self-Rated Health 3.08 3.01 3.21

Chronic Disease Self-

Efficacy 5.70 6.40 6.60

Pain 4.80 -- 4.60

Fatigue 5.03 -- 4.78

Emergency Room

Visits 1.44 0.72 0.35

Nights of

Hospitalization 7.62 4.56 1.88

These outcomes were instrumental in estimating the average health care cost savings for

participating in CDSMP. For example, Iowa estimated a one-year savings of $76,204 based on

the average charge per visit of $506.

Oregon reported similar results, with reduced emergency department visits from 0.8 to 0.7 visits

per year, hospitalizations from 0.4 to 0.3 visits per year, and hospital days from 2.4 to 1.9 days

per year. For the participants who have completed Oregon’s Living Well (CDSMP) program to

date, this translates to 557 fewer emergency department (ED) visits, 557 fewer hospitalizations,

18

2,783 fewer hospital days, and a savings of $634,980 in ED visits, and $6,501,088 in hospital

stays.

Finally, Massachusetts served as a prime example of a state that assessed the effectiveness of A

Matter of Balance. Survey tools were administered at the first (week one) and last (week eight)

classes. The survey tool measured outcomes in three key areas: 1) fall management; 2) fall

efficacy; and 3) fall control. Key findings showed that, as a result of completing the A Matter of

Balance class, 95% of participant responders noted they are more comfortable talking about their

fear of falling, 96% feel more comfortable increasing activity, 90% plan to continue exercising,

and 92% would recommend the program to other older adults.

3. NATIONAL PROGRAM IMPACTS

To examine the impacts of the programs on both the state and national levels, a rubric was

developed to score state successes as they “engaged state leadership in systems level strategic

planning,” “created aging service and health partnerships,” “reached rural, minority, or

underserved populations,” “increased capacity of local agencies to deliver EB programs,” “made

progress toward sustainability/funding,” “expanded geographic reach,” “created infrastructure

for program delivery, referrals and registration,” “aligned their goals, achievements and

successes with AoA,” “offered more programs than CDSMP,” and “measured outcomes.” Table

6 below provides a snapshot of the national impact made across the 10 abovementioned factors.

Table 6: What Impact Did the State’s Project Achieve?

No information

Limited

(Score = 1) Moderate

(Score = 2) Major

(Score = 3) Exemplary

(Score = 4) TOTAL (N = 24)

19

provided or

not done

(Score = 0)

Engaged state leadership

in systems level strategic

planning 4% (n=1) 13% (n=3) 33% (n=8) 25% (n=6) 25% (n=6) 24

Created aging service +

health partnerships 0% (n=0) 0% (n=0) 8% (n=2) 71% (n=17) 21% (n=5) 24

Reached rural, minority

or underserved

populations 13% (n=3) 38% (n=9) 25% (n=6) 16% (n=4) 8% (n=2) 24

Increased capacity of local

agencies to deliver EB

programs 0% (n=0) 4% (n=1) 38% (n=9) 50% (n=12) 8% (n=2) 24

Progress toward

sustainability/funding 0% (n=0) 0% (n=0) 33% (n=8) 38% (n=9) 29% (n=7) 24

Expanded geographic

reach 8% (n=2) 13% (n=3) 29% (n=7) 29% (n=7) 21% (n=5) 24

Created infrastructure for

program delivery,

referrals and registration 0% (n=0) 8% (n=2) 41% (n=10) 38% (n=9) 13% (n=3) 24

Goals, achievements and

successes aligned with

AoA 0% (n=0) 0% (n=0) 13% (n=3) 62% (n=15) 25% (n=6) 24

Offered more programs

than CDSMP 0% (n=0) 17% (n=4) 29% (n=7) 33% (n=8) 21% (n=5) 24

Notable impact was made in creating aging service and health partnerships, as 21% of grantees

proved “exemplary” in their accomplishments, 71% made a “major” impact and 8% made a

“moderate” impact. Key examples of partnerships are given in Figure 2 under the section,

“Types of Partners and their Roles.” Additionally, many states made significant progress toward

sustainability either by finding new funding, leveraging existing funding, or developing or

maintaining an infrastructure for offering programs and training leaders. 29 percent of states

were “exemplary” in their progress toward sustainability/funding, 38% made “major” progress,

and 33% made “moderate” progress.

Additionally, Table 6 shows that 13% of state grantees were “exemplary” in creating an

infrastructure for program delivery, referrals and registration, 38% made a “major” impact, 41%

made a “moderate” impact and 8% made a “limited” impact. Those scoring “exemplary” in this

category included: California (as mentioned above), Colorado and Hawaii.

4. PROGRAM FIDELITY AND QUALITY ASSURANCE

20

To evaluate the degree to which states maintained fidelity within the programs they were

offering and assured quality in all program implementation processes, raters focused on how well

states developed and carried out a quality improvement/fidelity plan and created an infrastructure

for program delivery, referrals and registration. Stellar state examples mentioned fidelity

monitoring processes embedded into their infrastructure to ensure that fidelity was kept

throughout the duration of the program. California, for example, created a Program Office and a

Steering Committee to facilitate program offerings across California and to ensure fidelity and

data collection.

Table 3 above shows 34% of state grantees were “exemplary” in developing and implementing a

quality improvement/fidelity plan, 4% “exceeded” their outcome goals in doing so, 50% “met”

their outcome goals, 4% “fell short,” and 8% did not provide information or did not specify a

goal. States that received “exemplary” marks included: Colorado, Hawaii, Massachusetts,

Michigan, New York, Oregon, South Carolina, and Texas. Colorado, for example, established

the Healthy Aging Service System for training, technical assistance, and evaluation/fidelity

checks. Using this system, they conducted 227 fidelity visits during the grant period. In Oregon,

grantees developed a statewide Living Well Quality Assurance and Fidelity workgroup, provided

fidelity tools, and observed 70% of all Living Well workshops over the course of a year.

5. BEST PRACTICES

Through analysis using the aforementioned scoring rubrics, best practices for implementing the

evidence-based programs emerged as state grantees received “exemplary” scores in project

outcome achievement and program impact. Specific attention was paid to best practices in the

areas of marketing/outreach, worker training, infrastructure development, fidelity, and quality

assurance and evaluation.

a. Marketing/Outreach

In order to determine best practices relating to marketing/outreach, states were scored on their

accomplishments in developing key partnerships, reaching rural populations and expanding their

geographic reach.

Four states (California, Colorado, Connecticut, and Ohio) received “exemplary” marks in key

partnerships. In California, over 70 health care organizations have invested in CDSMP and are

offering it internally, including 22 Kaiser sites, 17 physician groups and clinics, 12 Dignity

Health (formerly Catholic Healthcare West) hospitals and medical centers, five health care

districts, and three health plans. The grant has also contributed to a new collaboration between

the California Department of Aging and the California Department of Public Health, and at the

local level between sixteen local health departments and AAAs serving those counties.

The Connecticut Department of Social Services, Aging Services Division (DSS) and the

Connecticut Department of Public Health (DPH) served as the key partnership for implementing

all of the state’s programs. To implement CDSMP, they spearheaded an advisory council

comprised of their technical assistance consultant, project evaluator from the UCONN Center on

21

Aging, local partners from the Hartford Community including the City of Hartford Health and

Human Services and the City of Hartford Elderly Services, and the North Central AAA.

Two states (Massachusetts and Texas) were “exemplary” in reaching rural populations. Hebrew

Senior Life, a key partner in the state of Massachusetts, made focused efforts to reach diverse

populations, starting with a presentation at the Harvard Multicultural Coalition Annual Aging

Well Together Conference in 2008. At that conference, abbreviated workshops were presented

in several languages: English, Spanish, Portuguese, Chinese, Vietnamese, Haitian Creole, and

Cape Verdean Creole. A grant from the Tufts Health Plan Foundation in 2009 allowed for the

translation and piloting of the program into Spanish and Vietnamese. Translations of the

program into Haitian Creole and Portuguese are currently being conducted in collaboration with

the Cambridge Health Alliance (Haitian Creole) and the Visiting Nurse Association

(Portuguese).

Lastly, five states (California, Colorado, Florida, New Jersey, and Texas) were “exemplary” in

expanding their geographic reach. In Florida and New Jersey, programs are now being offered

in 100% of their counties, and in Texas program offerings increased from 10 counties to 58

counties over the course of this grant.

b. Worker Training

In order to increase capacity to deliver the programs, a number of states identified goals of

increasing their numbers of master trainers and lay leaders available to offer the programs. For

those states with this type of stated goal, we rated them using the rubric that ranged from “fell

short” in meeting goal, met goal, exceeded goal and exemplary in meeting their goal. It is

possible that for states who far exceeded their goal that they were creating excess capacity in

trainer availability, but we could not discern this from available data. What seemed to be

happening was that states needed two major things to increase the spread and number of

programs being offered. First, they needed the trained personnel to lead the classes; and second,

they needed the agencies being ready and willing to sponsor the programs. Sometimes it

appeared that one of these factors lagged behind the other, but oftentimes they were in sync.

For best practices in worker training, states were assigned scores for their achievements in the

number of trainers/leaders trained, and their ability to increase the capacity of local agencies to

deliver the evidence-based programs. Three states were “exemplary” in their number of

trainers/leaders by the end of the grant. These states were Colorado, Maryland, and

Massachusetts. Colorado stood out with two T-Trainers, 62 Master Trainers and 363 Lay

Leaders, and Massachusetts boasted stellar results by increasing their numbers of Master

Trainers and Lay Leaders in CDSMP, Healthy Eating, and A Matter of Balance. In 2011 alone,

the number of CDMSP Master Trainers in Massachusetts grew from 76 to 152, Lay Leaders

from 101 to 350, Healthy Eating Master Trainers from 5 to 86, Lay Leaders from 17 to 221, and

A Matter of Balance Master Trainers from 74 to 116 and Lay Leaders from 143 to 450.

Additionally, two states were “exemplary” in their ability to increase the capacity of their local

agencies to offer EB programs: California and Colorado. In Colorado, 23 partners provided

22

programs in 17 counties, and a Colorado Health Foundation grant supported program

implementation and infrastructure development.

c. Infrastructure Development

Development of a strong infrastructure was a key element to ensuring that agencies were able to

systematically recruit participants, deliver programs, and monitor fidelity through the provision

of technical assistance and data collection. The strength of this infrastructure was also telling in

state grantees’ ability to sustain their efforts beyond the duration of the grant. To look at best

practices in this area, raters assigned states scores focused on their achievement in building a

sustainable infrastructure, the aging service and health partnerships they created, and their

engagement of state leadership in systems level planning.

Seven states were “exemplary” in achieving a sustainable infrastructure: California, Colorado,

Connecticut, Hawaii, Massachusetts, New Jersey, and Ohio. All of these states made significant

headway in building and sustaining an infrastructure for offering EB programs. A few of them

are discussed below.

California’s development of a project office and statewide steering committee provided a central hub for guiding program implementation and facilitating relationships with

various organizations/entities.

In Colorado, a partnership was built with three health care systems and a state ADRC for referrals. Emphasis was placed on embedding programs into systems where they could

be sustained, and using Older American’s Act funding to support AAA programming.

In Connecticut, the Yale Connecticut Collaboration for Fall Prevention and the State Commission on Aging secured funding through the Connecticut State Legislature to

provide fall prevention training sessions to a larger network of home care and hospital

based clinicians in both regions and other parts of the state that were not covered by the

grant. In a September 2009 special legislative session, this statewide falls prevention

initiative became Public Act 09-5 and continues to receive yearly funding from the state

legislature.

In New Jersey, the statewide infrastructure for CDSMP is based on shared administration/oversight by state government and local implementation by community

partners. State level personnel costs will continue to be paid through other federal and

state funding streams.

Five states were “exemplary” in creating quality aging service and health partnerships to aid in

the process of developing a strong infrastructure. These states included: California, Connecticut,

Hawaii, Massachusetts, and Ohio. California and Connecticut’s stellar partnerships were

discussed in the “Marketing/Outreach” section above. Additionally, Hawaii created the Hawaii

Healthy Aging Partnership (HHAP), comprised of 63 partner organizations and 27 volunteers.

In Massachusetts, implementation of EBDPs is the top program and policy initiative for the

Councils on Aging and is actively being promoted by the Massachusetts Association of Councils

on Aging. A coalition of leaders from the Tufts Health Plan Foundation and the Massachusetts

Health Policy Forum formed the Healthy Aging Steering Committee to examine community-

23

based, environmental, and systematic approaches to promote healthy living and healthy aging.

Key partners in Ohio included their 12 AAAs, senior centers, behavior health organizations,

local/county health departments, meal providers, senior housing, neighborhood health centers,

health care providers, faith-based organizations, PASSPORT, hospitals, corrections facilities,

regional Alzheimer’s Association chapters, and many more.

d. Fidelity and Quality Assurance

The importance of maintaining fidelity to the programs as designed and incorporating quality

assurance measures was discussed in the “Program Fidelity and Quality Assurance” section

above. As mentioned, states received scores around their achievement of developing and

implementing a quality improvement/fidelity plan. Eight states received “exemplary” marks:

Colorado, Hawaii, Massachusetts, Michigan, New York, Oregon, South Carolina, and Texas.

For Massachusetts, in addition to recruiting and coordinating leader trainings in their three EBDP

programs, the lead community partners have provided mentoring, technical assistance (including

the development of websites for all programs), and support to newly trained program facilitators.

In addition, opportunities for continuing education and sharing of best practices have also been

available to leaders. Texas A&M, the university evaluators for the state’s grant, offered training

and technical support continuously, troubleshot programs for concerns or issues, and provided

remedial training and support to address challenges.

e. Evaluation

Finally, as discussed in the “Outcomes of Interventions on Program Participants” section above,

17 states provided extensive data on the effectiveness of the programs for participant health and

physical activity outcomes. Scoring to determine “exemplary” states was not done for this

construct since it was not an expectation of the funding initiative. Instead, qualitative data was

pulled to examine each of the 17 states’ efforts and individual outcomes. Premier examples of

evaluation efforts by those states are discussed above. It should be noted that the majority of the

states that provided these extensive evaluation studies included a university partner in their

project that managed the participant level evaluation studies.

In addition to those 17 states, two others emerged as key examples of strong evaluation efforts as

an analysis of solutions to remedy challenges was done. Towson University, the evaluation

partner in the Maryland grant, performed key evaluations in 2009 and 2010 to monitor the

fidelity of the program. Maryland implemented a fidelity consultant training in 2010-2011 with

each grantee, requiring the completion of a written fidelity plan. In Illinois, a robust evaluation

and fidelity monitoring of the CDSMP and Strong for Life programs was completed. The

evaluation monitored class participants, class leaders, and key informants (four agency

directors).

C. Challenges

24

In their final reports, States identified a number of challenges that they faced in executing their

plans and accomplishing activities to reach their goals. These challenges included marketing and

outreach, worker training, infrastructure development, fidelity and quality assurance, evaluation,

and other issues such as personnel changes, delays in hiring, and contract execution with

partners.

1. MARKETING/OUTREACH

Challenges in outreach and recruitment of both program participants and leaders/trainers were

common, being reported by over 70% of the states. This category of challenges represented

about 25% of types of all challenges reported.

a. Rural Issues

Many states’ challenges in this area were related to the rural nature of the state. Rural program

delivery was challenged because the scarcity of population made it difficult to recruit enough

people to be trained as leaders and also to recruit enough people to fill the classes. Arkansas

found that rural program delivery challenges made it difficult to accomplish their goal of “30-30”

– having a class available within 30 days and within 30 miles of any participant who desired to

enroll in a class. They employed an innovative GIS mapping tool to identify the geographic

distribution of leaders and implementation sites, thus revealing the areas of need for further site

development. They blanketed small rural communities’ senior centers, faith-based communities,

and other strategic locations with information on the evidence-based programs through radio,

free television spots, flyers, newsletters, and local newspapers. Through partnering with the

AAAs and the Arkansas Aging Initiative they were able to recruit local leaders and provide

programs in the geographic areas that represented locations for almost 90% of older adults in the

state. Michigan also used a GIS mapping tool effectively to document available leader locations

in order to target leader training programs in needed areas. Other states focused on developing

local champions to be trained as leaders and to help recruit members of their community to

classes.

b. Transportation

Transportation was also noted as a challenge, especially in rural areas, and frequently was the

reason for the difficulty in recruiting leaders and recruiting and retaining program participants.

Leaders were typically clustered in more densely populated areas and a number of states found it

difficult to address the challenge of leaders traveling to the more rural areas to deliver programs.

Participant travel to the programs was also an issue. Some older adults no longer drove or had

limited personal transportation access. Those residing in rural areas often had limited public

transportation options to travel to the program locations. Local agencies were very imaginative

in addressing transportation issues: they found funding to provide a transportation stipend to

leaders, they scheduled programs at leaders’ convenience, they assisted with carpooling, van

25

pickup of participants; and in some cases, dedicated leaders picked up participants on their way

to the programs.

c. Program Characteristics and Requirements

Other challenges in marketing and outreach had more to do with the programs themselves.

States reported that the falls management programs (e.g. A Matter of Balance) and physical

activity programs (like EnhanceFitness) seemed easier to recruit people to because participants

understood the goals of the programs and could identify how participating would address a

perceived risk or prevent a problem, whereas the Chronic Disease Self-Management programs

(CDSMP) were often a “harder sell.” On the face of it, participants often felt they were

managing just fine, and did not really understand the potential program benefits. States typically

changed the name of the CDSMP to one that was more upbeat and attractive: in California, the

program is called Healthier Living; whereas in Oregon and several other states it was known as

Living Well.

Participant retention problems for CDSMP were often addressed by adding a “class zero” for

orientation so that participants would understand more about the goals, benefits, and structure of

the program (six meetings) prior to signing up. Several states also addressed retaining

participants with an incentive such as Maryland did, by providing a book of county ride tickets

valued at $15. A very common problem in participant retention related to the characteristics of

the target audience for CDSMP – those with chronic illnesses. Participants’ health problems

often interfered with their ability to attend programs. The CDSMP curricula itself discusses

these issues and helps its participants to anticipate this potential problem in program attendance.

The expectation of a “completer” being a person who attends at least four of the six sessions also

recognizes the issue. It is an expectation that both the “buddy system” the program employs and

the expectation that leaders call participants after a missed session also addresses this challenge.

d. Outreach to Minority and Underserved Populations

26

The evidence-based health promotion and disease management programs are especially relevant

for ethnic, racial minorities and low-income populations due their high chronic disease

prevalence. States sought to specifically target these underserved groups for their programs and

often faced challenges in their efforts to do so. Often recruitment was a concern – these groups

have many life challenges and becoming involved in programs such as these may not be a

priority. They may question the relevance of the programs to their own life situations. States

found that their ability to successfully recruit these groups required a multifaceted approach:

they needed to engage peer group champions and local community leaders to support program

marketing, they needed to recruit leaders who are members of the participant groups they are

trying to recruit into the programs, they sometimes had to adapt the programs to make them more

relevant, and also needed to work with trusted agencies already serving these populations as

partners and referral agencies. Recruiting Native Americans by working with Tribal entities

typically took longer for the Tribal permission process and to engage agency personnel to be

recruited as leaders so they could provide the programs to their constituencies.

Many states selected key partners specifically to assist in recruiting participants for programs,

especially those from minority and underserved low-income populations. States who partnered

with Medicaid programs found these to be very effective referral sources. Several states,

including Colorado, Maine, Maryland, and New Jersey, noted how well their state’s Aging and

Disability Resource Center worked with them in making participant referrals. Minnesota

changed partners to work with Wisdom Steps, a preventive health organization that provides

services to all tribes in the state. By doing so, they were able to recruit and train Tribal leaders in

both A Matter of Balance and CDSMP and offer multiple sessions in both programs. By taking

the extra time to find the best partner and invest in the Tribes’ own capacity to provide its

members with programs, Minnesota has established the foundation to expand programs to

address the Tribal community needs.

2. WORKER TRAINING

State grantees in the Empowering People and Communities initiatives were tasked to expand the

capacity, geographic reach, and sustainability of evidence-based health promotion and disease

management programs. To accomplish this, adequate workforce capacity for trainers, leaders,

and program coordination staff was essential. Because the Empowering People and

Communities initiatives were building on previous years of program support in many states,

existing trainer and leader capacity varied. Thus the individual state goals and activities

regarding worker training were consistent with identified needs and expansion plan

requirements.

a. Trainer and Leader Recruitment

About 75% of all states experienced a common set of challenges centered around recruiting

enough of the “right people” as trainers and leaders and getting them started in leading the

programs. The issue of having enough people to be trained as leaders was often related to small

27

numbers of people living in rural regions (discussed above) and also related to having sufficient

numbers of people from underserved communities to model, lead, and relate to the target

audience of minority and low income populations. The “right people” to lead programs and train

others are those dedicated to the work, who believe in the mission and benefits of these

programs, and make the time to continue leading programs over time.

A number of states set up application and screening processes to make sure that people recruited

as potential leaders were suitable, committed, and available prior to making the investment in

their training. California was one of the states to identify these problems and they found that

selecting committed people was enhanced through two tools that the California Department of

Aging developed: 1) a website survey (SurveyMonkey) to assess prospective leaders’ readiness

and commitment level, and 2) a Leader Agreement stating that leaders would facilitate a

minimum of two workshops per year. In Illinois, staff employed multiple strategies for leader

recruitment. Strategies developed included on-site presentations at locations where potential lay

leaders work, and to volunteer organizations and community groups. A list of leader

responsibilities was developed so potential leaders understood the commitment. Peer leader

pairing was also established, and class leaders continued to identify former class participants as

potential class leaders as well.

Partnership selection also was identified as important to provide sufficient numbers from

minority affiliated organizations and communities. For example, in Oklahoma a key state level

partner is the Health Equity Resource Opportunity Network, an organization dedicated to

addressing issues of health disparities and providing services to underserved minority

populations. Other states recruited personnel from federally qualified community health centers

as leaders to serve their populations at the centers. In the four states actively targeting Native

Americans (Arizona, Minnesota, Oklahoma, and Oregon), Tribal entities were important partners

for both leader and participant recruitment.

For some programs, such as EnhanceFitness, the requirements to become a program leader were

more stringent. The more qualifications required to be a leader/trainer, the more challenging it

was to secure an adequate number of leaders. States addressed this particular challenge in

several ways. Some states, such as Arizona, had to limit program availability and expansion

opportunities due to the cost and availability of the certified fitness instructors who are required

to lead EnhanceFitness. Other states, such as Hawaii, utilized their agency networks to identify

and recruit people to become certified instructors to lead the program. Texas developed their

own fitness instructor training program to pipeline trainers into EnhanceFitness. Unfortunately,

Oregon noted that it replaced the program with an arthritis physical activity program because it

was less expensive and had less stringent leader qualification requirements.

b. Trainer and Leader Engagement and Retention

All states discussed the challenge of having a number of people trained to lead programs who

were not actively leading programs. A tremendous amount of effort was expended in getting

leaders scheduled to do the programs, and continuing to do the programs over time. States found

that leaders needed to be engaged immediately after being trained. California coordinators try to

28

schedule workshops within three weeks after leader trainings and pair new leaders with

experienced ones to increase the leaders’ comfort level and dedication. Centralized scheduling

systems like the ones developed by Colorado were very helpful in keeping the available leaders

actively leading programs.

Depending on whether the majority of leaders were volunteers or paid agency staff, different

challenges presented themselves. For volunteers, keeping people active and engaged was related

to the benefits they perceived from these activities. Recognition events, such as the luncheons

sponsored by Illinois to thank and honor volunteers, were very important in volunteer leader

retention. Travel reimbursements or small stipends provided to volunteers helped them to

continue their work.

North Carolina reported that leader turnover continued to be a challenge in EBHP development

with turnover averaging approximately 25-40% depending upon the program. They found it

incredibly time consuming to recruit and train leaders and also reported that it was a costly drain

of financial resources for programs with limited funding. They addressed this issue by having

the EBHP coordinator modify, develop, and increase screening tools for new leaders and spend

more time interviewing and discussing EBHP with prospective new leaders so that they knew

what to expect prior to certification. Other tools such as a memorandum of agreement with both

the leader and the organization they worked for (if not a volunteer from the community) were

helpful in communicating guidelines, designating resources, and defining expectations.

For paid agency personnel who were trained as leaders, often this was one more job to do for an

already busy person. As Title III-D funding began to be used to support programs and these

activities continue to become an expected part of the job, agency personnel will be more

effective in providing programs. The expansion and integration goals of the Empowering People

and Communities initiatives were nothing short of an organizational culture change mandate for

many community-based organizations. Embedding the programs into the fabric of the

organizations by having their personnel trained to lead and manage the programs was a goal in

almost every state. However, when state budget cuts or program budget redirection occurred,

this jeopardized the availability of paid staff to continue to do the programs (discussed in more

detail below under sustainability). Systems transformation is a slow process, but having a

qualified work force that sees this work as an important part of their job is essential. Integrating

the value and need for the programs into the agency’s protocols enhances staff buy-in and

solidifies role expectations. In Connecticut, when agency personnel found it difficult to

incorporate falls prevention activities into their work, the agency’s intake and follow-up tools

were modified to include falls prevention automatically, cross cutting agency fall prevention

committees were developed, and fall prevention was included as a part of orientation for all new

staff.

3. INFRASTRUCTURE DEVELOPMENT

There were two major challenges identified by the states regarding effective infrastructure

development. The biggest one was leveraging funding and helping to expand systems and

agencies to provide and manage the programs. A second challenge was identifying the partners

29

and getting them engaged in the mission. As noted in the introduction section of this report, the

current grant projects were building on previously funded projects that also focused on

programmatic infrastructure development. In addition, all states were required to mount their

projects using an aging services + public health partnership. By design, this identified the two

key partners who would anchor the infrastructure and lead the development efforts.

Due to previous funded projects that initiated the beginning steps of building the infrastructure,

almost all states began the grant period with some form of centralized coordination by staff at

state level agencies. A number of states were able to develop centralized resources to support

further infrastructure development. North Carolina utilized a centralized coordination approach

so that all regions of the state have access to consistent templates for various resources and

materials, as well as a database and leader tracking system. Maine state partners provided

financial resources, technical assistance, and a state business plan draft to serve as a framework

for local plans. Larger local and regional organizations obtained licenses to deliver CDSMP and

the state secured a multi-site, multi-program license to assist partners with licensure costs.

As local adoption and capacity was expanded, states often retained a centralized technical

assistance and data collection role, and made statewide resources available through their website.

Program coordination, fidelity monitoring, and program delivery was slowly transferred to the

local AAA’s and other community partners. New Jersey is a prime example, with their statewide

infrastructure for CDSMP based on shared administration/oversight within state government and

local implementation by community partners. State level personnel costs were covered by other

funding, with the bulk of New Jersey’s grant dedicated to providing seed money to local

agencies to establish local infrastructure for program delivery. Texas created the Texas Strategic

Health Partnership and reported that the state health department’s role was that of technical advisor and referral agent to local programs.

The states’ infrastructure to deliver the programs slowly expanded like a social network through

partnership building and seed funding. Bringing needed partners onboard was very time

consuming and required many meetings, strategic alliances, memorandums of understanding,

policy incentives, and tremendous state leadership. Reluctant partners were won over through

education about the programs’ value, demonstration of return on investment, and state funding

policies to provide public funding for programs. Most states started with small geographic

regions that grew over time to programs being available in the majority of the state. In Texas,

they expanded programs in three regions from 10 counties to 58, reaching older Hispanic and

rural populations with low-incomes and high diabetes rates. In Oregon, 29 of 36 counties have

Living Well or Tomando classes available now. The Ohio Department of Aging partners with all

12 of their AAAs to disseminate EB programs. Their initial goal was to offer programs in six

AAA regions, but with additional ARRA funding they were able to expand into the remaining

six so all are now involved. California’s infrastructure grew from an initial target of seven

counties to 32 counties; and New Jersey, Florida, and Maine report full state coverage engaging

all AAA’s and all counties. State partnerships, as shown in Table 2, represent all community

sectors: public and private service organizations, housing, faith-based communities, education,

health care, business, civic groups, and others.

30

4. FIDELITY AND QUALITY ASSURANCE

Protecting the fidelity of the EB programs is paramount in assuring that the potential beneficial

outcomes are protected. Fidelity monitoring and quality assurance activities are an essential

project activity for EB program delivery. For CDSMP, there are extensive fidelity tools (e.g.

check lists), and fidelity manuals and tool kits have been produced by Stanford and several states

including New York, Idaho, Colorado, Maine, Michigan, Oregon, Texas, and Wisconsin. A

number of states reported well thought out protocols and activities to protect fidelity. For

example, Massachusetts reported that in addition to recruiting and coordinating leader trainings

in all programs, the lead community partners provided mentoring, technical assistance (including

the development of websites for all programs), and support to newly trained leaders/coaches and

trainers. In addition, opportunities for continuing education and sharing of best practices have

also been available to leaders. Colorado established the Healthy Aging Service System for

training, technical assistance, and evaluation/fidelity checks. They conducted 227 fidelity visits

during the grant period and established Senior Assistant and Management System (SAMS).

During this grant period, very few states reported challenges in completing needed fidelity

activities. South Carolina noted the problem of not have enough well-trained staff at new

agencies to do fidelity monitoring. They addressed this problem by developing standardized

forms, as well as providing training, checklists, and technical assistance to the agencies.

Oklahoma reported a concern that it was difficult for them to assure that participants completed

necessary forms, but did not discuss a solution. More challenges in this area were possibly

averted due to the awareness of the necessity of fidelity monitoring and assuring program

quality, the wealth of tools and training materials available to states and organizations, and the

technical assistance provided at the national level on fidelity.

31

5. EVALUATION

There are several aspects of evaluation in the Empowering People and Communities initiatives.

Agencies offering programs were required to collect attendance and demographic information

from participants in order to enter the data into the national databases. State CDSMP programs

also collect pre- and post-program evaluation forms, monitor and report drop outs, and report

those who complete at least four of the six program sessions. Fidelity monitoring activities are

critical, and are addressed above. Challenges reported included the timely reporting of data from

partner organizations. California addressed this issue by adding a component to their leader

training on the data process. The lead state coordinating organization (Partners in Care) also

works with all licensed organizations in California on a quarterly basis to ensure CDSMP

activity and evaluation data are recorded. Massachusetts reported that partner organizations had

difficulty in conducting the six-month follow-up evaluations. This problem was managed by

suggesting that partner organizations use interns and volunteers to make the six-month follow up

phone calls. They also responded to this challenge by streamlining the survey tool and by using

foundation grants to provide stipends to organizations submitting data forms. New Jersey

revised the initial data collection protocol for CDSMP to eliminate the four-month follow-up

after it was determined that the data being collected was invalid.

While not required, states also implemented more rigorous evaluation studies, usually with the

help of a university research partner. Figure 2 shows that 11 states had university partners.

Typically, the evaluation studies measured the same participant outcome measures as were used

in the original EBP efficacy studies. In one state (South Carolina), the participant outcome

evaluation protocols were so rigorous that local agencies felt they were too burdensome and the

effort was discontinued because it was not practical for the agencies to collect.

D. Sustainability

32

The states approached their projects with the idea that they were building statewide partnerships

and delivery systems that would continue after Empowering People and Communities grants

ended. Sustaining the EB projects post-grant was a common goal identified in the original grant

applications. To assess how well the states’ efforts and approaches could be sustained, the

evaluators created an overall rubric that categorized a number of key features and assigned a

score ranging from 0 (not done or no information provided) to a 4 (exemplary efforts towards

sustaining the programs). These key features included a number of characteristics that identified

the approaches to sustainability and the activities undertaken to maximize funding for the

programs when the current grants ended. The features included how well states embedded the

programs into existing public health, aging and/or community based service systems; whether

new systems, units, or positions to support programs were established; whether new policies

were put in place to support programs; whether addressing sustainability was central in the

project’s design; how well documented plans for sustainability were described; and whether

additional funding was obtained during the grant period to sustain the programs.

Table 7: What evidence is there that the state’s programs will be sustained or replicated?

No info or not done

(Score = 0)

Limited

(Score = 1) Moderate

(Score = 2) Major

(Score = 3) Exemplary

(Score = 4) Mean

Programs were

embedded into systems 0% (n=0) 8% (n=2) 21% (n=5) 50% (n=12) 21% (n=5) 2.83

Established new systems,

units or positions to

support programs 12% (n=3) 17% (n=4) 25% (n=6) 29% (n=7) 17% (n=4) 2.21

New policies were put in

place to support

programs 33% (n=8) 8% (n=2) 29% (n=7) 25% (n=6) 5% (n=1) 1.58

Addressing sustainability

was central in the

project’s design 0% (n=0) 8% (n=2) 21% (n=5) 38% (n=9) 33% (n=8) 2.96

Sustainability plans were

documented 0% (n=0) 5% (n=1) 33% (n=8) 29% (n=7) 33% (n=8) 2.92

Additional funding was

obtained 8% (n=2) 12% (n=3) 21% (n=5) 38% (n=9) 21% (n=5) 2.50

1. EMBEDDING PROGRAMS INTO SYSTEMS OF SERVICES

As shown in Table 7, all states embedded programs into existing systems, with over 70% of the

states doing s