Home > Documents > National Council on Aging (NCOA) Empowering Older People ...activities were consistent with the...

National Council on Aging (NCOA) Empowering Older People ...activities were consistent with the...

Date post: 25-Dec-2019
Category:
Author: others
View: 0 times
Download: 0 times
Share this document with a friend
Embed Size (px)
of 103 /103
1 National Council on Aging (NCOA) Empowering Older People to Take More Control of Their Health Through Evidence-Based Prevention Programs: A Capping Report Administered: September 2011 December 2012 Prepared by: Janet C. Frank and Christy Ann Lau UCLA Multicampus Program in Geriatric Medicine and Gerontology Submitted to: National Council on Aging Revised Submission March 26, 2013
Transcript
  • 1

    National Council on Aging (NCOA)

    Empowering Older People to Take More Control of Their Health Through

    Evidence-Based Prevention Programs: A Capping Report

    Administered: September 2011 – December 2012

    Prepared by:

    Janet C. Frank and Christy Ann Lau

    UCLA Multicampus Program in Geriatric Medicine and Gerontology

    Submitted to: National Council on Aging

    Revised Submission

    March 26, 2013

  • 2

    TABLE OF CONTENTS

    I. EXECUTIVE SUMMARY ..................................................................................................... 4

    II. INTRODUCTION ............................................................................................................... 7

    A. Background .................................................................................................................... 7

    1. Program Purpose and Goals ........................................................................................ 7

    2. Program Evolution Over Time .................................................................................... 7

    a. The Early Years ...................................................................................................... 7

    b. National Expansion and Systems Integration ........................................................ 8

    c. Empowering Older People to Take More Control of Their Health Through

    Evidence-Based Prevention Programs .................................................................. 8

    d. Empowering Communities to Sustain Evidence-Based Disease and Disability

    Prevention Programs (Empowering Communities) ............................................... 9

    3. Approach to the Study ................................................................................................ 9

    a. Study Limitations ................................................................................................... 10

    4. Description of EBHP Programs Supported ................................................................. 10

    5. States Implementing Evidence-Based Health Promotion and Disease Management

    Programs (EBPs) ......................................................................................................... 12

    6. Types of Partners and Their Roles .............................................................................. 12

    B. Program Outcomes ....................................................................................................... 14

    1. Persons Served ............................................................................................................ 14

    2. Outcomes of Interventions on Program Participants .................................................. 16

    3. National Program Impacts .......................................................................................... 18

    4. Program Fidelity and Quality Assurance .................................................................... 20

    5. Best Practices .............................................................................................................. 20

    a. Marketing/Outreach ............................................................................................... 20

    b. Worker Training ..................................................................................................... 21

    c. Infrastructure Development ................................................................................... 22

    d. Fidelity and Quality Assurance ............................................................................. 23

    e. Evaluation .............................................................................................................. 23

    C. Challenges ...................................................................................................................... 24

    1. Marketing/Outreach .................................................................................................... 24

    a. Rural Issues ............................................................................................................ 24

    b. Transportation ....................................................................................................... 24

    c. Program Characteristics and Requirements .......................................................... 25

    d. Outreach to Minority and Underserved Populations ............................................ 25

    2. Worker Training .......................................................................................................... 26

    a. Trainer and Leader Recruitment ........................................................................... 26

    b. Trainer and Leader Engagement and Retention .................................................... 27

    3. Infrastructure Development ........................................................................................ 28

    4. Fidelity and Quality Assurance ................................................................................... 29

    5. Evaluation ................................................................................................................... 30

    D. Sustainability ................................................................................................................. 31

    1. Embedding Programs into Systems of Services ......................................................... 31

    2. Establishment of New Systems, Positions, Units or Programs ................................... 32

  • 3

    3. New Policy Development ........................................................................................... 33

    4. Sustainability as a Project Theme ............................................................................... 34

    5. Sustainability Plan Documentation ............................................................................. 35

    6. New Funding ............................................................................................................... 36

    E. Lessons Learned ............................................................................................................ 37

    1. Reach ........................................................................................................................... 37

    2. Effectiveness ............................................................................................................... 39

    3. Adoption ..................................................................................................................... 39

    4. Implementation ........................................................................................................... 40

    5. Maintenance and Sustainability .................................................................................. 41

    F. Products Developed ....................................................................................................... 43

    G. Conclusion ..................................................................................................................... 45

    APPENDICES .................................................................................................................... 47

    Appendix A: Data Extraction Tools and Scoring Rubrics ............................................. 47

    Appendix B: Case Studies of Five Grants ....................................................................... 56

    Appendix C: Products/Resources Developed .................................................................. 80

    Appendix D: Lessons Learned .......................................................................................... 100

  • 4

    I. EXECUTIVE SUMMARY

    In 2006 and 2007, the Administration on Aging made a major investment in the national

    expansion of evidence-based (EB) health promotion and disease management programs in the

    Empowering Older People to Take More Control of Their Health Through Evidence-Based

    Prevention Programs that provided funding to 24 states. This initiative was followed in 2010

    with additional funding from the American Recovery and Reinvestment (ARRA) entitled

    Empowering Communities to Sustain Evidence-Based Disease and Disability Prevention

    Programs, to expand the capacity and delivery of the EB programs in these states. These grants

    began in 2006, 2007 and all state grant projects under these two initiatives were completed in

    2012.

    Through the National Council on Aging, the Administration on Aging commissioned an

    evaluation to document the successes, challenges, accomplishments, lessons learned, and

    products produced through these two major grant initiatives. This report is based on data derived

    from the states’ Final Reports, and agency (AoA) and resource center (NCOA) administrative

    materials. The evaluation employed both quantitative (descriptive) and qualitative (content

    analyses) methods. The report is organized to answer the key questions of interest to the agency

    as outlined above.

    The states, in general, exceeded the grant goals they had set for themselves. All states goals and

    activities were consistent with the funding guidance and intent of the funding. The 24 states

    reported supporting 21 total evidence-based health promotion and disease management programs

    during the grant period. All states were expected to support the expansion of CDSMP program

    capacity and offerings, and all states also provided more than just the CDSMP programs. The

    most prevalent programs provided, besides the general CDSMP, included A Matter of Balance

    (MoB) offered by 14 states and EnhanceFitness (EF) offered by 10 states. Almost all states

    exceeded their goals for numbers of participants recruited into programs, and established

    important infrastructure protocols and partnerships. All 24 states involved a working partnership

    at the state level of aging services and public health departments for project leadership. Ten

    states identified their state’s Medicaid program and six included their Aging and Disability

    Resource Centers as key project partners. Four states identified their states’ Department of

    Corrections as a key partner. Four states engaged Tribal Entities as key partners and eleven

    states partnered with universities, primarily to provide evaluation expertise. The states excelled

    at building partnerships across many community sectors to impact policy, provide programs,

    provide referrals to programs, assure fidelity, and document outcomes.

    Program outcomes examined in this report included, but were not limited to, number of program

    completers and trainers/leaders, key partnerships developed, geographic coverage, the

    development of a quality improvement/fidelity plan and a sustainable infrastructure. Across all

    24 state grantees and 21 evidence-based programs offered, a total of 136,441 people were

    reached. About 25% of states were “exemplary” in reaching their target population goal, 17%

    “exceed” their goals for reaching their target population, 37% “met” their goals, 13% “fell short”

    of their goals, and 8% did not provide information or did not specify a goal. The majority of

    people reached were over the age of 65, female, about half lived alone and 68% were Caucasian,

    13% African-American and 11% Hispanic/Latino. Seventeen states went beyond grant

  • 5

    expectations to provide participant level program outcome and health improvements post

    programs, although typically follow up response rates were problematic. The most common

    program interventions that produced participant outcomes were EnhanceFitness, CDSMP and A

    Matter of Balance.

    Notable impact was made in creating aging service and health partnerships, as 21% of grantees

    proved “exemplary” in their accomplishments, Additionally, many states made significant

    progress toward sustainability either by finding new funding, leveraging existing funding, or

    developing or maintaining an infrastructure for offering programs and training leaders.

    Additionally, 51% of state grantees made substantial progress in creating an infrastructure for

    program delivery, referrals and registration. Eighty-eight percent of the States met or exceeded

    their goals for maintaining fidelity within the programs they were offering and assuring quality

    in all program implementation processes.

    All states identified challenges, with the predominance of them in the areas of implementation

    and sustainability. Because these programs are so highly relevant to addressing minority

    populations’ health needs, states made this a priority focus. States found that their ability to

    successfully recruit minority groups required a multifaceted approach: they needed to engage

    peer group champions and local community leaders to support program marketing, they needed

    to recruit leaders who are members of the participant groups they are trying to recruit into the

    programs, they sometimes had to adapt the programs to make them more relevant, and also

    needed to work with trusted agencies already serving these populations as partners and referral

    agencies. The evaluation report also documents best practices, solutions, lessons learned and

    exemplary case study states.

    We used the RE-AIM framework to capture and categorize some 120 key lessons learned. From

    the lessons learned, we identified a number of key recommendations for states to use in the

    future. Within the “Reach” category, states suggested that its best to pick partners already

    serving your target audience(s) to reach ethnic and underserved populations and to use GIS

    mapping tools for expansion planning to identify current program locations and trainer

    availability to indicate uncovered areas for development and expansion. Many states struggled

    with rural transportation problems, and recommendations emerged to try ride sharing programs,

    scheduling classes after other activities at times when people would already be at the site, and in

    some cases, consider whether program leaders can pick up participants on their way to the site.

    To assure “Effectiveness”, states recommended to get input and buy in for the scope and

    requirements of any planned evaluation with the agencies you expect to participate in advance

    and to assure the highest quality of leaders, set up a screening and/or interview protocol for

    potential leaders before enrolling them in a leader training program.

    In “Adoption”, it is best, when possible, to have a dedicated employee who specifically is in

    charge of heath promotion programs within the AAAs; to use an organizational readiness tool

    and spend time meeting with potential agencies before partnering; and to make sure staff at all

    levels in organization understand the commitment they are making in offering the programs.

  • 6

    For “Implementation”, a key role at the state level continues to be the development and oversight

    of protocols to foster fidelity monitoring and quality at all levels of program implementation.

    Another recommendation is to use the Cost Calculator not only to identify program costs but to

    also identify cost variations across programs and delivery sites/regions.

    In the “Maintenance and Sustainability” category, key recommendations included the use of

    business planning principles to approach sustainability and to diversify funding sources in

    sustainability planning and to have a paid program coordinator to manage program logistics –

    this is the best investment that can be made according to a number of states. Another innovative

    recommendation was to structure partner contracts that are based on the number of completers by

    grantee. In that way, there is a measurable outcome while incentivizing providers to engage

    participants and yield a high number of completers per workshop. Two final recommendations

    were to make sure that EBP is in the state plan for both aging and public health; and that having

    good outcome data on participant health improvement and costs provides the basis for state

    budget support and makes the business case for proposals for additional funding. The

    commitment of state leadership and local coordinators is evident and a critical ingredient to

    program success and sustainability.

    The evaluation report includes a number of tables with detailed specific information and a series

    of Appendices that provide the evaluation rubric tools, and complete lists of recommendations

    and products and resources produced by the states.

  • 7

    II. INTRODUCTION

    A. Background

    1. PROGRAM PURPOSE AND GOALS

    AoA funding support to states under the Evidence-Based Disease and Disability Prevention

    programs has been provided to empower older adults to take control of their health. In these

    programs, older adults learn to maintain a healthy lifestyle through increased self-efficacy and

    self-management behaviors (www.aoa.gov).

    A major expectation of the Empowering Older People initiative was to deliver high quality

    evidence-based programs that maintain fidelity to both the original design and to the research

    outcomes associated with the evidence-based models that are being deployed and to reach the

    maximum number of older adults that are at risk who can benefit from the programs. The AoA

    expectation reflected both the design and implementation of efficient and well-managed

    programs, and the need to find and commit funds from other public and private sources to these

    programs (as has occurred at the national level). By making these programs available in their

    communities, older adults were being empowered to take control of their health. Programs

    included:

    Physical activity programs, such as EnhanceFitness or Healthy Moves for Aging Well, which provide safe and effective low-impact aerobic exercise, strength training, and

    stretching.

    Falls management programs such as A Matter of Balance, which addresses fear of falling, and Stepping On and Tai Chi: Moving for Better Balance, which build muscle strength

    and improve balance to prevent falls.

    Nutrition programs, such as Healthy Eating for Successful Living among Older Adults, which teaches older adults the value of choosing and eating healthy foods, and

    maintaining an active lifestyle.

    Depression and/or Substance Abuse Programs, such as PEARLS and Healthy IDEAS, which teach older adults how to manage their mild to moderate depression.

    Medication Management Programs, such as HomeMeds.

    Stanford University Chronic Disease Self-Management Programs (CDSMP), which are effective in helping people with chronic conditions change their behaviors, improve their

    health status, and reduce their use of hospital services.

    2. PROGRAM EVOLUTION OVER TIME

    a. The Early Years

    The programs funded in 2006-7 that are the focus of this report represent a major expansion of

    the building blocks that had been put into place beginning in 2001. The programs began

    modestly in 2001 with John A. Hartford Foundation support of four demonstration projects led

    by the National Council on Aging to test the ability and interest of aging service organizations to

    http://www.aoa.gov/

  • 8

    actually lead and sponsor evidence-based health promotion (EBHP) programs. Making EBHP

    programs more accessible by placing them into community agencies was a major step, since

    previously the programs were based in a small number of research-oriented universities and a

    limited number of partnering organizations. In 2003 AoA funded 14 model projects that were

    primed by community agencies and included a major evaluation component, typically provided

    by a university partner. These model projects carefully documented the planning and

    implementation process of offering the programs in aging service community agencies and

    organizations while protecting the fidelity of the core components that made the programs

    effective.

    b. National Expansion and Systems Integration

    Since 2003, AoA has supported states as they have developed infrastructure, workforce, and

    capacity to deliver EBHPs through the aging services network and local partners (see Table 1).

    The Empowering Older People initiative, described below, was by far the largest program

    sponsored by AoA in support of the expansion of evidence-based programs and the integration of

    them into the fabric of community program delivery to support the health improvements of older

    people. From 2003 – 2012, AoA provided $23 million in funding for the Evidence-based

    Disease and Disability Prevention Program (EBDDP) to support programs aimed at keeping

    older adults healthy and engaged in their communities.

    Table 1: Evolution of AoA-Funded EBHP Programs in the United States

    • 2003: AoA model projects (14) served 5,000 people

    – Programs included CDSMP, falls management, depression, physical activity, medication management,

    and nutrition

    – Documented fidelity and focus on evaluation

    – Produced replication reports

    • 2006: AoA “Empowering Older People” funded in 16 states

    • 2007: “Empowering Older People” expands to 8 more states

    • 2010: AoA ARRA Projects: 47 states/territories

    c. Empowering Older People to Take More Control of Their Health through Evidence-Based

    Prevention Programs (Empowering Older People)

    In FY 2006, the Administration on Aging awarded cooperative agreement funding to 24 states to

    support dissemination of evidence-based programs. These grants were designed to mobilize the

  • 9

    aging, public health, and non-profit networks at the State and local level to accelerate the

    translation of HHS funded research into practice through the deployment of low-cost evidence-

    based disease and disability prevention programs at the community level. An AoA goal for the

    projects was for state units on aging and state health departments to collaborate on the provision

    of policy leadership and on-going support for local partnerships involving non-profit aging

    services providers, area agencies on aging, health organizations, the business sector, and other

    potential partners from the private and public sectors.

    d. Empowering Communities to Sustain Evidence-Based Disease and Disability Prevention

    Programs (Empowering Communities)

    In 2010, AoA issued a limited competition for one additional year of funding to the 24 states that

    had been funded under the 2006 program. The new initiative was to support the continued

    growth of partnership activities at both the state and community level. This opportunity allowed

    further advancement of collaborations with state units on aging and state health departments on

    the provision of health policy leadership, and the on-going strengthening of local partnerships

    involving area agencies on aging, local departments of public health, non-profit aging services

    providers, health and health insurance organizations, and other partners from the private and

    public sectors.

    3. APPROACH TO THE STUDY

    This report will present information and data provided through the 24 funded states’ Final

    Reports and related AoA and NCOA administrative and programmatic data. The state funded

    programs were scheduled to end in May 2011, but most states (75%) received no-cost

    extensions. The evaluators reviewed the original grant applications, final reports with their

    extensive appendices, state profiles, and grant management reports as the primary data sources

    for data extraction. Data extraction tools and evaluation rubrics were developed in six general

    areas of inquiry listed below. The tools and rubrics are provided in Appendix A.

    1. How well were project outcomes achieved? 2. What major challenges were encountered and what solutions for these challenges worked

    best?

    3. Taken together (across grantees) what was the major impact of the program and what lessons were identified that will assist future efforts by AoA in this program area?

    4. What features supported states in their own formal program evaluation efforts? 5. What evidence is there that programs will be sustained or replicated? What program

    features and/or partners support embedding the program into systems?

    6. What types of resources and products were developed by the projects?

    The data was available on a rolling basis as grantees completed their projects and subsequently

    provided their final reports. One evaluator reviewed state materials for 10 states (Frank) and the

    other evaluator reviewed the remaining 14 states (Lau). The two evaluators worked closely

    together in completing the data extraction and met weekly to review the evaluation matrices and

  • 10

    discuss any issues that were identified. The two evaluators also completed four state data

    extractions from each other’s state assignments to assure inter-rater reliability in score

    assignment and use of the tools.

    The data was then entered into an Excel database and included both rubric scores (quantitative)

    and extensive qualitative notes within the six areas of inquiry. Analyses included descriptive

    statistics (frequencies, means, and standard deviations), bivariate analyses and qualitative content

    analyses that included both conceptual groupings, frequency counts, and the creation of

    inventory lists (e.g. products list).

    a. Study Limitations

    The data presented in this report is derived from the written materials provided to the

    Administration of Aging by the states and other AoA and NCOA administrative materials (e.g.

    AoA grant monitoring reports, original state grant proposals). There is the potential that if the

    final reports did not include all relevant information sought during the data extraction, this

    missing information would result in an incomplete accounting of states’ accomplishments or

    incorrect scores assigned within the rubrics. The quality and validity of the evaluation data was

    dependent on the completeness and quality of the final reports and appendices provided to AoA

    by the states.

    4. DESCRIPTION OF EBHP PROGRAMS SUPPORTED

    The 24 states reported supporting 21 total evidence-based health promotion and disease

    management programs (Table 2 below) during the grant period. These programs can be

    organized into several general categories: Stanford University chronic disease self-management

    programs (CDSMP) (English and Spanish) both general and specialized (diabetes, pain,

    arthritis); falls management programs; physical activity programs; behavioral health, medication

    management and lifestyle improvement programs. All states were expected to support the

    expansion of CDSMP program capacity and offerings, and all states also provided more than just

    the CDSMP programs. The most prevalent programs provided besides the general CDSMP

    included A Matter of Balance (MoB) offered by 14 states and EnhanceFitness (EF) offered by 10

    states (see Table 2).

  • 11

    Table 2: EB Programs Offered by States During Grant Period

    EB Program Number of States Offering Number of Participants

    Trained by Program

    Stanford Chronic Disease Self-Management Programs

    Chronic Disease Self-Management Program 24 80,386*

    Diabetes Self-Management Program 10 35,278*

    Tomando Control de su Salud 5 9,889* **

    Programa de Manejo Personal de la

    Diabetes 1 0000

    Arthritis Self-Management Program 1 3,177*

    Chronic Pain Self-Management Program 1 0000

    Falls Management

    A Matter of Balance (MOB) 14 21,072

    Tai Chi: Moving for Better Balance 4 1,937

    Asunto de Equilibrio (Spanish MOB) 1 3,585

    Step-by-Step 1 172

    Stepping On 1 2,755

    Physical Activity Programs

    EnhanceFitness 10 11,320

    Active Living Every Day 2 623

    Fit & Strong! 2 94

    Active Choices 1 24

    Strong for Life 1 483

    Healthy Moves for Aging Well 1 345

    Behavioral Health Program

    Healthy IDEAS 4 5,288

    Medication Management

    HomeMeds 3 5,672

    Life Style Improvement Programs

    Healthy Eating for Successful Living among

    Older Adults 2 1,754**

    EnhanceWellness 1 131

    *States reported training numbers across multiple programs (e.g., CDSMP/Tomando/DSMP); exact numbers by

    individual program undeterminable/may be duplicative

    **Figure includes missing data, as some states omitted training numbers from reported programs

    0000 = state(s) did not report training numbers for specified program

  • 12

    5. STATES IMPLEMENTING EVIDENCE-BASED HEALTH PROMOTION AND DISEASE

    MANAGEMENT PROGRAMS (EBPS)

    The 24 states that were funded in this initiative are identified in Figure 1 below. Each state

    identified the geographic target areas (by region or county) in their grant proposal that they

    planned to focus their expansion efforts within. The planned expansion coverage is depicted by

    the blue color designation in the map. At the conclusion of the project, 50% of states (n = 12)

    had exceeded planned geographic coverage, 29% (n = 7) met their geographic coverage goals,

    and 21% (n = 5) of states fell short of meeting their geographic expansion plans.

    Figure 1: State grantee reach by county

    NOTE: Twenty-four grantee states’ targeted counties are shown in blue; unmarked states were not covered by this

    grant

    6. TYPES OF PARTNERS AND THEIR ROLES

    Within the grant guidance, states were required to demonstrate that the projects would involve a

    partnership between the state level aging services (e.g. State Unit on Aging) and the state level

    health department (e.g. State Department of Public Health). There was also encouragement in

    the grant guidance to include public health services funders (e.g. Medicaid) as a key partner. All

    24 states involved a working partnership at the state level of aging services and public health

  • 13

    departments for project leadership. Ten states identified their state’s Medicaid program and six

    included their Aging and Disability Resource Centers as key project partners. As shown in

    Figure 2, all states relied on local AAA’s as key partners and developed community

    organizations and local agencies as partners. The local organizations included health

    departments, parks and recreation departments, senior housing, faith based communities, and

    county and city service programs. Health care services providers (hospitals, clinics), health

    districts, physician groups, and health plans were all included in the “health care” category

    shown in Figure 2. Four states (Ohio, Oklahoma, Oregon, and South Carolina) identified their

    states’ Department of Corrections as a key partner. Four states engaged Tribal Entities (Arizona,

    Oklahoma, Oregon and Minnesota) as key partners and eleven states partnered with universities,

    primarily to provide evaluation expertise. Three states (Florida, Massachusetts, and Michigan)

    noted local foundations as key partners for their projects.

    Figure 2: Key Project Partners

    In almost all states, agencies and departments formed statewide collaborative networks for

    oversight and to serve as steering committees for the projects. Several states established formal

    statewide collaborative organizations. For example, Colorado formed the public-private

    Consortium for Older Adult Wellness; Hawaii developed the Hawaii Healthy Aging Partnership;

    and Massachusetts established the Massachusetts Disease Management Coalition. Texas

    developed the Texas Strategic Health Partnership, whereas Arkansas established the CDSMP

    Partners & Stakeholders’ Group. Wisconsin developed the Community-Academic Aging

    Research Network and the Evidence-based Coordinating Community. Wisconsin was the only

    state to establish a not-for-profit organization during the project, the Wisconsin Institute for

    Healthy Aging (WIHA). The WIHA’s steering committee provided project oversight and also

    applied for grant funding that was not available to governmental agencies. The states excelled

  • 14

    at building partnerships across many community sectors to impact policy, provide programs,

    provide referrals to programs, assure fidelity, and document outcomes.

    B. Program Outcomes

    The evaluation assessed how well state grantees achieved the goals they set forth in their original

    grant proposals. It also documents how well the programs retained their original design and

    were aligned with AoA’s overall goals for the initiative. Program outcomes examined in this

    grant included, but were not limited to, number of program completers and trainers/leaders, key

    partnerships developed, geographic coverage, the development of a quality improvement/fidelity

    plan and a sustainable infrastructure. For a complete list of the constructs that were assessed, see

    Appendix A, Tables 1 and 3.

    1. PERSONS SERVED

    Across all 24 state grantees and 21 evidence-based programs offered, a total of 136,441 people

    were reached. Among the most popular programs offered were the Chronic Disease Self-

    Management Program (CDSMP), A Matter of Balance (MOB), the Diabetes Self-Management

    Program, and EnhanceFitness, the four of which, combined, served 113,877 people in total. For

    each of the programs offered, grantees usually set goals for their targeted number of program

    participants. As noted in Table 3, 42% of grants were “exemplary” in achieving their participant

    goals, 29% “exceeded” their goals, 4% “met” their goals, 8% “fell short” of their goals, and 17%

    did not provide outcome information or did not designate a goal.

    Table 3: How Well Did the State Achieve Its Program Outcomes?

    Did not provide

    information

    or no goal (Score = 0)

    FELL

    SHORT of

    achieving

    outcome

    goals (Score = 1)

    MET

    outcome

    goals (Score = 2)

    EXCEEDED

    outcome

    goals (Score = 3)

    EXEMPLARY

    in achieving

    outcome goals

    (Score = 4)

    TOTAL (N = 24)

    Program Completers 17% (n=4) 8% (n=2) 4% (n=1) 29% (n=7) 42% (n=10) 24

    # of Trainers/Leaders 42% (n=10) 4% (n=1) 17% (n=4) 25% (n=6) 13% (n=3) 24

    Key Partnerships 0% (n=0) 0% (n=0) 42% (n=10) 42% (n=10) 17% (n=4) 24

    Geographic

    Coverage/Target

    Population 8% (n=2) 8% (n=2) 37% (n=9) 17% (n=4) 25% (n=6) 24

    Aging/Public Health

    Leadership 4% (n=1) 0% (n=0) 58% (n=14) 21% (n=5) 17% (n=4) 24

    Quality

    Improvement/Fidelity

    Plan 8% (n=2) 4% (n=1) 50% (n=12) 4% (n=1) 34% (n=8) 24

    Sustainable

    Infrastructure 4% (n=1) 4% (n=1) 29% (n=7) 34% (n=8) 29% (n=7) 24

  • 15

    Table 3 also indicates that 25% of states were “exemplary” in reaching their target population,

    17% “exceed” their goals for reaching their target population, 37% “met” their goals, 8% “fell

    short” of their goals, and 8% did not provide information or did not specify a goal. State

    grantees that fell short of reaching their target population were Maryland and Oklahoma. The

    Maryland grant noted that delays in contracting processes resulted in no workshops being offered

    in one area. Finally, Oklahoma’s report stated a goal of establishing 100+ permanent program

    sites. However, their state profile showed that they fell a little short of that goal with only 90

    workshop sites (which is still a substantial accomplishment). Those that did not provide

    information or did not specify a goal were Arkansas and Oregon.

    Table 4 below presents demographic data of program participants across all 24 states through

    five grant years. Note that completion of the demographic data form was not mandatory, so data

    is only provided for the 80,067 participants who provided this information.

    Table 4: Participant Demographics (N = 80,067)

    Demographic Construct 5-Year Total % of Known Statistics

    Age

    Under 60 5,328 9%

  • 16

    60-64 3,796 7%

    65-69 6,589 12%

    70-74 8,405 15%

    75-79 9,166 16%

    80-84 9,766 17%

    85-89 8,122 14%

    90 and Over 5,078 9%

    Unknown 23,817 30% (% of total)

    Gender

    Female 46,465 79%

    Male 12,097 21%

    Unknown 21,505 27% (% of total)

    Living Arrangement

    Living Alone 26,455 50%

    Living With Someone 26,166 50%

    Unknown 27,446 34% (% of total)

    Race/Ethnicity

    Native American 928 2%

    Asian 1,747 3%

    Black 7,027 13%

    Pacific Islander 194 0%

    Hispanic/Latino 6,232 11%

    White 38,109 68%

    Other Race 585 1%

    Multi-Racial 1,291 2%

    Unknown 24,040 30% (% of total)

    Source of data: The National Council on Aging (NCOA)

    2. OUTCOMES OF INTERVENTIONS ON PROGRAM PARTICIPANTS

    Seventeen grants conducted extensive evaluations of participant level outcomes in the programs

    they offered. While the majority of these efforts centered on measurements such as health

    outcomes, physical activity, and hospital readmissions, a few states chose to focus on program

    satisfaction and fidelity. These states included: Illinois, which focused on participant and leader

    satisfaction; Michigan, which conducted an extensive fidelity study in partnership with Michigan

    State University; and Texas, which performed an evaluation of program implementation

    processes across sites using the RE-AIM framework. This section, however, will focus on those

    state grantees that provided participant level health and wellness outcomes as a product of

    participating in the evidence-based program interventions. These states included: Arizona,

    Connecticut, Hawaii, Idaho, Iowa, Maine, Massachusetts, New Jersey, New York, Ohio,

    Oklahoma, Oregon, South Carolina, and Wisconsin. States with exemplar evaluation efforts are

    discussed below.

  • 17

    The most common program interventions that produced participant outcomes were

    EnhanceFitness, CDSMP and A Matter of Balance. States offering EnhanceFitness used tools

    such as the chair stand (lower body strength), arm curl (upper body strength), and timed Up and

    Go (transfer ability and risk of falls) tests to quantify participants’ health improvements while

    enrolled in the program. Both Arizona and Hawaii reported that participants improved at an

    average rate of 18% for the number of chair stands they could perform in 30 seconds, 22% for

    the number of arm curls they could do in 30 seconds, and 11% for the length of time in seconds

    it took to complete the Up and Go assessment.

    States that reported evaluation data for participants of CDSMP utilized survey instruments to

    measure constructs pre- and post-program such as:

    Participant self-rated health

    Number of times participants discussed health conditions with their doctors

    Health care utilization (e.g., physician visits, emergency department visits, nights of hospitalization)

    Fatigue, and

    Pain

    Iowa, as a key example, reported results at baseline, six months, and one year post-program for

    all constructs measured except pain and fatigue, which were reported at baseline and one year

    post-program. See Table 5 below.

    Table 5: Changes in Health Outcomes for Participants of Iowa’s CDSMP

    Baseline 6 Months Post- 1 Year Post-

    Self-Rated Health 3.08 3.01 3.21

    Chronic Disease Self-

    Efficacy 5.70 6.40 6.60

    Pain 4.80 -- 4.60

    Fatigue 5.03 -- 4.78

    Emergency Room

    Visits 1.44 0.72 0.35

    Nights of

    Hospitalization 7.62 4.56 1.88

    These outcomes were instrumental in estimating the average health care cost savings for

    participating in CDSMP. For example, Iowa estimated a one-year savings of $76,204 based on

    the average charge per visit of $506.

    Oregon reported similar results, with reduced emergency department visits from 0.8 to 0.7 visits

    per year, hospitalizations from 0.4 to 0.3 visits per year, and hospital days from 2.4 to 1.9 days

    per year. For the participants who have completed Oregon’s Living Well (CDSMP) program to

    date, this translates to 557 fewer emergency department (ED) visits, 557 fewer hospitalizations,

  • 18

    2,783 fewer hospital days, and a savings of $634,980 in ED visits, and $6,501,088 in hospital

    stays.

    Finally, Massachusetts served as a prime example of a state that assessed the effectiveness of A

    Matter of Balance. Survey tools were administered at the first (week one) and last (week eight)

    classes. The survey tool measured outcomes in three key areas: 1) fall management; 2) fall

    efficacy; and 3) fall control. Key findings showed that, as a result of completing the A Matter of

    Balance class, 95% of participant responders noted they are more comfortable talking about their

    fear of falling, 96% feel more comfortable increasing activity, 90% plan to continue exercising,

    and 92% would recommend the program to other older adults.

    3. NATIONAL PROGRAM IMPACTS

    To examine the impacts of the programs on both the state and national levels, a rubric was

    developed to score state successes as they “engaged state leadership in systems level strategic

    planning,” “created aging service and health partnerships,” “reached rural, minority, or

    underserved populations,” “increased capacity of local agencies to deliver EB programs,” “made

    progress toward sustainability/funding,” “expanded geographic reach,” “created infrastructure

    for program delivery, referrals and registration,” “aligned their goals, achievements and

    successes with AoA,” “offered more programs than CDSMP,” and “measured outcomes.” Table

    6 below provides a snapshot of the national impact made across the 10 abovementioned factors.

    Table 6: What Impact Did the State’s Project Achieve?

    No information

    Limited

    (Score = 1) Moderate

    (Score = 2) Major

    (Score = 3) Exemplary

    (Score = 4) TOTAL (N = 24)

  • 19

    provided or

    not done

    (Score = 0)

    Engaged state leadership

    in systems level strategic

    planning 4% (n=1) 13% (n=3) 33% (n=8) 25% (n=6) 25% (n=6) 24

    Created aging service +

    health partnerships 0% (n=0) 0% (n=0) 8% (n=2) 71% (n=17) 21% (n=5) 24

    Reached rural, minority

    or underserved

    populations 13% (n=3) 38% (n=9) 25% (n=6) 16% (n=4) 8% (n=2) 24

    Increased capacity of local

    agencies to deliver EB

    programs 0% (n=0) 4% (n=1) 38% (n=9) 50% (n=12) 8% (n=2) 24

    Progress toward

    sustainability/funding 0% (n=0) 0% (n=0) 33% (n=8) 38% (n=9) 29% (n=7) 24

    Expanded geographic

    reach 8% (n=2) 13% (n=3) 29% (n=7) 29% (n=7) 21% (n=5) 24

    Created infrastructure for

    program delivery,

    referrals and registration 0% (n=0) 8% (n=2) 41% (n=10) 38% (n=9) 13% (n=3) 24

    Goals, achievements and

    successes aligned with

    AoA 0% (n=0) 0% (n=0) 13% (n=3) 62% (n=15) 25% (n=6) 24

    Offered more programs

    than CDSMP 0% (n=0) 17% (n=4) 29% (n=7) 33% (n=8) 21% (n=5) 24

    Notable impact was made in creating aging service and health partnerships, as 21% of grantees

    proved “exemplary” in their accomplishments, 71% made a “major” impact and 8% made a

    “moderate” impact. Key examples of partnerships are given in Figure 2 under the section,

    “Types of Partners and their Roles.” Additionally, many states made significant progress toward

    sustainability either by finding new funding, leveraging existing funding, or developing or

    maintaining an infrastructure for offering programs and training leaders. 29 percent of states

    were “exemplary” in their progress toward sustainability/funding, 38% made “major” progress,

    and 33% made “moderate” progress.

    Additionally, Table 6 shows that 13% of state grantees were “exemplary” in creating an

    infrastructure for program delivery, referrals and registration, 38% made a “major” impact, 41%

    made a “moderate” impact and 8% made a “limited” impact. Those scoring “exemplary” in this

    category included: California (as mentioned above), Colorado and Hawaii.

    4. PROGRAM FIDELITY AND QUALITY ASSURANCE

  • 20

    To evaluate the degree to which states maintained fidelity within the programs they were

    offering and assured quality in all program implementation processes, raters focused on how well

    states developed and carried out a quality improvement/fidelity plan and created an infrastructure

    for program delivery, referrals and registration. Stellar state examples mentioned fidelity

    monitoring processes embedded into their infrastructure to ensure that fidelity was kept

    throughout the duration of the program. California, for example, created a Program Office and a

    Steering Committee to facilitate program offerings across California and to ensure fidelity and

    data collection.

    Table 3 above shows 34% of state grantees were “exemplary” in developing and implementing a

    quality improvement/fidelity plan, 4% “exceeded” their outcome goals in doing so, 50% “met”

    their outcome goals, 4% “fell short,” and 8% did not provide information or did not specify a

    goal. States that received “exemplary” marks included: Colorado, Hawaii, Massachusetts,

    Michigan, New York, Oregon, South Carolina, and Texas. Colorado, for example, established

    the Healthy Aging Service System for training, technical assistance, and evaluation/fidelity

    checks. Using this system, they conducted 227 fidelity visits during the grant period. In Oregon,

    grantees developed a statewide Living Well Quality Assurance and Fidelity workgroup, provided

    fidelity tools, and observed 70% of all Living Well workshops over the course of a year.

    5. BEST PRACTICES

    Through analysis using the aforementioned scoring rubrics, best practices for implementing the

    evidence-based programs emerged as state grantees received “exemplary” scores in project

    outcome achievement and program impact. Specific attention was paid to best practices in the

    areas of marketing/outreach, worker training, infrastructure development, fidelity, and quality

    assurance and evaluation.

    a. Marketing/Outreach

    In order to determine best practices relating to marketing/outreach, states were scored on their

    accomplishments in developing key partnerships, reaching rural populations and expanding their

    geographic reach.

    Four states (California, Colorado, Connecticut, and Ohio) received “exemplary” marks in key

    partnerships. In California, over 70 health care organizations have invested in CDSMP and are

    offering it internally, including 22 Kaiser sites, 17 physician groups and clinics, 12 Dignity

    Health (formerly Catholic Healthcare West) hospitals and medical centers, five health care

    districts, and three health plans. The grant has also contributed to a new collaboration between

    the California Department of Aging and the California Department of Public Health, and at the

    local level between sixteen local health departments and AAAs serving those counties.

    The Connecticut Department of Social Services, Aging Services Division (DSS) and the

    Connecticut Department of Public Health (DPH) served as the key partnership for implementing

    all of the state’s programs. To implement CDSMP, they spearheaded an advisory council

    comprised of their technical assistance consultant, project evaluator from the UCONN Center on

  • 21

    Aging, local partners from the Hartford Community including the City of Hartford Health and

    Human Services and the City of Hartford Elderly Services, and the North Central AAA.

    Two states (Massachusetts and Texas) were “exemplary” in reaching rural populations. Hebrew

    Senior Life, a key partner in the state of Massachusetts, made focused efforts to reach diverse

    populations, starting with a presentation at the Harvard Multicultural Coalition Annual Aging

    Well Together Conference in 2008. At that conference, abbreviated workshops were presented

    in several languages: English, Spanish, Portuguese, Chinese, Vietnamese, Haitian Creole, and

    Cape Verdean Creole. A grant from the Tufts Health Plan Foundation in 2009 allowed for the

    translation and piloting of the program into Spanish and Vietnamese. Translations of the

    program into Haitian Creole and Portuguese are currently being conducted in collaboration with

    the Cambridge Health Alliance (Haitian Creole) and the Visiting Nurse Association

    (Portuguese).

    Lastly, five states (California, Colorado, Florida, New Jersey, and Texas) were “exemplary” in

    expanding their geographic reach. In Florida and New Jersey, programs are now being offered

    in 100% of their counties, and in Texas program offerings increased from 10 counties to 58

    counties over the course of this grant.

    b. Worker Training

    In order to increase capacity to deliver the programs, a number of states identified goals of

    increasing their numbers of master trainers and lay leaders available to offer the programs. For

    those states with this type of stated goal, we rated them using the rubric that ranged from “fell

    short” in meeting goal, met goal, exceeded goal and exemplary in meeting their goal. It is

    possible that for states who far exceeded their goal that they were creating excess capacity in

    trainer availability, but we could not discern this from available data. What seemed to be

    happening was that states needed two major things to increase the spread and number of

    programs being offered. First, they needed the trained personnel to lead the classes; and second,

    they needed the agencies being ready and willing to sponsor the programs. Sometimes it

    appeared that one of these factors lagged behind the other, but oftentimes they were in sync.

    For best practices in worker training, states were assigned scores for their achievements in the

    number of trainers/leaders trained, and their ability to increase the capacity of local agencies to

    deliver the evidence-based programs. Three states were “exemplary” in their number of

    trainers/leaders by the end of the grant. These states were Colorado, Maryland, and

    Massachusetts. Colorado stood out with two T-Trainers, 62 Master Trainers and 363 Lay

    Leaders, and Massachusetts boasted stellar results by increasing their numbers of Master

    Trainers and Lay Leaders in CDSMP, Healthy Eating, and A Matter of Balance. In 2011 alone,

    the number of CDMSP Master Trainers in Massachusetts grew from 76 to 152, Lay Leaders

    from 101 to 350, Healthy Eating Master Trainers from 5 to 86, Lay Leaders from 17 to 221, and

    A Matter of Balance Master Trainers from 74 to 116 and Lay Leaders from 143 to 450.

    Additionally, two states were “exemplary” in their ability to increase the capacity of their local

    agencies to offer EB programs: California and Colorado. In Colorado, 23 partners provided

  • 22

    programs in 17 counties, and a Colorado Health Foundation grant supported program

    implementation and infrastructure development.

    c. Infrastructure Development

    Development of a strong infrastructure was a key element to ensuring that agencies were able to

    systematically recruit participants, deliver programs, and monitor fidelity through the provision

    of technical assistance and data collection. The strength of this infrastructure was also telling in

    state grantees’ ability to sustain their efforts beyond the duration of the grant. To look at best

    practices in this area, raters assigned states scores focused on their achievement in building a

    sustainable infrastructure, the aging service and health partnerships they created, and their

    engagement of state leadership in systems level planning.

    Seven states were “exemplary” in achieving a sustainable infrastructure: California, Colorado,

    Connecticut, Hawaii, Massachusetts, New Jersey, and Ohio. All of these states made significant

    headway in building and sustaining an infrastructure for offering EB programs. A few of them

    are discussed below.

    California’s development of a project office and statewide steering committee provided a central hub for guiding program implementation and facilitating relationships with

    various organizations/entities.

    In Colorado, a partnership was built with three health care systems and a state ADRC for referrals. Emphasis was placed on embedding programs into systems where they could

    be sustained, and using Older American’s Act funding to support AAA programming.

    In Connecticut, the Yale Connecticut Collaboration for Fall Prevention and the State Commission on Aging secured funding through the Connecticut State Legislature to

    provide fall prevention training sessions to a larger network of home care and hospital

    based clinicians in both regions and other parts of the state that were not covered by the

    grant. In a September 2009 special legislative session, this statewide falls prevention

    initiative became Public Act 09-5 and continues to receive yearly funding from the state

    legislature.

    In New Jersey, the statewide infrastructure for CDSMP is based on shared administration/oversight by state government and local implementation by community

    partners. State level personnel costs will continue to be paid through other federal and

    state funding streams.

    Five states were “exemplary” in creating quality aging service and health partnerships to aid in

    the process of developing a strong infrastructure. These states included: California, Connecticut,

    Hawaii, Massachusetts, and Ohio. California and Connecticut’s stellar partnerships were

    discussed in the “Marketing/Outreach” section above. Additionally, Hawaii created the Hawaii

    Healthy Aging Partnership (HHAP), comprised of 63 partner organizations and 27 volunteers.

    In Massachusetts, implementation of EBDPs is the top program and policy initiative for the

    Councils on Aging and is actively being promoted by the Massachusetts Association of Councils

    on Aging. A coalition of leaders from the Tufts Health Plan Foundation and the Massachusetts

    Health Policy Forum formed the Healthy Aging Steering Committee to examine community-

  • 23

    based, environmental, and systematic approaches to promote healthy living and healthy aging.

    Key partners in Ohio included their 12 AAAs, senior centers, behavior health organizations,

    local/county health departments, meal providers, senior housing, neighborhood health centers,

    health care providers, faith-based organizations, PASSPORT, hospitals, corrections facilities,

    regional Alzheimer’s Association chapters, and many more.

    d. Fidelity and Quality Assurance

    The importance of maintaining fidelity to the programs as designed and incorporating quality

    assurance measures was discussed in the “Program Fidelity and Quality Assurance” section

    above. As mentioned, states received scores around their achievement of developing and

    implementing a quality improvement/fidelity plan. Eight states received “exemplary” marks:

    Colorado, Hawaii, Massachusetts, Michigan, New York, Oregon, South Carolina, and Texas.

    For Massachusetts, in addition to recruiting and coordinating leader trainings in their three EBDP

    programs, the lead community partners have provided mentoring, technical assistance (including

    the development of websites for all programs), and support to newly trained program facilitators.

    In addition, opportunities for continuing education and sharing of best practices have also been

    available to leaders. Texas A&M, the university evaluators for the state’s grant, offered training

    and technical support continuously, troubleshot programs for concerns or issues, and provided

    remedial training and support to address challenges.

    e. Evaluation

    Finally, as discussed in the “Outcomes of Interventions on Program Participants” section above,

    17 states provided extensive data on the effectiveness of the programs for participant health and

    physical activity outcomes. Scoring to determine “exemplary” states was not done for this

    construct since it was not an expectation of the funding initiative. Instead, qualitative data was

    pulled to examine each of the 17 states’ efforts and individual outcomes. Premier examples of

    evaluation efforts by those states are discussed above. It should be noted that the majority of the

    states that provided these extensive evaluation studies included a university partner in their

    project that managed the participant level evaluation studies.

    In addition to those 17 states, two others emerged as key examples of strong evaluation efforts as

    an analysis of solutions to remedy challenges was done. Towson University, the evaluation

    partner in the Maryland grant, performed key evaluations in 2009 and 2010 to monitor the

    fidelity of the program. Maryland implemented a fidelity consultant training in 2010-2011 with

    each grantee, requiring the completion of a written fidelity plan. In Illinois, a robust evaluation

    and fidelity monitoring of the CDSMP and Strong for Life programs was completed. The

    evaluation monitored class participants, class leaders, and key informants (four agency

    directors).

    C. Challenges

  • 24

    In their final reports, States identified a number of challenges that they faced in executing their

    plans and accomplishing activities to reach their goals. These challenges included marketing and

    outreach, worker training, infrastructure development, fidelity and quality assurance, evaluation,

    and other issues such as personnel changes, delays in hiring, and contract execution with

    partners.

    1. MARKETING/OUTREACH

    Challenges in outreach and recruitment of both program participants and leaders/trainers were

    common, being reported by over 70% of the states. This category of challenges represented

    about 25% of types of all challenges reported.

    a. Rural Issues

    Many states’ challenges in this area were related to the rural nature of the state. Rural program

    delivery was challenged because the scarcity of population made it difficult to recruit enough

    people to be trained as leaders and also to recruit enough people to fill the classes. Arkansas

    found that rural program delivery challenges made it difficult to accomplish their goal of “30-30”

    – having a class available within 30 days and within 30 miles of any participant who desired to

    enroll in a class. They employed an innovative GIS mapping tool to identify the geographic

    distribution of leaders and implementation sites, thus revealing the areas of need for further site

    development. They blanketed small rural communities’ senior centers, faith-based communities,

    and other strategic locations with information on the evidence-based programs through radio,

    free television spots, flyers, newsletters, and local newspapers. Through partnering with the

    AAAs and the Arkansas Aging Initiative they were able to recruit local leaders and provide

    programs in the geographic areas that represented locations for almost 90% of older adults in the

    state. Michigan also used a GIS mapping tool effectively to document available leader locations

    in order to target leader training programs in needed areas. Other states focused on developing

    local champions to be trained as leaders and to help recruit members of their community to

    classes.

    b. Transportation

    Transportation was also noted as a challenge, especially in rural areas, and frequently was the

    reason for the difficulty in recruiting leaders and recruiting and retaining program participants.

    Leaders were typically clustered in more densely populated areas and a number of states found it

    difficult to address the challenge of leaders traveling to the more rural areas to deliver programs.

    Participant travel to the programs was also an issue. Some older adults no longer drove or had

    limited personal transportation access. Those residing in rural areas often had limited public

    transportation options to travel to the program locations. Local agencies were very imaginative

    in addressing transportation issues: they found funding to provide a transportation stipend to

    leaders, they scheduled programs at leaders’ convenience, they assisted with carpooling, van

  • 25

    pickup of participants; and in some cases, dedicated leaders picked up participants on their way

    to the programs.

    c. Program Characteristics and Requirements

    Other challenges in marketing and outreach had more to do with the programs themselves.

    States reported that the falls management programs (e.g. A Matter of Balance) and physical

    activity programs (like EnhanceFitness) seemed easier to recruit people to because participants

    understood the goals of the programs and could identify how participating would address a

    perceived risk or prevent a problem, whereas the Chronic Disease Self-Management programs

    (CDSMP) were often a “harder sell.” On the face of it, participants often felt they were

    managing just fine, and did not really understand the potential program benefits. States typically

    changed the name of the CDSMP to one that was more upbeat and attractive: in California, the

    program is called Healthier Living; whereas in Oregon and several other states it was known as

    Living Well.

    Participant retention problems for CDSMP were often addressed by adding a “class zero” for

    orientation so that participants would understand more about the goals, benefits, and structure of

    the program (six meetings) prior to signing up. Several states also addressed retaining

    participants with an incentive such as Maryland did, by providing a book of county ride tickets

    valued at $15. A very common problem in participant retention related to the characteristics of

    the target audience for CDSMP – those with chronic illnesses. Participants’ health problems

    often interfered with their ability to attend programs. The CDSMP curricula itself discusses

    these issues and helps its participants to anticipate this potential problem in program attendance.

    The expectation of a “completer” being a person who attends at least four of the six sessions also

    recognizes the issue. It is an expectation that both the “buddy system” the program employs and

    the expectation that leaders call participants after a missed session also addresses this challenge.

    d. Outreach to Minority and Underserved Populations

  • 26

    The evidence-based health promotion and disease management programs are especially relevant

    for ethnic, racial minorities and low-income populations due their high chronic disease

    prevalence. States sought to specifically target these underserved groups for their programs and

    often faced challenges in their efforts to do so. Often recruitment was a concern – these groups

    have many life challenges and becoming involved in programs such as these may not be a

    priority. They may question the relevance of the programs to their own life situations. States

    found that their ability to successfully recruit these groups required a multifaceted approach:

    they needed to engage peer group champions and local community leaders to support program

    marketing, they needed to recruit leaders who are members of the participant groups they are

    trying to recruit into the programs, they sometimes had to adapt the programs to make them more

    relevant, and also needed to work with trusted agencies already serving these populations as

    partners and referral agencies. Recruiting Native Americans by working with Tribal entities

    typically took longer for the Tribal permission process and to engage agency personnel to be

    recruited as leaders so they could provide the programs to their constituencies.

    Many states selected key partners specifically to assist in recruiting participants for programs,

    especially those from minority and underserved low-income populations. States who partnered

    with Medicaid programs found these to be very effective referral sources. Several states,

    including Colorado, Maine, Maryland, and New Jersey, noted how well their state’s Aging and

    Disability Resource Center worked with them in making participant referrals. Minnesota

    changed partners to work with Wisdom Steps, a preventive health organization that provides

    services to all tribes in the state. By doing so, they were able to recruit and train Tribal leaders in

    both A Matter of Balance and CDSMP and offer multiple sessions in both programs. By taking

    the extra time to find the best partner and invest in the Tribes’ own capacity to provide its

    members with programs, Minnesota has established the foundation to expand programs to

    address the Tribal community needs.

    2. WORKER TRAINING

    State grantees in the Empowering People and Communities initiatives were tasked to expand the

    capacity, geographic reach, and sustainability of evidence-based health promotion and disease

    management programs. To accomplish this, adequate workforce capacity for trainers, leaders,

    and program coordination staff was essential. Because the Empowering People and

    Communities initiatives were building on previous years of program support in many states,

    existing trainer and leader capacity varied. Thus the individual state goals and activities

    regarding worker training were consistent with identified needs and expansion plan

    requirements.

    a. Trainer and Leader Recruitment

    About 75% of all states experienced a common set of challenges centered around recruiting

    enough of the “right people” as trainers and leaders and getting them started in leading the

    programs. The issue of having enough people to be trained as leaders was often related to small

  • 27

    numbers of people living in rural regions (discussed above) and also related to having sufficient

    numbers of people from underserved communities to model, lead, and relate to the target

    audience of minority and low income populations. The “right people” to lead programs and train

    others are those dedicated to the work, who believe in the mission and benefits of these

    programs, and make the time to continue leading programs over time.

    A number of states set up application and screening processes to make sure that people recruited

    as potential leaders were suitable, committed, and available prior to making the investment in

    their training. California was one of the states to identify these problems and they found that

    selecting committed people was enhanced through two tools that the California Department of

    Aging developed: 1) a website survey (SurveyMonkey) to assess prospective leaders’ readiness

    and commitment level, and 2) a Leader Agreement stating that leaders would facilitate a

    minimum of two workshops per year. In Illinois, staff employed multiple strategies for leader

    recruitment. Strategies developed included on-site presentations at locations where potential lay

    leaders work, and to volunteer organizations and community groups. A list of leader

    responsibilities was developed so potential leaders understood the commitment. Peer leader

    pairing was also established, and class leaders continued to identify former class participants as

    potential class leaders as well.

    Partnership selection also was identified as important to provide sufficient numbers from

    minority affiliated organizations and communities. For example, in Oklahoma a key state level

    partner is the Health Equity Resource Opportunity Network, an organization dedicated to

    addressing issues of health disparities and providing services to underserved minority

    populations. Other states recruited personnel from federally qualified community health centers

    as leaders to serve their populations at the centers. In the four states actively targeting Native

    Americans (Arizona, Minnesota, Oklahoma, and Oregon), Tribal entities were important partners

    for both leader and participant recruitment.

    For some programs, such as EnhanceFitness, the requirements to become a program leader were

    more stringent. The more qualifications required to be a leader/trainer, the more challenging it

    was to secure an adequate number of leaders. States addressed this particular challenge in

    several ways. Some states, such as Arizona, had to limit program availability and expansion

    opportunities due to the cost and availability of the certified fitness instructors who are required

    to lead EnhanceFitness. Other states, such as Hawaii, utilized their agency networks to identify

    and recruit people to become certified instructors to lead the program. Texas developed their

    own fitness instructor training program to pipeline trainers into EnhanceFitness. Unfortunately,

    Oregon noted that it replaced the program with an arthritis physical activity program because it

    was less expensive and had less stringent leader qualification requirements.

    b. Trainer and Leader Engagement and Retention

    All states discussed the challenge of having a number of people trained to lead programs who

    were not actively leading programs. A tremendous amount of effort was expended in getting

    leaders scheduled to do the programs, and continuing to do the programs over time. States found

    that leaders needed to be engaged immediately after being trained. California coordinators try to

  • 28

    schedule workshops within three weeks after leader trainings and pair new leaders with

    experienced ones to increase the leaders’ comfort level and dedication. Centralized scheduling

    systems like the ones developed by Colorado were very helpful in keeping the available leaders

    actively leading programs.

    Depending on whether the majority of leaders were volunteers or paid agency staff, different

    challenges presented themselves. For volunteers, keeping people active and engaged was related

    to the benefits they perceived from these activities. Recognition events, such as the luncheons

    sponsored by Illinois to thank and honor volunteers, were very important in volunteer leader

    retention. Travel reimbursements or small stipends provided to volunteers helped them to

    continue their work.

    North Carolina reported that leader turnover continued to be a challenge in EBHP development

    with turnover averaging approximately 25-40% depending upon the program. They found it

    incredibly time consuming to recruit and train leaders and also reported that it was a costly drain

    of financial resources for programs with limited funding. They addressed this issue by having

    the EBHP coordinator modify, develop, and increase screening tools for new leaders and spend

    more time interviewing and discussing EBHP with prospective new leaders so that they knew

    what to expect prior to certification. Other tools such as a memorandum of agreement with both

    the leader and the organization they worked for (if not a volunteer from the community) were

    helpful in communicating guidelines, designating resources, and defining expectations.

    For paid agency personnel who were trained as leaders, often this was one more job to do for an

    already busy person. As Title III-D funding began to be used to support programs and these

    activities continue to become an expected part of the job, agency personnel will be more

    effective in providing programs. The expansion and integration goals of the Empowering People

    and Communities initiatives were nothing short of an organizational culture change mandate for

    many community-based organizations. Embedding the programs into the fabric of the

    organizations by having their personnel trained to lead and manage the programs was a goal in

    almost every state. However, when state budget cuts or program budget redirection occurred,

    this jeopardized the availability of paid staff to continue to do the programs (discussed in more

    detail below under sustainability). Systems transformation is a slow process, but having a

    qualified work force that sees this work as an important part of their job is essential. Integrating

    the value and need for the programs into the agency’s protocols enhances staff buy-in and

    solidifies role expectations. In Connecticut, when agency personnel found it difficult to

    incorporate falls prevention activities into their work, the agency’s intake and follow-up tools

    were modified to include falls prevention automatically, cross cutting agency fall prevention

    committees were developed, and fall prevention was included as a part of orientation for all new

    staff.

    3. INFRASTRUCTURE DEVELOPMENT

    There were two major challenges identified by the states regarding effective infrastructure

    development. The biggest one was leveraging funding and helping to expand systems and

    agencies to provide and manage the programs. A second challenge was identifying the partners

  • 29

    and getting them engaged in the mission. As noted in the introduction section of this report, the

    current grant projects were building on previously funded projects that also focused on

    programmatic infrastructure development. In addition, all states were required to mount their

    projects using an aging services + public health partnership. By design, this identified the two

    key partners who would anchor the infrastructure and lead the development efforts.

    Due to previous funded projects that initiated the beginning steps of building the infrastructure,

    almost all states began the grant period with some form of centralized coordination by staff at

    state level agencies. A number of states were able to develop centralized resources to support

    further infrastructure development. North Carolina utilized a centralized coordination approach

    so that all regions of the state have access to consistent templates for various resources and

    materials, as well as a database and leader tracking system. Maine state partners provided

    financial resources, technical assistance, and a state business plan draft to serve as a framework

    for local plans. Larger local and regional organizations obtained licenses to deliver CDSMP and

    the state secured a multi-site, multi-program license to assist partners with licensure costs.

    As local adoption and capacity was expanded, states often retained a centralized technical

    assistance and data collection role, and made statewide resources available through their website.

    Program coordination, fidelity monitoring, and program delivery was slowly transferred to the

    local AAA’s and other community partners. New Jersey is a prime example, with their statewide

    infrastructure for CDSMP based on shared administration/oversight within state government and

    local implementation by community partners. State level personnel costs were covered by other

    funding, with the bulk of New Jersey’s grant dedicated to providing seed money to local

    agencies to establish local infrastructure for program delivery. Texas created the Texas Strategic

    Health Partnership and reported that the state health department’s role was that of technical advisor and referral agent to local programs.

    The states’ infrastructure to deliver the programs slowly expanded like a social network through

    partnership building and seed funding. Bringing needed partners onboard was very time

    consuming and required many meetings, strategic alliances, memorandums of understanding,

    policy incentives, and tremendous state leadership. Reluctant partners were won over through

    education about the programs’ value, demonstration of return on investment, and state funding

    policies to provide public funding for programs. Most states started with small geographic

    regions that grew over time to programs being available in the majority of the state. In Texas,

    they expanded programs in three regions from 10 counties to 58, reaching older Hispanic and

    rural populations with low-incomes and high diabetes rates. In Oregon, 29 of 36 counties have

    Living Well or Tomando classes available now. The Ohio Department of Aging partners with all

    12 of their AAAs to disseminate EB programs. Their initial goal was to offer programs in six

    AAA regions, but with additional ARRA funding they were able to expand into the remaining

    six so all are now involved. California’s infrastructure grew from an initial target of seven

    counties to 32 counties; and New Jersey, Florida, and Maine report full state coverage engaging

    all AAA’s and all counties. State partnerships, as shown in Table 2, represent all community

    sectors: public and private service organizations, housing, faith-based communities, education,

    health care, business, civic groups, and others.

  • 30

    4. FIDELITY AND QUALITY ASSURANCE

    Protecting the fidelity of the EB programs is paramount in assuring that the potential beneficial

    outcomes are protected. Fidelity monitoring and quality assurance activities are an essential

    project activity for EB program delivery. For CDSMP, there are extensive fidelity tools (e.g.

    check lists), and fidelity manuals and tool kits have been produced by Stanford and several states

    including New York, Idaho, Colorado, Maine, Michigan, Oregon, Texas, and Wisconsin. A

    number of states reported well thought out protocols and activities to protect fidelity. For

    example, Massachusetts reported that in addition to recruiting and coordinating leader trainings

    in all programs, the lead community partners provided mentoring, technical assistance (including

    the development of websites for all programs), and support to newly trained leaders/coaches and

    trainers. In addition, opportunities for continuing education and sharing of best practices have

    also been available to leaders. Colorado established the Healthy Aging Service System for

    training, technical assistance, and evaluation/fidelity checks. They conducted 227 fidelity visits

    during the grant period and established Senior Assistant and Management System (SAMS).

    During this grant period, very few states reported challenges in completing needed fidelity

    activities. South Carolina noted the problem of not have enough well-trained staff at new

    agencies to do fidelity monitoring. They addressed this problem by developing standardized

    forms, as well as providing training, checklists, and technical assistance to the agencies.

    Oklahoma reported a concern that it was difficult for them to assure that participants completed

    necessary forms, but did not discuss a solution. More challenges in this area were possibly

    averted due to the awareness of the necessity of fidelity monitoring and assuring program

    quality, the wealth of tools and training materials available to states and organizations, and the

    technical assistance provided at the national level on fidelity.

  • 31

    5. EVALUATION

    There are several aspects of evaluation in the Empowering People and Communities initiatives.

    Agencies offering programs were required to collect attendance and demographic information

    from participants in order to enter the data into the national databases. State CDSMP programs

    also collect pre- and post-program evaluation forms, monitor and report drop outs, and report

    those who complete at least four of the six program sessions. Fidelity monitoring activities are

    critical, and are addressed above. Challenges reported included the timely reporting of data from

    partner organizations. California addressed this issue by adding a component to their leader

    training on the data process. The lead state coordinating organization (Partners in Care) also

    works with all licensed organizations in California on a quarterly basis to ensure CDSMP

    activity and evaluation data are recorded. Massachusetts reported that partner organizations had

    difficulty in conducting the six-month follow-up evaluations. This problem was managed by

    suggesting that partner organizations use interns and volunteers to make the six-month follow up

    phone calls. They also responded to this challenge by streamlining the survey tool and by using

    foundation grants to provide stipends to organizations submitting data forms. New Jersey

    revised the initial data collection protocol for CDSMP to eliminate the four-month follow-up

    after it was determined that the data being collected was invalid.

    While not required, states also implemented more rigorous evaluation studies, usually with the

    help of a university research partner. Figure 2 shows that 11 states had university partners.

    Typically, the evaluation studies measured the same participant outcome measures as were used

    in the original EBP efficacy studies. In one state (South Carolina), the participant outcome

    evaluation protocols were so rigorous that local agencies felt they were too burdensome and the

    effort was discontinued because it was not practical for the agencies to collect.

    D. Sustainability

  • 32

    The states approached their projects with the idea that they were building statewide partnerships

    and delivery systems that would continue after Empowering People and Communities grants

    ended. Sustaining the EB projects post-grant was a common goal identified in the original grant

    applications. To assess how well the states’ efforts and approaches could be sustained, the

    evaluators created an overall rubric that categorized a number of key features and assigned a

    score ranging from 0 (not done or no information provided) to a 4 (exemplary efforts towards

    sustaining the programs). These key features included a number of characteristics that identified

    the approaches to sustainability and the activities undertaken to maximize funding for the

    programs when the current grants ended. The features included how well states embedded the

    programs into existing public health, aging and/or community based service systems; whether

    new systems, units, or positions to support programs were established; whether new policies

    were put in place to support programs; whether addressing sustainability was central in the

    project’s design; how well documented plans for sustainability were described; and whether

    additional funding was obtained during the grant period to sustain the programs.

    Table 7: What evidence is there that the state’s programs will be sustained or replicated?

    No info or not done

    (Score = 0)

    Limited

    (Score = 1) Moderate

    (Score = 2) Major

    (Score = 3) Exemplary

    (Score = 4) Mean

    Programs were

    embedded into systems 0% (n=0) 8% (n=2) 21% (n=5) 50% (n=12) 21% (n=5) 2.83

    Established new systems,

    units or positions to

    support programs 12% (n=3) 17% (n=4) 25% (n=6) 29% (n=7) 17% (n=4) 2.21

    New policies were put in

    place to support

    programs 33% (n=8) 8% (n=2) 29% (n=7) 25% (n=6) 5% (n=1) 1.58

    Addressing sustainability

    was central in the

    project’s design 0% (n=0) 8% (n=2) 21% (n=5) 38% (n=9) 33% (n=8) 2.96

    Sustainability plans were

    documented 0% (n=0) 5% (n=1) 33% (n=8) 29% (n=7) 33% (n=8) 2.92

    Additional funding was

    obtained 8% (n=2) 12% (n=3) 21% (n=5) 38% (n=9) 21% (n=5) 2.50

    1. EMBEDDING PROGRAMS INTO SYSTEMS OF SERVICES

    As shown in Table 7, all states embedded programs into existing systems, with over 70% of the

    states doing s


Recommended