Date post: | 31-Dec-2015 |
Category: |
Documents |
Upload: | quynn-petersen |
View: | 32 times |
Download: | 1 times |
Using RE-AIM as a tool for Program Evaluation
From Research to Practice
What is RE-AIM
• RE-AIM is an acronym that consists of five elements, or dimensions, that relate health behavior interventions:– Reach the target population– Efficacy or effectiveness– Adoption by target settings or institutions – Implementation - consistency of delivery of
intervention – Maintenance of intervention effects in individuals
and populations over time
• Commonly used for evaluating efficacy trials (Phase III Research)
Defining Elements
Reach — The absolute number, proportion, and representativeness of individuals who participate in a given program.
Representativeness refers to whether participants have characteristics that reflect the target population's characteristics. For example, if your intent is to increase physical activity in sedentary people between the ages of 35 and 70, you wouldn't test your program on triathletes.
Defining Elements
Efficacy/Effectiveness — The impact of an intervention on important outcomes. This includes potential negative effects, quality of life, and costs.
Defining Elements
Adoption — The absolute number, proportion, and representativeness of settings and staff who are willing to offer a program.
Defining Elements
Implementation — At the setting level, implementation refers to how closely staff members follow the program that the developers provide. This includes consistency of delivery as intended and the time and cost of the program.
Defining Elements
Maintenance — The extent to which a program or policy becomes part of the routine organizational practices and policies. Within the RE-AIM framework, maintenance also applies at the individual level.
At the individual level, maintenance refers to the long-term effects of a program on outcomes after 6 or more months after the most recent intervention contact.
How do elements relate to planning?
• As you design, plan, or evaluate a health behavior intervention, there are questions that you should ask yourself.– Reach: HOW DO I REACH THE TARGETED
POPULATION FOR INTERVENTION?– Efficacy or effectiveness: HOW DO I KNOW
THAT MY INTERVENTION IS EFFECTIVE?– Adoption HOW DO I DEVELOP
ORGANIZATIONAL SUPPORT TO DELIVERY THE INTERVENTION?
How do elements relate to planning?
• As you design, plan, or evaluate a health behavior intervention, there are questions that you should ask yourself.– Implementation: HOW DO I ENSURE THE
INTERVENTION IS DELIVERED PROPRLY?– Maintenance: HOW DO I INCOPROATE THE
INTERVENTION SO IT IS DELIVERED OVER THE LONG TERM?
How is RE-AIM different from other evaluation approaches?
RE-AIM draws upon previous work in several areas including:
Diffusion of innovations, multi-level models, and Precede-Proceed.
The primary ways that it is different is that it (a) is intended specifically to facilitate translation of research to practice, (b) it places equal emphasis on internal and external validity issues and emphasizes representativeness, and (c) it provides specific and standard ways of measuring key factors involved in evaluating potential for public health impact and widespread application.
Is RE-AIM used to design programs, or just to evaluate them?
• It is both. Although used more commonly at present to report results or compare interventions, it is also intended as a planning tool.
An Example
• Physical Activity Promotion in Primary Care: Bridging the Gap Between Research and Practice– Eakin, Brown, Marshall et al. (2004).
Translating and Disseminating Evidence-based Falls Prevention Programs into Community
Li et al. (2008). American Journal of Public Health
An Example of Application
Primary Aim:
(a) To translate an evidence-based Tai Chi exercise fall intervention into a community-based program for implementation with older adults; and
(b) Using the RE-AIM framework (Glasgow et al., 1999), to pilot implement the program with a primary focus on reach, uptake (adoption), and implementation.
Secondary Aim:• To evaluate the effectiveness of the program
with respect to improvements in physical performance measures germane to falling.
Tertiary Aim:• To evaluate program maintenance with
respect to the extent to which older adults would continue to practice Tai Chi beyond the period of the initial evaluation.
Translation
Translate an evidence-based Tai Chi exercise fall intervention into a community-based program for implementation with older adults
• Identify training objectives and elements• Identify end users and dissemination partners• Develop a dissemination package• Expert evaluation • Pilot testing
Program Evaluation
Reach: A total of individuals/providers responded to the program promotion
Effectiveness: defined as change in physical performance outcome measures taken at baseline and again at 12 weeks termination.
Adoption (or uptake): defined as the proportion of local community (senior activity) providers that agreed to participate and implement the program.
Implementation: defined as the extent to which providers' implemented a Tai Chi class to participating older adults and the ability to conduct the various elements of program protocols, including the use of implementation plan, a 2-times weekly program schedule, distribution of program supplements (i.e., videotape and a guidebook), a class attendance rate of 75% or better over the 12-week class period.
Maintenance: defined at both the service provider level and participant level. At the provider’s level, it was defined as providers’ willingness to consider the program to be part of routine organizational provisions (assuming adequate financial resources). At the participant level, it was defined as the extent to which improvements in participants’ physical performance were sustained, and their continued practice of Tai Chi, 8-weeks after completion of the class.
REACH: 87% (by study criteria); 45% by client attendance
Dissemination Outcome
ADOPTION: Six senior activity centers from five communities: 100% adoption
IMPLEMENTATION: 75% completed; >85% class attendance; average 32 min. of home practice
Effectiveness: Improved physical performance and quality-of life measures (a)Functional Reach; (b) Up and Go; (c) Chair Stands, (d) 50-foot speed walk, and SF-12.
Maintenances: Five centers continued; 87% participants continued
Dissemination Outcome
ImpactAdopted by the State of Oregon: being implemented in four counties; three more starting this year
Conclusion-Tai Chi
The evidence-based Tai Chi program is practical to disseminate and can be effectively implemented and maintained in community settings