Date post: | 20-Jan-2016 |
Category: |
Documents |
Upload: | theodora-terry |
View: | 219 times |
Download: | 0 times |
INSTITUTE OF MEDICINE
Obesity Evaluation Toolkit: Resources for Evaluating Community-Level Obesity
Prevention EffortsWebinar
August 25, 2015
INSTITUTE OF MEDICINE
INSTITUTE OF MEDICINE
2
1. Introduction
Leslie Sim, M.P.H. Institute of Medicine of The National Academies of Science, Engineering, and Medicine
2. Overview of report, Evaluating Obesity Prevention Efforts: A Plan for Measuring Progress
Lawrence W. Green, Dr. P.H., M.P.H. University of California, San Francisco
INSTITUTE OF MEDICINE
3
3. Steps in evaluating community-level obesity prevention efforts
Stephen Fawcett, Ph.D.Work Group for Community Health and Development, University of Kansas
4. Web-based resources to support your efforts
Christina Holt, M.A.Community Tool Box, Work Group for Community Health and Development, University of Kansas
5. Questions
INSTITUTE OF MEDICINE
5
Evaluating Obesity Prevention Efforts:
A Plan for Measuring Progress
Lawrence W. Green, Chair; and 13 members of
The Committee on Evaluating Progress of Obesity Prevention Efforts
INSTITUTE OF MEDICINE
6
Committee on Evaluating Progress of Obesity Prevention Efforts
INSTITUTE OF MEDICINE
7
AN URGENT
NEED FOR
EVALUATION
INSTITUTE OF MEDICINE
8
What we need to know from evaluation…
• Where are we in making progress and with whom? (current status)
• How are we doing in making progress? (trends over time in assessing needs and implementation of policies and strategies)
• What works in which populations?• What are the unintended consequences?
INSTITUTE OF MEDICINE
9
Committee on Evaluating Progress of Obesity Prevention Efforts
Study Charge: “to develop a concise and actionable plan for
measuring progress in obesity prevention efforts for the nation and adaptable
guidelines for community assessments and evaluation.”
INSTITUTE OF MEDICINE
10
INSTITUTE OF MEDICINE
11
The Concise and Actionable Plan
• Enhanced Leadership• Enhanced Roadmap• Enhanced Capacity and Infrastructure
Included resources: • Indicators for the evaluation plans• Tools and methods for assessing progress
in populations at greater risk for obesity• Community health assessment,
surveillance, and monitoring of interventions
INSTITUTE OF MEDICINE
12
Interdependence of National and Community Obesity Evaluation
Plans
Community Obesity
Evaluation Plan
National Obesity
Evaluation Plan
Core indicators, Data sources, Resources,
Methodologies
Contextual data, Feasibility, Local
innovation
INSTITUTE OF MEDICINE
13
Recommendations
1. Improve Leadership and Coordination
2. Improve Data Collection
3. Provide Common Guidance
4. Improve Access to and Dissemination of Information
5. Improve Workforce Capacity
6. Address Disparities and Health Equity
7. Support a Systems Approach
INSTITUTE OF MEDICINE
14
To access the full report and related dissemination materials(url: http://iom.nationalacademies.org/Reports/2013/Evaluating-Obesity-Prevention-Efforts-A-Plan-for-Measuring-Progress.aspx)
4-page report brief
Pull-out summary of indicators
Interactive indicator widget
INSTITUTE OF MEDICINE
15
Indicators of Progress (Excerpt from Table 4-1)
Indicator Topic ObjectiveOVERARCHING/SYSTEM-LEVEL INDICATORSObesity-adult Reduce the proportion of adults who are obese (body mass
index (BMI) ≥ 30)
APOP GOAL AREA 1: PHYSICAL ACTIVITY ENVIRONMENT
Adult physical activity Increase the proportion of adults who meet current federal
physical activity guidelines for aerobic physical activity and for muscle-strengthening activity
…
INSTITUTE OF MEDICINE
16
National and State Level Indicators (excerpt, Table 6-4)
INSTITUTE OF MEDICINE
17
Community Level Indicators (excerpt, Table 7-2)
Indicator Topica Data Source Current Availability by Community Sizeb
Larger Smaller
Overarching/System-Level
Obesity-adult BRFSS
Overweight-adult BRFSS
Obesity-adolescent YRBSS, School reports
Goal Area 2: Food and Beverage Environment
Sugar-sweetened beverage policies in schools (school district)
SHPPS
Sugar-sweetened beverage consumption YRBSS (adolescent)
School policies to facilitate access to clean drinking water SHPPS
Note: Larger – population >50,000; Smaller – population < 50,000
INSTITUTE OF MEDICINE
18
Some Steps in Evaluating Community-Level Obesity
Prevention Efforts
Stephen Fawcett, Work Group for Community Health and Development, University of Kansas
INSTITUTE OF MEDICINE
19
Vision for IOM Report
Assure collection and analysis oftimely and meaningful data
to inform and improve obesityprevention efforts at national,
state,and community levels
INSTITUTE OF MEDICINE
20
CONTEXT for Evaluation
• Multi-component, multi-sector, and multi-level interventions
• Indicators/measures of varying quality & utility
• Varying capacity, capabilities, leadership, and resources
INSTITUTE OF MEDICINE
21
ACTIVITIES: Develop Resources
for Training, Technical Assistance,
and Dissemination
INSTITUTE OF MEDICINE
22
Steps/Components of a Community Evaluation Plan (Box
8-1)1. Design stakeholder involvement. 2. Identify resources for the monitoring and
summative evaluation.3. Describe the intervention’s framework,
logic model, or theory of change.4. Focus the monitoring and summative
evaluation plan.5. Plan for credible methods.6. Synthesize and generalize.
INSTITUTE OF MEDICINE
23
1. Design stakeholder involvement• Identify stakeholders
• Consider the extent of stakeholder involvement
• Assess desired outcomes of monitoring and summative evaluation
• Define stakeholder roles in the monitoring and summative evaluation
INSTITUTE OF MEDICINE
24
Participatory Evaluation (CBPR)
INSTITUTE OF MEDICINE
25
2. Identify resources for monitoring and summative
evaluation• Person-power resources• Data collection resources
INSTITUTE OF MEDICINE
26
Some Web-Based Data Sources(Table 7-5)
• Community Commons & CHNA.org
• County Health Rankings
• USDA Food Atlas
• CDC Diabetes Interactive Atlas
• Census ACS and County/Zip Business Patterns
• HHS Community Health Status Indicators
INSTITUTE OF MEDICINE
27
What indicators available at local level?Available for all/ many communities:
• Adult obesity/overweight
• Activity: Active transport by walking, bicycling, density of recreational facilities, leisure-time PA
• Nutrition: Adult F/V, food outlet density, farmers’ market density, food deserts, SNAP/WIC store density
Some larger communities also have:• Youth obesity/overweight
• Activity: Youth PA, screen time, school PA participation
• Nutrition: SSB consumption, youth F/V
INSTITUTE OF MEDICINE
Data Availability (Table 7-2 – Example)
Large Small
INSTITUTE OF MEDICINE
29
3. Describe the intervention’s framework or logic model, or
theory of change.• Purpose or mission• Context or conditions• Inputs: resources and barriers• Activities or interventions• Outputs of activities• Intended effects or outcomes
INSTITUTE OF MEDICINE
30
Generic logic model for community obesity prevention (Figure 8-1, adapted).
INSTITUTE OF MEDICINE
31
4. Focus the monitoring and summative evaluation plan.
• Purpose or uses: What does the monitoring and summative evaluation aim to accomplish?
• Set priorities by end-user questions, resources, context
• What questions will the monitoring and summative evaluation answer?
• Ethical implications (benefit outweighs risk)
INSTITUTE OF MEDICINE
32
End-User Focused Evaluation Questions—Some examples:
• How fully was intervention implemented?
• Did the intervention have desired effects?
• What was the impact on participants/ population?• With whom? • Under what conditions?
INSTITUTE OF MEDICINE
33
5. Plan for credible methods.
• Stakeholder agreement on methods• Indicators of success• Credibility of evidence
INSTITUTE OF MEDICINE
34
Some Emerging Methods forData Collection
Environmental change data• Documentation of initiatives• Unobtrusive observations• Secondary data (e.g., GIS)
Policy change data• Documentation of initiatives• Surveillance
Systems change data• Mapping changing relationships
INSTITUTE OF MEDICINE
35
Indicators of Success• Translate expected effects (logic model) into specific
measurable units • Examples include:
– Program Outputs—units of activities delivered– Intermediate Outcomes—changes in communities and
systems (program, policy, environment)– Behavioral Outcomes—changes in diet and physical
activity – Population-level Outcomes—reduced prevalence of
obesity
INSTITUTE OF MEDICINE
36
Evaluation Designs – Match to Goal/Context
Qualitative methods: interviews, focus groups, photo-voice, etc.
INSTITUTE OF MEDICINE
37
6. Synthesize and generalize.
• Disseminating and compiling studies• Learning more from implementation• Ways to assist generalization• Shared sense-making and cultural
competence• Disentangling effects of interventions
INSTITUTE OF MEDICINE
38
Logic Model Design and Shared Sensemaking
Obesity
INSTITUTE OF MEDICINE
39
Obesity Evaluation Toolkit:Web-based Resources
for Community Evaluation
CONTEXT: Distributed evaluation workforce • People we will never see• In places we will never be
TOOLKIT: Just-in-time resources for:• Training• Technical Assistance
INSTITUTE OF MEDICINE
40
Web-based resources to support your efforts
Christina Holt, M.A.Community Tool Box, Work Group for Community Health and Development, University of Kansas
INSTITUTE OF MEDICINE
41
Navigating to the Toolkit
The Obesity Evaluation Toolkit is available online:http://iom.nationalacademies.org/activities/nutrition/obesityprevprogress/resources-evaluating-community-level-obesity-prevention-efforts
INSTITUTE OF MEDICINE
42
INSTITUTE OF MEDICINE
43
What you will find
INSTITUTE OF MEDICINE
44
INSTITUTE OF MEDICINE
45
INSTITUTE OF MEDICINE
46
INSTITUTE OF MEDICINE
47
Evaluation Toolkit
INSTITUTE OF MEDICINE
48
Community Tool Box – Example Resource
INSTITUTE OF MEDICINE
49
Troubleshooting Guide
INSTITUTE OF MEDICINE
50
INSTITUTE OF MEDICINE
51
INSTITUTE OF MEDICINE
52
INSTITUTE OF MEDICINE
53
INSTITUTE OF MEDICINE
54
INSTITUTE OF MEDICINE
55
INSTITUTE OF MEDICINE
56
INSTITUTE OF MEDICINE
57
Questions/ Discussion