52
Key Performance Measures, Evaluation Plans, and Work Plan Shelley Kuklish Epidemiologist Arizona Nutrition Network
Key Performance Measures, Evaluation Plans,
and Work Plan
Shelley Kuklish
Epidemiologist
Arizona Nutrition Network
Session Objectives
By the end of this session, participants will be able to:
1. Complete the Key Performance Measures, Evaluation Plans, and Work Plan Sections of the application
2. Write S.M.A.R.T. objectives
3. Write succinct action steps
4. Identify an evaluation plan for each objective
Application
• It is recommended that you:– Complete Work Plan (Appendix B) before
completing Key Performance Measures and Evaluation Plans on the “Project Overview” sheet
Housekeeping
• We will be working on a ‘sample’ work plan one step at a time.
• Goal is that you leave here with at least one objective, one action step and one evaluation plan for that objective.– Can be used as an example for writing the rest of your FY10
work plan
• During the practice, Network staff will walk around to help as necessary.
• Be polite and tolerant of other’s opinions. – The goal of this training is to teach you HOW to write your work
plan, not WHAT you are doing to teach SNAP-ED.
Work Plan
Writing Objectives
Goals vs. Objectives
• A GOAL is:– A broad statement of intent– Non-specific, non-measurable
– Example:• Increase the proportion of SNAP recipients and
eligibles aged two years and older who meet dietary recommendations for fruit and vegetable consumption.
Goals vs. Objectives
• An OBJECTIVE is:– A realistic target for the program– Answers: Who will do how much of what by
when?
– Example:• By the end of the class series, 50% of low income
3rd grade students participating in the FVMM classes will report an increase in fruit and vegetable consumption.
S.M.A.R.T. Objectives
• S.M.A.R.T. objectives are:– Specific
– Measurable
– Achievable
– Relevant
– Time Bound
• Specific • Measurable• Achievable• Relevant• Time Bound
• What are you going to do, with or for whom?– Focus on results, not
how they are achieved.
• Specific
• Measurable• Achievable• Relevant• Time Bound
• Is the objective measurable?– Quality– Resources– Quantity– Time
• Specific • Measurable
• Achievable• Relevant• Time Bound
• Can you achieve this objective? – Set objective after
considering the baseline
– Keep objectives within reach
• Is this objective realistic?– Can we do this?– Why are we doing
this?
• Specific • Measurable• Achievable
• Relevant• Time Bound
• Is this objective logical as it relates to SNAP-Ed goals?
• Specific • Measurable• Achievable• Relevant
• Time Bound
• What is the timeframe for this objective? – For the Network
purposes, the time frame generally is by September 30, 2010
Objectives Apply to 3 Areas
• Process (Operational)
• Impact (Knowledge)
• Outcome (Behaviors)
Process Objectives(Operational)
• Tells what you are doing and how you will do it– Tracks participants, interactions and activities– Example:
• By September 30, 2010 at least 5,000 Fun Food News will be distributed to SNAP-Ed participants at community events.
Impact Objectives(Knowledge)
• Tells how you will change attitudes and knowledge– Example:
• By the end of the class series, 80% of low income 3rd grade students participating in the BBB classes will increase their knowledge of activities that help build strong bones.
Outcome Objectives(Behavior)
• Tells what the long term implications of your program will be. – Example:
• By the end of the class series, 50% of low income 3rd grade students participating in the FVMM classes will report an increase in fruit and vegetable consumption.
Easy Way to Write a SMART Objective
• Answer the following question:– Who will do– How much– Of What– By When?
• Example:– By the end of the class series, 50% of low income 3rd
grade students participating in the FVMM classes will report an increase in fruit and vegetable consumption.
Your Turn!Write An Objective Statement
• Make sure that it meets the SMART requirements– Specific – Measurable– Achievable– Relevant– Time Bound
• Answer the following questions:– Who will do– How much– Of What– By When?
By the end of the class series, 50% of low income 3rd grade students
participating in the FVMM classes will report an increase in fruit and vegetable consumption.
(FVMM survey baseline 48%)
By September 30, 2010, teach the importance of eating fruits and
vegetables through the FVMM classes to at least 500 low-income 3rd grade
students.
Writing Action Steps
Action Steps vs Objectives
• Action Step– Focus on the activities– Provide the ‘How’
• Objective– Focuses on the results– Provide the ‘What’
Action Steps
– Should work towards the objective.– Should provide how you will attain your
objective.– Be brief and succinct.
ObjectivesObjectives
Act
ion
Ste
ps
Act
ion
Ste
ps
Example:
Objective: – By the end of the class series, 50% of low
income 3rd grade students participating in the FVMM classes will report an increase in fruit and vegetable consumption.
Action Step: – 1.Teach FVMM classes to low income 3rd
grade students in 18 schools.– 2. Conduct pre-/post-surveys
Your Turn!Write an Action Step
• Using the objective statement you wrote in section 1, write at least 1 action step for this objective.
• Remember:– Action steps should work towards the objective.– Should provide how you will attain your objective.– Be brief and succinct.
By the end of the class series, 50% of low income 3rd grade students
participating in the FVMM classes will report an increase in fruit and vegetable consumption.
(FVMM survey baseline 48%)
Coordinate with 18 schools to conduct FVMM classes.
Meet with teachers to schedule FVMM classes.
Teach FVMM classes to low income 3rd grade students in 18 schools.
Conduct pre/post-test evaluations.
By September 30, 2011, teach the importance of eating fruits and vegetables
through the FVMM classes to at least 500 low-income 3rd grade
students.
Teach FVMM classes to low income 3rd grade students in 18 schools.
Writing Evaluation Plans
Evaluation Measures
• Formative – Collects data for improvement of
program/processes
• Process – Evaluates that the program is being delivered
as planned
• Impact/Outcome – Measure change in knowledge, attitudes or
behaviors
Formative Evaluation
• Formative research is conducted before a program is designed and implemented, or while a program is being conducted.
• It helps you:– Decide on and describe your target audience, – Understand the needs of the audience and the factors
which influence their behavior– Helps you to design programs, services, and products
to address those needs.
Examples of Formative Evaluation
• Needs assessments
• Focus group discussions
• In-depth interviews
• Market assessments
• Teacher evaluations
• Pilot programs where content of nutrition education is tested and evaluated, etc.
Process Evaluation
• Process evaluation focuses on how the program operates, the services it delivers, and the function it carries out.
• It addresses whether the program was implemented and is providing services as intended.
Examples of Process Evaluation
• Tracking:– Number of materials distributed – Number of clients viewing a food
demonstration – Number of classes held – Number of participants reached
Outcome/Impact Evaluation
• An outcome or impact evaluation assess the change in knowledge or behaviors as a result of a particular program or intervention.
• The goal of an outcome or impact evaluation is to learn how effective the intervention was in changing the target populations' attitudes, awareness, and behaviors.
Examples of Outcome/Impact Evaluation
• Impact– Pre-/post-test showing an increase in
knowledge
• Outcome– Pre-/post-surveys showing an improvement in
self-reported behaviors
• Point-in-time surveys, tracked over time
Example:
• Objective: – By the end of the class series, 50% of low income 3rd
grade students participating in the FVMM classes will report an increase in fruit and vegetable consumption.
• Action Steps: – 1. Teach FVMM classes to low income 3rd grade
students in 18 schools.– 2. Conduct pre/post-test evaluations.
• Evaluation Plan:– Conduct pre- and post-surveys using FVMM
consumption survey. (Baseline 48%)
Baselines
• Clearly defined starting point from where implementation begins, improvement is measured, or comparison is made
• The first data point for your evaluation– Allows you to create achievable objectives by
providing a starting point.– Allows for tracking trends over time.
Evaluation Plans Should Tie Directly to The Objective
• Process objective=process evaluation– Example:
• Objective: Teach FVMM curriculum to 1,000 students in FY 2010• Evaluation: Track number of students participating via attendance
records
• Impact objective=impact evaluation– Example:
• Objective: 5th graders increasing knowledge of osteoporosis prevention
• Evaluation: Pre/Post-test measuring knowledge
• Outcome objective=outcome evaluation– Example:
• Objective: 3rd graders increasing fruit and vegetable consumption• Evaluation: Pre/Post-test measuring consumption
Your Turn!Write An Evaluation Plan
• Remember:– Evaluation plans tie to the OBJECTIVE– Process objective=process evaluation– Impact objective=impact evaluation– Outcome objective=outcome evaluation
By the end of the class series, 50% of low income 3rd grade students
participating in the FVMM classes will report an increase in fruit and vegetable consumption.
(FVMM survey baseline 48%)
Teach FVMM classes to low income 3rd grade students in 18 schools.
Conduct pre/post-test evaluations.Conduct pre- and post- surveys using FVMM consumption survey
Coordinate with 18 schools to conduct FVMM classes.
Meet with teachers to schedule FVMM classes.
Track number of students using attendance records
Teach FVMM classes to low income 3rd grade students in 18 schools.
By September 30, 2011, teach the importance of eating fruits and vegetables
through the FVMM classes to at least 500 low-income 3rd grade
students.
Things to Note
• You can include evaluation plans for each action step, however at least ONE evaluation plan must directly apply to the objective.
Project Overview
Appendix F
• Send in samples of each evaluation tool used for impact/outcome evaluations
• Stephanie will give you more information in her presentation.
