Date post: | 29-Dec-2015 |
Category: |
Documents |
Upload: | bryce-hamilton |
View: | 219 times |
Download: | 2 times |
Program Evaluation
Homeless Education State Coordinators Meeting
February 3, 2009
Key Bridge Marriott
Arlington, VA
Program Evaluation
“Program Evaluation” and
“Performance Evaluation”
are comprehensive
terms encompassing two
critical components:
Project Design Project Evaluation
Project Design - Outline
Review performance evaluation purposes and processes
Create sample outcomes and activities with criteria and measures
Assess/create your outcome measures with SMART processes
Performance Evaluation
Performance measurement:
a process that
systematically evaluates whether your efforts are making an
impact/change on/to the clients you are serving or the problem you are targeting.
Why should programs be interested in performance measurement?
We are all in the business of helping people; we need to…
• understand whether current activities are working to achieve intended results.
We are all in the business of helping people; we need to…
• understand whether current activities are working to achieve intended results.
• drive program improvement and share information on effective practices with others.
Why should programs be interested in performance measurement?
Why should programs be interested in performance measurement?
We are all in the business of helping people; we need to…
• understand whether current activities are working to achieve intended results.
• drive program improvement and share information on effective practices with others.
• acknowledge that high-performing programs receive credibility/funding especially in competitive processes.
Creating SMART Objectives With Measurable Outcomes
S pecific
M easurable
A ctivity-oriented
R ealistic/feasible
T ime-oriented
Building Blocks of Performance Measurement
Inputs
include resources dedicated to, or consumed by the program—
e.g., money, staff and staff time, volunteers and volunteer time, facilities, equipment and supplies.
Building Blocks of Performance Measurement
Activities
are what the program does with the inputs to fulfill its mission–
e.g., providing school supplies or staff training, conducting a tutoring or summer program.
Building Blocks of Performance Measurement
Outputs
are the direct products of program activities, usually represented in terms of the volume of work accomplished—
e.g., number of students served and the number of staff trained—
and are often confused for outcomes.
Building Blocks of Performance MeasurementOutcomes
are benefits or changes among clients during or after participating in program activities. Outcomes relate to measurable change in student knowledge, behavior, skills, conditions, or other attributes. Outcomes impact the target you are addressing, which for us in generally the kids we serve.
Building Blocks of Performance Measurement
Outcomes (continued)
How does the activity that is engaged in impact on the target?
Performance Measurement Process
ActivitiesTrainingTutoring
Summer programField trips
School supplies
Outputs# staff trained on MV
# of students in tutoring# of students in summer
program# of students receiving school
supplies
Outcomes30% more identified40% increase TAKS 25% incrs reading
retention50%increased
attendance
How do we document
our efforts?
What did our efforts achieve?
Inputs
ProgramsInfrastructure
StaffPartnersSupplies
Should we adjust how we spend
our resources?
Should we add or
change use of resources
to expand our impact?
Outputs vs. Outcomes
An outcome is:An output is: Focused on what the
student will gain/how will change
Focused on what the program will do (activity) to achieve the outcome. A way to measure the
student-level impact with clear targets and methods for measuring change.
A way to quantify the frequency and intensity of the activity. Reasonably attributable
(a result of) to an output or outputs
Specific to the activity described
Meaningful and attainable.
Feasible and attainable.
Be mindful to distinguishbetween outputs and outcomes.
If outcomes show the program works, then outputs are needed to understand how to replicate results
Outcome, Output or Neither?
School supplies distributed to 150 campuses
90% of homeless students pass the state assessments
75% of all LEAs’ staff will be MV trained
40% of homeless students improve reading by one grade level
Homeless student attendance improves by 50%
Examples Answers
Output
Outcome
Output
Outcome
Outcome
Linking Outputs to Outcomes
Outputs Outcomes
75% of homeless students who attend after school tutoring will increase reading by one grade level
Homeless students’ attendance will increase by 50% statewide
100 % of students in homeless situations are enrolled immediately
95% of homeless students in LEAs attend after school tutoring
100 percent of homeless students receive school supplies and uniforms
LEA staff statewide staff are MV trained
What data will you need to collect?
How will you collect it?
What system will you put in place to collect the data?
What criteria will tell you that you are on the right track?
When do you need to begin collecting data?
Collecting the Necessary Data
Budget Support and Alignment
Do large categories and line items support activities?
How will you know if your money is spent in the most effective way? How will you stay on top of anticipated expenses? What will you do to build a relationship with your business office? Are there business office practices that make it difficult for you to work from current figures, or know what invoices are paid?
Project Design: Summary
1. Need (with baseline reference)
2. Objective(s)
3. Anticipated Successful Target Outcome(s)
4. Timeline(s)
5. Inputs
6. Activities
7. Anticipated Outputs
Project Design: Summary
See sample schedule for
converting this project
design into a planning and
implementation document.
Project Evaluation: Summary
Objective(s)
Actual Outcome compared to Anticipated Target Outcome
Activity(ies)
Criteria used to measure activities
Project Evaluation: Summary
Actual Outputs compared toAnticipated Outputs
Analyze Status (mid-year and end of the year)
What worked? What didn’t?
Changes
What will you change to moreclosely align anticipatedoutcomes to actual outcomes –mid year or for next year?
Project Design and Evaluation: Developing M-V Outcomes
Objective:
What do you hope to accomplish?
Outcome (measurable):
How will you know you’ve accomplished it? What will tell you that you have been successful?
Project Design and Evaluation: Developing M-V Outcomes
Activities:
What activities will you put in place to accomplish your outcome?
Criteria:
What criteria will you look at to see if the activities are being completed?
Project Design and Evaluation: Developing M-V Outcomes
Measure/Output:
What will be the measure (output) of each criteria/activity to tell you that you that you are on track to meet your outcome?
Project Design and Evaluation: Developing M-V Outcomes
See sample Project Design
and Project Evaluation
handout developing a
statewide M-V outcome for
Florida.
Project Design and Evaluation: Group Activities
See sample M-V outcome
activity sheets, or develop
outcomes selected by the
group.
Special Thanks To:
Brooke Spellman &
Michelle Abbenante
For sharing their knowledge and expertise
regarding program evaluation, and for
granting their permission for to use/revise
much of their presentation delivered at the
NAEH Conference on Ending Family
Homelessness - February 8, 2008