Step 2. Describe ProgramLogic Models & Outcome Measurement

Luann D’Ambrosio, MEd

Exercise

• Improving on a hobby…

• Expectations by you and your agency

CDC Evaluation Framework

Overview

• Measuring and articulating what we do and what we accomplish

• Why?

• What does this process look like?

• What does it mean?

• How might I do this?

• What next?

Exercise

• When did you have to articulate what you do at work and why? To whom?

• How did it go?

Goals

• Understand the basics of performance/outcome measurement for our services

• Appreciate the importance of measurement systems for public health activity

• Identify components and uses of a logic model

• Describe outcome measures and why they are important

Objectives

Outcome measurement is a systematic way to assess the extent to which a program has achieved its intended results.

Yesterday: More often focused on what we did.

Today: Focusing on what happens when we do it!

Ways to Think About Outcome Measurement

• Performance measurement

• Evaluation

• Logic models

These are not mutually exclusive!

Why Do We Need to Do This?

• To provide accountability

• To improve program quality

• To make decisions about resource allocation

• To help programs market themselves

• To understand our contribution to our community’s health

“Everyday” Example: Marathon Runner

• Goal: To do better than last year’s time

• Resource: New running shoes

• Activities: Running workouts

• Outputs: Train 5x per week; run increasing #s of miles

• Outcome: Training times/mile, overall time/distance, energy level

How Do We Do This?

• Set a goal

• Define activities

• Identify inputs or resources

• Define outcomes

• Develop outcome measures

• Build consensus around these measures

Activities Activities OutputsOutputs OutcomesOutcomes GoalsGoalsResources Resources

Logic Models: “So That” Chains

so that

Activities Activities OutputsOutputs OutcomesOutcomes GoalsGoalsResources Resources

so that so that so that

Logic Models: Public Health Example

StaffStaff Immunization clinicImmunization clinic

ShotsShots Less VPDLess VPD Healthier communitiesHealthier communities

RESOURCES ACTIVITIES OUTPUTS GOALSOUTCOMES

so thatso that so thatso that so thatso that so thatso that

Exercise

Create a So That chain for your project or program.

Outcome Evaluation Logic Model

OutputsHow much do you do for how many people?

Program inputs. Elements or ingredients that constitute the program.

Program inputs. Elements or ingredients that constitute the program.

Methods for providing the program. Specific processes or events undertaken.

Methods for providing the program. Specific processes or events undertaken.

Units of service or product units. How many, over what duration?

Units of service or product units. How many, over what duration?

Short, intermediate, or longer-term changes anticipated in participants’ lives or in organizational or community conditions.

Short, intermediate, or longer-term changes anticipated in participants’ lives or in organizational or community conditions.

Ultimate impact(s) expected to occur, usually beyond what one program alone can achieve.

Ultimate impact(s) expected to occur, usually beyond what one program alone can achieve.

Developing Logic Models

Process

RESOURCES

Outcomes

ACTIVITIES OUTPUTS GOALSOUTCOMES

Why a Logic Model?

A logic model

• Clarifies each program element

• Shows the relationship of resources and activities to expected outcomes

• Summarizes how program parts relate to the whole

• Makes explicit the theory of how the program works

• Identifies categories to measure in the program evaluation

Exercise

Start your own (work related) logic model (individually)

Logic Models: “So That” Chains

so that

Activities Activities OutputsOutputs OutcomesOutcomes GoalsGoalsResources Resources

so that so that so that

Exercise

Logic model scramble (everyone)

Outcome Levels

• Individual performance outcomes

• Program outcomes

• Agency outcomes

• System outcomes

• Community outcomes

Program inputs. Elements or ingredients that constitute the program.

Program inputs. Elements or ingredients that constitute the program.

Methods for providing the program. Specific processes or events undertaken.

Methods for providing the program. Specific processes or events undertaken.

Units of service or product units. How many, over what duration?

Units of service or product units. How many, over what duration?

Short, intermediate, or longer-term changes anticipated in participants’ lives or in organizational or community conditions.

Short, intermediate, or longer-term changes anticipated in participants’ lives or in organizational or community conditions.

Ultimate impact(s) expected to occur, usually beyond what one program alone can achieve.

Ultimate impact(s) expected to occur, usually beyond what one program alone can achieve.

Developing Logic Models

Process

RESOURCES

Outcomes

ACTIVITIES OUTPUTS GOALSOUTCOMES

Exercise

Further develop your own (work related) logic model (individually)

Reading Logic Models

11

Certain resources are needed tooperate your Program.

Certain resources are needed tooperate your Program.

If you have access to them,then you canuse them toaccomplish your plannedActivities.

If you have access to them,then you canuse them toaccomplish your plannedActivities.

If you accomplishyour plannedactivities, thenyou willhopefullydeliver the amount of product or service that you intended.

If you accomplishyour plannedactivities, thenyou willhopefullydeliver the amount of product or service that you intended.

If you accomplishyour plannedactivities tothe extent you intended, then your participantswill benefit in certain ways.

If you accomplishyour plannedactivities tothe extent you intended, then your participantswill benefit in certain ways.

If these benefitsto participantsare achieved, then certainchanges inorganizations, communities, or systems might be expected to occur.

If these benefitsto participantsare achieved, then certainchanges inorganizations, communities, or systems might be expected to occur.

Resources/InputsResources/Inputs

ActivitiesActivities OutputsOutputs OutcomesOutcomes ImpactImpact

Your planned work Your intended results

22 33 44 55

What Next With Outcome Measures?