Sponsored by:A Service
Of:
Evaluation Quick Tricks and Easy Tools for Nonprofits
Sheri Chaney Jones
February 13, 2012
Sponsored by:A Service
Of:
Protecting and Preserving the
Institutional Memories of
Nonprofits Since 1993
www.cjwconsulting.com
(866) 598-0430
Sponsored by:A Service
Of:
Affordable collaborative data
management in the cloud.
Sponsored by:A Service
Of:
Today’s Speaker
Sheri Chaney Jones President
Measurement Resources Company
Hosting:
Cheri J Weissman, CJW Consulting & Services, Inc.Assisting with chat questions: April Hunt, Nonprofit Webinars
Measurement Resources Company www.measurementresourcesco.com 614-893-0773
Facts are better than dreams”◦ -Winston Churchill
“Great organizations gather, confront, and use the brutal facts to make organizational decisions.”◦ -Jim Collins “Good to
Great”
Measurement Resources Company www.measurementresourcesco.com 614-893-0773
Good Measurement Practices
Measurement Resources Company www.measurementresourcesco.com 614-893-0773
Measurement Resources Company www.measurementresourcesco.com 614-893-0773
Comparison of Measurement Culture and Effective Accomplishment of Organizational Outcomes
Organizational Outcomes% High Culture
Reporting Effectiveness
% Moderate Culture
Reporting Effectiveness
% Low Culture Reporting
Effectiveness
Organizational Efficiencies83% 50% 17%
External Relations75% 50% 33%
Internal Relations83% 27% 17%
Increased Revenues67% 23% 17%
Strengthening Organizational Culture 100% 64% 17%
Successful Organizational Change 75% 27% 17%
Not Enough Time 68%
Not Enough Funding 51%
Not Enough Evaluation
Expertise 50%
Not Enough Staff 49%
Not Enough Trained Staff 46%
Measurement Resources Company www.measurementresourcesco.com 614-893-0773
Source: Carman, J. & Fredericks, K. (2010) Evaluation Capacity and Non-Profits Organizations. American Journal of Evaluation. 31 (1) 84-101
45%
45%
10%
Non-Profit
Evaluation
Fully Utilized
Challenges
Signifcant Struggle
Evaluation is Essential
◦ It is easy
◦ It is fun
◦ It can fit in your budget
◦ Done with your staff
◦ Lead to positive outcomes
Measurement Resources Company www.measurementresourcesco.com 614-893-0773
Situation 1: FY10-11 Budget: Significant
budget cuts resulting in long waitlists for
programs
Measurement Resources Company www.measurementresourcesco.com 614-893-0773
11
Culture
Clarify
Capture
Communicate
Change and/or Celebrate
Measurement Resources Company www.measurementresourcesco.com 614-893-0773
Measurement Resources Company www.measurementresourcesco.com 614-893-0773
“Take away my people, but leave my factories, and soon grass will grow on the factory floors. Take away my factories, but leave my people, and soon we will have a new and better factory.”-- Andrew Carnegie (1835-1919)
Evaluation more than external accountability tool
Variety of data collected and used
Positive internal management
Results educate and bond
Measurement Resources Company www.measurementresourcesco.com 614-893-0773
High Measurement Culture– Already been
working towards this -- staff satisfaction
surveys, 360 degree leadership assessments,
performance goals, regularly generating
feedback from staff, “coins in the cushion
program”
Measurement Resources Company www.measurementresourcesco.com 614-893-0773
15
Measurement Resources Company www.measurementresourcesco.com 614-893-0773
Measurement Resources Company www.measurementresourcesco.com 614-893-0773
“Most communicated and relevant measures get done”
Must have clear mission/ vision in order to evaluate and measure what is important.
Measurement Resources Company www.measurementresourcesco.com 614-893-0773
Why do you do what you do?◦ Why, so that◦ Why, so that◦ Why, so that◦ Why, so that◦ Why, so that
Why do your funders/ partners what to partner with you?
Why do your participants/ clients participate? What do they hope to accomplish?
Before After
Long and lengthy
mission and vision
statement that no one
could remember
Very siloed and each
program thought of
goals separately
“Promoting choice, independence, and quality of life for all Aging Ohioans”
Measurement Resources Company www.measurementresourcesco.com 614-893-0773
19
Measurement Resources Company www.measurementresourcesco.com 614-893-0773
Logical linkages among ◦ program resources (inputs), ◦ activities (outputs), ◦ Audiences (participants), ◦ and short-, intermediate-, and long-term outcomes.
This is the framework for demonstrating how your program or services will achieve the funders expected outcomes (your “whys”)
Resources: http://www.wkkf.org/knowledge-center/resources/2010/Logic-Model-Development-Guide.aspx
20
Measurement Resources Company www.measurementresourcesco.com 614-893-0773
21
Measurement Resources Company www.measurementresourcesco.com 614-893-0773
Measurement Resources Company www.measurementresourcesco.com 614-893-0773
Demographic data on participants
Activities/ services received◦ Start and end date
Pre-test, post-test data◦ Knowledge, skill, attitudes, readiness, etc.
Costs
Participant and stakeholder feedback (satisfaction data)
Outcomes (Your True Profit)
1. Increased knowledge and learning
2. Changed attitudes
3. Increased readiness
4. Reduction of undesirable behavior
5. Increased desirable behavior
6. Maintenance of new behavior
7. Increased social status
8. Increased economic conditions
9. Increased health conditions
10. Reduction in administrative costs
11. Increased economic development
12. Participant, employee, and/ or stakeholder satisfaction
Measurement Resources Company www.measurementresourcesco.com 614-893-0773
24
1. Are these measures related to important policy and practice issues?
2. Are these measures relevant and apply to practitioners?
3. Can these measures be communicated in a way that will influence outcomes
4. Are incentives built into the system for collecting and acting on these data?
5. Are the measures linked to the stakeholders interests?
Measurement Resources Company www.measurementresourcesco.com 614-893-0773
25
18% 25%
82%
75%
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
FY 2008 FY2011 Goal
Ratio of Expenditures for Home and Community Based Care/ Nursing
Facility Utilization for Clients over 60
Home/ Community Care Nursing Facility Care
39% 44%
61%
56%
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
FY 2008 FY2011 Goal
Ratio of Individuals over 60 for Home and Community Based Care/ Nursing
Facility Utilization
Home/ Community Care Nursing Facility Care
Measurement Resources Company www.measurementresourcesco.com 614-893-0773
26
Measurement Resources Company www.measurementresourcesco.com 614-893-0773
Measurement Resources Company www.measurementresourcesco.com 614-893-0773
“The greatest problem with communication is the illusion it has been accomplished.”--George Bernard Shaw
Share your story with all stakeholders
Try different methods
Measurement Resources Company www.measurementresourcesco.com 614-893-0773
Measurement Resources Company www.measurementresourcesco.com 614-893-0773
Use results to confidently make improvements
Celebrate your impact
Change Celebrate
Increased communication and awareness around the relationship between home and community-based waivers and nursing facility funding.
Clearly showed savings
A partnership between state agencies formed to use this framework to estimate diversion and transition efforts.
Program funding fully restored in March 2010
“If mom and dad want to stay at home or go into a nursing home, we should make that happen…and taxpayers will save a bundle of money” Governor Kasich 3/7/11
Measurement Resources Company www.measurementresourcesco.com 614-893-0773
31
Measurement Resources Company www.measurementresourcesco.com 614-893-0773
Culture
◦ Measurement Culture Survey
Clarify
◦ Five “Whys”
Capture
◦ Excel
◦ Survey Monkey, Zoomerang
Communicate
◦ Excel graphs
◦ Survey Monkey Reports
◦ Meetings, website, press release, funding applications, bulletin board
Change and Celebrate!
Measurement Resources Company www.measurementresourcesco.com 614-893-0773
Measurement Resources CompanySheri Chaney Jones
614-893-0773
www.measurementresourcesco.com
Sponsored by:A Service
Of:
Find listings for our current season of webinars and register at:
NonprofitWebinars.com