Grantmaking 101: Developing Assessment Measures that provide meaningful
information for grant makers and nonprofit leaders
Paul Watson, MSHSWilliam T. Oswald, Ph.D.
Global Action Research Center
January 17, 2019
www.theglobalarc.org
Key Questions
Who will be best served by different types of data, with discussions on
the perspectives of nonprofits, funders, and community stakeholders?
How you go about developing assessment measures that accurately
reflect the work of your grantees, while respecting your grantees’ time
and resources?
How do you use different qualitative and quantitative data points to paint
a more holistic picture of impact.
What are some different approaches to collecting the data you need?
Pros and cons of different data types and collection methods.
Workshop AgendaSome Basics
Authentic PartnershipsTheory of Change
Logic Model
Appreciative Inquiry
Connecting Outcomes to GoalsOperational Definitions
We Do What We Measure
Assessing Product v ProcessAssessing Connection between Goal and Deliverables
Basic Assumptions 1. Its all about the funder-grantee relationship
2. Numbers don’t lie people do
3. Numbers have no meaning except what we give them
4. We do what we measure
5. There is an uneasy alignment between our work and models of
assessment
6. Good data is everywhere
Why We Do AssessmentsBECAUSE
we want to capture what is happening in ways that inform us about the
connection across our
INVESTMENTS(the funding)
OUTPUTS(what we do/produce)
OUTCOMES(how it affects those we touch)
IMPACTS(how it changes conditions)
There are many ways to capture what we want to know – there is no one method that fits all situations.
How data is collected and who collects it impacts the quality of the data.
Be very clear as to why you want the data and how you will use it before you design your assessment.
Authentic Partnerships Require Trust
Appreciative Inquiry & Becoming a Learning Organization
Appreciative Inquiry (AI) is a change management approach that focuses on identifying what is
working well, analyzing why it is working well and then doing more of it. The basic tenet of AI is
that an organization will grow in whichever direction that people in the organization focus their
attention.
A Learning Organization…is one where people continually expand their capacity to create the
results they truly desire, where new and expansive patterns of thinking are nurtured, where
collective aspiration is set free, and where people are continually learning to see the whole
together.
How to move beyond “Bean Counting” and move goals forward while
getting a wholistic assessment of the impact of the foundation’s
investment
Building a Trusted Partnership
The five main characteristics that
Peter Senge identified are said to
innovate a learning organization.
These are:
1. Systems thinking,
2. Personal mastery,
3. Mental models,
4. Building shared vision,
5. Team learning.
Requires:
1. Information is Shared and
Accessible
2. Learning is Emphasized and
Valued
3. Mistakes and Failures are not
Punished
4. People are Expected to Learn
Constantly
By building relationships that feed the conditions for creating Learning Organization
Collectively setting of benchmarks and jointly assessing progress towards those
benchmarks on a regular basis
Logic Model
The LOGIC MODEL, because of
its focus on “outcomes”
provides a good road map for
accomplishing goals and
identifying the resources
needed to achieve the goals.
Theory of Change
Long-Term
Outcome
Preconditions
Preconditions
Preconditions Preconditions
PreconditionsPreconditions
Intervention Intervention Intervention
Interven
tion In
terv
enti
on
Indicators
IndicatorsIndicators
Indicators
IndicatorsIndicators Indicators
A
B
Connecting Outcomes to Goals
Numbers Don’t Lie – People Do
Numbers have no meaning except that which we give them
Operational Definitions
Concept
Indicators
Measure Measure
Indicators
Measure Measure
What is the impact you are
hoping to achieve?
What will tell you that you’ve
reached or are moving toward
your desired impact?
How will you assess the
existence and strength of the
Indicators
Operational Definition: To test an idea, one must gather data. This means relating your
concept to concrete, measurable events in the world. To bridge that gap, scientists need
operational definitions. These are definitions that specify how to measure or detect
something.
We Do What We MeasureConcept
Indicators
Measure Measure
Indicators
Measure Measure
Civic Engagement
Interest in public affairs
Increase in the number of people voting
Increase in the number of people attending
specifically identified public meetings
Connected to the community
The number of people volunteering at the
local schools
Self-report on "feelings of connected" to
community
DeliverablesShape
the
work
Informs
What is the
relationship between
the deliverables & the
goal?
Appreciative Inquiry within a Learning Organization constantly assesses alignment and
direction of movement ~AND~ Changes the nature of the relationship, laying the ground
work for an authentic partnership
• Voter turn out will increase by 5%
• Attendance at Town Council meetings will increase by 10
attendees
• Pre-Post Survey of at least 100 residents shows increase
in feelings of connectedness to the neighborhood
Assessing Product v Project
Caution: Programs/Projects create products AND impact process
Products are easily measured – Process is not
Products are the number of voters, attendance at meeting, etc.
Process relates to changes in how decisions are made and who makes them – a change in the
power dynamics of the community.
Notice that two of the three deliverables on the previous screen are “outputs” or “products” only one
attempts to assess process – and even that does not assess actual change in community power.
If it is true that
We do what we measure?
How do we shape our measures and deliverables
in ways that are true to the project’s goal?
Exercise
Concept
Indicators
Measures
Deliverables
Questions:1. Assume all deliverables are met:
• How did/will the project achieve or move
toward its goal? (What is the Chain of
Change?)
• If there is change – how sustainable is it?
2. How is the assessment benefiting:
• The project
• The Foundation
• The Targeted Population
• The people working on the project
We Do What We Measure
Concept
Indicators
Measure Measure
Indicators
Measure Measure
Civic Engagement
Interest in public affairs
Increase in the number of people voting
Increase in the number of people attending
specifically identified public meetings
Connected to the community
The number of people volunteering at the
local schools
Self-report on "feelings of connected" to
community
DeliverablesShape
the
work
Informs
What is the
relationship between
the deliverables & the
goal?
• Voter turn out will increase by 5%
• Attendance at Town Council meetings will increase by 10
attendees
• Pre-Post Survey of at least100 residents shows increase
in feelings of connectedness to the neighborhood
Operational Definitions
Concept
Indicators
Measure Measure
Indicators
Measure Measure
What is the impact you are
hoping to achieve?
What will tell you that you’ve
reached or are moving toward
your desired impact?
How will you assess the
existence and strength of the
Indicators
Operational Definition: To test an idea, one must gather data. This means relating your
concept to concrete, measurable events in the world. To bridge that gap, scientists need
operational definitions. These are definitions that specify how to measure or detect
something.
Assessing Our Assessments
Closure
Name one thing you will take with you – that you
will think about as you go back to work
