Data Collection**Some content courtesy of the Mississippi Service Commission

Presenter: Octavia Kuransky, MSP

Funders do not fund great causes – they fund great results.

Great results

Great Causes

Funding

Agenda• Why Collect High Quality Data?• What is Good Data?• Choosing a Data Collection (DC) Tool• Creating a DC Implementation Plan• Go Forth and Data.

Why Collect High Quality Data?• Data - evidence to support your program intervention• Accountability• Solid case for future funding• Tell your story

• See what’s working and what’s not to inform change• Allow you to isolate elements• Comparable to other program interventions• Tweak implementation

• It’s required

What is Good Data?• Reliable: instrument & implementation produce

consistent results under the same conditions• Implement the instrument in a uniform way• Make sure the instrument is clear and unambiguous

• Valid: instrument & implementation accurately measure what you intended• Instrument should reflect the real info to answer your questions• All dimensions of the outcome• Corroborated by other evidence

• Unbiased: DC should not include any systematic distortion of results• Look at who responded, how the tool is worded, the timing &

location and ensure an even distribution of data is received

How Do You Get the Data?• Many kinds of instruments to record data (not just

surveys!)• Surveys• Interviews• Observation• Standardized tests• Tracking sheets• Focus Groups• Diaries or journals• Secondary Data• Databases

Where Do I Start?• Revisit your Theory of Change• What did you say you would do? (Outputs)

• Dosage• Activities

• What did you say will change? (Outcomes)• Reasonably measurable within the scope of the funding• Include all elements of changes

• By how much? (Outcomes)• Your target numbers

• The data that is reasonable for you to measure (at a minimum, Performance Measures) is what you are looking for.

Who Has the Data?• What is the best source of the data you need?

Consider:• What KIND of data is reflected in your TOC (change in

knowledge, attitudes, conditions, behavior)?• Attitude/Knowledge = person experiencing the change• Condition/Behavior = EITHER person exp. change OR other qualified

observers

• Who is in a position & qualified to provide that data?• Who do you have access to? • Who has a vested interest in providing you with data?

• Consider existing data sets: will they accurately reflect your intended outcomes?• List the Pros and Cons of each data source

How Do You Get the Data? Part 1• Find an Instrument• CNCS Performance Measures• Field Best Practices • Other Studies on programs similar to yours

OR

• Create an Instrument• Engage a professional• Do it yourself

How Do You Get the Data? Part 2Evaluating Instruments

• Consider feasibility• Ease of use/implementation

• Making sure the instrument can reasonably be used within your program structure

• Consider any training for staff/data collectors• Consider privacy concerns

• Ease of data analysis• Objective vs subjective data• Converting collected data to answers about outcomes

How Do You Get the Data? Part 3Evaluating Instruments • Does it measure your outcomes adequately?• Matches intervention• Appropriate for beneficiaries• Measures all relevant dimensions of the outcomes

• Is the tool designed well?• Too many, too few questions• Clear, unbiased questions that will be interpreted uniformly• Consistent question format

Implementing Data Collection • Identify DC plan participants• Who should be involved & what will their roles be?

• Set a schedule for data collection• Who will administer DC instruments?• When will you collect, analyze and report on data?

• Prepare your team for DC• Plan to train DCers on the instrument and how to administer

Implementing Data Collection

• Pilot the instrument• Small group similar to respondents• Analyze pilot data

• Make any necessary changes to the instrument or plan and then implement

Review• Data collection should provide the info you need to provide

evidence that your Theory of Change is correct• Instruments should be chosen carefully to ensure they

• Collect the correct info• Can be administered ensuring reliability, validity, and freedom from

bias

• Each program should craft a collective plan to:• Engage all stakeholders & understand why data collection is

important & how it will be used• Choose a strong instrument• Ensure that data collection is implemented uniformly and timely• Ensure that data is analyzed and used in decision making &

reporting

One accurate measurement is worth more than a thousand expert opinions.

Admiral Grace Hopper