GET WHAT YOU NEED FROM

ONLINE SURVEYSJanelle Commins, MSPH

Research AssociateEMT Associates, Inc.

15720 Ventura Blvd., PenthouseEncino, California 91436

Topic Overview

2

When is an online survey appropriate? What do I have to choose from?

Commercial or custom-built How do I get people to take MY survey? Case Study: LA County Service Provider

Survey.

When are Online Surveys Appropriate?

3

Respondent factors. Internet access, geographic spread.

Questionnaire factors. Type of question, length of questionnaire.

Evaluator factors. Time frame, budget, technological

expertise.

What Are My Options?

4

Commercial

Software

Custom-Built

Web-hosted or off-the-

shelf

JAVA

Convenience (for evaluator)

X

Convenience (for user) X X

PC Compatibility X

Complex Skip Logic X

Foreign Languages X

Real-time Data Feeds X X

Look and Feel X

Endless Possibilities X

How Do I Get a Good Response?

5

What IS a good response? Depends on population and relevance. For service providers: 2-45%; median 13%. For market research < 2%.

Recruiting Strategies

Get executive buy-in Send a personal letter to the Executive or

Research Director Provide summary sheet with quick facts.

Present at collaborative meetings. Cold-calling.

Response Strategies

Distribute survey invitations through a trusted intermediary.

Send a pre-announcement email. Emphasize relevance and simplicity

Avoid using the word “survey” in the Subject line.

Case Study: The Los Angeles County Service Provider Online Survey

What is the Los Angeles County Service Provider Online Survey?

9

• First 5 LA-sponsored.• Countywide survey of agency decision-

makers.• Custom-built (JAVA).• Evidence-based recruitment,

distribution, and formatting.

Recruitment Strategy

10

Executive buy-in at public institutions. Presentation at collaborative meetings. Cold calling.

Outcome: Distributed to 1,800 service providers

(150 agencies)

Response Strategy

11

Subject line asked for “help” and “assistance.”

7 Day Contact Day 0: Pre-announcement email. Day 2: Invitation email. Day 7: Reminder email.

Outcome: 23% response rate (N=461)

Item Response Strategy

Survey loaded upfront allowing users to “glide” through questions.

Uncluttered appearance. Invisible drop-downs appeared depending

on users’ selection.

Potential Obstacles

#1: Coordinating Revisions

14

Development Problem: Unpredictable turnaround time.

Solution: Pilot survey and bundle revisions for programmer, understand complexity of revisions, set a realistic deadline.

#2: Limited incentive

Incentive: What do agencies stand to gain from participating?

Solution: Offer agency administrators the opportunity

to include a question of interest. Provide a summary of findings.

#3: Technical incompatibility

16

User Problem: Slow loading time for those operating old PC’s or early web browsers.

Solution: Know the user IT environment! What kind of

browsers are they operating? Use graphics conservatively.

Scenario

17

Very important stakeholder. 600+ candidates Problem: Dated web browsers took 1-2

minutes to load survey. Solution: Eventually loaded survey in

department’s commercial software. Resolved lag time Lost smooth look and feel Required more data cleaning 25% response rate (N=165 respondents)

Typically institution receives a 2-12% response to internal surveys.

Comparison of Labor Time

• 26 developer hours

• 461 responses

• 0.06 developer hours per response

• 8 developer hours• 165 responses

• 0.05 developer hours per response*

18

Custom built Off-the-shelf software

Lessons Learned

19

Facilitating Recruitment. Get executive buy-in.

Boosting Response. Use trusted intermediary. Send pre-announcement. Emphasize relevance and simplicity. Know the IT environment EARLY.

Take-Home Message

Unless you are conducting a survey with unique questionnaire options, special formatting requirements, or complex data analysis, commercial software will probably suit your needs.

Questions?

Contact information:Janelle Commins, M.P.H.818-990-8301jcommins@emt.org