Date post: | 27-Dec-2015 |
Category: |
Documents |
Upload: | dwain-daniel |
View: | 219 times |
Download: | 2 times |
INTRODUCTION
Government administrators and elected officials love to claim that they possess a profound understanding of their public’s needs, desires, and disaffection.
INTRODUCTION Unfortunately, the administrators and
officials are learning that storms of controversy provide meager evidence of the workaday values of the everyday people they govern.
Surveys of the public, conducted following the basic precepts of survey design and analysis, are fast becoming the vehicle for genuine connection to the public will.
INTRODUCTION
Uses. Evaluations of government services. Changing demographics that may signal shifts in
service demand. Patterns of service utilization. Problem identification. Customer service.
INTRODUCTION
Surveys have several important qualities. Anonymity to respondents. Point of view, characteristics, or use patterns can
be characterized with little confusion. Good surveys provide input from a
representative cross-section.
BEGIN BEFORE THE BEGINNING
The best surveys grow from well-conceived and well-articulated reasons for conducting them.
Resist the temptation to hit the ground running.
Be certain of the purposes of the survey.
BEGIN BEFORE THE BEGINNING
Identify the appropriate audiences. Identify the political and personal will for
doing the survey. Determine whether the questionnaire to be
developed is better as a one-time or periodic survey.
Think about the usefulness of comparative data.
GETTING STARTED
Convene a steering committee with key stakeholders.
Enlist the help of top government officials or administrators.
DESIGNING THE SURVEY
Sampling. Choose the appropriate sampling frame: about
what population do you wish to generalize? A sampling plan must give every respondent in
the sampling universe an equal chance of ending up in the sample. Simple random sample. Stratified sampling. Stratified random cluster sampling.
DESIGNING THE SURVEY
Targeting the individual in the household. If no list exists, you may only have addresses or
phone numbers. If so, use household member with most recent birthday.
MAIL, PHONE, OR IN-PERSON INTERVIEWS
The best ways to conduct surveys vary by accuracy, speed, and cost. Most common are mail and phone surveys.
MAIL, PHONE, OR IN-PERSON INTERVIEWS
Criteria Mail Phone In-Person
Accuracy
Response rate 45-55% 65-75% 75%+
Permits anonymity High Moderate Low
Is free from interviewer bias High Moderate Low
Handles various question types
Long/ complex Low Moderate High
Visual aids High Low High
Ensures question order Low High High
Permits widest coverage
Targets geographic areas High Low High
Avoids education bias Low Moderate High
Gives easy access to target population
High High Moderate
Speed of administration Month Week Month
Cost per interview $8-12 $15-20 >$20
MAIL, PHONE, OR IN-PERSON INTERVIEWS
Increasing response rates. Multiple mailings (up to three) with stamped,
return address envelope. Press coverage. Combination of methods often best: Mail survey
with telephone and in-person followup.
MAIL, PHONE, OR IN-PERSON INTERVIEWS Selecting sample size.
The size of sample depends on desired precision of estimates.
Generally speaking, if opinions are split as much as possible, than 100 residents will have a margin of error of +/- 10% with 95 percent confidence. Four hundred residents the margin is +/- 5%.
In general, 100 is a good minimum number, especially for subgroups.
QUESTIONNAIRE CONSTRUCTION
Each question should be judged against the purposes of the survey and the uses to which it will be put.
Steal widely. National Citizen Survey from International City
Management Association and National Research Center.
QUESTIONNAIRE CONSTRUCTION
Major principles.Consistency.Clarity.
Vague wording. Double-barreled questions. Assumed knowledge. Overlapping response categories.
Simplicity. Specificity. Brevity (30 min. Phone, 60 min. In-person, 10
page mail). Context sensitivity.
QUESTIONNAIRE CONSTRUCTION
Major principles. Security.
Demographic at end. General to specific.
Fairness. Option symmetry (balanced responses). Option wording and order.
Background info, pros and cons, opinion. Randomize pros and cons in a complicated survey.
QUESTIONNAIRE CONSTRUCTION
Open-ended versus closed-ended questions. Commonality versus depth.
Broad categories of questions. Factual. Opinion. Attitude. Motive. Knowledge. Action or behavior.
QUESTIONNAIRE CONSTRUCTIONA Classification of Question Types by Content
Cognitive Awareness Stage
Type of Question Information Acquired
Cognitive Stage Factual The facts about people and things
Knowledge What people know about things
Affective Stage Opinion What people say about things
Attitude What people believe about things
Motive Why people act the way they do
Action Stage Behavior How people act, what they do; how they will react to certain stimuli
QUESTIONNAIRE CONSTRUCTION Sections.
Title and identification of the survey sponsors (including human subjects information).
Instructions. Warm-up questions (simple and direct, factual or
knowledge). Body of the questionnaire (more complex)
First third – Awareness and knowledge of factors, indicators, and causes.
Second third – attitude, opinion, motive scales. Lifestyle information (nonresponse goes up).
Last third – Focused questions, controversial, personally embarrassing. Partially completed is better than not completed.
Classification items. Demographics.
Pretest and revise the instrument.
WRITING QUESTIONS Open-ended questions.
Unstructured (free to answer as they will). Projective.
Association – React to a particular stimulus. Construction – Create a story or self-portrait. Completion – Finish an already started stimulus or
picture. Ordering – Arrange or select items as important or
salient. Expressive – Freely express themselves by drawing a
picture or something similar. Closed-ended questions.
Structured Answer. Dichotomous. Multichotomous.
Scale (see next slide).
DEVELOPING AND USING SCALES The types of scales most commonly used in
public administration are attitude scales, importance scales, rating scales, and readiness-to-act scales.
The most common are attitude scales. Types of attitude scales.
Thurstone – 100 or more opinion statements ranked by informed judges. Assigned scale values by median ranking of judges. Time consuming and rarely used.
Likert Scales – Statements ranked on a five-point scale ranging from Strongly Agree, Agree, Neutral or Undecided, Disagree, or Strongly Disagree. Usually summed or average across multiple items.
Semantic differential – Five- or seven-point scales with opposing adjectives.
CONDUCTING THE SURVEY The survey steering committee.
Double check questionnaire with steering committee.
Frequency of surveys. For most multipurpose surveys, no more than
once per year. Pretest.
Test on twenty people at random. Ask questions about format and clarity.
CONDUCTING THE SURVEY
Training.Survey assistants must be trained. All
must operate uniformly, asking the questions in the same way, coding in the same way.
Consistent open-ended coding.10% recontact of survey respondents.
Trying hard and keeping track.Three contacts by telephone for each
number.Warning and at least two mailings for mail
surveys.
REPORTING RESULTS
Data analysis.For most government surveys, percentages,
average responses, simple cross-classifications.
The most complicated analysis will be to get accurate population estimates – weighting.
Report writing and presentation.Executive summary.Bulleted lists.Document survey methods in appendix.Augment tables with bar and pie charts.Powerpoint for in person presentation.