Date post: | 03-Jan-2016 |
Category: |
Documents |
Upload: | paloma-hall |
View: | 29 times |
Download: | 0 times |
ABCs of Satisfaction & ABCs of Satisfaction & Impact Surveys Impact Surveys
Elizabeth Harris, Ph.D.Elizabeth Harris, Ph.D.Evaluation, Management & Training Evaluation, Management & Training
Associates, Inc.Associates, Inc.
May 21May 21stst , 2008 , 2008
ABCs of Satisfaction & Impact ABCs of Satisfaction & Impact SurveysSurveys
ObjectivesObjectives– Understand the importance and relevance of science-Understand the importance and relevance of science-
based surveyingbased surveying– Understand method of developing telephone and Understand method of developing telephone and
Internet surveys to measure inquirer satisfaction and Internet surveys to measure inquirer satisfaction and key outcomes important to contact centerskey outcomes important to contact centers
– Leave the workshop with the tools necessary to Leave the workshop with the tools necessary to develop surveys tailored to the needs of each contact develop surveys tailored to the needs of each contact center center
Workshop exerciseWorkshop exercise
““Lessons LearnedLessons Learned” ” from surveying for: from surveying for:
Centers for Disease Control & PreventionCenters for Disease Control & Prevention– Consumer Response Services Center (CDC-INFO)Consumer Response Services Center (CDC-INFO)– Monitor calls & e-mail responses Monitor calls & e-mail responses
First 5 LAFirst 5 LA– Countywide Parent Helpline Countywide Parent Helpline
We will discuss:We will discuss:
Survey: Survey: – DesignDesign
To Do’sTo Do’s
Common PitfallsCommon Pitfalls
– Administration Administration OptionsOptions
– Data UseData Use
Satisfaction & Impact Satisfaction & Impact
Satisfaction:Satisfaction:– ImmediateImmediate– Caller’s opinion about Caller’s opinion about
Overall ServiceOverall Service
Various Aspects of the Service (Drill Down)Various Aspects of the Service (Drill Down)
Impact:Impact:– What happened as a result of the call?What happened as a result of the call?
Satisfaction & Impact Surveys: Satisfaction & Impact Surveys: Step 1Step 1
What is important to us when we consider What is important to us when we consider the caller’s perspective and experience? the caller’s perspective and experience?
What questions are our key stakeholders What questions are our key stakeholders asking that we need to answer? asking that we need to answer?
Satisfaction & Impact Surveys: Satisfaction & Impact Surveys: Step 2Step 2
Determine method of administration Determine method of administration – Interactive Voice ResponseInteractive Voice Response– Live SurveyorLive Surveyor– Web-Based Web-Based
FeasibilityFeasibility
Balance between cost & knowledge gainBalance between cost & knowledge gain
Satisfaction & Impact Surveys: Satisfaction & Impact Surveys: Step 3 Step 3
Develop questions Develop questions Response options Response options – ClosedClosed
Yes/NoYes/NoLikert ScaleLikert Scale
– e.g. Very Satisfied to Very Dissatisfied, Press 1 for Very e.g. Very Satisfied to Very Dissatisfied, Press 1 for Very SatisfiedSatisfied
– Allow a middle option? Neither Satisfied or Dissatisfied Allow a middle option? Neither Satisfied or Dissatisfied
– Open Open ProPro
– Information Information
ConCon– Labor considerationLabor consideration
Satisfaction & Impact Surveys: Satisfaction & Impact Surveys: Step 3Step 3
Develop questions Develop questions Handout example of closed and open-Handout example of closed and open-ended questions ended questions
Satisfaction & Impact Surveys: Satisfaction & Impact Surveys: Step 4Step 4
Design Survey Design Survey – Question wordingQuestion wording
NeutralNeutral
– Response order effect Response order effect Carefully consider which question is first and which Carefully consider which question is first and which is last is last
Satisfaction & Impact Surveys: Satisfaction & Impact Surveys: Step 4Step 4
Design SurveyDesign Survey– IVR RulesIVR Rules
Brief and to the pointBrief and to the point
No more than six response optionsNo more than six response options
– Consider lengthConsider lengthLonger = more respondents drop off Longer = more respondents drop off
– Language options Language options
Satisfaction & Impact Surveys: Satisfaction & Impact Surveys: Step 5Step 5
Determine Survey ProtocolDetermine Survey Protocol– Who is surveyed-Immediate SatisfactionWho is surveyed-Immediate Satisfaction
Every callerEvery caller– Works well if no automated transfer optionWorks well if no automated transfer option
TrainingTraining
BiasBias
Random selectionRandom selection– Works well with a robust systemWorks well with a robust system
program algorithm that automatically chooses program algorithm that automatically chooses
Satisfaction & Impact Surveys: Satisfaction & Impact Surveys: Step 5Step 5
Who is surveyed—Impact (Call Back) Who is surveyed—Impact (Call Back) – Again consider:Again consider:
Purpose of the surveyPurpose of the survey– Caller preferenceCaller preference– Part of service follow up effortPart of service follow up effort
How the information will be used How the information will be used
– Other considerations:Other considerations:Labor allocation/resourcesLabor allocation/resources
– To surveyTo survey– To interpret/make use of information To interpret/make use of information
Satisfaction & Impact Surveys: Satisfaction & Impact Surveys: Step 6Step 6
Pilot Testing Pilot Testing – Works out the bugsWorks out the bugs– Improves introductory instructionsImproves introductory instructions– May revise questions and responses based May revise questions and responses based
on pilot test results on pilot test results
What to Expect: Response Rate What to Expect: Response Rate
Immediate Satisfaction SurveyImmediate Satisfaction Survey– 22% to 33%22% to 33%
Follow Up SurveyFollow Up Survey– Will reach half of those you try to callWill reach half of those you try to call
If you try more than onceIf you try more than once
Vary time/day called Vary time/day called
Results: Results:
Over 90% give the First 5 LA Parent Helpline the Over 90% give the First 5 LA Parent Helpline the highest satisfaction rating for overall service!highest satisfaction rating for overall service!
Over half who received a referral successfully Over half who received a referral successfully accessed servicesaccessed services
Results: Results:
Over 75% give CDC-INFO the highest satisfaction rating for Over 75% give CDC-INFO the highest satisfaction rating for overall serviceoverall service
Over 75% report learning something new from their callOver 75% report learning something new from their call
Over half report that what they learned makes them want to Over half report that what they learned makes them want to change a behavior change a behavior
Questions? Questions?
Exercise: Exercise:
See handoutsSee handouts
Select one question you would like to answer Select one question you would like to answer with a satisfaction survey with a satisfaction survey
Exercise—Break Out: Exercise—Break Out:
Develop a survey question and responses:Develop a survey question and responses:– Doable/feasible to implement in your setting Doable/feasible to implement in your setting
Exercise—Debriefing: Exercise—Debriefing:
Share examples from each group Share examples from each group
Questions?Questions?
Contact information:Contact information:
Elizabeth Harris, Ph.D.Elizabeth Harris, Ph.D.
818-990-8301818-990-8301
[email protected]@emt.org