+ All Categories
Home > Design > Introduction to UX Research: Designing Surveys That Don't Suck!

Introduction to UX Research: Designing Surveys That Don't Suck!

Date post: 17-Oct-2014
Category:
View: 4,636 times
Download: 1 times
Share this document with a friend
Description:
 
29
Will Evans Yana Kuchirko
Transcript
Page 1: Introduction to UX Research: Designing Surveys That Don't Suck!

Will Evans

Yana Kuchirko

Page 2: Introduction to UX Research: Designing Surveys That Don't Suck!

"They've done studies, you know. 60% of the time, it works every time. ”

- Brian Fantana

04/07/23 2

Page 3: Introduction to UX Research: Designing Surveys That Don't Suck!

04/07/23 3

Surveys Aren’t Objective

They are created by people who are biased and analyze their results based on their own (mis)perceptions.

Page 4: Introduction to UX Research: Designing Surveys That Don't Suck!

Epistemological Dead End?

No. Awareness that the researcher plays an integral role in the process of measuring any given phenomena by deciding how to measure is key.

There are ways to minimize researcher bias by creating better questions.04/07/23 4

Page 5: Introduction to UX Research: Designing Surveys That Don't Suck!

Methods are not “just methods”

04/07/23 5

How you measure what you are studying shapes what you find.

Page 6: Introduction to UX Research: Designing Surveys That Don't Suck!

Let’s say you are studying…

04/07/23 6

How often teenagers use your website

Page 7: Introduction to UX Research: Designing Surveys That Don't Suck!

You might ask teenagers…

“How often do you my website?”

a.Very rarelyb.Rarelyc.Occasionallyd.Frequentlye.Very frequently

04/07/23 7

Response options source: http://www.dataguru.org

Page 8: Introduction to UX Research: Designing Surveys That Don't Suck!

You might ask teenagers…

“How often do you my website?”

a.Very rarelyb.Rarelyc.Occasionallyd.Frequentlye.Very frequently

04/07/23 8

Response options source: http://www.dataguru.org

And what’s wrong with these????

Page 9: Introduction to UX Research: Designing Surveys That Don't Suck!

You might ask teenagers…

“How often do you my website?”

a.Very rarelyb.Rarelyc.Occasionallyd.Frequentlye.Very frequently

04/07/23 9

Response options source: http://www.dataguru.org

And what’s wrong with these????

SUBJECTIVITY!!!

Page 10: Introduction to UX Research: Designing Surveys That Don't Suck!

“Hmmm, for me “frequently” really means….”

A few times a week A few times a day

So what are we really measuring? NOISE.

04/07/23 10

Page 11: Introduction to UX Research: Designing Surveys That Don't Suck!

What is “noise”?

What we observe…

04/07/23 11

Page 12: Introduction to UX Research: Designing Surveys That Don't Suck!

We WANT this!We WANT this!

We DON’T WANT this!We DON’T WANT this!

04/07/23 12

What is “noise”?

Page 13: Introduction to UX Research: Designing Surveys That Don't Suck!

Let’s really stretch our thinking a bit here to provide a more concrete understanding of “noise” in your data.

Imagine you have AT&T phone service and you’re trying to make a call. The signal isn’t clearly going through and you end up hearing everything but the other person’s voice. Frustrating, right? Thank God this is only a pretend scenario.

04/07/23 13

An imaginary scenario

Page 14: Introduction to UX Research: Designing Surveys That Don't Suck!

How is AT&T related to “Noise”?

When you develop bad questions, you don’t “hear” the message of your data clearly.

04/07/23 14

Page 15: Introduction to UX Research: Designing Surveys That Don't Suck!

So…why is “noise” bad???

• Badly structured and poorly worded questions that obfuscate meaning for participants provide bad data.

• Bad data– Doesn’t answer your research question–Makes it difficult to interpret results– Is pretty much useless.

04/07/23 15

Page 16: Introduction to UX Research: Designing Surveys That Don't Suck!

More bad examples of frequently used response

optionsa. Completely satisfiedb. Very satisfiedc. Fairly well satisfiedd. Somewhat dissatisfiede. Very dissatisfied

a. Totally likeb. Very much likec. Moderately liked. Somewhat likee. Not like

What’s the difference between “fairly well” and “somewhat”?

What’s the difference between “moderately like” and “somewhat like”?

04/07/23 16

Page 17: Introduction to UX Research: Designing Surveys That Don't Suck!

a. Completely satisfiedb. Very satisfiedc. Fairly well satisfiedd. Somewhat dissatisfiede. Very dissatisfied

a. Totally likeb. Very much likec. Moderately liked. Somewhat likee. Not like

What’s the difference between “fairly well” and “somewhat”?

What’s the difference between “moderately like” and “somewhat like”?

Semantics04/07/23 17

More bad examples of frequently used response

options

Page 18: Introduction to UX Research: Designing Surveys That Don't Suck!

Words like….

• Sometimes• Often• Moderately• Very• Not very much

Are subjective = they mean different things for different people

04/07/23 18

Page 19: Introduction to UX Research: Designing Surveys That Don't Suck!

A better alternative?

Actually asking people how often they do certain activities.

04/07/23 19

Page 20: Introduction to UX Research: Designing Surveys That Don't Suck!

In order to gather objective data…

…your questions must mean the same thing for everyone.

How often do you use my website?a.Neverb.A few times a yearc.Once a monthd.2-3 times a monthe.Once a weekf.A few times a weekg.Every day

“Never” means never for everyone!04/07/23 20

Page 21: Introduction to UX Research: Designing Surveys That Don't Suck!

Importance of “Anchors”

Responses options serve as “anchors” for each question, determining the “location” of each responses as qualitatively distinct from the other.

04/07/23 21

Page 22: Introduction to UX Research: Designing Surveys That Don't Suck!

Responses options serve as “anchors” for each question, determining the “location” of each responses as qualitatively distinct from the other.

04/07/23 22

Importance of “Anchors”

Strongly disagree Disagree Neutral Agree

StronglyAgree

Page 23: Introduction to UX Research: Designing Surveys That Don't Suck!

(Ideal) Response Symmetry

• Good questions aim to have symmetrical quantitative/qualitative distance between anchors a. Neverb. A few times a yearc. Once a monthd. 2-3 times a monthe. Once a weekf. A few times a weekg. Every day

The distance between “never” and “a few times a year” is proportionate to “every day” and “a few times a week”.

04/07/23 23

Page 24: Introduction to UX Research: Designing Surveys That Don't Suck!

Adherence to Logic & Linearity

People are accustomed to paradigms that are intuitive, and often “linear”.

Anchors should not be the exception.04/07/23 24

Page 25: Introduction to UX Research: Designing Surveys That Don't Suck!

Intuitive Anchor Directions

Never Always

Disagree Agree

Very poor Very good

Not very important

Very Important

04/07/23 25

Page 26: Introduction to UX Research: Designing Surveys That Don't Suck!

Value of Clarity

If questions are clear and concise, participants would spend less time analyzing the questions themselves and more time on answering them. Mystic Arts, LLC

04/07/23 26

Page 27: Introduction to UX Research: Designing Surveys That Don't Suck!

So do good questions guarantee valid results?

• No. But good questions offer more assurance that you are listening to signal and not the noise.

• But answering your research question can be done in other ways

• Behavioral Observations• Open ended qualitative questions• Many more…

04/07/23 27

Page 28: Introduction to UX Research: Designing Surveys That Don't Suck!

The End.

So was this presentation simply awesome?

a. Hell yeah!b. Definitely!c. Totally rocked!

04/07/23 28

Page 29: Introduction to UX Research: Designing Surveys That Don't Suck!

Thanks!


Recommended