+ All Categories
Home > Technology > UX Assessment Techniques (from NOVA UX Psychology of UX Panel: Dec 11, 2013)

UX Assessment Techniques (from NOVA UX Psychology of UX Panel: Dec 11, 2013)

Date post: 10-May-2015
Category:
Upload: jennifer-romano-bergstrom
View: 560 times
Download: 4 times
Share this document with a friend
Popular Tags:
19
UX Assessment Techniques Jen Romano Bergstrom December 11, 2013 NOVA UX @forsmarshgroup @romanocog
Transcript
Page 1: UX Assessment Techniques (from NOVA UX Psychology of UX Panel: Dec 11, 2013)

UX Assessment Techniques

Jen Romano Bergstrom

December 11, 2013

NOVA UX

@forsmarshgroup @romanocog

Page 2: UX Assessment Techniques (from NOVA UX Psychology of UX Panel: Dec 11, 2013)

2

@forsmarshgroup @romanocog

Measuring the UX

• How does it work for the

end user?

• What does the user

expect?

• How does it make the user

feel?

“the extent to which a

product can be used by

specified users to

achieve specified goals

with effectiveness,

efficiency, and

satisfaction in a

specified context of

use.” ISO 9241-11

+ emotions

Page 3: UX Assessment Techniques (from NOVA UX Psychology of UX Panel: Dec 11, 2013)

3

@forsmarshgroup @romanocog

Why is it important?

• Put it in the hands of the end user.

• Things may seem straightforward to you but maybe not to

your users.

Page 4: UX Assessment Techniques (from NOVA UX Psychology of UX Panel: Dec 11, 2013)

4

@forsmarshgroup @romanocog

Why is it important?

• Put it in the hands of the end user.

• Things may seem straightforward to you but maybe not to

your users.

Page 5: UX Assessment Techniques (from NOVA UX Psychology of UX Panel: Dec 11, 2013)

5

@forsmarshgroup @romanocog

Why is it important?

• Put it in the hands of the end user.

• Things may seem straightforward to you but maybe not to

your users.

Page 6: UX Assessment Techniques (from NOVA UX Psychology of UX Panel: Dec 11, 2013)

6

@forsmarshgroup @romanocog

Where to test

• Controlled environment

• All participants have the

same experience

• Record and

communicate from

control room

• Observers watch from

control room and provide

additional probes (via

moderator) in real time

• Incorporate physiological

measures (e.g., eye

tracking, EDA)

• No travel costs

LABORATORY REMOTE IN THE FIELD

• Participants tend to be

more comfortable in

their natural

environments

• Recruit hard-to-reach

populations (e.g.,

children, doctors)

• Moderator travels to

various locations

• Bring equipment (e.g.,

eye tracker)

• Natural observations

• Participants in their

natural environments

(e.g., home, work)

• Use video chat

(moderated sessions)

or online programs

(unmoderated)

• Conduct many sessions

quickly

• Recruit participants in

many locations (e.g.,

states, countries)

Page 7: UX Assessment Techniques (from NOVA UX Psychology of UX Panel: Dec 11, 2013)

7

@forsmarshgroup @romanocog

Where to test

• Controlled environment

• All participants have the

same experience

• Record and

communicate from

control room

• Observers watch from

control room and provide

additional probes (via

moderator) in real time

• Incorporate physiological

measures (e.g., eye

tracking, EDA)

• No travel costs

LABORATORY REMOTE IN THE FIELD

• Participants tend to be

more comfortable in

their natural

environments

• Recruit hard-to-reach

populations (e.g.,

children, doctors)

• Moderator travels to

various locations

• Bring equipment (e.g.,

eye tracker)

• Natural observations

• Participants in their

natural environments

(e.g., home, work)

• Use video chat

(moderated sessions)

or online programs

(unmoderated)

• Conduct many sessions

quickly

• Recruit participants in

many locations (e.g.,

states, countries)

Page 8: UX Assessment Techniques (from NOVA UX Psychology of UX Panel: Dec 11, 2013)

8

@forsmarshgroup @romanocog

How to test

• In-depth feedback from

each participant

• No group think

• Can allow participants to

take their own route and

explore freely

• No interference

• Remote in participant’s

environment

• Flexible scheduling

• Qualitative and

Quantitative

ONE-ON-ONE SESSIONS FOCUS GROUPS SURVEYS

• Representative

• Large sample sizes

• Collect a lot of data

quickly

• No interviewer bias

• No scheduling sessions

• Quantitative analysis

• Participants may be

more comfortable with

others

• Interview many people

quickly

• Opinions collide

• Peer review

• Qualitative

Page 9: UX Assessment Techniques (from NOVA UX Psychology of UX Panel: Dec 11, 2013)

When to test

9

@forsmarshgroup @romanocog

Page 10: UX Assessment Techniques (from NOVA UX Psychology of UX Panel: Dec 11, 2013)

10

@forsmarshgroup @romanocog

What to measure

OBSERVATIONAL + Ethnography

+ Time to complete task

+ Reaction time

+ Selection/click behavior

+ Ability to complete tasks

+ Accuracy

IMPLICIT + Facial expression analysis

+ Eye tracking

+ Electrodermal activity (EDA)

+ Behavioral analysis

+ Linguistic analysis of verbalizations

+ Implicit associations

+ Pupil dilation

EXPLICIT + Post-task satisfaction

questionnaires

+ In-session difficulty ratings

+ Verbal responses

+ Moderator follow up

+ Real-time +/- dial

Page 11: UX Assessment Techniques (from NOVA UX Psychology of UX Panel: Dec 11, 2013)

11

@forsmarshgroup @romanocog

1. Participant repeatedly fixated the upper right hand corner. Participant said that he/she was looking for a search tool on the

page. The search tool was in a disappearing banner on the page.

2. Participants had similar fixation counts across bottom links, indicating uncertainty of where to click to get started.

Case 1

• Problems: What do users want? Does the new design work?

• Methods: Focus groups, one-on-one interviews, in-lab

usability testing with eye tracking

Page 12: UX Assessment Techniques (from NOVA UX Psychology of UX Panel: Dec 11, 2013)

12

@forsmarshgroup @romanocog

1. Participant repeatedly fixated the upper right hand corner. Participant said that he/she was looking for a search tool on the

page. The search tool was in a disappearing banner on the page.

2. Participants had similar fixation counts across bottom links, indicating uncertainty of where to click to get started.

Case 1

• Problems: What do users want? Does the new design work?

• Methods: Focus groups, one-on-one interviews, in-lab

usability testing with eye tracking

Page 13: UX Assessment Techniques (from NOVA UX Psychology of UX Panel: Dec 11, 2013)

13

@forsmarshgroup @romanocog

Case 2 • Problem: What parts of the form do people actually read?

• Method: In-lab usability testing with eye tracking

Page 14: UX Assessment Techniques (from NOVA UX Psychology of UX Panel: Dec 11, 2013)

14

@forsmarshgroup @romanocog

Case 2

0

20

40

60

80

100

120

Page 1 Page 2 Page 3

Tim

e (

se

con

ds)

Participants did not read the

instructions in their entirety (page 1:

left; page 3: right); rather, they skimmed

and then moved on to the form where

they needed to enter information.

Length of time spent on each page of the instructions

before working on form.

Aggregate fixation count heat map across all

participants, Page 1. Participants looked at

‘Purpose of Form’ section the most often.

Page 15: UX Assessment Techniques (from NOVA UX Psychology of UX Panel: Dec 11, 2013)

15

@forsmarshgroup @romanocog

Case 3

Prototype Grid Old Grid

Page 16: UX Assessment Techniques (from NOVA UX Psychology of UX Panel: Dec 11, 2013)

16

@forsmarshgroup @romanocog

Case 4

Tablet

Smartphone

Gaze Plot: After getting an error message, the participant

had to search all over the screen to find the missing field.

“How do I advance to the next screen?”

“It seems like it's stuck on the screen.”

Page 17: UX Assessment Techniques (from NOVA UX Psychology of UX Panel: Dec 11, 2013)

17

@forsmarshgroup @romanocog

Many ways to evaluate the UX

• Surveys

• Focus groups

• In-person one-on-one with eye tracking

• Analytics

Page 18: UX Assessment Techniques (from NOVA UX Psychology of UX Panel: Dec 11, 2013)

18

@forsmarshgroup @romanocog

Obstacles to UX Testing

• There is no time.

– Start early in development process

– One morning a month with 3 users – Krug

– 12 people in 3 days – Anderson Reimer

– 12 people in 2 days – Lebson & Romano Bergstrom

• Can’t find representative users

– Everyone is important

– Travel

– Remote testing

• We don’t have a lab

– Test anywhere

Page 19: UX Assessment Techniques (from NOVA UX Psychology of UX Panel: Dec 11, 2013)

Thank you!

• Twitter: @forsmarshgroup

• LinkedIn: http://www.linkedin.com/company/fors-marsh-group

• Blog: www.forsmarshgroup.com/index.php/blog

Jennifer Romano Bergstrom

@romanocog

[email protected]

NOVA UX


Recommended