Measuring usabilityHow to design experiments for prototype

evaluation

Stefan Ivanov

• Senior UX Architecto User Research

o Concept Design

o Prototyping

o User Testing

• Educatoro Human-Computer Interaction

o UX processes

o Design thinking

UX has taught us a lot

• User researchers in the room?

• People who have ever prototyped?

• Anyone who starts prototyping early on?

• Outliers that also got feedback on their prototypes from real users?

“UX is not order taking”

Kevin Richardson

Evaluation

• Literature Review

Evaluation

• Literature Review

• Cognitive Walkthrough

Evaluation

• Literature Review

• Cognitive Walkthrough

• Heuristic Evaluation

Evaluation

• Literature Review

• Cognitive Walkthrough

• Heuristic Evaluation

• Model Based Evaluation

Evaluation

• Model Extraction

Evaluation

• Model Extraction

• Silent Observation

Evaluation

• Model Extraction

• Silent Observation

• Think Aloud

Evaluation

• Model Extraction

• Silent Observation

• Think Aloud

• Constructive Usage

Evaluation

• Model Extraction

• Silent Observation

• Think Aloud

• Constructive Usage

• Retrospective Testing

Evaluation

• Model Extraction

• Silent Observation

• Think Aloud

• Constructive Usage

• Retrospective Testing

• Controlled Experiments

Synthesizing hypothesis

• Color grabber

• Zoom-able UI

• Pixel-perfect accuracy

• Ease and speed of use

• Picking smaller and larger areas of the same color

• Zoom focal point design

Null hypothesis

Easy and accurate color grabbing through a zoom-able UI is independent of the zoom configuration, target size and focal point design.

Null hypothesis

Easy and accurate color grabbing through a zoom-able UI is independent of the zoom configuration, target size and focal point design.

Independent variables

Controlled variables

Dependent variables

Measured variables

Experiment concept

Experiment concept

Zoom configuration

Target size

Focal point design

#Conditions = #ZoomConfigs * #TargetSizes * #FocalPointDesigns

Experiment concept

Ease of use

Accuracy

• Error rate

• Task completion time

• Number of errors until success

• Distance of click from target

• Cursor behavior over the target

• Number of errors until success

Eliminating learning effects

Eliminating learning effects

Executing the study

Analyzing the data

Further learning

https://www.r-project.org/

http://randomization.com/

http://www.ats.ucla.edu/stat/mult_pkg/whatstat/

What statistical analysis to use cheat sheet

Randomization to avoid learning effects

R-project for running statistical analysis

http://www.ats.ucla.edu/stat/r/whatstat/whatstat.htm

Remote usability studies

http://bit.ly/2ehYiqG

Remote usability studies

Stefan Ivanovsivanov@infragistics.com

stefanivanovux

#uxify is brought to you by

Indigo Script2:00

2:052:102:20

2:30

2:45

2:50

• Who was in the Indigo Prototyping workshop in the morning?

• I pick one person with a prototype ready to use as a showcase and connect him to the screen.

• The rest form smaller or larger teams with at least one person with a prototype.

• We go through the prototype

• Build some more screens of it

• We define the goals for our study on the whiteboard (a couple of minutes for the showcase and the rest of the time they work)

• We upload it in indigodesigned and I show the usability study creation flow. (5 minutes for the showcase and the rest of the time they work)

• I pick another person/team to send the link to their study to everyone in the room and they do his study

• We go through the results together and discuss them