@amir_ansari @dius_au1

@amir_ansari @dius_au2

6’3”

@amir_ansari @dius_au3

@amir_ansari @dius_au4

@amir_ansari @dius_au5

product and technology strategy

application-specific devices

web applications

mobile applications

@amir_ansari @dius_au6

What is UX?

@amir_ansari @dius_au7

Study and Design of ALL aspects of a person’s EXPERIENCE with a product or service

@amir_ansari @dius_au8

@amir_ansari @dius_au9

http://www.uxisnotui.com/

@amir_ansari @dius_au10

Pain points

Usability

Likelihood to recommend

Customer loyalty

Current behaviours Right direction

Alternate designs

Mental model

Why measure?

@amir_ansari @dius_au11 www.theimprovegroup.com

@amir_ansari @dius_au12

@amir_ansari @dius_au13

What?What? Why?Why?

@amir_ansari @dius_au14

@amir_ansari @dius_au15

@amir_ansari @dius_au16 http://lyndabuitrago.com/tag/leaky-gut/

@amir_ansari @dius_au17

• Picture of me measuring UX

@amir_ansari @dius_au18

@amir_ansari @dius_au19

@amir_ansari @dius_au20www.r-66.info

Completion rate

Task time

Task level satisfactionConversions

Page viewsSystem Usability Scale (SUS)

Net Promoter Score (NPS)

@amir_ansari @dius_au21

Why?Why?

@amir_ansari @dius_au22http://dralexkent.files.wordpress.com/2011/12/gut.jpg

We feel we know…

• who our customers are• what their pain points & barriers are • what will trigger them to use our product • this product will be a good idea

@amir_ansari @dius_au23

How do we measure and

validate these? And When?

@amir_ansari @dius_au24

Visioning Iterations Iterations Iterations Iterations MVPLaunch

Inception & Planning

1 2 3

@amir_ansari @dius_au25

Technique 1

@amir_ansari @dius_au26

The sooner we can find which features are worth investing in, the sooner we can focus our limited resources on the best solution to our business problems.

Jeff Gothelf – LEAN UX

“

@amir_ansari @dius_au27

@amir_ansari @dius_au28

Visioning Iterations Iterations Iterations Iterations MVPLaunch

Inception & Planning

1 2 3

@amir_ansari @dius_au29

30 – 60 mins 3 One on one interviews

@amir_ansari @dius_au30

• Concerns• Fears• Pain points• Ideal experience• Ideal channel

@amir_ansari @dius_au31

• Stakeholders observing• Call centre Staff felt engaged

and included • More insights than we had

anticipated

@amir_ansari @dius_au32

@amir_ansari @dius_au33

@amir_ansari @dius_au34

@amir_ansari @dius_au35

Visioning Inception & Planning

1 2 3

Iterations Iterations Iterations Iterations MVPLaunch

@amir_ansari @dius_au36

Focus group 3

2 hours

Focus group 1

Focus group 224

@amir_ansari @dius_au37

@amir_ansari @dius_au38

Feature 1 Feature 2 Feature 3

@amir_ansari @dius_au39

• Stakeholders buy-in through observation• Validate assumptions• Client was anonymous

@amir_ansari @dius_au40

• 2 hours wasn’t quite enough• Running sessions back to back

in the same evening!

@amir_ansari @dius_au41

@amir_ansari @dius_au42

Visioning Inception & Planning

1 2 3

Iterations Iterations Iterations Iterations MVPLaunch

@amir_ansari @dius_au43

1 hour One on one testing

6

@amir_ansari @dius_au44

http://www.nngroup.com/articles/how-many-test-users/

@amir_ansari @dius_au45

@amir_ansari @dius_au46

Lean Stack framework by Spark 59(www.spark59.com/leanstack)

@amir_ansari @dius_au47

1. Background

2. Falsifiable Hypotheses

3. Details

6. Next Actions

5. Validated Learning

4. Results

User 1 User 2 User 3 etc.

Hypothesis 1

Hypothesis 2

Etc.

• What feature or assumption to test• What questions to ask• How well does X perform• Etc. • What to test next

• What hypothesis do we want to validate

1. That majority of users think X2. That majority of users perform Y 3. That majority of users need Z4. Etc.

• Previous experiment taught us …• We want to find out…• We want to test our solutions…

1. We can proceed with X2. We need to tweak Y3. We need to pivot and get rid of Z

Title: Experiment 2: XXX Author: Created:

@amir_ansari @dius_au48

1. Background

2. Falsifiable Hypotheses

3. Details

6. Next Actions

5. Validated Learning

4. Results

User 1 User 2 User 3 etc.

Hypothesis 1

Hypothesis 2

Etc.

• What feature or assumption to test• What questions to ask• How well does X perform• Etc. • What to test next

• What hypothesis do we want to validate

1. That majority of users think X2. That majority of users perform Y 3. That majority of users need Z4. Etc.

• Previous experiment taught us …• We want to find out…• We want to test our solutions…

1. We can proceed with X2. We need to tweak Y3. We need to pivot and get rid of Z

Title: Experiment 2: XXX Author: Created:

• From focus groups, it was clear that one barrier with other online providers has been the lack of trainer support

• That providing access to trainers in the classroom and telling students upfront about trainer availability will eliminate barriers and increase conversion

• Show users the marketing pages that talk about trainer availability

• Show users the classroom with the trainer being online

• Users indeed look for trainer availability and are comforted by knowing OTI provides access to them

• Test something else! ; )

@amir_ansari @dius_au49

bit.ly/rapid-debrief

@amir_ansari @dius_au50

@amir_ansari @dius_au51

@amir_ansari @dius_au52

@amir_ansari @dius_au53

@amir_ansari @dius_au54

• Speed of testing• The feedback loop• Stakeholder buy-in

(live stream)• Rapid de-brief• One designs and one tests

@amir_ansari @dius_au55

@amir_ansari @dius_au56

@amir_ansari @dius_au57

@amir_ansari @dius_au58

5

One on one testingFocus groups

= $800

2 FTEs

X $160

X $375*

* Daily gross equivalent based on a gross annual salary of $90k

= $750

$1,550$1,550

16 = $3,200

3 FTEs

X $200

X $375*= $1,125

$4,325$4,325

If doing this internally…

@amir_ansari @dius_au59

@amir_ansari @dius_au60http://stamfordinteractive.com.au/conversations/benefits-of-guerrilla-testing/

@amir_ansari @dius_au61

You can quantify qualitative data!

bit.ly/quantifying-qualitative bit.ly/t-distribution

@amir_ansari @dius_au62

Completion rateCompletion rate

Task timeTask time

Task level satisfactionTask level satisfaction

ConversionsConversions

Page viewsPage views

System Usability Scale (SUS)System Usability Scale (SUS)

Net Promoter Score (NPS)Net Promoter Score (NPS)

{Visioning Inception & Planning Iterations Iterations Iterations Iterations MVP

Launch

@amir_ansari @dius_au63

@amir_ansari @dius_au64

Define why, what, when who, where how, how often

1

@amir_ansari @dius_au65

Become friends with the PM (or the person with the budget)

2

@amir_ansari @dius_au66

Keep it lean and measure often

3

@amir_ansari @dius_au67

Get stakeholder buy-in(Get them to observe)

4

@amir_ansari @dius_au68

Be consistentin how you measure and capture measurements

5

@amir_ansari @dius_au69

Define when to Quant and when to Qual

6

@amir_ansari @dius_au70

https://www.measuringusability.com/topics/UX

@amir_ansari @dius_au71

Thanks to

@amir_ansari @dius_au72http://dralexkent.files.wordpress.com/2011/12/gut.jpg

We feel we know…

• who our customers are• what their pain points & barriers are • what will trigger them to use our product • this product will be a good idea

@amir_ansari @dius_au73

Thank you

Qualitative data can in fact be converted into quantitative measures even if it doesn't come from an experiment or from a large sample size. 

Jeff Sauro – Measuring Usability“