52
"Successful user experiences deliver a firm's value proposition
-- the brand promise -- to customers in the most effective and
appropriate way.”
Karen Donoghue, Built For Use
research methods
TALK ASK WATCH OBSERVE
focus groups interviews usability testing ethnography
Typically 4-9 people 1-on-1 1-on-1 1-on-1 or group
Discussion about critical aspects
Questionnaire
Brainstorming
Users complete representative tasks
Field observation of sociocultural phenomena
Moderator Conducted by Interviewer
Observer/Moderator Observing in natural environment
1-2 hours 1-2 hours 1- 1.5 hours 1 hour- many years
WHAT PEOPLE THINK HOW PEOPLE LIVE
concepttesting
customer interviews
cardsorting
focus groups
expertanalysis
usabilitytesting
contextualinquiry
verification testing
concepttesting
customer interviews
cardsorting
focus groups
expertanalysis
usabilitytesting
Contextualinquiry
verification testing
http://www.slis.indiana.edu/technology/usability_lab.html
• Formal facility• 8-20 carefully screened applicants• Formal test plan• Complex video monitoring (picture in picture) practices, including high-light videos
• Participant's work environment• 3-10 recruited participants meeting criteria• Impartial monitor• Informal test plan• (Simple screen capturing, with voice, using programs like Camtasia or Morae)
http://www.adaptivetechnologies.com
formal informalsemi-formal
• In-house (or remote)• 3-5 Friends, co-workers or family members• Impartial monitor• Informal test plan• No video taping, only notes and observations
rapid research
traditional
alpha release(get it up quick!)
product beta(trial users)
live product(free trial, sign up today)
focus groups(how are we doing?)
iterative
proof of concept(what do they want?)
beta release(do they like it?)
beta2 release(can we simplify it?)
live product(does it meet their needs?)
Rapid usability testing is characterized by rapid cycles of testing between iterations of
prototyping
Characteristics:
+ Informal testing of 5 participants and one floater per day (one or two days of testing scheduled)+ Modifications to prototype (paper or HTML) between testing days (or sometimes during testing, but not in this case)+ Team present for testing in order to observe first hand what is being found+ Raw notes/ rough observations delivered immediately following testing + Informal report (word format) delivered approx. 48-hours following testing
toolkit: essentials
Windows Media Encoder (download.com) – FREE easy to use full-
motion screen capture tool.
CamStudio & Camtasia Player – FREE full-
motion screen capture & player (NOT the usability software)
Participant ID:Date: Start time: End time:
Task Time total
Video highlight at
Notes/Main Observations Quotes Events
1 2:33 1:30 (time
stamp)
User struggles to sign up due to unclearinstructions;
“I don’t understand why I can’t sign in!”
2 3:20 User can not find account information to edit the password; looks in options instead
3 7:45 User can not find support and FAQ section to change his order preferences; gives up after looking everywhere, including the search function
“This is really frustrating! I would stop trying now!”
4 … No problem to find shopping cart “I like the shopping cart”
Legend: Task failed Prompt by moderator Struggle Task solved Insight
Notetaking Format (with macros)
Notetaking Format (with macros)
A classic overview of the testing process – surprisingly straightforward although there is very little mention of the web!
A classic – humorous and practical approach to usability. Steve has created the true ‘must read’ book for teams and stakeholders.
contextual inquiry
idea!
proof-of-concept
alpha-release
usability testing
When to test?
beta release
beta release 2
launch
launch 1
alpha-release
alpha-release
alpha-release
alpha-release
Contextual Inquiry
1 determineaudience 2 develop
interview script
3 observe
64 photo diary 5 follow up compile & report
Sample Project: Technorati
1 determineaudience 2 develop
interview script
3 observe
64 photo diary 5 follow up compile & report
16 participants in 2 citiesthe more specific profiles, the better
2 hours to a full day with notes and photos in context
Ask participants to take photos of daily life over next 3 days to 1 week
Interview 2-hours and review photo diaries
Sample Project: Technorati
Online Survey in 5 cities
Photo Diaries & Contextual Interviews in L.A. & Minneapolis
Interview Summary
Audience Segmentation Grid
Audience Segmentation Grid
Contextual Findings Report
rapid usability testing
Rapid Usability Testing
1 determineaudience 2 develop
test plan 3 recruit
64 conduct testing 5 high-level findings
informal report
6
1 determineaudience 2 develop
test plan 3 recruit
4 conduct testing 5 high-level findings
informal report
Sample Project: Technorati Testing
5 target audience users
5 one-hour testing sessions
Extract high-level findings, 2-4 pages
Observations and recommendations with screenshots, 8-12 pages
develop test plan including task list
define recruiting criteria with client
Recruiting
Confirmation Email
Informal Usability Testing
Participant ID:Date: Start time: End time:
Task Time total
Video highlight at
Notes/Main Observations Quotes Events
1 2:33 1:30 (time
stamp)
User struggles to sign up due to unclearinstructions;
“I don’t understand why I can’t sign in!”
2 3:20 User can not find account information to edit the password; looks in options instead
3 7:45 User can not find support and FAQ section to change his order preferences; gives up after looking everywhere, including the search function
“This is really frustrating! I would stop trying now!”
4 … No problem to find shopping cart “I like the shopping cart”
Legend: Task failed Prompt by moderator Struggle Task solved Insight
Note Taking
Pre- and Post Test Surveys:Paper vs. Online
Recording Testing Sessions
rapid, iterative cycles during development process
“sprints” are approx. 3-4 weeksrounds of testing should take place
during this timeframe
rele
aseco
nce
pt
rele
ase
Concept/Prototype
focusgroups
contextualinquiry
con
cep
t
Beta Release
assessmentusabilitytesting
expertanalysis
rele
aseco
nce
pt
AlphaRelease
expertanalysis
verification usabilitytesting
rele
aseco
nce
pt
ProductLaunch
site stats
post-launchusabilitytesting
rele
aseco
nce
pt
recommended reading
The first guide to linking business strategy with the art and science of the online user experience
resources
usability resourceshttp://www.gotomedia.com/resources.html
presentation postedhttp://www.gotomedia.com/goto/usability/
this presentation can be found at:www.gotomedia.com/goto/lifestyle
sign up for our quarterly newsletter on user experience and usability: www.gotomedia.com/gotoreport
mobile user experience:www.gotomobile.comthank you!
