138
User Research in the Wild Jim Ross Senior UX Architect, Infragistics [email protected] @anotheruxguy User Research in the Wild Flickr: Katy Nicolson
User Research in the Wild
Jim Ross
Senior UX Architect, Infragistics
@anotheruxguy
User Research in the Wild
Flickr: Katy Nicolson
User Research in the Wild
Jim Ross
Senior UX Architect, Infragistics
@anotheruxguy
zz
What's the key to designing
an excellent user experience?
June 11, 2015 3
User Research in the Wild
Jim Ross
Senior UX Architect, Infragistics
@anotheruxguy
Understanding the users and their needs
Flickr: Highways England
zz
But how do you get that understanding
of people and their needs?
June 11, 2015 5
Stakeholder interviews and workshops
Traditional Methods of Understanding Users
June 11, 2015 6Flickr: Jim Larrison
Interviews with subject-matter experts or user representatives
Traditional Methods of Understanding Users
June 11, 2015 7Flickr: National Assembly for Wales
User interviews
Traditional Methods of Understanding Users
June 11, 2015 8Flickr: Nicholas Wang
Focus groups
Traditional Methods of Understanding Users
June 11, 2015 9Flickr: Sebastiaan ter Burg
Surveys
Traditional Methods of Understanding Users
June 11, 2015 10
Usability testing
Traditional Methods of Understanding Users
June 11, 2015 11Flickr: K2_UX
But what do these methods lack?
• Most of them focus on what people say, rather than what they do.
• They occur out of context.
Understanding Users and Their Needs
June 11, 2015 12
zz
/ / /
So how do you get that understanding
of people and their needs?
Go out into the field to observe people in their natural context.
June 11, 2015 13
Understanding Users
June 11, 2015 14Flickr: Medill DC
Understanding Users
June 11, 2015 15Flickr: Intel Free Press
Understanding Users
June 11, 2015 16Flickr: State Farm
Field studies
• Observation and interview methods
• Solving common problems
• Overcoming objections to field studies
What We’ll Discuss
June 11, 2015 17
zz
/ / /
Field Studies
What are Field Studies?
June 11, 2015 18
Going out “into the field” to observe and interview people in their natural context…
What are Field Studies?
June 11, 2015 19Flickr: West Midlands Police
To observe firsthand their context of use:
• Users
• Tasks
• Tools and technology
• Environment
What are Field Studies?
June 11, 2015 20
To observe firsthand their context of use:
• Users
• Characteristics
• Knowledge
• Expectations
• Experience
• Technical skills
• Needs
What are Field Studies?
June 11, 2015 21
Flickr: UC Davis College of Engineering
To observe firsthand their context of use:
• Tasks
• Tasks, sub-tasks, and individual steps
• Frequency of tasks
• Workflow across people
• Goals
What are Field Studies?
June 11, 2015 22
Flickr: Anne-Sophie Leens
To observe firsthand their context of use:
• Tools and technology
• Technology
• Physical tools
• Documents, paperwork, manuals, etc.
• Other software and applications
What are Field Studies?
June 11, 2015 23
Flickr: Corey Seeman
To observe firsthand their context of use:
• Environment – physical and social
• Location: office, cubicle, outdoors, mobile
What are Field Studies?
June 11, 2015 24
Flickr: Jason Devaun
To observe firsthand their context of use:
• Environment – physical and social
• Location: office, cubicle, outdoors, mobile
• Activity: sitting, standing, lying, walking, etc.
What are Field Studies?
June 11, 2015 25
Flickr: Kun-chia Wu
To observe firsthand their context of use:
• Environment – physical and social
• Location: office, cubicle, outdoors, mobile
• Activity: sitting, standing, lying, walking, etc.
• Environmental conditions: lighting, noise, etc.
What are Field Studies?
June 11, 2015 26
Flickr: MoDOT Photos
To observe firsthand their context of use:
• Environment – physical and social
• Location: office, cubicle, outdoors, mobile
• Activity: sitting, standing, lying, walking, etc.
• Environmental conditions: lighting, noise, etc.
• Interaction with other people
What are Field Studies?
June 11, 2015 27
Flickr: Highways England
zz
/ / /
Field Studies
Advantages of Field Studies
June 11, 2015 28
You see firsthand what people really do, instead of what they say they do.
Advantages of Field Studies
June 11, 2015 29Flickr: essers
It’s easier for people to show you what they do, than to discuss it in the abstract.
Advantages of Field Studies
June 11, 2015 30Flickr: MoDot Photos
Allows you to see all the other things involved in tasks, outside of a user interface.
Advantages of Field Studies
June 11, 2015 31Flickr: pltauk
Participants feel more comfortable in their own environment and act more naturally.
Advantages of Field Studies
June 11, 2015 32Flickr: Sylvain Kalache
zz
Field Study Methods
June 11, 2015 33
Field studies include
• Interviews
• Observation
• Contextual inquiry
Field Study Methods
Interviews Contextual inquiry Observation
June 11, 2015 34
Observation
Field Study Methods
June 11, 2015 35Flickr: COD Newsroom
Interviews
Field Study Methods
June 11, 2015 36Flickr: Dave Fayram
Contextual Inquiry
Field Study Methods
June 11, 2015 37Flickr: UGA College of Ag & Environmental Sciences - OCCS
You can combine these methods.
Field Study Methods
June 11, 2015 38Flickr: Highways England
zz
How to Perform a Field Study
June 11, 2015 39
Steps in the Process
• Planning
• Conducting the Study
• Analyzing the Data
• Creating Deliverables
How to Perform a Field Study
June 11, 2015 40
zz
/ / /
How to Perform a Field Study
Planning
June 11, 2015 41
Conduct user research at the beginning or before a project begins.
Planning
June 11, 2015 42
Plan what you want to learn from the research.
• Why are you conducting the research?
• What is the goal?
• What do you want to learn?
Planning
June 11, 2015 43
Review any existing information about the users and their domain.
Planning
June 11, 2015 44
Flickr: Terry FreedmanFlickr: Jim Larrison
Choose the techniques you’ll use to gather that information.
Planning
June 11, 2015 45
Interviews Contextual Inquiry Observation
Determine how many participants to include by considering:
• The number of user groups
• The number of tasks you need to observe
• The need for repetition
Planning
June 11, 2015 465 5 5 5
Determine how many participants to include by considering:
• The limitations of time, budget, and travel
Planning
June 11, 2015 47
Flickr: 401(K) 2013
Recruit and schedule Participants.
Planning
June 11, 2015 48
Create a discussion guide.
Planning
June 11, 2015 49
Create a discussion guide.
• Introduction
• Questions
• Tasks to observe
Planning
June 11, 2015 50
Create a discussion guide.
• Keep it as a loose guide.
Planning
June 11, 2015 51
Plan how you will capture the data.
• Handwritten notes
Planning
June 11, 2015 52
Flickr: Geek Calendar
Plan how you will capture the data.
• Typing notes in a laptop or tablet
Planning
June 11, 2015 53
Flickr: ilouque
Plan how you will capture the data
• Recording video
Planning
June 11, 2015 54
Flickr: Gunnar Bothner-By
Plan how you will capture the data
• Recording video
Planning
June 11, 2015 55
Plan how you will capture the data
• Recording audio
Planning
June 11, 2015 56
Plan how you will capture the data
• Livescribe Smartpen
Planning
June 11, 2015 57
Decide whether you will allow additional observers.
• Two is the ideal number of people to conduct field studies.
Planning
June 11, 2015 58
If you have additional observers, give them rules:
• You’re the leader
• Refrain from talking, making noises, or sending signals with body language
• Where to sit
• When to ask questions
• How to ask neutral, non-leading questions
Planning
June 11, 2015 59
Prepare the participants for your visit.
• Individual session
• Not an interview
• Observing tasks
• In your usual location
• Save work to show during the session
Planning
June 11, 2015 60
zz
/ / /
How to Perform a Field Study
Contextual Inquiry
June 11, 2015 61
Contextual Inquiry
June 11, 2015 62Flickr: Terry Whalebone
General format
• Greeting
• Introduction
• Initial interview
• Observing tasks
• Follow-up questions
Contextual Inquiry
June 11, 2015 63
Greeting
• Make the participant comfortable
• Be informal
• Establish a good rapport
Contextual Inquiry
June 11, 2015 64
Greeting
Contextual Inquiry
June 11, 2015 65
Greeting
Get people to open up and talk to you (Oprah):
• Make the participant comfortable
• Establish trust and rapport
• Listen and let the participant talk most of the time
• Be curious
• Be interested
Contextual Inquiry
June 11, 2015 66
Greeting
Be neutral (Freud):
• Avoid biasing or leading the participants.
• Get the participant talking.
• Listen.
• Don’t judge.
Contextual Inquiry
June 11, 2015 67
Introduction
• Purpose of the research
• Your philosophy – you represent the user
• Explain what you’ll be doing
• Get permission to record
Contextual Inquiry
June 11, 2015 68
Initial interview
• Background information
Contextual Inquiry
June 11, 2015 69
Initial interview
• Background information
• Their experience and involvement in the subject
you’re researching
• What kind of car are you interested in?
• Where are you at in the car buying process?
• What kind of research have you done?
• How do you determine what type of car you’ll buy?
Contextual Inquiry
June 11, 2015 70
Initial interview
• Background information
• Their experience and involvement in the subject
you’re researching
• Discuss their tasks
Contextual Inquiry
June 11, 2015 71
Transition from the interview to contextual inquiry…
Contextual Inquiry
June 11, 2015 72
Flickr: J J
Transition from the interview to contextual inquiry
Conducting the Study
June 11, 2015 73Flickr: Rhys A.
Transition from the interview to contextual inquiry
• Requires the participant to change their mindset from:
Contextual Inquiry
June 11, 2015 74
Interview
Passive, answering questions
Task Demonstration
Actively teaching
Transition from the interview to contextual inquiry
• Make it clear that you’re going to do something
very different.
Contextual Inquiry
June 11, 2015 75
Transition from the interview to contextual inquiry
• Tell the participant:
• Actually perform the task.
• Go through the entire task, step-by-step.
• Tell me what you’re doing.
• You’re the master, and I’m the apprentice.
Contextual Inquiry
June 11, 2015 76
Flickr: Esther Dyson
At the start of each task
• Find out what led up to this point.
• What steps came before?
• Who else was involved?
• How does the task get to them?
Contextual Inquiry
June 11, 2015 77
Flickr: Fabrizio Salvetti
The participant performs the task.
• Goes through each step in the task
• Describes what he/she is doing
Contextual Inquiry
June 11, 2015 78
Flickr: Southern Arkansas University
Flickr: Southern Arkansas University
Ask probing questions as needed.
• “Why?”
• “Tell me more about that.”
• “What happens next?”
• “Can you show me how that works?”
Contextual Inquiry
June 11, 2015 79Hall, Erika. "Interviewing Humans." A List Apart The Full. N.p., 10 Sept. 2013. Web. 21 May 2015.
Flickr: Southern Arkansas University
During the tasks, pay attention to:
• Actions and explanations
• Tools and technology used
• Information, paperwork, documents
• People they interact with
• The environment
Contextual Inquiry
June 11, 2015 80
Flickr: For Inspiration Only
Facilitating a contextual inquiry can be very difficult.
Contextual Inquiry
June 11, 2015 81Flickr: Sasquatch I
Facilitating a contextual inquiry can be very difficult.
Contextual Inquiry
June 11, 2015 82
Flickr: Sasquatch I
Facilitating the session involves:
• Observing
• Listening
Contextual Inquiry
June 11, 2015 83Flickr: You Belong In Longmont
Facilitating the session involves:
• Assessing whether you understand
• Asking questions
• Should I ask a question about this?
• How should I phrase it to avoid bias?
• When should I ask the question?
• Noting questions to ask later
Contextual Inquiry
June 11, 2015 84
Flickr: Ethan Lofton
Facilitating the session involves:
• Assessing the answer
• Did I understand the answer?
• How should I rephrase it?
Contextual Inquiry
June 11, 2015 85
Flickr: Ethan Lofton
Facilitating the session involves:
• Ensuring the participant keeps showing and
explaining the task
Contextual Inquiry
June 11, 2015 86
Flickr: Ethan Lofton
Facilitating the session involves:
• Assessing how well the session is going
• Are you getting the information you need?
• Is this really the task you want to see?
• Is the participant staying on track?
• Do you need to lead the participant in another direction?
• Are there other tasks that you want to see next?
Contextual Inquiry
June 11, 2015 87
Flickr: Anders Sandberg
Facilitating the session involves:
• Keeping track of the time
• How much time is left in the session?
• Is this task taking too long?
• Will we still have time to see the other tasks?
• Maintaining a positive rapport with the participant
• Taking notes
Contextual Inquiry
June 11, 2015 88Flickr: AaltoFablab
How can you handle the competing demands of facilitation?
• Include repetition in participants and tasks
• Don’t take notes during the session
• Review recordings and take notes from those
• Delegate tasks to observers
• Learn and improve with experience
Contextual Inquiry
June 11, 2015 89
Follow-Up Questions
After each task and at the end of the session
• What happens next?
• Who performs the next step?
• Reconfirm your understanding
• Ask any remaining questions
Contextual Inquiry
June 11, 2015 90
Flickr: You Belong In Longmont
Closing
• Thank the participant
• Take photos of the environment,
tools, documents, technology, etc.
Contextual Inquiry
June 11, 2015 91
Flickr: rallenhill
Flickr: Mike Beltzner
zz
/ / /
How to Perform a Field Study
Analyzing the Data
June 11, 2015 92
Type up your notes between sessions.
• Helps you assess what you’ve learned so far
• Avoids too much to type up at the end
Analyzing the Data
June 11, 2015 93
Flickr: Rachel Johnson
Analysis methods: spreadsheet
Analyzing the Data
June 11, 2015 94
Analysis methods: affinity diagram
Analyzing the Data
June 11, 2015 95
zz
/ / /
Yeah, But What If…?
Solutions to Common Problems
June 11, 2015 96
The participants have tasks that can’t be interrupted.
Solutions to Common Problems
June 11, 2015 97Flickr: World Bank Photo Collection
The participants have tasks that can’t be interrupted.
Solutions to Common Problems
June 11, 2015 98Flickr: yooperann
The participants have tasks that can’t be interrupted
Solutions
• Interview before
• Observe the tasks without interruption
• Interview after
Solutions to Common Problems
June 11, 2015 99Flickr: Andy G
People want to do the session in a conference room.
Solutions to Common Problems
June 11, 2015 100
Flickr: Chris Miuccio
People want to do the session in a conference room
Solutions
• Explain the importance of observing tasks in their usual context.
• If that doesn’t work, cancel the session.
Solutions to Common Problems
June 11, 2015 101
Participants try to change it to a group session.
Solutions to Common Problems
June 11, 2015 102Flickr: Pat Kight
Participants try to change it to a group session
Solutions
• Emphasize ahead of time that these are individual sessions.
• Offer to schedule the additional people for their own sessions.
• Be flexible to adapt when it does make sense to include a second participant.
Solutions to Common Problems
June 11, 2015 103
Participants try to change it to a group session
Solutions
• Be flexible to adapt when it does make
sense to include a second participant.
Solutions to Common Problems
June 11, 2015 104
Flickr: IAEA Imagebank
The participant keeps slipping into interview mode.
Solutions to Common Problems
June 11, 2015 105Flickr: You Belong In Longmont
The participant keeps slipping into interview mode.
Solutions
• Make a dramatic transition from the interview to the observation of tasks.
• Keep reminding that this is not an interview, you want to observe tasks in detail.
• Instead of asking, “How do you…?” Say, “Show me how you….”
Solutions to Common Problems
June 11, 2015 106
It becomes a complaint session.
Solutions to Common Problems
June 11, 2015 107Flickr: Penumbra
It becomes a complaint session.
Solutions
• Clarify that you’re there to observe and understand their tasks.
• Ask them to save their list of problems or improvements for the end of the session.
Solutions to Common Problems
June 11, 2015 108
The participant can’t show you a task.
Solutions to Common Problems
June 11, 2015 109Flickr: Penumbra
The participant can’t show you a task.
Solutions - ask the participant to:
• Meet when the participant normally performs that task
• Save work to show you during the session
• Walk you through what they normally do, as best they can
Solutions to Common Problems
June 11, 2015 110
The subject matter is very complicated and unfamiliar.
Solutions to Common Problems
June 11, 2015 111Flickr: Faculty of Medicine NTNU
The subject matter is very complicated and unfamiliar.
Solutions
• Learn as much as you can before you conduct your sessions.
• Interview stakeholders first, to get an overview.
• Observe multiple participants performing the same tasks.
• Review the recordings.
Solutions to Common Problems
June 11, 2015 112
The participant talks above your level of understanding.
Solutions to Common Problems
June 11, 2015 113Flickr: NASA HQ Photo
The participant talks above your level of understanding.
Solutions
• Remind them that they’re the expert and you’re the apprentice.
• Ask dumb questions to remind them how much you don’t know.
Solutions to Common Problems
June 11, 2015 114
zz
Objections to User Research
June 11, 2015 115
“It takes too long. We don’t have the time.”
Objections to User Research
June 11, 2015 116
Flickr: TaxRebate.org.uk
“It costs too much. We can’t afford it.”
• Time for the researcher and designer
• Recruiting
• Incentives
• Travel
Objections to User Research
June 11, 2015 117
“We can’t travel to the participants.”
Objections to User Research
June 11, 2015 118Flickr: Brian Fagan
zz
Overcoming Objections to User Research
June 11, 2015 119
Ensure that projects are scoped with user research as an activity.
Overcoming Objections to User Research
June 11, 2015 120Rosenfeld Media: Leah Buley
Perform user research outside of projects.
Overcoming Objections to User Research
June 11, 2015 121
Sy, Desiree. "Adapting Usability Investigations for Agile User-centered Design." Journal of Usability Studies 2.3 (2007): n. pag. UPA. UPA, May 2007. Web. 6 June 2014.
Start with a small study to show the value of user research.
Overcoming Objections to User Research
June 11, 2015 122
Find participants in your local area to limit travel.
Overcoming Objections to User Research
June 11, 2015 123
Divide the research between several people in different locations, each conducting the
research with local participants.
Overcoming Objections to User Research
June 11, 2015 124
Limit the number of participants by focusing only on the primary user group.
Overcoming Objections to User Research
June 11, 2015 125
Doctors Nurses Hospital Administrators Technicians
Narrow the scope of your research and shorten the sessions.
• Fewer questions you want to answer
• Fewer tasks to observe
Overcoming Objections to User Research
June 11, 2015 126
Flickr: Juhan Sonin
Shorten the analysis and deliverables
• Don’t type up your notes
• Produce only high-level, informal deliverables
Overcoming Objections to User Research
June 11, 2015 127
Flickr: chicgeekuk
Conduct remote contextual inquiries to reach people that you can’t travel to.
Overcoming Objections to User Research
June 11, 2015 128Flickr: HeatherLWilliams
zz
/ / /
The key to designing
an excellent user experience
Understanding the users and their needs
June 11, 2015 129
June 11, 2015 130Flickr: Fly For Fun
Flickr: U.S. Army RDECOM
Flickr: Leonardo RizziFlickr: Va DOT
Flickr: wistechcolleges
June 11, 2015 131
Thank You
zz
Resources for More Information
June 11, 2015 132
Recruiting participants
• Recruiting Better Research Participants: Jim Ross
• Recruiting User Research Participants by Email: Jim Ross
User research methods
• Observing the User Experience: A Practitioner's Guide to User Research: Elizabeth
Goodman, Mike Kuniavsky, Andrea Moed
Resources for More Information
June 11, 2015 133
Interviewing
• Interviewing Users: How to Uncover Compelling Insights: Steve Portigal
• Interviewing Humans: Erika Hall
Resources for More Information
June 11, 2015 134
Contextual inquiry
• Contextual Design: Hugh Beyer, Karen Holtzblatt
• Rapid Contextual Design: Karen Holtzblatt, Jessamyn Burns Wendell, Shelley Wood
• Why are Contextual Inquiries So Difficult?: Jim Ross
• Remote Contextual Inquiry: A Technique to Improve Enterprise Software: Lynn
Rampoldi-Hnilo
Resources for More Information
June 11, 2015 135
Recording user research data
• Capturing User Research: Jim Ross
Handling observers in user research sessions
• Observing User Research: Jim Ross
User Research Deliverables
• Communicating User Research Findings: Jim Ross
Resources for More Information
June 11, 2015 136
Getting buy-in to conduct user research
• It’s Our Research: Getting Stakeholder Buy-In for User Experience Research Projects:
Tomer Sharon
Resources for More Information
June 11, 2015 137
June 11, 2015 138
Thank You
