Qualitative Research Methods in User Experience Design
UX Tokyo Talk February 2014
What is Qualitative Research? The data is not numeric – Photos – Videos – Direct observation – Comments on message boards – Recordings of phone transactions – Interview transcripts
What is Qualitative Research? Although this data can be converted into
quantitative data, doing so is not always useful or appropriate. – Data of this type is usually very difficult to collect
consistently across a representative sample – Meaningful valuable outcomes can be achieved
without using numbers – This is especially good at providing you insights into
things you didn’t already know
What is Qualitative Research? Analysis, interpretation, and presentation often are done
using very different techniques from quantitative analysis.
Examples: – personas - A "typical" user presented in a way that a
development team can focus on – process maps - An understanding of the actual steps in a
process
What is Qualitative Research? Some (mostly) qualitative research traditions from
the social sciences: – Ethnography – Biography – Phenomenology – Grounded Theory – Case Study – Ethno-methodology
Source: Qualita've Inquiry and Research Design: Choosing Among Five Tradi'ons by John Creswell
Why do Qualitative Research? It is sometimes the best way to address research needs:
– Discovering issues that you don't know exist – Sorting out messy information ( ie the critical knowledge only
exists in the heads of a privileged few) – To complement quantitative research. – Some projects do not allow for easy collection of quantitative
data (ie a B2B with a limited number of customers, hardware development, or “sensitive” markets)
How to Start Research should be designed carefully.
You need to consider: – Budget – Goals – Available resources – Schedule – Etc...
How to Start Good research starts with questions Examples:
– How can we improve app X? • What pain-points do current users have? • What else would users like to do with the app?
– Which of these 5 innovation projects should we productize? • What are the potential market, competitors, technical risks,
etc...? • Which ones excite people?
How to Start Know why your doing it Often, you just want to bring back something for the
design team to make brain-storming more productive.
But there can be other strategies
How much data to collect How many visits to end users’ homes/offices is
enough? It depends… – Until you seen so much that you aren’t learning anymore – Until you run out of time or budget – Until you meet the objectives of the study
Some of the Tools Semi-structured interviews
User observation
Photo & video “ethnography”
Semi-structured interviewing Goldilocks interviewing style Structured interviews tend to miss out: – What you don’t know to ask – Getting good detail without consuming too much time
Unstructured interviews tend to gather data that is – Inconsistent – Off-topic
Semi-structured interviewing Usually requires more planning & refining Create a research protocol
– An outline of what you want to talk about – Format it so that you can be flexible while talking – Pay attention to how you print it and use it in the field
Update it as you research
Like many other skills, it requires talent and practice.
Semi-structured interviewing Recording Where possible, audio-record Get good at note-taking too Working in pairs is also recommended, when possible
Semi-structured interviewing Steering the conversation Because it’s an interview, you are allowed to drive it in the
direction you want to. But you also want to get them talking as openly, freely, &
much as possible.
Always be very aware of the time, but don’t make it too obvious!!!
Semi-structured interviewing Tricks: Use pauses and interjections (相槌) effectively Get them to tell stories Get them to show you things
Observation "What people say, what people do, and what they say
they do are entirely different things." Margaret Mead
Observation Depending on your topic, ability to observe and
video or photograph end users varies greatly
Strategies typically follow these constraints
You should always try to collect as much intelligible data as you can, including what you store in your head
Observation
When you can’t use a camera….
Observation
Note what you can & re-sketch before you forget details
Observation Frameworks Typically useful for novice researchers or when
including engineers/designers in the research process – AEIOU (Actions, Environments, Interactions, Objects,
Users) – POEMS (People, Objects, Environments, Message,
Services) – Who, What, Where, Why, When
What to do with the data Process it while it is fresh
If possible, everyone that participated should sit down immediately following the interviewing/observation, to discuss what was seen.
Copious notes should be taken during this session.
What to do with the data If the goal of the research is deeper
understanding… Deep analysis is needed. Methods vary by researcher. – Post-It sorts – Video tagging – Conversation analysis
Various, sometimes lengthy, reports result
What to do with the data If the goal is quick fodder for brain-storming…
Make the data as clear and visual as possible Try to bring the design team into the user’s
shoes