Top 10 User Research Pitfalls
Dan Arra Managing Director
2
PART 1 About the Speaker PART 2 What is Research? PART 3 10 Research Pitfalls
Agenda
3
ABOUT THE SPEAKER
About The Speaker: Dan Arra
• Began career as an engineer in the aerospace industry.
• Spent 10 years at software companies in systems engineering, product management and professional services.
• Started software & technology companies and was responsible for leading numerous successful projects. Another 10 years.
• Joined Macadamian, a software design & development studio. Work closely with design and software engineering teams to improve and create new products.
4
5
WHAT IS CUSTOMER RESEARCH?
User-centered Design
6
Good design is built on a solid understanding of your audience: specific goals, tasks and context of use.
UCD: understanding users and tasks
Confidential 3/22/13 7
Research Methods: Qualitative versus Quantitative
QUALITATIVE QUANTITATIVE attitude about
your product and competition
Focus groups Interviews
Surveys (online, telephone)
Behavior related to your
product offering
Ethnographic research (Observation, shadowing)
A/B Testing
usability of product
Usability testing
Usability testing
Confidential 3/22/13 8
9
10 RESEARCH PITFALLS
The most dangerous things about research!
10
Myth: Being Smart means you can do good UX Research & Design. Anybody can ask a question… but relevant, appropriate, constructive?
Design research IS real science!
11
Sci-ence [sahy-uh ns] noun • systematic knowledge of the physical or material world gained through
observation and experimentation.
Reality!
12
Observe Think Design Test
! Rinse, repeat.
Our users don’t tell us what they need
13
Myth: We asked users what they wanted, so we’re covered.
Users are NOT designers
• Users are focused on solutions • Don’t consider context and other mitigating factors
• Why? Why? Why?
14
Avoid jumping to Solutions too quickly
15
Staying in problem space leads to INNOVATION
16
Go to: http://www.macadamian.com/ for full story.
Staying in problem space leads to INNOVATION
17
Go to: http://www.macadamian.com/ for full story.
Staying in problem space leads to INNOVATION
18
Go to: http://www.macadamian.com/ for full story.
Staying in problem space leads to INNOVATION
19
Go to: http://www.macadamian.com/ for full story.
Talking to customers " Research
20
Who to ask... AND How to “ask”.
Consider Triangulating Research !
21
Stating, without knowing! “We must have this feature”! can be very to EXPENSIVE
22
Go to: http://www.macadamian.com/ for full story.
Numbers can be deceiving!
23
Sometimes numbers are not appropriate!!
24
Qualitative Quantitative Attitude
(Preference) • Focus Groups • Interviews
• Surveys • A/B Testing
Behavior (Performance)
• Usability Walkthrough • Ethnographic Research • Job Shadowing
• Usability Test • Benchmark Test
Focus Groups & Usability Testing!NOT the same!
25
What people say, what people do, and what they say they do are entirely
different things.
Photo credit: http://www.webusability.co.uk/what-we-do/mobile-testing/
Tale of Two methods
26
Usability Testing Focus Groups Method One-on-one testing; observing representative
samples of end users using the product and measuring their performance on pre-determined tasks (n= # of individuals being tested)
Group discussion/exploration with up to 10 participants per group (n=1)
When to use Throughout development phase and during deployment
Pre-development; early in program verification
Key focus What will make the product easy for end users to learn and use?
What will motivate potential customers to buy the product? Determining market need.
Key motivation
How well can end users use the functionality a product provides? Can they realize the value promised?
What are the customers' wants and desires ?
Eight is sometimes enough
27
It just depends
28
Faulkner, Laura (2003) Beyond the Five-user Assumption: Benefits of Increased Sample
sizes in Usability Testing Behavior Research Methods, Instruments and Computers Volume
35 (3) 379-383
Raw data is NOT enough
29
Translate data into meaningful requirements
30
Don’t rely only on the Spec!
31
More than words can say!
32
Not just anyone can do research
33
UX Researcher – Prevents “heard what I wanted to hear.”
• Consider bringing someone on board with experience, a Degree in Human factors, HCI and/or soft sciences (e.g., Psychology, Sociology).
• If conducting true user research, you’ll want:
− someone with knowledge and experience in research methods and/or experimental practices.
− someone with the expertise to turn insights into design direction
• Partner user researcher with SMEs, business analysts, and product managers for maximum success!
34
A Good Researcher!
! can identify the best methods to get the right information ! is “familiar enough” with the subject material to determine the
right questions ! avoids leading questions ! has strong listening skills ! follows the the natural flow of the conversation ! can steer the conversation in the case of digression and guide
user to discuss new topics when one is exhausted ! remains neutral and puts the interviewee at ease ! focuses on the most recent experiences ! looks for ways to visualize and summarize results to be
actionable
35
36
Ask the right questions! Use the right method(s) Analyze appropriately Use the right people!
Remember
37
Improving AND growing business. Developing relationships = gold. Establishing credibility = save time. Learning/refining your craft.
Remember, also that you are!
Thank-You
[email protected] For more information, visit: www.macadamian.com Follow Macadamian on Twitter: @macadamianlabs
38
Confidential 3/23/13 39
”Innumeracy: Mathematical Illiteracy and Its Consequences”, by John Allen Paulos “Ambient Findability: What We Find Changes Who We Become”, by Peter Morville “Communicating Design: Developing Web Site Documentation for Design and Planning,” by Dan Brown The Design of Everyday Things Don Norman Designing Interfaces Jennifer Tidwell Do You Matter? How Great Design Will Make People Love Your Company Robert Brunner “Don't Make Me Think: A Common Sense Approach to Web Usability,” by Steve Krug
Design and Research Resources
Confidential 3/22/13 40
“The Elements of User Experience: User-Centered Design for the Web,” by Jesse James Garrett Emotional Design Don Norman Mental Models: Aligning Design Strategy With Human Behavior by Indi Young “Observing the User Experience: A Practitioner’s Guide to User Research,” by Mike Kuniavsky “Presentation Zen Garr Reynolds Sketching User Experiences: Getting the Design Right and the Right Design,” by Bill Buxton Universal Principles of Design William Lidwell “Visual Explanations: Images and Quantities, Evidence and Narrative,” by Edward Tufte
Design and Research Resources
Confidential 3/22/13 41
We create compelling user experiences for: • Consumer and enterprise desktop applications
• Interfaces embedded into physical products • Mobile interfaces and applications on multiple platforms
• Web applications and cloud-based applications
About Macadamian
Design + Development!
42 Confidential 3/22/13
43
Ottawa!
San Mateo!Yerevan!
Cluj, Romania!
Confidential 3/22/13
• Over 200 employees!• 15 + years in business!• Integrated design & development teams!
44
What We Do: UX services
• UX Research − Stakeholder Interviews − Product Strategy Workshops − Task Analysis − On-line Surveys − Usability Testing
• Software Development − Stand alone or combined with design − PoC or market ready − Technologies: Web, Cloud, Mobile,
Desktop − Agile methodology − Integrated with customer teams
• UX Design − Information Architecture − Content Modeling − Interaction Design − Visual Design
• Software Quality Assurance-QA − QA Review: performance or process − QA/QC for customer lead development − QA/QC within our agile process − Localization QA − QA Script/Automation