GETTING BUY-IN FOR UX RESEARCH
CHRISTINE PERFETTI & EZRA GILDESGAME ACQUIA
THE LOGISTICS
Ask questions as they occur to you
The schedule
You’ll be working in groups of 2
Lean UX NYC Perfetti & Gildesgame, © 2014 1
EXERCISE
CONDUCT A USABILITY TEST
USABILITY TEST
A usability test involves putting your users in front of the product and observing what they do
Lean UX NYC Perfetti & Gildesgame, © 2014 2
One person is the user
‣ Think aloud as you work
One person is the observer
‣ Observe silently and take notes
After we perform the task, we’ll ask you to switch roles and do another
EXPLORATION: USABILITY TEST
‣ We’re evaluating the design, not you
‣ The answer is somewhere on the site
‣ If you experience problems, it’s not your fault
‣ You’re helping the observers learn more about the design problems
AS THE USER
Lean UX NYC Perfetti & Gildesgame, © 2014 3
Take notes as the user works‣ Was the user successful?
‣ How did the user go about performing the task?
‣ What obstacles did the user encounter?
‣ What confusion did they experience?
‣ Note things that worked well
When you complete a task, say, “Got it!”
AS THE OBSERVER
‣ How many users found the answer?
‣ What helped users succeed?
‣ What obstacles prevented them from finding what they wanted?
‣ What do you expect will happen?
DISCUSSION
Lean UX NYC Perfetti & Gildesgame, © 2014 4
Planning your research program‣ The first 90 days
‣ How to choose your first project
‣ Identifying your allies and champions
Involving stakeholders in planning‣ Planning and running usability tests
‣ Quick testing techniques for buy-in
WHAT WE’LL COVER
Analyzing the data‣ Rolling list of observations
‣ KJ technique
‣ Journey maps
Communicating results effectively‣ Weekly research days
‣ Formal and informal reporting strategies
‣ Effective communication techniques
DISCUSSION
What prevents you fromgetting buy-in for your research?
Lean UX NYC Perfetti & Gildesgame, © 2014 5
Text
THE FIRST 90 DAYS
Meet with all stakeholders
Know your audience
Write a study plan
Conduct your first usability test
Lean UX NYC Perfetti & Gildesgame, © 2014 7
‣ Meet with all of your product stakeholders to understand their priorities
‣ Treat these meetings just like any other research project‣ Who are your users?
‣ What are their main goals?
‣ What challenges do they face today?
‣ What do they currently know about their users?
‣ What do they know about the product? What are the areas of risk?
KNOW YOUR AUDIENCE
What are all of the components of your product?
Lean UX NYC Perfetti & Gildesgame, © 2014 8
‣ Interview anyone who has knowledge of the product or the business
‣ The product team includes:‣ Engineering
‣ Product Management
‣ UX
‣ Marketing
‣ Sales
‣ Support
‣ Legal
THE PRODUCT TEAM
‣ What is your role with respect to the product?
‣ What did you do before this?
‣ Who is the product for?
‣ How would you describe the client onboarding process?
‣ Is there anything that concerns you about the product?
‣ What should the user research accomplish for the business?
‣ How will you define success for the research?
‣ How would you like to be involved in the research?
GENERAL QUESTIONS
Lean UX NYC Perfetti & Gildesgame, © 2014 10
QUESTIONS: SPECIFIC TO AUDIENCE
Sales‣ Why do customers buy the product?
‣ Why do you win/lose sales?
‣ Who are the users today?
‣ How should they be different in 5 years?
‣ What do customers complain about most?Who
Support‣ Who typically contacts you?
‣ Why do they contact?
‣ What are the top problems for customers?
‣ What areas of the product cause problems?
‣ How can we best reduce calls?
The research team is at your service
Lean UX NYC Perfetti & Gildesgame, © 2014 11
SELLING THE VALUE: FRAME BENEFITS IN THEIR TERMS
Sales‣ Increase win rates
‣ Make selling easier
Product‣ Research can inform debate about product strategy
‣ Come armed with great data & insights
Support‣ Decrease # of tickets
‣ Decrease ticket response time
Finance and leadership‣ Increase renewals
‣ Improve win/growth rate
‣ Improve margins
Identify your Executive Champion
‣ Who drives the product and business decisions?
‣ What leader most values and understands user research and design?
Conduct a research project to:
‣ Answer the Executive Champion’s questions
‣ Focus on their risks and concerns
‣ Accomplish a quick win
CHOOSE YOUR FIRST PROJECT
Lean UX NYC Perfetti & Gildesgame, © 2014 12
CREATE A STUDY PLAN
THE STUDY PLAN
Serves as a blueprint for the test
Provides a systematic approach to testing
Facilitates buy-in from stakeholders
Lean UX NYC Perfetti & Gildesgame, © 2014 13
‣ Study goals
‣ Research questions and issues to explore
‣ User characteristics
‣ Schedule
‣ Task list
‣ Data to collect
‣ Reporting strategy
THE TOPICS
THE PLANNING MEETING
At the beginning of the project, we hold a planning meeting with all product team members and stakeholders to discuss:‣ An overview of the testing process
‣ Project scope
‣ Issues (risks and concerns) with the design
‣ Number of users to test
‣ Testing schedule
GOAL: No surprises! Avoid the game of telephone
Lean UX NYC Perfetti & Gildesgame, © 2014 14
MEETING ATTENDEES
‣ Product and Design
‣ Marketing and Engineering
‣ Anyone involved in design decisions
PROJECT SCOPE
What are you testing?
‣ Complete working prototype
‣ Prototype with only some screens
‣ Paper mockup
‣ Just the navigation
Lean UX NYC Perfetti & Gildesgame, © 2014 15
‣ What do you want to learn?‣ Most studies aren’t purely exploratory
‣ The issues and research questions will drive the tasks and recruitment
‣ Work with the team to outline the research questions
‣ Vague statement: Will this work for users?
‣ Ask increasingly specific questions
RISKS AND CONCERNS
‣ What are you hoping to learn from the study?
‣ What areas of the design are you most concerned about?
‣ Why are you concerned?
‣ Are you worried about a specific group of users?
‣ What are the greatest risks for the organization?
QUESTIONS TO ASK
Lean UX NYC Perfetti & Gildesgame, © 2014 16
‣ What do you want to learn?
‣ What areas are you most concerned about?
‣ Is our mortgage application too long?
‣ Will users know all fields are required?
‣ Will they use account lookup? (It will be hard to build.)
‣ Do our users understand what “conforming” means?
EXAMPLE: MORTGAGE APPLICATION
TECHNIQUES FOR BUY-IN
Lean UX NYC Perfetti & Gildesgame, © 2014 17
THE EXCUSES
Not enough time
No resources and staff
No money No buy-in from management
Unsure where to start
Testing is too scientific
We already know the product
TECHNIQUES FOR STAKEHOLDER BUY-IN
First Click Test
5 Second Test
Comprehension Test
Ethnographic Interviews
Lean UX NYC Perfetti & Gildesgame, © 2014 18
Quick and dirty technique for measuring content pages
Takes less than 10 minutes to run
Measures if content pages quickly convey their purpose
5 SECOND PAGE TESTS
You want to post pictures online from your last vacation
You are concerned that the upload process will be difficult
How confident are you that you can upload photos quickly and easily?
SHARING PICTURES
Lean UX NYC Perfetti & Gildesgame, © 2014 19
You want to post pictures online from your last vacation
You are concerned that the upload process will be difficult
How confident are you that you can upload photos quickly and easily?
SHARING PICTURES
Lean UX NYC Perfetti & Gildesgame, © 2014 21
You want to post pictures online from your last vacation
You are concerned that the upload process will be difficult
How confident are you that you can upload photos quickly and easily?
SHARING PICTURES
Lean UX NYC Perfetti & Gildesgame, © 2014 22
Useful method to assess where users first click on your site’s home or entry page
Provide users with a specific task to complete when they arrive at the site
By observing where users first click, it’s a clear indicator whether they’ll succeed
FIRST CLICK TEST
Lean UX NYC Perfetti & Gildesgame, © 2014 24
Pages containing complex content
Such as policies or procedures
How your product works
User comprehension is imperative to their success
Questions determine if users understand content
COMPREHENSION TEST
Lean UX NYC Perfetti & Gildesgame, © 2014 26
Excellent technique for understanding users’ goals
ETHNOGRAPHIC INTERVIEWS
Context of use
Pain points
Workarounds
Motivations
Lean UX NYC Perfetti & Gildesgame, © 2014 27
With a field study is, we learn about:‣ How technology fits into users’ life
‣ Tasks that users haven’t talked about
‣ Pain points and user frustration
‣ Evidence of technology that isn’t working for the user
“If I had asked people what they wanted, they would have said faster horses.”- Henry Ford (maybe)
FIELD STUDIES
VISIT USERS IN THEIR NATURAL SETTING
Lean UX NYC Perfetti & Gildesgame, © 2014 28
‣ Typically takes at least two hours at the users’ site
‣ If it’s not possible to visit users onsite, we use a remote meeting tool, such as GoToMeeting or WebEx
‣ Equipment‣ Digital recorder or audio recorder
‣ When approaching the interview, take the mindset of an apprentice‣ The user is the expert
INTERVIEW LOGISTICS
‣ Purpose of interview or site visit
‣ Demographics and background
‣ Overview of work process‣ What is your typical workday?
‣ Can you give an overview of your work?
‣ Who do you interact with?
‣ What are the most important and frequent tasks?
RECOMMENDED TOPICS
‣ Current behavior relevant to project without focusing on technology‣ Overview of technology and software user
works with
‣ How does technology support your work?
‣ What are you trying to accomplish by performing the tasks? (Goals)
‣ Follow up on key user feedback
‣ Closing
Lean UX NYC Perfetti & Gildesgame, © 2014 29
Avoid leading questions‣ Users like to respond, “yes”
Avoid:‣ Do you like this feature?
‣ Are you confused by this content?
BEWARE: LEADING QUESTIONS
‣ What activities waste your time or drive you crazy?
‣ How did you learn about the product?
‣ What kind of training did you receive?
‣ What are you most important activities?
‣ Who do you provide information to? What information do you collect?
SAMPLE QUESTIONS
‣ What are your most important and frequent activities with the product?
‣ What are the two things you like best about the product?
‣ What are the two things you would like to see improved?
‣ Are there any activities you currently perform that you’d like to see automated?
‣ What shortcuts help you?
Lean UX NYC Perfetti & Gildesgame, © 2014 30
‣ Understanding of work environment
‣ Tasks that users haven’t talked about
‣ “Cheat sheets”
‣ Evidence of technology that isn’t working for the user
WHAT TO OBSERVE
‣ Product pain points that were previously unknown
‣ Customers successfully using product in unexpected ways
‣ Customer needs that were previously unknown
‣ Examples that help tell the story (supplementing data)
‣ Identify clients who want to continue to partner
WHAT STAKEHOLDERS LEARN
Lean UX NYC Perfetti & Gildesgame, © 2014 31
ANALYZING DATA WITH STAKEHOLDERS
ANALYZING DATA
The KJ Method
Rolling list of observations
Journey mapping
Lean UX NYC Perfetti & Gildesgame, © 2014 32
‣ Gather all observations
‣ Identify problems
‣ Prioritize problems based on importance and frequency
‣ Iterate on the design
ANALYZE DATA
ROLLING LIST: OBSERVATIONS
Lean UX NYC Perfetti & Gildesgame, © 2014 33
‣ Named after Kawakita Jiro
‣ Consensus method for grouping and prioritizing usability findings
‣ Quickly determines most important observations
‣ Captures everyone’s perspective and observation
THE KJ: METHOD FOR PRIORITIZING DATA
‣ Call together the group
‣ Only invite members of the team who observed a test session
‣ KJ takes one hour
STEP 1: ORGANIZE GROUP
Lean UX NYC Perfetti & Gildesgame, © 2014 34
‣ Every KJ has a focus question that drives the prioritization exercise:
‣ What are the biggest usability problems we observed in the test?
‣ What needs to be fixed in the product to improve the user experience?
STEP 2: DETERMINE FOCUS QUESTION
KJ EXERCISEWhat are the biggest problems
that need to be fixed with the airline experience?
Lean UX NYC Perfetti & Gildesgame, © 2014 35
STEP 3: LIST USABILITY PROBLEMS OBSERVED
STEP 4: PUT OBSERVATIONS ON WALL
Lean UX NYC Perfetti & Gildesgame, © 2014 36
STEP 9: ORDER GROUPS AND DISCUSS
‣ We order the groups based on number of votes, highest numbers at the top
‣ Participants combine groups and discuss identified priorities
‣ Everyone is involved
‣ Writing down observations lets all team members contribute
‣ Prevents too much influence from select team members
‣ Identifies top priorities
METHOD ADVANTAGES
Lean UX NYC Perfetti & Gildesgame, © 2014 39
‣ Up-front study planning‣ Focus question: What are your research questions?
‣ Brainstorming sessions with stakeholders?
‣ Focus question: What are the biggest user problems we should address?
‣ Usability test debriefs
‣ Focus question: What are the biggest problems we observed in the tests?
‣ Immediately following field studies
‣ Focus question: What are the biggest pain points you observed? What solutions could help?
WHEN TO USE THE KJ METHOD
JOURNEY MAPPING
Lean UX NYC Perfetti & Gildesgame, © 2014 40
‣ Excellent technique for identifying pain points in the user journey
‣ Clearly communicates the opportunities for improvements and innovation
‣ Prioritizes the areas of the product that need the most attention
‣ Go from abstract to actionable
“World-class product” “Delight users” “Leading user experience”
JOURNEY MAPPING
Lean UX NYC Perfetti & Gildesgame, © 2014 41
Carbonite.com Journey Map
Mapping out the current experience
Lean UX NYC Perfetti & Gildesgame, © 2014 48
Task profile: Well performing areas
Task profile: Areas to improve
Lean UX NYC Perfetti & Gildesgame, © 2014 49
Q1 Update: 11 Critical Tasks Tested
Q1 2013 compared to Q4 2012
Lean UX NYC Perfetti & Gildesgame, © 2014 50
CREATIVE BRIEF
Benefits
‣ Helps people to fully articulate their ideas
‣ Shared understanding of goals and risks (and even disagreement)
‣ Help establish a shared understanding before starting a project
Components:
‣ Product statement
‣ Business problem
‣ Success criteria
‣ Risks
‣ Personas
‣ Deadlines and milestones
‣ Inspiration
COMMUNICATING RESEARCH
Lean UX NYC Perfetti & Gildesgame, © 2014 51
COMMUNICATING RESULTS
Formal methods
Informal methods
Involve stakeholders
‣ Communicate the results quickly
‣ Debrief often
‣ Report to stakeholders and project team when they need it
‣ Don’t report too much information
‣ Determine best delivery method considering the needs of your different audiences
‣ No surprises!
SUCCESS FACTORS
Lean UX NYC Perfetti & Gildesgame, © 2014 52
Informal
‣ Emails or internet dashboard with short memos of findings and recommendations
‣ Working meetings with project teams
‣ Integration with existing problem/bug reporting strategy
Formal
‣ Presentations and workshops
‣ Usability reports
METHODS: COMMUNICATING RESULTS
‣ Effective communication tool for sharing test results immediately
‣ After each day of tests, send a short email to design team with major observations from the day
EMAIL / INTERNET DASHBOARD
Lean UX NYC Perfetti & Gildesgame, © 2014 53
‣ When quick changes are required after a round of tests, schedule a one-hour meeting to share and prioritize the test results
‣ Review the usability test observations
‣ Determine next steps and short-term fixes
WORKING MEETING
‣ Most organizations already have a method for collecting problem or bug reports
‣ Merge usability problems with other bugs
‣ Prevents having two tracking systems
‣ Helps determine relative importance of a problem
INTEGRATE WITH BUG TRACKING
Lean UX NYC Perfetti & Gildesgame, © 2014 55
‣ 1-2 hour presentation with design team, engineers, marketing, and key stakeholders
‣ Share most important usability findings based on prioritization of problems
‣ Effective communication technique
‣ People often ignore written communications
‣ Active involvement with design team and stakeholders
PRESENTATIONS/ WORKSHOPS
EXECUTING ON UX IMPROVEMENTS: MORE THAN BUY-IN
‣ Nobody says, “Let’s have a poor UX”
‣ “Experience Rot” as a risk
‣ It’s about the ideas, not the people
‣ Encourage open discussion of data
‣ Avoid sacrificing long-term quality for short-term goals
‣ Document commitment to UX and get buy-in for when things deviate
‣ Eat vegetables, floss daily, follow through on UX goals
Lean UX NYC Perfetti & Gildesgame, © 2014 56