Usability Testing

Usability testing is the bookend to user research. !Both are needed to support what goes in between.

Usability Testing

What is usability testing? !— First point: We are not testing the user (you will hear the term "user testing", but that may make Jared Spool sad) !— Second point: It’s not a science !!

Usability Testing

What is usability testing? !"“It takes only five users to uncover 80 percent of high-level usability problems." !"… debugging a design." !

— Jakob Nielsen !

Usability Testing

What is usability testing? !"Every time a person has a great experience with a website, a web app, a gadget, or a service, it’s because a design team made excellent decisions about both design and implementation — decisions based on data about how people use designs. And how can you get that data? Usability testing." — Dana Chisnell

Usability Testing

What is usability testing? !(Dana co-wrote this book)

http://www.wiley.com/WileyCDA/WileyTitle/productCd-0470185481.html

Usability Testing

What is usability testing? !"Watching people try to use what you’re creating/designing/building (or something you’ve already created/designed/built), with the intention of (a) making it easier for people to use or (b) proving that it is easy to use." — Steve Krug

Usability Testing

What is usability testing? !(Steve wrote this book)

Usability Testing

What is usability testing? !"In a usability test, individual users work through a series of fairly realistic tasks. Some tasks may be timed, but most often, each participant talks out loud to describe his thought process as he uses an actual product or prototype." — Kim Goodwin !

Usability Testing

What is usability testing? !— It can demonstrate a problem exists — It can help us understand how something we design should behave !!

Usability Testing

What is usability testing? !— There are lots of online resources, even some you might not expect !!!

Usability Testing

What is usability testing? !— It can be time-consuming and not easy; you can outsource it to specialized companies at different scales !— usertesting.com — AnswerLab — blinkUX.com — others

Usability Testing

What is usability testing? !Please note: it is NOT a focus group !People want to agree and show favor. If you basically ask them if they like the work you’ve done, they’ll try to please you (and give useless results).

Usability Testing

What is usability testing? !Please note: it is NOT a focus group !The difference? !"I’m showing you this. Do you like it?" !"Show me how you do this."

Usability Testing

What is usability testing? !It is NOT

http://blog.templatemonster.com/2011/09/14/usability-testing-basics/

Usability Testing

When to do what? !!

by Hong Qi, Donna Lichaw; Adapted & taught by Christina Wodtke 11/13/13

What do we know? User Interviews Contextual Inquiry, Field Studies & Ethnography Diary & Journal Studies Benchmark Usability Tests Card Sorting Surveys

How are we doing? Usability & Concept Tests Eye Tracking Diary & Journal Studies

How did we do? Usability & Concept Tests

Eye Tracking A/B & Multivariate Tests

Data Analysis

Usability Testing

This isn’t rocket science or deep thinking !

But this does require keeping many, things in mind, and self-control, as well as practice.

Usability Testing

The "classic model" !— Develop a test plan — Choose a testing environment — Find and select participants — Prepare test materials — Conduct the sessions — Debrief with participants and observers — Analyze data and observations — Create findings and recommendations

Usability Testing

When to go classic !— Your team/company does it !— You need to deliver lots of data/stats !— Your team/company works with extensive requirements documents !

Usability Testing

Plan !— Discuss with the whole team !— Agree on a test objective (something besides “determine whether users can use it”) !— The methods and measures you’ll use to learn the answers to your research questions !— This can take < 1hr

Usability Testing

Plan !One template: !http://www.usability.gov/how-to-and-tools/resources/templates/usability-test-plan-template.html

Usability Testing

Environment !— You can do it in a formal or dedicated lab !!

Usability Testing

Environment !— You can do it in a more casual setting !!

Usability Testing

Environment !— It helps if there is recording equipment (even just software recording the screen and user on the laptop or desktop being used), but if you only have a person to record observations and take notes, that can work, too. !!

Usability Testing

Environment !— You can even do it remotely (screen sharing via Skype, JoinMe, GoToMeeting; these can record both the user’s screen and the user via webcam) !— And there’s always the ambush method: "Hi. I’m ______________. Do you have a minute?"

Usability Testing

Usability Testing

Recruiting !— Can be hard and time-consuming !— Focusing on the behavior that you’re interested in observing is easier than trying to select for market segmentation or demographics !— If you can, offload. Many services will charge $75-150 per participant and take care of everything

Usability Testing

Recruiting !— mention the gift certificate — friends and family and friends of friends — post to user groups of similar apps/sites — professional mailing lists — did we mention the gift certificate? — Facebook fan pages/Twitter hashtags/LinkedIn groups — probably not people at the same company (see: Google Buzz) — NOT people supplied by the marketing department

Usability Testing

Three may be a magic number

Usability Testing

Screening !— "If you want to test a kiosk for checking people into and out of education programs, you want people who are attending those programs. Make sense?  Don’t make recruiting harder than it has to be." — Dana Chisnell !— make sure they’re available !— make sure they meet your qualifications !— make sure you’ve explained the process to them

Usability Testing

Prep for tests !— Guide or checklist for the moderator !— Includes the research questions !— I any specific interview questions you might want to ask, prompts for follow-up questions, as well as closing, debriefing questions that you want to ask each participant

Usability Testing

Prep for tests !See Hdotcom_test_plan.pdf

Usability Testing

Choosing tasks for the test !— Translate the research questions into task scenarios that represent realistic user goals !— Tasks = user goals, not app/site features — e.g., "How would you book a hotel room?", not "How do you navigate our hotel interface?" !— Give a task that realistically represents a user goal

Usability Testing

Conducting the test

Usability Testing

How to structure the test !(See "session_script.pdf") !

Prep reminders for moderator — Rehearsal and preparation needs — Before each session ("Set up computer", etc.) — After each session ("Turn off recording", etc.) !

Usability Testing

How to structure the test !(See "session_script.pdf") !

Session introduction — Introduce yourself nicely — Think-out-loud protocol — "You can’t do anything wrong" — "You’re helping us find where we confuse you." !

Usability Testing

How to structure the test !(See "session_script.pdf") !

Tasks !

Usability Testing

How to structure the test !(See "session_script.pdf") !

Close the session !This often involves overall closing questions. !

Remember: Thank the participant. !

Usability Testing

How to conduct yourself !— Be nice, empathetic !— Ask open-ended questions: "What would you expect to happen? Where would you look for that?" !— Be reassuring: "We’re trying to make this better, so you’re helping us by pointing out where it fails you."

Usability Testing

How to conduct yourself !— Do NOT say, "Click this/go there" !— Remind participant to think out loud !— Mirror their statements, allow them to clarify !— Don’t interrupt. LISTEN.

Usability Testing

Don’t lead the witness (phrasing and idea: Hong Qi, Donna Lichaw, Christina Wodtke)

Usability Testing

How to take notes !— A dedicated observer/notetaker helps !— Note participants’ facial or non-verbal expressions, body language !— Add time labels to notes ("This at 9:42") !

NOTES OR IT DIDN’T HAPPEN

Usability Testing

How to take notes !

http://silverbackapp.com

Usability Testing

Conducting the test !— The moderator as MC: sees to the safety and comfort of the participants, manages the team members observing, and handles the data collected. !

Usability Testing

Debrief !— Take a step back with the participant and ask, “How’d that go?” Invite the trained observers to pass follow-up questions to the moderator. !— Thank the participant, compensate, and say good-bye. !

Usability Testing

Debrief !— Now, the team observing should talk briefly about what they saw and what they heard. (This discussion is not about solving design problems, yet.)

Usability Testing

The RITE way !Rapid Iteration Testing Environment — integrates well into Lean UX and agile development in general — Boxes and Arrows podcast "Rapid-Turnaround Usability Testing" by Kyle Soucy and Holly Phillips: https://itunes.apple.com/us/podcast/rapid-turnaround-usability/id275459507?i=82723199&mt=2 — Keys include: small testing sample, rapid design iterations, debriefs of all stakeholders time-limited and tied to physical observations

Usability Testing

When to test

Usability Testing

The Lean UX "3-12-1" !!!

Usability Testing

The Lean UX "3-12-1" !!!

Usability Testing

Analyze !— Can be stories (qualitative), can be stats (quantitative) !

http://www.fanxingkong.com/digital-music-notepad/

Usability Testing

Findings and recs !— Concentrate on the most significant and serious findings, not all the details !— It’s a good idea to include a summary of the main points !

Usability Testing

Findings and recs !— Think visual. You ideally want lots of screen shots and perhaps even video clips to showcase issues and findings !— Be ready to discuss in person

Usability Testing

Krug says: You can do this.

Usability Testing

Any testing helps: !Why? !— The "second set of eyes" principle !— The "it’s always been right in front of me" principle !— Everything is broken, even if everything is awesome !(Can still be a pain.)

Usability Testing

Hey, ho, let’s go !A: Create a script !B: Pair up !C: Each person interviews the other. Then switch. !D: Report back to the group what you learned

Review

Do = !! !

Why > Do

Review

Know the Why !

Then Do

Review

And thanks

ddt@twoangstroms.com

twitter web