Date post: | 02-Jul-2015 |
Category: |
Design |
Upload: | kyle-soucy |
View: | 1,889 times |
Download: | 3 times |
What do you want to learn?
How do teachers shop for books?
Do they use these slider dots?
Do they notice these arrows?
Qualitative vs. Quantitative Research• Qualitative Research
- Data gathered directly by observing the user
- Researcher can ask follow-up questions, probe on behavior
- Analysis of data is not mathematical (Why? How? Compare...)
• Quantitative Research
- Data gathered indirectly through surveys, log files, support calls, etc.
- Data can be averaged or statistically testedSource: Christian Rohrer Move, Inc. | Realor.com
1. Write a list of the most important tasks that users need to accomplish using the product.
2. Turn each task into a scenario.a) Provide some context.b) Supply needed information.c) Don’t give any clues or hints!
3. Decide on the order of tasks.a) Make the first task easier.
Good Task: You need to get a book about genealogy that covers the reliability of different sources. You want this book in your hands by Thursday. Find and purchase this book.
What makes a good task:
• You’re asking users to do something that they would actually do in real life.
• Good scope. Not too broad, not too specific.
• Has a clear end point.
• Elicits action, not opinion.
Don’t start with “how would you” or “where would you”
Good Task: You need to get a book about genealogy that covers the reliability of different sources. You want this book in your hands by Thursday. Find and purchase this book. Find and purchase this book.
✓ Has a clear end point✓ Elicits action, not opinion✓ Good scope
Your Turn...
1. Take out a sheet of paper (or your computer, phone, or iPad)
2. Write down the most important task your users perform on your product.
3. Turn the task into a scenario.a) Provide some context.b) Supply needed information.c) Don’t give any clues or hints!
Restrictions: Consider whether or not you want to place restrictions on how participants try and accomplish the tasks (e.g. “don’t use search” or “stay on the website”).
Printing:If you’re conducting an in-person test, you may want to print out the scenarios so the participant can refer back to it.
Time-Fillers: Create more tasks than you think you have time to complete just incase someone finishes early. Every minute with a participant is an opportunity to learn something.
Visual Aids:If you’re worried that revealing the name of something will bias the results, consider showing the participant a picture instead.
Visual Aid Example:A friend of yours uses on of these to make baby food,
so you’d like to buy one...
Alternating Tasks/Prototypes:Depending on what you’re testing, you may need to alternate tasks. Participants are learning the interface as they go, so the last task may always be the easiest to accomplish unless you alternate them.
Interview-based Tasks
1. Interview participant about how they use the product.
2. Use the answers given in the interview to create tasks on the fly.
When in doubt just ask, “what was the last thing you did when you used this product? Can you show me?”
Remote Testing Pros & Cons
Participants stay in their native environment.
Accessible to a larger & more diverse pool of participants.
Recruiting is easier. Less no shows & easier last minute replacements.
Opportunity to gather ethnographic data.
Cost & time savings. No travel & lab rental fees.
Easier for observers to “attend” a session.
Risk of compromising data because the participant is using a foreign PC is removed.
Pros:Can’t see the user’s facial expressions.
Can’t see if the user is using peripheral devices.
Dependent on an Internet connection.
Can only recruit participants with a high-speed Internet connection.
Cons:
Remote Testing Tips• If using a cordless phone, have multiple handsets
charged and ready
• Use a phone recording controller
• Use as few tools as possible
• Back up, back up, back up!
• USB Modem
• Screen sharing backup
• Create a test meeting to make sure the participant can connect beforehand
• Instruct participants to log on early and use a landline (no VOIP)
• Turn off entry/exit chimes for conference call
• Allow only one observer to chat to you
• Use a separate chat/IM client
Satisfaction Ratings/Opinion Rankings
Image source: http://wikidoc.org/index.php/Pain
It is simple to use:Strongly Disagree ---1---2---3---4---5---6---7 Strongly Agree
In-context Feedback
Image source: Loop11
“...I’m only offered a search by model number.
How ridiculous.”
“...I would like to see the products availability and delivery/
shipping information listed.”
Process
1. Define the study
2. Recruit participants
3. Launch test and send email invites
4. Analyze results
3. Run a Pilot test!>
4 /5 /
Nothing beats watching participants in real time and being able to ask probing questions about what they are doing as it’s happening.
Some participants may only be interested in earning the honorarium you’ve provided as an incentive.
$$$
It is simple to use:Strongly Disagree ---1---2---3---4---5---6---7 Strongly Agree
Image source: http://www.hiero.com/web-analytics.html
What participants report on surveys can be very different in comparison to what they actually do.
Image source: http://www.hiero.com/web-analytics.html
It’s possible for participants to think they’ve successfully completed a task when they haven’t.
Image source: http://www.hiero.com/web-analytics.html
Does it matter if it took someone longer to complete a task?
Not a replacement for moderated testing or research.
Best used to compliment your qualitative research.
Video Recording Options
• Techsmith Morae - $1,995 USD
• Techsmith Camtasia - $99 USD
• Silverback (Mac only) - v2 Free
• Online Meeting Built-in Recorders
Reminder Prompting
• So...?
• So, what are you thinking?
• What are you seeing here?
• What are you looking at?
• Can you tell me what you think is happening here?
Source: Moderating Usability Testing by Joe Dumas and Beth Loring
Probing Questions
• Is that what you expected?
• What would you do next?
• What did you think about that task?
• Help me to understand more about...
Source: Moderating Usability Testing by Joe Dumas and Beth Loring
Encouraging Statements
• This feedback is really helpful.
• Thank you for thinking out loud.
• That’s good to know.
• You’re doing fine.
Source: Moderating Usability Testing by Joe Dumas and Beth Loring
Remain Neutral
• When in doubt turn the question around:
• “I don’t know what do you think?”
• “What would you do if I wasn’t here?
• Keep acknowledgements limited to: “ah”, “Mm Hmm”, “Ok”, etc.
Source: Moderating Usability Testing by Joe Dumas and Beth Loring and Rocket Surgery Made Easy by Steve Krug
“...the mobile web will be bigger than desktop Internet use by 2015.”
- Morgan Stanley Study
Source: http://mashable.com/2010/04/13/mobile-web-stats/Source: http://mashable.com/2010/04/13/mobile-web-stats/
Image source: http://3.bp.blogspot.com/-Oh6rZE2fG1k/Uct4T4KhE0I/AAAAAAAAARo/3c_HFDkMwAE/s1600/girls-on-their-phone.jpg
Mobile matters!
Mobile In-person Usability Testing
Image source: http://unmatchedstyle.com/news/usability-testing-on-mobile-devices.php
Image source: http://www.90percentofeverything.com/2010/05/07/quick-tip-make-your-own-iphone-usability-testing-sled-for-5/
Testing on paper prototypesImage source: http://www.90percentofeverything.com/2010/05/07/quick-tip-make-your-own-iphone-usability-testing-sled-for-5/
Waterfall
Analysis Design Coding Testing
Analysis
Design Coding
Testing
Agile
Analysis
Design Coding
Testing
Analysis
Design Coding
Testing
1-4 week sprints
Source: http://www.upassoc.org/upa_publications/jus/2007may/agile-ucd.pdf
Tip: Choose a feature to develop first that has a lengthy dev sprint.
• Conduct your own recruiting
• Don’t offer large honorariums
• Don’t use a lab
• Don’t video record
• Test less people
• Use surrogate users or just conduct cognitive walkthroughs
• Conduct unmoderated studies
• Don’t write formal reports
Learn more...
• Book: Interviewing Users by Steve Portigal
• Book: Moderating Usability Tests by Joe Dumas & Beth Loring
• Book: Paper Prototyping (chapter on task design)by Caroline Snyder
• Book: Rocket Surgery Made Easy by Steve Krug
• Book: The Moderator’s Survival Guide by Donna Tedesco & Fiona Tranquada
• Article: Seven Common Usability Testing Mistakesby Jared Spoolhttp://www.uie.com/articles/usability_testing_mistakes/
• Article: Interview-Based Tasks: Learning from Leonardo DiCaprioby Jared Spoolhttp://www.uie.com/articles/interview_based_tasks/
• Article: Mobile Testing Toolbox: Part 1&2by Amber Derosahttp://uxmag.com/articles/mobile-testing-toolbox-part-1
• Article: How To Conduct A Usability Test On A Mobile Device by Jeff Saurohttp://www.measuringusability.com/blog/mobile-usability-test.php
• Article: Why And How To Segment Your Customers by Jeff Saurohttp://www.measuringu.com/blog/segment-customers.php