34
1 Tips and Tricks for Remote Unmoderated Usability Testing Alfonso de la Nuez Founder at UserZoom May 2009
| Date post: | 27-Jan-2015 |
| Category: |
Technology |
| Upload: | alfonso-de-la-nuez |
| View: | 110 times |
| Download: | 1 times |
www.userzoom.com
1
Tips and Tricks for
Remote Unmoderated Usability Testing
Alfonso de la NuezFounder at UserZoom
May 2009
www.userzoom.com
Overview
• Introduction to Remote Unmoderating User Testing (RUUT)
• What is the value and how does it work?
• When and how to use it as a research method
• Best practices and examples
www.userzoom.com
What does the market say about web research & analytics?
Usability testing in the lab is great, but it has its limitations
Web analytics tells you a lot about the WHAT, not the WHY
Conducting international testing is expensive
Benchmarking and measuring UX is more important than ever
www.userzoom.com
Overall conclusion:
Today’s Web and E-business Managers seek more sophisticated and innovative research and analytics products
to help them compete
www.userzoom.com
Test REAL tasks
LargeSamples Reasons WHY
BehaviouralData
Natural Setting
Focus Groups
Usability Labs
Online surveys
Web analytics
RUUT
Comparing various research methods for usability research
www.userzoom.com
RUUT: What is the value and how does it work?
1. No face-to-face, human moderation (moderation through software)
2. Hundreds of users participate simultaneously…
3. …from geographically spread locations (local & int’l) and…
4. …in their natural context (using their own PCs)
5. Software captures both usability data and users’ behavior
Very cost-effective and faster
www.userzoom.com
Define your study
Choose the type of studyyou want to conductWrite your study scriptChoose the recruitmentmethod & audiencePre-test before launch
Invite users to participate(or ask your visitors to do it)Collect data automaticallyMonitor the progress real-time
Launch study, collect data & monitor progress
Perform advanced data analysisUse filtering capabilitiesCopy & paste resultsExport automatically intoWord or Excel
Analyze results & generate a report
www.userzoom.com
Option 1: Panel companies*
• Million users, spread across the world
• Wide variety of profiles, great segmentation capabilities
• Strict, professional control and panel mgmt.
Option 2: Intercept method
• Ask users who visit the website to participate in a study.
• Insert a javascript in the page
Option 3: Private panels, mailing lists
• We can also work with private databases
What about user recruitment?
* Use a panel company
www.userzoom.com
When to use this research method
Whenever you need to quantify usability problems, not just identify them
If you need to test users in various locations (local or international)
For benchmarking and UX measurement projects
To obtain user behavior data (search strategy, click-streams and click-maps)
When you want to combine usability and market research
For quick pre-testing during the development cycle (prototype testing)
www.userzoom.com
Task based studies: Real users attempting real tasks
Websites
www.userzoom.com
Prototypes
Task based studies: Test any web-based interface
www.userzoom.com
Task based studies: Test any web-based interface
Mobile interfaces
www.userzoom.com
Also, you can conduct…
• Online Card sorting exercises
• True Intent studies
• Free search studies
Let’s take a look…(all videos are taken from pilot studies)
www.userzoom.com
Card sorting: Open and closed categories
www.userzoom.com
True intent studies (through intercept method)
www.userzoom.com
Online user behavior research (through free search studies)
www.userzoom.com
Important! Remote does not mean you’re not in control
About scripting capabilities:
• High-level of flexibility
• Advanced task validation criteria (by question, by page interception, by time...)
• Advanced questionnaires tool (all type of questions: Choice, scales, open...)
• Insert multimedia (images, videos, flash, applets, sounds...)
• Manage advanced logic/branching & conditions
• Behavioral questionnaires
• Task randomization
• Time-out options
• Sample quality control
www.userzoom.com
Ok… What about the results you get?
www.userzoom.com
Effectiveness & efficiency ratios, plus answers to the questionnaire
www.userzoom.com
You have filtering capabilities
www.userzoom.com
www.userzoom.com
www.userzoom.com
Text clouds to analyze open-ended questions
www.userzoom.com
Clickstreams per task
www.userzoom.com
Clickmaps/heatmaps per page
www.userzoom.com
Finally, about reporting & exporting capabilities
Generate automatic reports (.doc, .excel, .ppt)
Copy & paste capabilities
Export raw-data for further advanced analysis
Share results online
www.userzoom.com
Any examples?
www.userzoom.com
Testing users in various locations (local or international)
Monster’s UX Director:
“Need to test various local sites, in 5 countries, but
I have a limited budget and time plan”
They tested 200 users online, in 5 markets
Data was collected in approximately 1 month
A year later the study was repeated
Visitbritain.co.uk
www.userzoom.com
Need to quantify usability issues, not just identify them
Spanair’s Marketing Director:
“Loved the results from the lab, but I simply can’t
make a decision based on the feedback from 10 users”
The airline ended up testing 15 users in the lab (included
eye-tracking), then added 229 users online
Tasks were identical, results were not…
Spanair is Spain’s 2nd largest airline
www.userzoom.com
Task
Nºof users
(Lab)
Nºof users(remote)
Successrate
(Labs)
Successrate(UZ)
Find a flight of a special class 3/15 185/229 20 % 71 %
Register on the web 4/5 5/12 70% 42 %
Check your current balance (loyaltyprogram) 8/15 65/73 53.3% 88 %
Spanair is Spain’s 2nd largest airline
www.userzoom.com
For UX benchmarking & brand proposition research
Arsys’s Marketing Director:
“We’d like to measure both user experience and
branding metrics over a period of 3 years”
They test 200 users each semester
Upload results in a UX Dashboard
Arsys is Spain’s leading ISP
UK’s Training & Development Agency
www.userzoom.com
Testing users in their natural context of use
Badoo’s Management had 3 goals and issues in mind:
1. Were concerned about testing in the lab
2. Wanted to check differences between cultures
3. Wanted to combine usability testing with brand-value
proposition testing
RUUT was the perfect solution for them:
Were able to ask private matters openly
Were able to test in 5 markets at once and
compare results
Asked users about their brand-value perceptions
When to use unmoderated RUT Examples
Badoo is a UK-based, fast-growing
online dating and social network site
www.userzoom.com
Would you like a demo?Contact us and we’ll offer you a chance to see and try out our technology
Do you have any questions?We’re sure you have many questions. Don’t hesitate to contact us and we’ll be glad to answerthem.
Alfonso de la Nuez
Founder & Int’l Business Dev
Virginie Glaenzer
Sales Manager, USA
Xavier Mestres
Founder & CTO / R&D
USA Office440 N. Wolfe Rd. - Sunnyvale, CA 94085 (USA) · [t] +1 (408) 524 7445
Spain officesAv. Diagonal 419, 3º 2ª - 08008 Barcelona - T. +34 93 414 75 54 - F. +34 93 368 42 95
Gran Vía 62, 7º Izda. - 28013 Madrid - T. +34 91 542 62 52 - F. +34 91 298 08 92
Contact information
Members of
Thanks and have a great day!
www.userzoom.com
Introduction: Our ‘elevator pitch’
1. What is UserZoom and what do we do?• International market research firm, specializing in online
user experience and usability research. • We help brands boost website performance and improve
online business revenues.
2. How do we do it?• We create a unique web-based software application
that empowers researchers to conduct highly sophisticated online research.
• We offer both a self-service solution and professional services.
3. Bottom line… Why should you work with us?• You’ll be able to capture & measure the critical and
actionable data about your users’ online experience that you need to manage your business, and no other tool can provide!
