Web Analytics and Usability Testing:

Triangulate your way to better design

recommendations by Eva Kaniasty

What’s your favorite method?

Web Analytics Explosion

Who wins the battle?

Usability Testing Sucks

Usability Testing

Small SampleResults not Generalizable

Artificial SituationTask Design Bias

Recruiting BiasUnrepresentative Participants

Moderator Required

Moderator Bias

Usability Testing

3 users experienced X…Consider changing X to Y.

Usability Testing is Disconnected

Web Analytics are Awesome!

Web Analytics

Representative/ Large Sample Size

Real GoalsReal Tasks

Real Users“In the Wild”

No ModeratorAutomatic Data Collection/Analysis

Numbers and graphs: universally beloved

Analytics rundown

Visitor/Session Tracking

Visitor/Session Tracking

Data Gathered Uses•Some visitor characteristics: locations, browsers, OS•Visit & page view trends•Engagement measures: time on page•Search data: keyword referrals•Custom tracking: detailed information on paths/goal conversion

•Confirm Usability Test findings•Inform usability testing tasks

Clickmaps/Heatmaps

Clickmaps/Heatmaps

Data Gathered Uses

•Info not on just which pages are being accessed, but from where•Full picture of users’ interactions with the page•Measure of user interest/engagement •Some segmentation by browser/screen size/time to click

•IA•Visual hierarchy•Interface refinement

Email Data Mining

Email Data Mining

Data Gathered Uses

•Demographics•Geography•User Profiles•Site usage data: accounts on amazon, facebook, linkedin, etc.

•Utest Recruiting Profiles•Personas

Session Recording

Session Recording

Data Gathered Uses•View all interactions with page, including mouse movements •Scroll maps•Form analytics

•Identify potential usability problems•QA/Browser Testing

A/B or Multivariate Testing

A/B or Multivariate Testing

Data Gathered Uses

•Relative success of design alternatives: impact on conversion and other engagement measures

•IA•Visual hierarchy•Interface refinement

Unmoderated Usability Testing

Unmoderated Usability Testing

Data Gathered Uses

•Immediate feedback from real users as they engage with the website

•Alternative to usability testing, when large samples are required•Compensate on weaknesses of survey methods

Custom Metrics

Custom Metrics

Data Gathered Uses

•Functionality usage•Form errors•Patterns of use•Conversion measurement

•Sky is the limit

The what.

But why?

Analytics Suck

Web Analytics

Limited information about context No info about user intent

One-dimensional view (i.e. Page-level data only from Clickmaps)

Overwhelming amounts of data to analyze

Data on current state only

Usability Testing is Awesome

Usability Testing

Rich data/ holistic view

Info about goals

Moderator can probe to clarify intent

Ability to test new interfaces/ prototypes

Triangulate!

Triangulate

• Focus• Deep Data• Goals and Intent

• Confirm Problems• Extent• Severity

Usability Testing

Analytics

Case Study

• Form errors• No abandonment• Confusion about

messaging

• Detail on types/extent of errors

• True abandonment rates

• Visual hierarchy problems

Usability Testing

Analytics

Triangulate

Surveys

AnalyticsUsability Testing

Questions first

CSI or Law & Order?

Questions?

Twitter: @kaniastykaniasty@gmail.comkaniasty.com

Resources

• Google Analytics• Crazyegg• Clicktales • Keynote Systems