AlanAlan Woolrych Woolrychalan.woolrych@sunderland.ac.ukalan.woolrych@sunderland.ac.uk

My BackgroundMy BackgroundCurrently – Research & Liaison Officer (DMN)Currently – Research & Liaison Officer (DMN)

From 1From 1stst January 2003 Usability Researcher with January 2003 Usability Researcher with

NITRO - NITRO - NNorth East orth East ITIT RReach each OOut Projectut Project

MPhil on Assessment of Usability Inspection MPhil on Assessment of Usability Inspection Methods (Currently part – time PhD)Methods (Currently part – time PhD)

co-author, International Handbook on HCI, UIM Chapterco-author, International Handbook on HCI, UIM Chapter Usability Consultancy and collaboration with a variety of Usability Consultancy and collaboration with a variety of

local and international companieslocal and international companiesLeighton Internet, domainnames, IBMLeighton Internet, domainnames, IBM

publications in international conferences and journalspublications in international conferences and journals

HEFCE capital project managementHEFCE capital project management usability lab (x2)usability lab (x2) Multimedia Lab (x2)Multimedia Lab (x2)

This WeekThis Week

Evaluation Methods (…or Inspection Evaluation Methods (…or Inspection Methods)Methods)

My ResearchMy Research

Evaluation ExerciseEvaluation Exercise

Poor UsabilityPoor Usability

What is ‘bad usability?’What is ‘bad usability?’ Something ‘happens’ that you don’t understandSomething ‘happens’ that you don’t understand Something happens outside of your controlSomething happens outside of your control

Why does it happen?Why does it happen? Lots of reasons!Lots of reasons!

Misunderstanding the user, out of context…Misunderstanding the user, out of context…

Leads to:Leads to: Frustration, anger, confusion etc.Frustration, anger, confusion etc.

When it happens what do you do?When it happens what do you do?

Strange But True…Strange But True…

Cannot delete tmp150_3.tmp: There is not Cannot delete tmp150_3.tmp: There is not enough free disk space. Delete one or more enough free disk space. Delete one or more files to free disk space, and then try again. files to free disk space, and then try again.

Error: Keyboard not found. Press F1 to Error: Keyboard not found. Press F1 to continue.continue.

Error 0000: No errors found, restarting Error 0000: No errors found, restarting computer.computer.

Windows has found an unknown device and Windows has found an unknown device and is installing a driver for it.is installing a driver for it.

Approaches to EvaluationApproaches to Evaluation

AnalyticalAnalytical deduction, inference, constructing arguments deduction, inference, constructing arguments

based on inspection of web-sitesbased on inspection of web-sites

EmpiricalEmpirical factual, evidence gathered from real usage by factual, evidence gathered from real usage by

real peoplereal people

Inspection MethodsInspection Methods

Heuristic EvaluationHeuristic Evaluation

Heuristic WalkthroughHeuristic Walkthrough

Expert InspectionExpert Inspection

Cognitive WalkthroughCognitive Walkthrough Novice users, learning the site for the first Novice users, learning the site for the first

timetime 4 questions4 questions

And many more…And many more…

Cognitive WalkthroughCognitive Walkthrough

Novice users, learning the site for the first timeNovice users, learning the site for the first time 4 questions4 questions

1.1. Will the user be trying to achieve the right Will the user be trying to achieve the right effect?effect?

2.2. Will the user notice that the correct action Will the user notice that the correct action is available?is available?

3.3. Will the user associate the correct action Will the user associate the correct action with the desired effect?with the desired effect?

4.4. If the correct action is performed, will the If the correct action is performed, will the user see that progress is being made?user see that progress is being made?

Heuristic Evaluation 1Heuristic Evaluation 1

Visibility of system status (Nielsen) The system should always keep users

informed about what is going on, through appropriate feedback within reasonable time.

Heuristic Evaluation 2Heuristic Evaluation 2

Match between system and the real world The system should speak the users'

language, with words, phrases and concepts familiar to the user, rather than system-oriented terms. Follow real-world conventions, making information appear in a natural and logical order.

Heuristic Evaluation 3Heuristic Evaluation 3

User control and freedom Users often choose system

functions by mistake and need a clearly marked "emergency exit" to leave unwanted states without having to go through an extended dialogue. Support undo and redo.

Heuristic Evaluation 4Heuristic Evaluation 4

Consistency and standards Users must not wonder whether

different words, situations, or actions mean the same thing. Follow platform conventions.

Heuristic Evaluation 5Heuristic Evaluation 5

Error prevention Even better than good error

messages is a careful design which prevents a problem from occurring in the first place.

Heuristic Evaluation 6Heuristic Evaluation 6

Recognition rather than recall Make objects, actions, and options

visible. The user should not have to remember information from one part of the dialogue to another. Instructions for use of the system should be visible or easily retrievable whenever appropriate.

Heuristic Evaluation 7Heuristic Evaluation 7

Flexibility and efficiency of use Accelerators - unseen by novices - may

speed up interaction for experts so that systems can cater to both inexperienced and experienced users. Let users tailor frequent actions.

Heuristic Evaluation 8Heuristic Evaluation 8

Aesthetic and minimalist design Dialogues should not contain

information that is irrelevant or rarely needed. Every extra unit of information in a dialogue competes with relevant units of information and diminishes relative visibility.

Heuristic Evaluation 9Heuristic Evaluation 9

Help users recognise, diagnose, and recover from errors Express error messages in plain

language (no codes), precisely indicate the problem, and constructively suggest a solution.

Heuristic Evaluation 10Heuristic Evaluation 10

Help and documentation Even though systems are best used

without documentation, it may be necessary to provide help. This should not be too large, be easy to search, focused on user tasks, listing concrete steps to be carried out.

Empirical TestingEmpirical Testing

Paper prototype testingPaper prototype testing

Lab TestingLab Testing

Field TestingField Testing

Remote observation and Remote observation and instrumentationinstrumentation

Site FeedbackSite Feedback

My Research…and yours!My Research…and yours!

To Accurately Assess UIMs in particular: Heuristic EvaluationWhy? Inspection methods are unreliable! (but would be

great if they weren't)Previous research quantative approach

HE can find 75% of usability problems in an interface - Nielsen and others…)

Fundamentally flawed!How can we know what is 100%?What if the most serious usability problems are in the undiscovered 25%

My ResearchMy Research

Heuristic Evaluations

Predicted Problems Task Sets

Assess Quality

Actual Problems

User Testing

Compare

My ResearchMy Research

Your WorkYour Work

To be usability experts

Inspect a web site

Produce a usability problem Report In groups or individual

Evaluation ExerciseEvaluation Exercise

You will be supplied with:You will be supplied with: Lecture…Lecture… HE training Manual (available on HE training Manual (available on

module website)module website) Problem report template (available on Problem report template (available on

module website)module website) Problem report template guide Problem report template guide

(available on module website)(available on module website)

RecommendationsRecommendations

Read the training manual thoroughlyRead the training manual thoroughly You are welcome to consult other You are welcome to consult other

material if you wishmaterial if you wish

Read and understand the Read and understand the requirements of the problem report requirements of the problem report formatformat

Perform heuristic evaluationPerform heuristic evaluation

Problem Reporting 1Problem Reporting 1

Section 1 - Problem DescriptionSection 1 - Problem Description Brief DescriptionBrief Description Specific Likely/Actual DifficultiesSpecific Likely/Actual Difficulties Specific ContextSpecific Context Assumed CausesAssumed Causes

Problem Reporting 2Problem Reporting 2

Section 2 - Discovery MethodSection 2 - Discovery Method Individual/Group TestingIndividual/Group Testing Adopted MethodAdopted Method

ScanningScanning

SearchingSearching

Goal PlayingGoal Playing

Method FollowingMethod Following

ExplainExplain

Problem Reporting 3Problem Reporting 3

Section 3 - Heuristic Section 3 - Heuristic ApplicationApplication Heuristic BreachedHeuristic Breached Evidence of non-conformanceEvidence of non-conformance

Important!Important!

Confirmation rationaleConfirmation rationale

Problem Reporting 4Problem Reporting 4

Section 4 - Exclusion RationaleSection 4 - Exclusion Rationale Elimination DiscussionElimination Discussion

URL For ExerciseURL For Exercise

http://www.tyneandwearmetro.co.uk/