Post on 29-Dec-2015
transcript
Usability evaluation
Big picture as lead into Heuristic Evaluation
and situating think-aloud evaluation
Overview
• Core• Heuristic evaluation
– What is it– How to do it
• Practice in preparation for homework and Assignment
• Beyond the basics– Broader picture of HE
• Homework
Core methods
What do the next graphs tell you?
Technology Transfer of Heuristic Evaluation and Usability Inspectionby Jakob Nielsen on June 27, 1995
http://www.nngroup.com/articles/technology-transfer-of-heuristic-evaluation/
Source as above
What is Heuristic Evaluation
• Heuristic Evaluation• Select set of design heuristics• Recruit experts in HE• For each expert:
– Expert taken through tasks– Meanwhile team member records violations of the heuristics– Expert reviews all the problems found and rate them
• Combine all expert reports• Compare this with think-aloud:
– How is this different?– What is similar?
Guidelines (heuristics) for design and evaluation
Heuristic evaluation aka Usability Inspection
and guidelines
About heuristics
Rules of thumbGuidelines
Example
Copyright MKP. All rights reserved. 10
Aside
• These heuristics/guidelines– Are helpful for design– And for evaluation
Why is this so?
Core: Use Neilsen Heuristics to do HE
Hard for novices to use effectively
Cockton, Gilbert (2014): Usability Evaluation. In: Soegaard, Mads and Dam, Rikke Friis (eds.). "The Encyclopedia of Human-Computer Interaction, 2nd Ed.". Aarhus, Denmark: The Interaction Design Foundation. Available online at https://www.interaction-design.org/encyclopedia/usability_evaluation.html
14
Nielsen’s original heuristics
• Visibility of system status.• Match between system and real world.• User control and freedom.• Consistency and standards.• Error prevention. • Recognition rather than recall.• Flexibility and efficiency of use.• Aesthetic and minimalist design.• Help users recognize, diagnose, recover from errors.• Help and documentation.http://www.nngroup.com/articles/ten-usability-heuristics/
• Visibility of system status The system should always keep users informed about what is going on, through appropriate feedback within reasonable time.
• Match between system and the real world The system should speak the users' language, with words, phrases and concepts familiar to the user, rather than system-oriented terms. Follow real-world conventions, making information appear in a natural and logical order.
• User control and freedom Users often choose system functions by mistake and will need a clearly marked "emergency exit" to leave the unwanted state without having to go through an extended dialogue. Support undo and redo.
• Consistency and standards Users should not have to wonder whether different words, situations, or actions mean the same thing. Follow platform conventions.
• Error prevention Even better than good error messages is a careful design which prevents a problem from occurring in the first place. Either eliminate error-prone conditions or check for them and present users with a confirmation option before they commit to the action.
• Recognition rather than recall Minimize the user's memory load by making objects, actions, and options visible. The user should not have to remember information from one part of the dialogue to another. Instructions for use of the system should be visible or easily retrievable whenever appropriate.
• Flexibility and efficiency of use Accelerators -- unseen by the novice user -- may often speed up the interaction for the expert user such that the system can cater to both inexperienced and experienced users. Allow users to tailor frequent actions.
• Aesthetic and minimalist design Dialogues should not contain information which is irrelevant or rarely needed. Every extra unit of information in a dialogue competes with the relevant units of information and diminishes their relative visibility.
• Help users recognize, diagnose, and recover from errors Error messages should be expressed in plain language (no codes), precisely indicate the problem, and constructively suggest a solution.
• Help and documentation Even though it is better if the system can be used without documentation, it may be necessary to provide help and documentation. Any such information should be easy to search, focused on the user's task, list concrete steps to be carried out, and not be too large.
Class activity
18
Activity: Identify potential trade-offs
• Visibility of system status.• Match between system and real world.• User control and freedom.• Consistency and standards.• Error prevention. • Recognition rather than recall.• Flexibility and efficiency of use.• Aesthetic and minimalist design.• Help users recognize, diagnose, recover from errors.• Help and documentation.
19
Main stages of heuristic evaluation
• 1. Preliminaries– Agreed set of heuristics to use;– Programming team member overviews system;– And team member is available throughout;– Set of tasks;
20
Main stages of heuristic evaluation
• 2 Evaluation:– Each expert works independently through the UI;– Team member records problems, so expert can simply
state them (single person works well, so they bring all details together);
– Also answers any questions (eg expert gets stuck, cannot find how to do a task, may need help with domain expertise aspects)
– Multiple passes (overview, then detailed)– Results in a set of identified failures to match the
heuristics (part of interface, violated heuristic)
21
Main stages of heuristic evaluation
• 3. Concluding summary– Summarise all the flaws– Rate these in terms of severity
Severity ratingshttp://www.nngroup.com/articles/how-to-rate-the-severity-of-usability-problems/
• “The severity of a usability problem is a combination of three factors:
• The frequency with which the problem occurs: Is it common or rare?
• The impact of the problem if it occurs: Will it be easy or difficult for the users to overcome?
• The persistence of the problem: Is it a one-time problem that users can overcome once they know about it or will users repeatedly be bothered by the problem?”
• 0 = I don't agree that this is a usability problem at all 1 = Cosmetic problem only: need not be fixed unless extra time is available on project 2 = Minor usability problem: fixing this should be given low priority 3 = Major usability problem: important to fix, so should be given high priority 4 = Usability catastrophe: imperative to fix this before product can be released
Severity ratingshttp://www.nngroup.com/articles/how-to-rate-the-severity-of-usability-problems/
Summarising the resultsDescription of problem Heuristics violated S1 S2 S3
Hard to find the Archive button Consistency … 0 1 2
Scroll bar not visible Visibility…, Match system and real world
3 2 4
User has to remember ID from last screen
Recognition rather than recall 4 4 4
Interface uses language that is not meaningful to user – select bundle for student data
Match system and real world 2 2 3
Instructions say “Circle all selections” but there are check boxes
Match system and real world
Shows which evaluators found which usability problems in HE of a banking system. Each row is an evaluator (n=19) and each column is a flaw (n=16)Black squares show where evaluator found the problem.The rows are sorted with most successful evaluators at the bottom.The columns are sorted with easiest to find problems at right.
http://www.nngroup.com/articles/how-to-conduct-a-heuristic-evaluation/
What does this graph tell you?
Give a definition of an “Easy” and “Hard”.State some reasons a problem may be easy … hardIs there any problem that every evaluator found?What are the main lessons?
http://www.nngroup.com/articles/how-to-conduct-a-heuristic-evaluation/
27www.id-book.com
Discount evaluation
• Heuristic evaluation is referred to as discount evaluation when 5 evaluators are used.
• Empirical evidence suggests that on average 5 evaluators identify 75-80% of usability problems.
28www.id-book.com
No. of evaluators & problems
http://www.nngroup.com/articles/how-to-conduct-a-heuristic-evaluation/
Critical to success
• Set of heurstics• Set of tasks• Prototype interface• Experts!!!
31www.id-book.com
Advantages and problems
• Few ethical & practical issues to consider because users not involved.
• Can be difficult & expensive to find experts.• Best experts have knowledge of application domain &
users.• Can find both important and minor problems• Best if experts are not the designers.• Biggest problems:
– Important problems may get missed;– Many trivial problems are often identified;– Experts have biases;– May encourage tinkering with the interface detailed in the
identified problems.
How would one validate a set of heuristics?
And how would you know about this?
How to select appropriate heuristics?
ContextValidity
Usability (broad versus detailed)
Summary of HE
• No users needed• Heuristics needed• Experts needed (NOT the designers…)• Relatively cheap
Class activity in your groups
• More practice in creating tasks– For CUSP, define 2 important abstract tasks– For each of these, define 2 concrete tasks
• Conduct HE on CUSP using pods– Using Nielsen’s Heuristics – Using the above 4 tasks– As you are getting started allocate different heuristics to team
members to focus on– Write down problems as you see them (Sketch the CUSP
screen and annotate it)– Make a table of the problems and your assessment of their
severity
About guidelines
H&P Conclusions about guidelines
• Be cautious using guidelines• Use careful thought and interpretation • In application, guidelines can conflict and
overlap• Guidelines do not guarantee a good user
experience • Using guidelines does NOT eliminate need
for fuller UX evaluation
Copyright MKP. All rights reserved. 37
How to Write User-Friendly Contenthttp://www.usability.gov/how-to-and-tools/methods/writing-for-the-web.html
• Use the words your users use. • Chunk your content. • Front-load the important information. • Use pronouns. • Use active voice. • Use short sentences and paragraphs. • Use bullets and numbered lists. • Use clear headlines and subheads. • Use images, diagrams, or multimedia• Use white space.
• Use the words your users use. By using keywords that your users use, you will help them understand the copy and will help optimize it for search engines.
• Chunk your content. Chunking makes your content more scannable by breaking it into manageable sections.
• Front-load the important information. Use the journalism model of the “inverted pyramid.” Start with the content that is most important to your audience, and then provide additional details.
• Use pronouns. The user is “you.” The organization or government agency is “we.” This creates cleaner sentence structure and more approachable content.
• Use active voice. “The board proposed the legislation” not “The regulation was proposed by the board.”• Use short sentences and paragraphs. The ideal standard is no more than 20 words per sentence, five
sentences per paragraph. Use dashes instead of semi-colons or, better yet, break the sentence into two. It is ok to start a sentence with “and,” “but,” or “or” if it makes things clear and brief.
• Use bullets and numbered lists. Don’t limit yourself to using this for long lists—one sentence and two bullets is easier to read than three sentences.
• Use clear headlines and subheads. Questions, especially those with pronouns, are particularly effective.• Use images, diagrams, or multimedia to visually represent ideas in the content. Videos and images
should reinforce the text on your page.• Use white space. Using white space allows you to reduce noise by visually separate information.
http://guidelines.usability.gov/
• Section 1, Make Action Sequences Clear• Section 2, Avoid Jargon• Section 3, Use Familiar Words• Section 4, Define Acronyms and Abbreviations• Section 5, Use Abbreviations Sparingly• Section 6, Use Mixed Case with Prose• Section 7, Limit the Number of Words and Sentences• Section 8, Limit Prose Text on Navigation Pages• Section 9, Use Active Voice• Section 10, Write Instructions in the Affirmative• Section 11, Make First Sentences Descriptive
Ben Shneiderman's golden rules for dialogue
• Consistency– eg location of “quit”
• short cuts for frequent users• informative feedback
– HTTP Error 404 !!!• closure in dialogues
– (ie clear when action is complete)• simple error handling• easy reversal of actions
– undo• support internal locus of control
– Users should feel in control• reduce short term memory load
Bruce Togazzini• http://www.asktog.com/basics/firstPrinciples.html• Anticipation• Autonomy• Color Blindness• Consistency• Defaults• Efficiency of the User• Explorable Interfaces• Fitts' Law• Human Interface Objects • Latency Reduction• Learnability• Metaphors, Use of• Protect Users' Work• Readability• Track State• Visible Navigation
Many other guidelines
• Company specific• Device/OS specific
– eg https://developer.apple.com/library/mac/documentation/userexperience/conceptual/applehiguidelines/Windows/Windows.html
• ISO, ANSI Standards• National Standards• Military Standards• Accessibility Standards
OS X Example
• “mental model your users have should infuse the design … support the user’s mental model by striving to incorporate the following characteristics
• Familiarity• Simplicity• Availability (functionality available)• Discoverability• https://developer.apple.com/library/mac/documentation/userexperience/conceptual/applehiguidelines/Intr
o/Intro.html#//apple_ref/doc/uid/TP30000894-TP6 (visited 2013)
Class activity
• Take the OS X set and map them to Neilsen– Where and how do they match– And not
• Visibility of system status.• Match between system and real
world.• User control and freedom.• Consistency and standards.• Error prevention. • Recognition rather than recall.• Flexibility and efficiency of use.• Aesthetic and minimalist design.• Help users recognize, diagnose,
recover from errors.• Help and documentation
• Familiarity• Simplicity• Availability
(functionality available)
• Discoverability
Homework for next week• Reading
– Reinecke, K., & Gajos, K. Z. (2014, April). Quantifying visual preferences around the world. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (pp. 11-20). ACM.
– Create concept map, including the concepts: methods, key results, links to HE
• Work on Assignment 1 Heuristic evaluation on the e-book– Same tasks as for TA– Each team member is an expert– Using Nielsen’s Heuristics – Bring a printout of screenshots, annotated to show problems you have found– Make a table of the problems and your assessment of their severity