Lecture #1INTRODUCTION*39TUR Winter 2013/2014

Testování uživatelských rozhraní

Adam J. Sporka

TUR 2013 – INTRODUCTION – [2]

INTRODUCTION

TUR 2013 – INTRODUCTION – [3]

Motivation

User Interface– The part of a system (software, machine, device, …)

that enables the user to control this system– User carries a task through this interface

A user interface must not:– disable or obstruct the purpose of the system– make users feel frustrated or stupid

Why all this?– Are not software engineers and developers skilled enough to make

software that works AND is easy to use?

http://xkcd.com/773/

TUR 2013 – INTRODUCTION – [5]

Motivation

User Interface Testing– Verifying that real users can use the system

• Identification of problems that prevent real users from using the system

“What good is a phone call if you’re unable to speak.”

– Verifying that the system reflects the user’s needs• Understanding how the real users (will) interact with the

system

• … and whether they will

TUR 2013 – INTRODUCTION – [6]

Humanists and Mechanists

C. P. Snow (1959) The Two Cultures Two “camps” in the Western Civilization:

– Humanists• Artists• Social scientists• Study the humans, praising humans• Having unrealistic trust in humans’ abilities• No interest in technology

– Mechanists• Physicists, computer scientists, engineers, …• Study the technology, praising technology• Having unrealistic trust in performance of the technology• No interest in human, in her natural conditions• “Human is also some kind of a machine”

TUR 2013 – INTRODUCTION – [7]

Humanists and Mechanists

Split apart: “They can’t talk to each other.” Situation:

– They do not perceive the world as a combination of people AND technology

– No perception of the interaction between the people and the technology

Clash:– Humanists have no clue how technology works– Mechanists have no clue how the human works

“Somebody Else’s Problem”– Douglas Addams

Little understanding of what happens in between

TUR 2013 – INTRODUCTION – [8]

“In Technology We Trust”

London, 26 October 1992 Situation

– Automated emergency response telephone service– No assumption of duplicate messages– System overloaded, units dispatched late– 30~45 people died

Cause– Trust in technology with lives of people

• No load testing, 81 known software issues, no contingency plan

– Limited knowledge of how people call for help– Limited knowledge of how other people respond to these calls

Kim Vincente (2004) The Human Factor http://www.wired.com/thisdayintech/2009/10/1026london-ambulance-computer-meltdown/

TUR 2013 – INTRODUCTION – [9]

“In Human We Trust”

80-hr working week for the US medical doctors Trust in super-human abilities

TUR 2013 – INTRODUCTION – [10]

Paradigm Shift – 1980s, 1990s

“It’s always users’ fault.”– Machines considered more expensive than people.– “The problem is between the chair and the screen.”– “Read The F***ing Manual.”

"Always be wary of any helpful item that weighsless than its operating manual."

– Terry Pratchett (Jingo)

TUR 2013 – INTRODUCTION – [11]

Paradigm Shift – 2000s, 2010s

“Nobody reads manuals!”– People are finally valued more than machines.

• Time people spend with devices

• People’s emotional state

– “Systems should be self-explaining.”– “Don’t make me think.”

– User-centered Design (UCD)

TUR 2013 – INTRODUCTION – [12]

Motivation

Courses in informatics so far:– How to create applications that perform function– Given inputs, produce outputs– Little care about the users– Technology viewpoint

The TUR course:– The applications are used by real users– How to detect that users will not be able to use the application by

the authors’ expectations?– How to detect problems and how to provide a feedback to the

authors? User Testing• How to report problems in a trustworthy way?

TUR 2013 – INTRODUCTION – [13]

Motivation

It’s not code testing– Unit tests– Quality assurance, etc.

It’s not code debugging– We identify the problems of usability– We provide suggestions at the level of design, not implementation

It’s not performance tests– (at least most of the time)

TUR 2013 – INTRODUCTION – [14]

Motivation

When to test?– Don’t test the obvious– Well-placed test may show counterintuitive findings

TUR 2013 – INTRODUCTION – [15]

Examples of Obviously Bad Design

TUR 2013 – INTRODUCTION – [16]

Examples of Obviously Bad Design

TUR 2013 – INTRODUCTION – [17]

Examples of Obviously Bad Design

TUR 2013 – INTRODUCTION – [19]

Why test?

These were things that were obviously wrong.– The problems could be prevented at the design stage

provided that good practices.

But are problems always this obvious?– Sometimes even apparently good designs fail

TUR 2013 – INTRODUCTION – [20]

USA Presidential Elections 2000

Palm Beach County's butterfly ballots

TUR 2013 – INTRODUCTION – [21]

USA Presidential Elections 2000

The arrows clearly indicate the candidates What can go wrong then?

TUR 2010

TUR 2013 – INTRODUCTION – [23]

USA Presidential Elections 2000

TUR 2013 – INTRODUCTION – [24]

USA Presidential Elections 2000

Pat Buchanan received more votes– 0.29 % … state-wide average– 0.8 % … Palm Beach County

Likely explanation:(when rejecting various conspiracy theories)

– User interface problem– People who intended to vote for Al Gore actually voted

for Pat Buchanan

TUR 2013 – INTRODUCTION – [25]

TUR: THE COURSE

TUR 2013 – INTRODUCTION – [26]

Lectures

Theoretical lectures– Introduction– Background

3 Guest lectures– Usability testing in the lab

TUR 2013 – INTRODUCTION – [27]

Seminars

Devoted almost entirely to the semester project– regular seminars– lab work– homework / consultations

Assignments– Task A: Cognitive Walkthrough

(1 person per assignment)– Task B: A simple quantitative study– Task C: Test in the usability lab (4 people per

assignment)

TUR 2013 – INTRODUCTION – [28]

Examination

A combination of theoretical questions and practical problems

Written preparation Oral exam

2 or 3 dates of exams

TUR 2013 – INTRODUCTION – [29]

Scoring

Semester projects: Up to 50 points Exam: Up to 50 points

– Some fundamental questions are “mandatory non-zero”

Some bonus points along the way– Activity in the class

A Truly awesome 90+

B Awesome 80+

C Sorta awesome 70+

D Could be more awesome 60+

E Not awesome 50+

F You did not make the cut <50

TUR 2013 – INTRODUCTION – [30]

Bachelor program:– PDA … mobile devices

Master program:– NUR … User Interface Design (Návrh uživatelského

rozhraní)– PSY … Psychology of User Interfaces

Follow-up Courses

TUR 2013 – INTRODUCTION – [31]

HUMAN—COMPUTER INTERACTION

TUR 2013 – INTRODUCTION – [32]

Human-Computer Interaction (HCI)

“A science and an art of how not make the user angry while using technology”

TUR 2013 – INTRODUCTION – [33]

Human-Computer Interaction (HCI)

Design, Implementation, and Evaluation of the interactive systems from the perspective of use by the human. Humans

Technology

Tasks

Design

A. Holzinger, TU Graz

TUR 2013 – INTRODUCTION – [35]

Why study the HCI?

You will create real systems– … used by real people

User interface takes majority of the source code– Over 50% (Some authors report as much as 80%)– More than 50% of the implementation efforts go to the UI

Risks of the bad UI– Financial (your product won’t sell)– Lives (air or factory disasters, …)

Successful UI requires– Good knowledge of the human’s abilities– Good knowledge of the principles of the UI design

TUR 2013 – INTRODUCTION – [36]

Usability (Nielsen, 1984)

Simplicity of learning to use the system– To learn to use a similar system faster

Recall– To remember the way from one situation to another

Efficiency– To carry out the task quickly and efficiently

Minimum amount of errors– If encountered, inform the users on the cause and an advice how to

proceed

Satisfaction of the user– The users is convinced that the task has been successfully

achieved

TUR 2013 – INTRODUCTION – [37]

UsabilityUsability

Socialacceptability

Practicalacceptability

Cost

Compatibility

Reliability

Etc.

Sys

tem

acc

epta

bilit

y

Usefulness

Utility

Usability Easy to learn

Efficient to use

Easy to remember

Few errors

Subjectivelypleasing

Source: Jacob Nielsen, Usability Engineering

TUR 2013 – INTRODUCTION – [38](38)

Usability

Simplicity of learning to use the system– System can be learned easily and fast

Memorability = Recall (Easy to remember)– Whether the user is able to use the system after a

longer time

Efficiency– Once the user learned how to use the system, the

system will be fast to use– To carry out the task quickly and efficiently

TUR 2013 – INTRODUCTION – [39](39)

Usability

Minimum amount of errors– Preventing users from making errors– If encountered, inform the users on the cause and give

an advice

Satisfaction of the user– Subjectively pleasing– The users is convinced that the task has been

successfully achieved

TUR 2013 – INTRODUCTION – [40]

UI Design Cycle

Design

Evaluation Implementation

TUR 2013 – INTRODUCTION – [41]

Phases of the UI Development

Design– “Every-day psychology”– User-centered design: Understanding users and their needs– Task-centered design: The execution of the task should not be split

in multiple UIs

Evaluation– Rapid prototyping– UI evaluation in cooperation with the users

• Qualitative• Quantitative

Implementation– Techniques of informatics

TUR 2013 – INTRODUCTION – [42]

Usability Testing

Verifying that a product is usable We need some structure / framework

– We need metrics– So that testers and designers of the UI can discuss the results– So that the results of the test can be trusted

Human behavior and actions are complex– Difficult to analyze– Individual differences

Need for objective measurements– Qualitative– Quantitative

TUR 2013 – INTRODUCTION – [43]

Usability Test is an Experiment

Defined hypotheses / research questions Defined conditions

Many kinds of usability tests– First impression tests– Migration of experienced users– …

Many possible objects of the usability tests

TUR 2013 – INTRODUCTION – [44]

Contexts of Evaluation

(1) Design– Testing in order to provide feedback to the designers

and developers• Purpose: Gain feedback for design

(2) Product assessment– Does a product fit the user’s needs?

• Purpose: “To buy or not to buy?”

– Product comparison• Purpose: “Which model is better for us?”

TUR 2013 – INTRODUCTION – [45]

Objects of usability testing

Software tools– “Will people accomplish the tasks for which the tools are

created?”

Web pages– “Will people find the information that needs to be

presented?”

Hardware devices– “Can people use the device in the physical conditions for

which it has been created?”

TUR 2013 – INTRODUCTION – [46]

Objects of usability testing

Non-IT objects– “Can people use their common sense to find out how to

open / close the door / bottle / …”

(Public) Services– “Is the process of obtaining OpenCard reasonable and

sensible?”

Buildings– “Can people navigate easily? Can people safely escape

the building during an emergency?”

TUR 2013 – INTRODUCTION – [47]

Software products

“Is the software easy for the user to install?”

“Is the software suitable for the task?”

“Is the software easy to use?”

“Does the software recover from errors?”

“Does the user understand the underlying processes?”

etc. http://mike-austin.com/blog/uploaded_images/badui2-747337.jpg

TUR 2013 – INTRODUCTION – [48]

Web pages

“Will visitors understand what the web page contains?”

“Are visitors able to navigate?” “Are visitors able to perform an on-line purchase?” etc.

TUR 2013 – INTRODUCTION – [49]

Hardware devices

“Does a need to conserve electric power affect the user experience?”

“Can the telephone be used by a snowboarder wearing gloves?”

“Can the GPS be operated safely while driving?”

“Do I get the changeback?”

TUR 2013 – INTRODUCTION – [50]

Non-IT Objects

E.g.: Italian IC/EC trains – the emergency brake handle is too close to the luggage holder

Similar examples athttp://www.baddesigns.com

TUR 2013 – INTRODUCTION – [51]

Services

A bureau is also a system– User = customer or client– Users come with clear goals

• Open a bank account, send a letter, buy groceries

“Do the clients understandthe process?”

“Are they being informedin a clear way?”

“Or are they being sentaround for no reason?”

Bruno Bozzetto

TUR 2013 – INTRODUCTION – [52]

Buildings

14 12 10 8 4 2 1 3

Platform 1Plat-form

2

Plat-form

3

Plat-form

4

Track #

“Where to wait for the train?”– “The train approaches Platform #4, Track #3.” – Train station Pardubice, Feburary 2007

TUR 2013 – INTRODUCTION – [53]

SOME MORE “SUBOPTIMAL”USER INTERFACES

TUR 2013 – INTRODUCTION – [54]

Eudora Pro 4.0 Special menu

What’s the difference between“FIND” and “SEARCH”?

TUR 2013 – INTRODUCTION – [55]

Oracle’s ORS CLEATS web-application

Error message is easy to miss.

TUR 2013 – INTRODUCTION – [56]

Two commands “Add…”

Do both do the same? Are they different?

TUR 2013 – INTRODUCTION – [57]

Opencard Kiosk

Instructions to the user:

1. Please insert yourcard

2. During the entireoperation, leave thecard in the reader

3. …

4. …

TUR 2013 – INTRODUCTION – [58]

Opencard Kiosk

“Your coupons”<dates><table>

“You may remove yourcard”

TUR 2013 – INTRODUCTION – [59]

Opencard Kiosk

“You card has been activatedfor DOS-DPP”A cryptic acronym.

“You card does not containany valid coupons”No instructions to remove thecard! User: “Do I have to waitor what?”

TUR 2013 – INTRODUCTION – [60]

DPP (2012)

Tramvaje, co mění čísla za jízdy

http://aktualne.centrum.cz/domaci/regiony/praha/clanek.phtml?id=755407

[…] Čili vám chybí informace o tom pokračování? Tam je to jako příjemný bonus, kdy je cestující příjemně překvapen tím, že tramvaj jede ještě dál.

A.cz: Já myslím, že cestující nechce být překvapen, ani příjemně. Že chce už na Vltavské vědět, do které zastávky dojede.

M.J.: On má garantovanou trasu té 24.

Tramvaje, co mění čísla za jízdy

A.cz: Proč tam nemůže být napsáno, kam tramvaj pokračuje po změně čísla?

M.J.: Já nerozumím, o co vám jde. Vždyť tam máte seznam zastávek čísla 24, která jede na Dělnickou.

A.cz: Ano, ale já přijdu na zastávku Vltavská a chci jet na zastávku U Průhonu. A já u té 24 nenajdu, že se změní na 14 a doveze mě tam. Tam prostě není zastávka U Průhonu uvedená.

M.J.: Není, protože to už je linka 14.

TUR 2013 – INTRODUCTION – [63](Oslo Subway)(Oslo Subway)

TUR 2013 – INTRODUCTION – [64]

TUR 2013: The Big Picture

TUR 2013 – INTRODUCTION – [65]

Určení cílové skupiny Určení uživatelských požadavků Určení požadavků na nastavení testu

a výběr metodologie provádění testu

Vytvoření screeneru•kritéria výběru

•sada otázek (uzavřených)

Nábor účastníků testu• výběr podle screeneru

Test s uživatelem

• verze kvalitativní:• 6-12 uživatelů

• verze kvantitativní:• 20 a více uživatelů

Příprava testu• dotazníky (před/po testu)

• test setup• seznam úkolů

Pilotní test• prověřuje kvalitu "připravy

testu"• provede se celý test

• účastník nesmí vědět nic o testu

Příprava testu• test setup

• seznam úkolů (průchodů)

Test bez uživatele

Příprava testu• test setup

• určení heuristik• seznam úkolů (průchodů)• nábor externích expertů

Test bez uživatele• test provádí externí experti

(cca 3-5)

Zpracování výsledku• identifikované problémy

• doporučení na odstranění identifikovaných problémů

Cognitive walkthrough

Heuristic Evaluation

Usability test with users

Testovaná aplikace• prototyp

• low-fidelity• high-fidelity

• finální aplikace

Remote testing

TUR Course: The Big Picture

Analysis Methodology Results

TUR 2013 – INTRODUCTION – [66]

Phase 1 – Analysis

Determining the target group

“Which people are going to use the

product?”

Determining the user requirements

“What do people expect from the

product?”

Choosing the right method of test.

Conditions on the test setup

“Lab? Street? What device?”

Tested application• Prototype

• Low-fidelity • High-fidelity

• Final application

TUR 2013 – INTRODUCTION – [67]

Phase 2 – Methodology

Screener• Selection criteria• Set of questions

• Predefined answers(multiple choices)

Selection of participants

• According to the screener

Test with users

• Qualitative• 6-12 users

• Quantitative• 20+ users

Preparing the test• Questionnaires

(before/after the test)• Test set-up• List of tasks

Pilot test• Verifies the quality

• The participant must not know anything

about the test

Usability test with users

TUR 2013 – INTRODUCTION – [68]

Preparing the test• Test setup

• List of tasks (walkthroughs)

Test without user

Preparing the test• Test setup

• Determining the heuristics• List of tasks (walkthroughs)• Recruiting external experts

Test without user• Performed by the

external experts (cca 3-5)

Cognitive walkthrough

Heuristic evaluation

Phase 2 – Methodology

TUR 2013 – INTRODUCTION – [69]

Remote testingFormal tests where users don’t need to visit the lab

Phase 2 – Methodology

TUR 2013 – INTRODUCTION – [70]

Phase 3 – Results

Processing the results• Identified problems • Recommendations

TUR 2013 – INTRODUCTION – [71]

TUR

People Adam J Sporka Jakub Franc Ivo Malý Zdeněk Míkovec, Pavel Slavík, Ondřej Poláček

Websites https://cent.felk.cvut.cz/courses/Y39TUR/ http://hcisemestralky.felk.cvut.cz