Interaction Design

Session 12

LBSC 790 / INFM 718B

Building the Human-Computer Interface

Agenda

• Questions

• Interaction design

• Some examples

• Project presentations

Interaction Design

• Play to the strengths of machine and human

• Place the locus of control with the user

• Make it easy to do the right thing

• Support multiple interaction styles

Strengths

• Machine– Speed– Storage– Repeatability

• Human– Initiative– Flexibility– Recognition

Putting the User in Control

• Familiar metaphor

• Visible objects– Non-modal design

• Predictable behavior

• Feedback on progress

• Explicit user models– Basic and expert modes

• Optional “wizards”

Making it Easy

• Visible context

• Understandable icons and messages– Tool tips and drill-down

• Atomic actions

• Obvious results

• Previews and reversability

Multiple Interaction Styles

• Point-and-click

• Keyboard shortcuts

• Command line

• Spoken dialog

Interactive Voice Response Systems

• Operate without graphical interfaces– Hands-free operation (e.g., driving)– Telephone access

• Built on three technologies– Speech recognition (input)– Text-to-speech (output)– Dialog management (control)

• Example: TellMe (1-800-555-TELL)

Speech Recognition

• Isolated words recognition is easy– Specialized dictation and telephone applications

• Continuous speech is slow and error prone– Hands-free tasks, dictation, speech retrieval

• Performs best when trained for one speaker• Limited vocabulary and language coverage• Does not work well in noisy environments

Speech Recognition

PhonemeDetection

WordConstruction

WordSelection

PhonemeString

PhonemeLattice

WordString

Pronunciationdictionary

Word n-gramlanguage

model

One-best phoneme transcription

N-best phoneme sequences

One-bestword transcript

Speech Recognition Lattice

Dialogue Management

• Turn-taking– User initiative– System initiative (allows smaller vocabulary)– Mixed initiative (e.g., barge in)

• Interaction style– Direct answers

• Achieving conversational goals

System Initiative

• Finite state control automates scripts– Restaurant, airline reservation, …

• A “state” encodes everything you know– What prompt to offer– What to do for each possible answer

• Loops allow for compact representations

Finite State Control Example

Where areyou departing

from?

Where doyou want

to go?

What daydo you want

to travel?

VerificationGoodbye

Wrong

Confirmed

BaltimoreNationalDulles

San FranciscoOaklandSan JoseAnywhere

else

Day when thereare flights

Not a day

Sorry

AnotherdayAnywhere

else

Cooperative Responses

• I want to fly to Tysons Corner on Friday– Completion

• All of the flights are sold out

– Correction• There is no airport in Tyson’s Corner

– Suggestion• Dulles is the closest airport

– Conditional answer• The only flight is on Tuesday

– Summary answer• I have flights on US carriers or KLM

The CSLU Toolkit

• IVRS development environment

• Graphical finite state dialog editor

• Text-to-speech, plus an animated face

• Isolated-word speech recognition

• Available at http://cslu.cse.ogi.edu/toolkit/

Supporting Information Access

SourceSelection

Search

Query

Selection

Ranked List

Examination

Recording

Delivery

Recording

QueryFormulation

Search System

Query Reformulation and

Relevance Feedback

SourceReselection

NPR Online

SpeechBot

Thesaurus-Based Search

Design Critique

• Select any 3 GUI’s you know and can use here– e.g., Windows XP, Google, USMAI catalog

• Work in in groups of 3 to critique each– Using IBM design guidelines

• http://www-3.ibm.com/ibm/easy/eou_ext.nsf/publish/6

– What are the 3 best features of each?– What are the 3 principal weaknesses of each?

An Example

• http://www.philipglass.com/