1DSIC/UPV

COMPUTER GRAPHICS

CHAPTER 2. Interaction

2.1. Devices2.2. User interfaces

2DSIC/UPV

ObjectivesA general vision of current state of interaction devicesTo know interaction devices of desktop computersTo learn about interaction devices of advanced applications, like virtual realityTo learn the basic interaction techniquesTo learn about the different applications of these interaction techniques

3DSIC/UPV

2.1 DevicesDevices• Keyboard and screen• Pointing devices• Scanners and cameras• Voice and sound• Haptic devices• Virtual and Augmented Reality• Advances devices

Tablet PC’sUbiquitous (pervasive) computingFinger tips devicesOcular scannersSmellsInteractive housesWearing computing

4DSIC/UPV

Pointing devicesThey allow to change the position of the screen cursorThey allow to select items (e.g. using buttons)Examples• Mouse• Trackball• Joystick• Touch screen• Computers with pen

5DSIC/UPV

MouseMade up by Doug Englebart at Xerox PARC (1968)Allows to chage cursor position and to select using buttonsTwo types• Mechanical• Optical

They can have an extra wheel to “scroll”

6DSIC/UPV

Trackball and JoystickTrackball: similar to a mouse, but it doesn’t require extra spaceTouchpad:• Similar to trackball• Detects finger capacity

Joystick• Used in computer games• They usually have several buttons• They can have motion feedback

7DSIC/UPV

Touch screenTo change cursor position and to select directly over the screenApplications• Sales terminals• Cellular phones, PDAs• Cash points, information points

8DSIC/UPV

Computers with penPen:• A pointing device, able to select and move information• It can be used to draw and write, creating a bitmap

This bitmap can be processed using optical recognitionTwo types of pen:• Optical: a photo-detector detects the light emitted by a pixel• Digitizer: a grid over the screen detects pen position

9DSIC/UPV

Scanners and Cameras2D Scanners• They allow to acquire text or imagery• Generate a bitmap• Text needs OCR (Optical Character Recognition)• Characterized by:

Spatial resolution: dots per inch (DPI)Color resolution: bits per pixel

3D Scanners• Obtain a 3D point cloud• They can also generate the geometry (polygons) of the

object (faces)

10DSIC/UPV

Scanners and CamerasCameras• They allow to capture images and video• USB, Firewire, or Flash memory connection• They require of a frame grabber (analog to digital conversion)• Characterized by:

Pixel resolutionImages per second able to capture

11DSIC/UPV

Voice and SoundMicrophone• In order to introduce multimedia information: sounds, words, songs,

etc.• Natural language supported (still under research)

Speakers• To transmit information to the user

12DSIC/UPV

Voice and SoundVoice technologies: interaction methods• Speech recognition• Voice synthesis• People identification and verification using voice• Natural language interpretation

Speech recognition (from voice to text)• Isolated words recognition

They require a pause between words• Continuous speech recognition

No pauses required, continuous speech• Dependency with speaker

They require training with the specific user• Independency with speaker

Able to recognize the voice of any user• Applications:

Commands by words: most common onedocument introduction by voice

13DSIC/UPV

Haptic devicesThe current systems don’t provide sense of touch to the usersExample: taking a virtual of wine• How to inform when the glass is touched• How to avoid the hand to cross the glass

Types of systems• Kinestetic: force feedback

Interacts with tendons and muscles: applying forces• Touch feedback

Interaction with skin nerves (heat, pressure, texture): they give the feeling of being in contact with a virtual object

14DSIC/UPV

Augmented and Virtual RealityVirtual Reality• To allow simulation with special interaction devices:

Input: 3D mice, virtual gloves, trackers (orientation)Output: 3D sound, head-mounted displays (HMD’s), projection systems, caves

• The objective is the total immersion of the user into a virtual environment

Augmented Reality• Imagery or video from the real world• Adds synthetic or virtual graphics and text in real time

Elements• Interaction devices• Trackers (position and orientation)• Navigation• Visualization • Hardware and software support

15DSIC/UPV

Virtual and Augmented RealityManipulation and interaction devices• 3D mouse:

It allows 3D position and orientation determinationIt allows navigation and object selection

• Joystick (3D):It can have force feedback

• GloveMost intuitive interaction deviceThey capture finger flexionThey allow to “take” and manipulate objectsThey allow to point with the fingerDataglove: optic fiber sensingPowerglove: extensiometric gauges

• Dextrous Hand MasterDermaskeletor attached to the fingersIt can detect side-to-side movement

16DSIC/UPV

Virtual and Augmented RealityHelmets• HMD: head mounted display• Stereo visualization: an image for each

eye• Types

With LCD screensWith CRT screensWith a columns of LEDs and a mirror

Glasses• For augmented reality• They merge real word view with a

synthetic image with a LCD

17DSIC/UPV

Virtual and Augmented RealityWorkbenchMulti-projection systems• Front view projection• Back view projection

18DSIC/UPV

Augmented and Virtual RealityCueCaveva• CAVE: Cave Automatic Virtual Environment• Multiprojection system• It allows several users• Setups

Three walls and floorThree walls and ceiling

19DSIC/UPV

2.2 User InterfaceUser interface• User interface introduction

DefinitionsGeneral conceptsUsability and accessibility

• Tools• Components• Interaction paradigms

20DSIC/UPV

ObjectivesTo learn ‘interface’ conceptTo know the general principles of UITo know the principles of universal designTo know what are UI and the basis of the UI creation toolsTo know UI architectures

To have a general vision and a comparison among the different interaction paradigms

21DSIC/UPV

Introduction to UI

Definition and JustificationHCI: Human-Computer Interaction• Discipline related with the design, implementation

and evaluation of interactive computer systems to be used by human beings

Why to study HCI?• Interface: very important part of an application

success or failure• About the 50% of an application code is centered on

the interface• In spite of that, few time is dedicated to that in

computer science

22DSIC/UPV

Introduction to UI

Why to study HCI?

Currently, more than the 70% of developementeffort in interactive application is focused on the interfaceA programming paradigm change:• Interaction paradigm

versus• Algorithm paradigm

Data management

Application logicUser interface

Multimedia3D GraphicsVoice input etc..

23DSIC/UPV

Introduction to UI

Basic definitionsUser:• Person who interacts with a computer system

Interaction• All the exchanges between user and

computer

24DSIC/UPV

Introduction to UI

Interface. Everyday object

25DSIC/UPV

Introduction to UI

Interface. Everyday objects

Obviousness: Visible and intuitive comprehension

26DSIC/UPV

Introduction to UIInterfaces

27DSIC/UPV

Introduction to UI

Interface. Definitions

• It is everything the user experiments, sees and does with a computer system

• It is the parts of a system with which the users is in contact

• Interfaces have to be usable and accessible

28DSIC/UPV

Introduction to UI

Factors to consider• Communication• Consistency• Flexibility• Feedback• Help• Robust• Error minimizing• Attractive• Standard• Portable

General principles of a UI

29DSIC/UPV

Introduction to UI

Communication• Dialogue between the user and the application• Aspects to consider:

Application controlComplete functionality (set of activities that can be done)Easy access and comprehensible

Consistency• The interface and dialogue aspects are uniform in its operation

and philosophyPresentation mode (color, messages types, data request)sequence of actions …

Flexibility• To facilitate different dialogue modes• Possibility of a wide range of different users

Different degree of experience, knowledge, purposes …

General principles of a UI

30DSIC/UPV

Introduction to UI

Feedback• Keep the user informed about the task he is carrying out• Progress information about the activities being carried out

Visual representation of the action (cursor changes,file deletion …)

Help• Information to the system user and its operation

General help, context sensitive, about future actions …

Robust• System reliability

To carry out tasks correctly, exception management

Minimizing errors• To avoid user’s errors

To facilitate recovery mechanisms (undo)

General principles of a UI

31DSIC/UPV

Introduction to UI

Attractive• Use fulfillment

Easy to useCustomizable

Standard• Common interface characteristics among applications

Better training times (Familiarity)

Portable• Data conversion between applications• Multiplatform

General principles of a UI

32DSIC/UPV

Introduction to UIParadigm changeChanging the idea of computer:•A paradigm change is going on• From desktop computer in which we

interact in front of it, sat down and centered in the interaction, we are changing to a interaction anywhere, at any time and concurrently to other activities (ubiquitous computing)

•Examples: agenda, internet mobile, on board computer.

33DSIC/UPV

Introduction to UIUI creation tools

What are they ?:• The UI editing tools allow the development of

interactive systems, minimizing development effort• A 48% of the application code is centered in the

interface development

Basis• Application independence• Device independent• User independence

34DSIC/UPV

Introduction to UIUI creation tools

Windows based managers• Graphics interfaces• Desktop metaphor• Icons and task bar

35DSIC/UPV

Introduction to UIUI creation tools

Controls/Widgets• Interactive elements• Compoundable: they can be compound between them

36DSIC/UPV

Components

Seeheim Model• Divided by levels (1985)

Architecture

37DSIC/UPV

ComponentsDialogue layer

Objective• Function: Control of dialogue with the user

Functions• Interaction protocol establishment• Events control• Events management and application connection

38DSIC/UPV

Components

Input modes: Request• Dialogue directed by the application

Dialogue layer

39DSIC/UPV

Components

Input mode: Events• Control of the dialogue by the user

Dialogue layer

40DSIC/UPV

Components

Events• Information about particular happenings• They include relevant information

Type of events• User

Mouse (buttons, movement, …)Keys (press, modifiers)Component In/Out (focus)Action (of each component)

• SystemWindows (resize, closing, …)Application (inserting new elements)

Dialogue layer

41DSIC/UPV

ComponentsDialogue layer

Event management• Strategies on event capture and application notification

The most common method: (callbacks• OpenGL (glut) system• Component connection (visual)-event-application code

42DSIC/UPV

ComponentsPresentation layer

Presentation• I/O information organization and its appearance

Importance• Represents the user’s perception of the system• Managing the available resources• Suitable object representation

43DSIC/UPV

ComponentsPresentations layer

Aspects of study• Visual clarity

Emphasizing the logic organization of the information through ananalysis on how the visual stimuli is organized by the observer

• Visual codesVisual distinction between different nature elementsIconic language: Graphic representation of a conceptRecognition, memory and discrimination

• Visual consistencyOrganization of different nature elementsColor: Using color as a coding technique conservatively, modificablecolor combination

• Screen formatManagement and organization of available spaceBalancedSimmetryRegularity and stressGrouping and alignment

44DSIC/UPV

Interaction paradigms

Models from which the interaction system is derived and it establishes the communication characteristicsTypes:• Desktop computer• Virtual reality• Ubiquitous computing• Augmented reality

Interactions:• Human-computer• Computer-environment• Human-environment

45DSIC/UPV

Interaction paradigmsComparative diagram

A) Desktop

D) Agumented realityC) Ubiquotous computing

B) Virtual realityC Computer

R Real world

Human - Computer

Human – Real world

Real world - Computer

Comparative diagram