Human Factors and User Interfaces in Energy Efficiency

Lin ZhongELEC518, Spring 2011

2

Motivation

Operating system

ApplicationSoftware

Hardware

User interface

User

Processor MemoryMassive storage

Network interface

Display & other interface hardware

3

Energy efficiency: definition

Energy efficiency = User productivity

Avg. power consumption

= (User productivity) ×(Power efficiency)

Human-computer interaction (HCI)

Low-power design

4

Limits

• Minimal power/energy requirements

• Human speeds

5

Speed mismatch

1

10

100

1000

10000

100000

1000000

1968 1972 1976 1980 1984 1988 1992 1996 2000 2004

Year

Tim

es

of

imp

rov

em

en

t

Olympic Gold Metal winner: 100m dash (men)

Olympic Gold Metal winner: 100m dash (women)

# of transistors for Intel processor

Processor performance measured in MIPS

A constantly slow user

An increasingly powerful computer

Sources: intel.com and factmonster.com

6

Slow-user problem

0

0.2

0.4

0.6

0.8

1

0 1 2 3 4 5

Time (s)

Po

we

r (W

att

)A computer spends most of its energy in interfacing

Slow-user problem cannot be alleviated by a “better” or more powerful interface

7

Model Human Processor

Cognitive process

Perceptual process

Motor process

Model Human Processor: Card, Moran & Newell’83

Three processes involved in the user reaction to a computer

Perceptual process• Fixations and saccades

– Fixation: information absorbed in the fovea (60ms)

– Saccades: quick movements between fixations (30ms)

– Each GUI object requires one fixation and one saccade

• Rauding rate– Raud: read with understanding– 30 letters/second (Carver, 1990)

8

9

Cognitive process

• Hick-Hyman Law– N distinct and equally possible choices

• Applicable only to simple cognitive tasks– Selection: menu, buttons, list

(s) 1Nlog7

1delay Cognitive 2

10

General form

• Hick-Hyman Law

– pi : the probability that the ith choice is selected

– pi can be estimated based on history

)1

(1 log7

1delay Cognitive

i1 pp

N

ii

11

Motor process

• Stylus operation

• Fitts’ Law– A: distance to move– W: target dimension along the moving direction

– Parameters adopted from (MacKenzie and Buxton, 1992)

(s) )1(log166.023.0delayMotor 2 W

A

12

0 5 10 15 20 25 30 35 40 45 50

0

5

10

15

20

25

30

35

40

45

50

Power Law of practice

• Speed on nth trial – Sn = S1 na, where a ≈0.4 – Applies to perceptual & motor processes– Does not apply to cognitive process or quality

Learning curve of text entry using Twiddler, Lyons, 2004

Power Law predictionMeasurement

13

Human capacity limitations

Human capacity

• Perceptual• Cognitive• Motor• ……

14

Cache

Frequent interactions

Frequently accessed data

Task to outsource

Interfacing energy

Memory access latency

Cost to reduce

Computer & user

CPU & memorySpeed mismatch

Interface cacheMemory cache

Alleviate slow-user problem with a “worse” or less powerful interface

15

Interface cache: examples

Flip phones

Average time spent on laptop per day declined from 11.1 hours to 6.1 hours 5 months after Blackberry deployment

-----Goldman Sachs Mobile Device Usage Study

16

Human thermal comfort

Starner & Maguire, 1999 and Kroemer et al, 1994

17

A hot case: 3-Watt Nokia 3120

Phone case temperature will be 40 deg C higher.

Every One Watt increases surface temperature by about 13 deg C

18

Minimal power/energy requirement

D

Ω

Visual and auditory output

Emin ≈ Ω·D2·10-13 (Joule)

About 10-14 (Joule) for most handheld usagePoint source

Minimal energy requirement for 1-bit changewith irreversible computing

10-21 (Joule) (Landauer, 1961)

19

Insights for power reduction

D

ΩPoint source

P∝Ω·D2

η(λ)·V(λ)

η(λ): conversion efficiency from electrical power

V(λ): relative human sensitivity factor

Reflective layer to control Ω

λ: wavelength of light/sound

20

Text entry speed (productivity)

150

2313 15

25 2212

7

0

20

40

60

80

100

120

140

160

180

Speaking mini hardware keyboard Software keyboard withstylus

Handwriting

Spe

ed (w

ords

per

min

ute) Raw speed

Corrected speed

21

Impact of human factors

0

0.2

0.4

0.6

0.8

1

0 1 2 3 4 5

Time (s)

Po

we

r (W

att

)

Length of idle periods cannot be significantly reduced

Power consumption in idle periods is dominated by interfacing devices

Using Calculator on Sharp Zaurus PDA

99% time and 95% energy spent in idle periods during interaction

22

Experimental setup

Intel Xscale 400Mhz

240X320, 16-bit color

mic., speaker & headphone jack

WindowsTransflective/back lightBluetoothSpeech recog.

Linux/QtReflective/front light

DevicesHP iPAQ 4350 Sharp Zaurus SL-5600

23

Experimental setup (Contd.)

iPAQ H3870

RsVsVdd

5V

Host machine GPIB card GPIB cable Agilent 34401A multimeter

Measurement

200 samples/second

24

0

0.4

0.8

1.2

1.6

0 0.5 1 1.5

Time (s)

Po

we

r (W

)

Experimental setup (Contd.)

0

0.4

0.8

1.2

1.6

0 0.5 1 1.5

Time (s)

Po

we

r (W

)

Extra energy/power consumption of an event is obtained through differential measurement

Extra energy consumption by writing “x”

Write “x” with stylus/touchscreen

25

Power breakdown

A handheld usually spends most time being idle but the display has to be on most time

If the display is not on, the speaker subsystem is usually on

0

1

2

3

4

iPAQ Zaurus

Pow

er c

onsu

mpt

ion

(mW

) Earphone

Speaker

Lighting

LCD

Computing

Basic idle

Computing: carrying out DCT repetitively

26

Energy characterization

• Visual interfaces– Graphical user interfaces (GUIs)– Digital camera

• Auditory interfaces– Recording/playback– Speech recognition & synthesis

• Manual text entry

27

GUIs• Stylus/Touch-screen• Most energy/time spent in idle periods

– Energy consumed by computing negligible

• Task time determines energy consumption

0

0.2

0.4

0.6

0.8

1

0 1 2 3 4 5

Time (s)

Po

we

r (W

att

)

280

0.4

0.8

1.2

1.6

2

1 207 413 619 825 1031 1237 1443 1649 1855 2061 2267

Time (1/206 s)

Po

we

r (W

)

Speech synthesis & recognition• Infer the behavior of Voice Command by

comparing voice recording and power trace

• Computing is not demanding• Used as baseline for comparison

Voice recording

Power trace

29

Comparison: Output

0

1

2

display off earphone

display on earphone

display offloudspeaker

display onloudspeaker

Different scenarios

r output

Lighting required for text

Lighting not required for text

• Speech is better only when– display is turned off – earphone is used – nighttime usage

iPAQ

spk

txt

rd

spk

P

P

R

Rr Energy efficiency

ratio

If r >1, speech output is more energy-efficient

30

Comparison: Text entry

0.1

1

10

100

0 20 40 60 80 100 120 140 160

Speech recog. input rate (cwpm)

r input

HW MKB-ideal VKB-ideal Letter Recog.-ideal

HW MKB VKB Letter

If r >1, speech recognition is more energy-efficient

State of the art

Near future

Ideal

31

Comparison: Text entry (Contd.)

0.1

1

10

100

0 20 40 60 80 100 120 140 160

Speech recog. input rate (cwpm)

r input

HW MKB-No LCD VKB-No LCD Letter Recog.-No LCD

HW MKB-No LCD/Night VKB-No LCD/Night Letter Recog.-No LCD/Night

Handwriting recognition is inferior to alternatives

Speech recognition can be the most energy-efficient

32

Comparison: Command & control• Speech vs. GUI operation

0

1

2

3

4

5

6

7

8

9

1 2 3 4 5

No. of taps

Ma

xim

al n

o. o

f wo

rds

pe

r co

mm

an

d

Ideal

95% accurate

95% accurate/No LCD

95% accurate/No LCD/LightAssume each stylus tapping takes 750ms

Single word voice command is more energy-efficient than GUI operation with 2 taps

33

Observations

• User productivity (speed) is critical – energy consumed being idle is significant

• Handwriting-based text entry is inferior• Speech-based text entry can be superior

– Turning off display is important– Accuracy

• Loudspeaker consumes significant power– Earphone incurs usability issue– Wireless audio delivery not energy-efficient

• “Computing” usually consumes trivial energy

34

Examples of energy inefficient interfaces

Kyocera KX2325 LG VX 6100 Microsoft Voice Command 1.01

35

Energy efficiency: definition

Energy efficiency = User productivity

Avg. power consumption

= (User productivity) ×(Power efficiency)

Human-computer interaction (HCI)

Low-power design

Model of Man

• Herbert Simon – Turing Award (1975) – Nobel Prize in Economics (1978)

• Human mind is simple; its apparent complexity is due to the environment’s complexity– Short-term memory is fast but small (~7)– Long-term memory is unlimited but writing takes time

(10 to 30 seconds)– Retrieval from long-term memory is associative and

depends on the storage structure

Bounded rationality

• Limitation on ability to plan long behavior sequences

• Tendency to set aspiration levels for each goal• Tendency to operate on goals sequentially

rather than simultaneously• Satisficing rather than optimizing search

behavior

http://www.princeton.edu/~smeunier/JonesBounded1.pdf