+ All Categories
Home > Documents > Goldberg Visual Scanpath Representation

Goldberg Visual Scanpath Representation

Date post: 05-Dec-2014
Category:
Upload: kalle
View: 1,701 times
Download: 1 times
Share this document with a friend
Description:
Eye tracking scanpaths contain information about how people see, but traditional tangled, overlapping scanpath representations provide little insight about scanning strategies. The present work describes and extends several compact visual scanpath representationsthat can provide additional insight about individual and aggregate/multiple scanning strategies. Three categories of representations are introduced: (1) Scaled traces are small images of scanpaths as connected saccades, allowing the comparison of relative fixation densities and distributions of saccades. (2) Time expansions, substituting ordinal position for either the scanpath’s x or y-coordinates, can uncover otherwise subtle horizontal or vertical reversals in visual scanning. (3) Radial plots represent scanpaths as a set of radial arms about an origin, with each arm representing saccade counts or lengths within a binned set of absolute or relative angles. Radial plots can convey useful shape characteristics of scanpaths, and can provide a basis for new metrics. Nine different prototype scanning strategies were represented by these plots, then heuristics were developed to classify the major strategies. The heuristics were subsequently applied to real scanpath data, to identify strategy trends. Future work will further automate the identification of scanning strategies to provide researchers with a tool to uncover and diagnosescanning-related challenges.
8
Copyright © 2010 by the Association for Computing Machinery, Inc. Permission to make digital or hard copies of part or all of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, to republish, to post on servers, or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from Permissions Dept, ACM Inc., fax +1 (212) 869-0481 or e-mail [email protected] . ETRA 2010, Austin, TX, March 22 – 24, 2010. © 2010 ACM 978-1-60558-994-7/10/0003 $10.00 Visual Scanpath Representation Joseph H. Goldberg and Jonathan I. Helfman Applications User Experience Oracle USA [email protected]; [email protected] Abstract Eye tracking scanpaths contain information about how people see, but traditional tangled, overlapping scanpath representations provide little insight about scanning strategies. The present work describes and extends several compact visual scanpath represen- tations that can provide additional insight about individual and aggregate/multiple scanning strategies. Three categories of rep- resentations are introduced: (1) Scaled traces are small images of scanpaths as connected saccades, allowing the comparison of relative fixation densities and distributions of saccades. (2) Time expansions, substituting ordinal position for either the scan- path’s x or y-coordinates, can uncover otherwise subtle horizon- tal or vertical reversals in visual scanning. (3) Radial plots repre- sent scanpaths as a set of radial arms about an origin, with each arm representing saccade counts or lengths within a binned set of absolute or relative angles. Radial plots can convey useful shape characteristics of scanpaths, and can provide a basis for new metrics. Nine different prototype scanning strategies were represented by these plots, then heuristics were developed to classify the major strategies. The heuristics were subsequently applied to real scanpath data, to identify strategy trends. Future work will further automate the identification of scanning strate- gies to provide researchers with a tool to uncover and diagnose scanning-related challenges. CR Categories: H.1.2. User/Machine Systems: Software Psy- chology, H.5.2. User Interfaces: Graphical User Interfaces. Keywords: Eye Tracking, Usability Evaluation, Scanpath, Scanning Strategy, Visualization 1 Introduction It has been said that proper representation of a problem is the single most important step towards its eventual solution. The problem we are investigating is the comparison of visual scan- ning strategies between individuals, groups, and conditions, in order to make inferences for improving the visual design of software and digital media. This paper introduces and extends several representations for scanpaths, then demonstrates how these representations can be used to develop heuristics and met- rics for detecting and labeling scanning strategies. 1.1 Scanpaths Eye tracking methods typically sample gaze locations at 50-120 Hz. Samples are then reduced to fixations, periods of visual attention at particular positions, and saccades, rapid eye move- ments with suppressed visual perception. Fixation positions are specified with respect to the background stimulus image or dis- play that was visible when the samples were collected. A se- quence of consecutive fixations and saccades results in a scan- path – a trace through time and space that may overlap itself. Scanpaths, along with other ocular-based metrics, may indicate higher cognitive strategies and states [e.g., Marshall 2007; Hor- nof and Halverson 2003]. A challenge for the researcher is to compare multiple scanpaths and quickly comprehend each par- ticipant’s scanning strategy in each tested condition. Typical visual representations of scanpaths use circles to repre- sent fixations and lines to represent saccades. Figure 1 displays a hypothetical scanpath with 5 fixations and 4 saccades. A scan- path may be replayed using animation, but to show a scanpath as a static image, fixation duration is typically coded by circle ra- dius. Figure 1. A hypothetical scanpath with 5 fixations and 4 sac- cades, showing fixation order and varying saccade lengths. Actual scanpath records are usually quite complex, and can be difficult to interpret and compare. Figure 2 shows a scanpath from a single observer trying to locate content within the lower right hand area of a web page. The scanpath visited multiple content columns as well as the search area before locating the content. Frequent crossing of saccades and revisiting of loca- tions contribute to scanpath complexity. 1.2 Scanpath Comparison Numerical techniques have been presented for summarizing and comparing scanpaths.. The length of a scanpath can be a meas- ure of productivity, and average fixation duration may measure of cognitive complexity [Goldberg and Kotval 1999]. The sum and distribution of the inter-fixation angles within a scanpath have been proposed as a measure of efficiency on a task, with simple, direct scanpaths signaling higher efficiency [Goldberg and Kotval 1998]. String editing distance computes the mini- mum number of editing steps required to transform one scanpath into another [West, et al. 2006]. Sequence alignment techniques first attempt to align sequences as closely as possible, before computing their disimilarity [Josephson and Holmes 2002]. Matching alignments from multiple sequences may also repre- sent the ‘averaged’ scanpath from a set of users [Hembrooke, et 203
Transcript
Page 1: Goldberg Visual Scanpath Representation

Copyright © 2010 by the Association for Computing Machinery, Inc. Permission to make digital or hard copies of part or all of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, to republish, to post on servers, or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from Permissions Dept, ACM Inc., fax +1 (212) 869-0481 or e-mail [email protected]. ETRA 2010, Austin, TX, March 22 – 24, 2010. © 2010 ACM 978-1-60558-994-7/10/0003 $10.00

Visual Scanpath Representation

Joseph H. Goldberg and Jonathan I. Helfman

Applications User Experience

Oracle USA

[email protected]; [email protected]

Abstract

Eye tracking scanpaths contain information about how people see, but traditional tangled, overlapping scanpath representations provide little insight about scanning strategies. The present work describes and extends several compact visual scanpath represen-tations that can provide additional insight about individual and aggregate/multiple scanning strategies. Three categories of rep-resentations are introduced: (1) Scaled traces are small images of scanpaths as connected saccades, allowing the comparison of relative fixation densities and distributions of saccades. (2) Time expansions, substituting ordinal position for either the scan-path’s x or y-coordinates, can uncover otherwise subtle horizon-tal or vertical reversals in visual scanning. (3) Radial plots repre-sent scanpaths as a set of radial arms about an origin, with each arm representing saccade counts or lengths within a binned set of absolute or relative angles. Radial plots can convey useful shape characteristics of scanpaths, and can provide a basis for new metrics. Nine different prototype scanning strategies were represented by these plots, then heuristics were developed to classify the major strategies. The heuristics were subsequently applied to real scanpath data, to identify strategy trends. Future work will further automate the identification of scanning strate-gies to provide researchers with a tool to uncover and diagnose scanning-related challenges.

CR Categories: H.1.2. User/Machine Systems: Software Psy-chology, H.5.2. User Interfaces: Graphical User Interfaces.

Keywords: Eye Tracking, Usability Evaluation, Scanpath, Scanning Strategy, Visualization

1 Introduction

It has been said that proper representation of a problem is the single most important step towards its eventual solution. The problem we are investigating is the comparison of visual scan-ning strategies between individuals, groups, and conditions, in order to make inferences for improving the visual design of software and digital media. This paper introduces and extends several representations for scanpaths, then demonstrates how these representations can be used to develop heuristics and met-rics for detecting and labeling scanning strategies.

1.1 Scanpaths

Eye tracking methods typically sample gaze locations at 50-120 Hz. Samples are then reduced to fixations, periods of visual

attention at particular positions, and saccades, rapid eye move-ments with suppressed visual perception. Fixation positions are specified with respect to the background stimulus image or dis-play that was visible when the samples were collected. A se-quence of consecutive fixations and saccades results in a scan-path – a trace through time and space that may overlap itself. Scanpaths, along with other ocular-based metrics, may indicate higher cognitive strategies and states [e.g., Marshall 2007; Hor-nof and Halverson 2003]. A challenge for the researcher is to compare multiple scanpaths and quickly comprehend each par-ticipant’s scanning strategy in each tested condition.

Typical visual representations of scanpaths use circles to repre-sent fixations and lines to represent saccades. Figure 1 displays a hypothetical scanpath with 5 fixations and 4 saccades. A scan-path may be replayed using animation, but to show a scanpath as a static image, fixation duration is typically coded by circle ra-dius.

Figure 1. A hypothetical scanpath with 5 fixations and 4 sac-

cades, showing fixation order and varying saccade lengths.

Actual scanpath records are usually quite complex, and can be difficult to interpret and compare. Figure 2 shows a scanpath from a single observer trying to locate content within the lower right hand area of a web page. The scanpath visited multiple content columns as well as the search area before locating the content. Frequent crossing of saccades and revisiting of loca-tions contribute to scanpath complexity.

1.2 Scanpath Comparison

Numerical techniques have been presented for summarizing and comparing scanpaths.. The length of a scanpath can be a meas-ure of productivity, and average fixation duration may measure of cognitive complexity [Goldberg and Kotval 1999]. The sum and distribution of the inter-fixation angles within a scanpath have been proposed as a measure of efficiency on a task, with simple, direct scanpaths signaling higher efficiency [Goldberg and Kotval 1998]. String editing distance computes the mini-mum number of editing steps required to transform one scanpath into another [West, et al. 2006]. Sequence alignment techniques first attempt to align sequences as closely as possible, before computing their disimilarity [Josephson and Holmes 2002]. Matching alignments from multiple sequences may also repre-sent the ‘averaged’ scanpath from a set of users [Hembrooke, et

203

Page 2: Goldberg Visual Scanpath Representation

al., 2006]. Statistical tests are also available for testing signifi-cant differences between sets of scanpaths. In one comparison approach, Heminghous and Duchowski [2006] provide a method to automate the process of assigning areas of interest. They first cluster scanpaths using a mean shift algorithm [Santella and DeCarlo, 2004], then tabulate similarity coefficients within and between participants and images. In another approach, Feusner and Lukoff [2008] compute the distance between all pairwise combinations of scanpath sequences using editing distance, then determine the difference between the average distances within and between groups to be compared. Using Monte Carlo simula-tion methods, they generate significance values for observed differences.

Scanpaths can also be summarized and compared visually. This paper provides a set of compact visual representations of scan-paths that can help usability professionals identify and compare scanning strategies in eye-tracking data. Visual representations can provide rich summaries of scanpath metrics, such as the distribution of inter-fixation angles and saccade lengths. Visual representations are also useful for validating and comparing the expressiveness of different metrics, as well as associating ca-nonical visual search strategies with scanpath data.

2 Methods: Scanpath Visual Representations

Several compact visual representations that summarize scan-paths are described and compared below. These show scanpath fixation density and complexity. Some of the representations show aggregate values associated with revisited locations and aren’t subject to the visual complexity problems associated with saccade crossings. The representations can be used to help vali-date associations between actual data and prototypical strategies.

Scanpaths may also be compressed or simplified, prior to con-verting to any of the visual representations. Compression algo-rithms, defined for specific studies, could merge or delete dupli-cate fixations within close proximity, could remove duplicate sub-sequences, or could remove sub-sequences that match in reverse order [Tsai 2001].

2.1 Scaled Traces

A scaled trace is a spatial representation of a scanpath sequence that shows saccadic lengths and inter-fixation angles, but not fixation durations. In classic studies of the influence of intent on eye movements, Yarbus [1967] demonstrated that eye move-ment scanpaths gathered over 3 minute periods differed dramati-cally depending on the demands of specific tasks. Our represen-tation of scaled traces, derived from this work, is shown in Fig-ure 3.

Figure 3. Representation of scanpath (left) as a trace image

(right) by excluding fixations and minimizing scale.

In addition to showing a standard trace line (Figure 4A), sequen-tial fixation order can also be represented by changing graphics properties of the line as it is drawn from beginning to end, from light to dark (Figure 4B), or from thin to thick (Figure 4D). Multicolored segments can also be used (Figure 4C), but it may be more difficult to convey fixation order with color than with brightness or thickness.

Figure 4. Alternate strategies for coding temporal order of sac-

cades in a scaled trace.

Though not intended to communicate detailed scanning tenden-cies, traces can effectively differentiate very horizontal from very vertical scanning tendencies. Figure 5 shows 12 different trace examples, each with 8 fixations and 7 saccades. This array of traces is an example of a small multiple, in which minimized graphics allow the viewer to compare many variables simultane-ously [Tufte 1983]. In this case, these scanpaths can be com-pared for relative aspect ratios, horizontal/vertical extent of scanning, and density of fixations. Saccade crossings in scaled traces (Figure 5) may provide a rough approximation of scan-path complexity, but such conclusions are much less reliable for longer scanpaths or when comparing scanpaths of different lengths.

While traces provide compact scanpath representations for rapid comparison, they cannot accurately convey number of fixations, lengths of saccades, or search area. Rather, they only provide relative comparisons of scanpath fixation density and complex-ity.

Figure 2. A scanpath from a single observer trying to locate

content within the lower right hand area of a web page.

204

Page 3: Goldberg Visual Scanpath Representation

Figure 5. Array of scanpath traces, forming a small multiple.

Each trace contains 8 fixations and 7 saccades.

2.2 Time Expansion

Scanpaths and scanpath traces are often tangled with significant crossing and overlap, making it hard to separate horizontal and vertical tendencies in scanning. By plotting the scanpath x or y-component against time, ‘time expanded’ representations of scanning tendencies are obtained. The general concept of substi-tuting time for the horizontal axis of a scanpath has been previ-ously introduced for cases of searching programming code [Uwano et al. 2006] and search result listings [Raiha et al., 2005, Aula et al. 2005].

Separation of the scanpath into vertical and horizontal time ex-pansions can broaden this concept to scanning whole pages. Figure 6 shows an 11-fixation scanpath trace that is separated into x and y time expansion graphs. Frequent horizontal shifts are clearly seen in the x graph, and vertical shifts are apparent in the y graph. Neither of these tendencies is as easily observed in the original scanpath trace.

The time expansion representations are similar to ‘sparklines;’ when plotted as small multiples, they are much like those used for stock prices and temperature [Tufte 2006]. General move-ment trends and back-tracking frequency become apparent when several time expansions are compared.

Figure 6. Time expanded x and y representations of scanpath.

2.3 Radial Plots

Radial plots are another compact way to represent scanpaths visually. Radial plots represent scanpath angles and magnitudes with bars that radiate from a center point like spokes in a wheel.

To construct a radial plot, a scanpath is first broken into its indi-vidual saccadic segments, much like in Noton and Stark’s origi-nal scanpath theory [Noton and Stark 1971]. As shown in Table 1, each of these segments can be assigned a length, an absolute inter-fixation angle, and a relative inter-fixation angle. Rantala [2008] introduced a radial plot (‘saccade star’) that presents a distribution of gaze statistics, such as fixation duration, arranged by angle. The present concept extends this representation to several other statistics and aggregation methods.

Table 1. Computation of angles in example scanpath.

Saccade ID

Length

(pixels)

Absolute

Angle (o)

Relative Angle

(o)

1 130 45 --

2 55 290 260

3 52 30 90

4 206 240 220

5 305 320 100

6 239 110 135

7 138 170 40

8 122 320 150

9 43 75 90

10 145 30 310

Absolute angles are measured relative to a global coordinate system (see Figure 7A), whereas relative angles are measured relative to the scanpath’s current direction of saccadic move-ment (see Figure 7B). Scanpath metrics using absolute angles reveal spatial tendencies relative to element layout on a page, whereas metrics using relative angles highlight veering from a set course.

Figure 7. Scanpath saccade angles measured counter-clockwise

from an absolute or global reference frame (A), and a frame that

is relative to the direction of motion (B).

The angles are then aggregated into angle ranges, or bins, of a predetermined resolution, as shown in Table 2. Here, bins are of width 45o; smaller widths provide greater precision in the plots,

205

Page 4: Goldberg Visual Scanpath Representation

at the expense of few samples per bin. Fewer samples per bin, in turn, result in less matching because there is less opportunity for different distributions to share the same bins. On the other hand, bin sizes that are too large result in less precision for individual distributions, but provide more information about how different distributions are similar.

Each bin is then shown as a bar in the radial plot. Bar angle cor-responds to the range of angles in the bin. We typically set each bar angle to the angle at the center of the range of angles in each bin, but alternate values may be chosen (e.g. the average of the actual angles in the bin). The bar length for each bin may be computed in a number of ways; it may be proportional to:

• Count: The number of saccade segments in the bin (i.e.

whose inter-fixation angle lies within the angle range of the

bin). These plots can also be termed radial histograms.

• Average The average length of the saccade segments in the

bin.

• Sum: The sum total length of the saccade segments in the

bin.

• Max (or Min): The maximum (or minimum) saccade seg-ment length in the bin.

Table 2. Computation of saccade bin counts, with bins of 45o

width and saccade segments from Table 1.

Number of Saccades within Bin Bin Angles

(degrees)

Absolute

Angle (o)

Relative Angle (o)

1-45 3 1

46-90 1 2

91-135 1 2

136-180 1 1

181-225 0 1

226-270 1 1

271-315 1 1

316-360 2 0

In addition to bin lengths, the mean shift indicates the geometric or spatial mean of each radial plot. It is represented as a single dot on the radial plot, and is computed from the mean of the x and y components of the graph. This represents the overall movement tendency for the scanpath. The coordinates of this dot could, optionally, be computed from the weighted means of each bin length, where weights are defined by the count (the number of scanpaths contained) in each bin. Compared to an unweighted mean shift, this weighted mean shift would show greater move-ment in the direction of bins containing greater numbers of scanpath segments.

Example radial plots, based upon the scanpath in Table 1, are shown in Figure 8. Plots based on absolute angles are shown in the left column, and those based upon relative angles are in the right column. The radial histograms, using saccade segment counts, indicate movement in most of the binned directions, with some bias to rightward movement (absolute angles). From the relative angles, however, there was a small bias in the direction of 90o movement, left-hand turns, or counter-clockwise scan-ning. The longest average saccade lengths appeared to be in upward and downward directions, although large rightward sac-cades were also noted. Notice that only short average saccade lengths were made at a relative 0o angle, indicating that the scanpath was indeed quite twisted. The relative angle, summed lengths were even more sensitive to the twisting than the relative angle, average saccade length plots.

Figure 8. Radial plot representations of scanpath from Table 1,

based either on absolute or relative angles. The number indi-

cates the maximum value along each axis.

3 Results: Strategy Comparison using Repre-sentations

3.1 Prototype Strategies

The proposed scanpath representations can be applied to a few prototypical scanning strategies, to assess the representations’ diagnostic potential for identifying these prototype patterns. A software tool was developed to render the scanpath representa-tions, and to provide a basis for their qualitative and quantitative comparison. The tool was developed in HTML and JavaScript, and runs in either Firefox or Chrome browsers. It allows cursor-based entry of a prototype scanpath trace, which is then repre-sented by each of the plots. Radial plots use default bin widths of 20o, but this value is easily modified.

Table 3 shows nine scanning traces with their compact represen-tations. These should be considered as examples in a much lar-

206

Page 5: Goldberg Visual Scanpath Representation

ger library of prototypical scanning strategies. Sequential traces (the first three columns) are drawn with ink that transitions from light to dark over each temporal sequence. The last two columns contain an example metric, the number of bins containing data in the radial plots, that could lead to a programmatic solution for teasing apart the scanning strategies.

By studying these prototypical traces, we can derive a few quali-tative indicators of various scanning strategies. These indicators can, in turn, lead to appropriate metrics to enable more auto-mated classification of the scanning strategies. These heuristics are discussed below, with references to the appropriate row(s) from Table 3.

• Linear scanning (rows 1 and 2). Whether scanning

downward or rightward, both time expansions are linear,

without reversals, and count and length radial plots are

identical. The absolute angle plots contain data and mean

shifts that are in the direction of scanning, while the bin an-

gle from the relative angle plots was 0o. Mean shift location

relative to origin can indicate the direction of scanning.

• Downward, followed by rightward scanning (row 3). X

and Y-time expansions both move downward and to the

right. Relative angle radial plots show mostly 0o data, due

to the fact that most saccades are in the same direction as

preceding saccades. Absolute angle radial plots are more balanced between rightward and downward data.

• Square scanning (rows 4 and 5). Scanpaths that traverse a

closed loop are indicated by absolute angle radial plots con-

taining data spanning many bins. If scanning is primarily in

one direction, relative angle radial plots show most data in

one angular bin. The more looping that is present in a scan-

path, the closer the absolute angle mean shifts are to the

Table 3. Sample prototype scanning strategies, with their compact representations.

207

Page 6: Goldberg Visual Scanpath Representation

origin, compared to relative angle mean shifts. In the case

of a square-like scanning strategy, time expansions show a

single horizontal and vertical reversal pattern. The radial

plots are primarily single bin (relative angle) and four bins

(absolute), whether clockwise (CW) or counter-clockwise

(CCW). Teasing apart CW from CCW direction can be

based upon the primary angle of the mean shift associated

with average length, relative angle radial plots. Note that

radial plots that are based on count data don’t necessarily

have equal length arms. This is caused by differences in

cursor velocity when inputting the square scanning strat-

egy. Here, the absolute angle 90o arm, moving upward, was

shorter because it contained fewer fixation counts than the

other arms. This, in turn, was due to faster upward cursor movement, compared to the other directions.

• Circular scanning (rows 6 and 7). Both CW and CCW

circles showed complex, correlated movement patterns in

time expansions. Both Relative angle radial plots contain

directional data primarily in one or two angular bins. Abso-

lute angle radial plots typically contain data in all bins, al-

though the length of individual bins depend on cursor input

velocity (fixation count data) or relative differences in sac-

cade lengths (average length data). Both absolute angle

plots contain data that are generally more distributed

among multiple bins than for relative radial plots. If the

mean shift is located between 0o-180o, there is a tendency

towards counter-clockwise scanning, and clockwise scan-

ning otherwise.

• Triangular scanning (row 8). Absolute angle radial plots

clearly show data in three bins, corresponding to the three

primary angles, whereas relative plots were primarily one

or two bins, in correspondence with few turns. As for

square scanning, the length of the arms in the absolute an-

gle, fixation count plots depends heavily upon cursor input

speed. Whereas the absolute angle plots contain 3 arms

with data, the relative angle plots contain 2 arms for an

equilateral triangle. These relative angles correspond to 0o, and 60o (leftward).

• More complex tendencies (row 9). Although not nearly as

complex as ‘real’ scanpath data, a ‘Figure-8’ prototype

scanpath starts to approximate the subtle complexities of

real data. Here, there are more reversals in the X than the Y

time expansion plot, indicating more complex horizontal

than vertical movement. Absolute radial plots show all an-

gular bins are filled, much like circular scanning. Their

mean shifts are close to the origin, indicating closed loop

scanning. The fact that the relative angle mean shifts were

positive and close to 0o indicates that equal movements

were made rightward and leftward from current scanning

direction. In even more complex scanpaths, radial plots

based on fixation counts indicate sub-segments with many

longer, saccades in a specified direction. Radial plots based

on saccade lengths could indicate a few long saccades in a specified direction, within a larger set of saccades.

3.2 Potential Metrics

Based upon the preceding analysis of prototypical strategies, metrics can be obtained from the scanpath representations to help identify prototype strategies within real scanpath data. Many metrics are possible; although these are not further util-

ized here, the following are likely to separate scanpath strategies in classification analyses:

• Number of time expansion reversals. Measures amount of reversal movement in X or Y-components of scanpath movement. Circles, for example, sweep 1 or 2 reversals, depending on starting and ending locations. This measure can also be further subdivided into positive and negative directions, to provide further classification precision.

• Number of bins containing data in radial plots. Com-puted for both absolute and relative angle plots, based on either counts or scanpath lengths. This metric measures directness and amount of circularity versus squareness in scanpaths. Circles contain many bins, and lines only con-tain a single bin.

• Angle and Length of mean shift in radial plots. Also computed for both absolute and relative angle plots, based on either counts or scanpath lengths. Measures position and distance of the mean shift, relative to 0o origin. Indi-cates primary direction and magnitude of scanpath move-ment.

3.3 Validation: Dataset Analysis

The process of using the aforementioned heuristics to diagnose scanning tendencies can be illustrated using the results from a recent study, in which over 120 participants scanned the image of a novel iPhone application, shown in Figure 9, in order to locate the ‘Status’ of the ‘CHI Services Deal’, located in the yellow shaded cell. This relatively simple scanning task resulted in a very rich set of varied scanning strategies.

Figure 9. Emulated iPhone application task image, in which

participants had to report the status of the CHI Service Deal

(shaded yellow here).

3.3.1 Similar Time Expansions

Table 4 shows a subset of participant scanpaths that contain similar time expansion representations. Due to space constraints, we only show the time expansion representations and the scan-paths; other figures will show pertinent representations where there are similarities. Each X-component time expansion indi-cates an initial leftward movement, followed by a rightward movement. Indeed, each scanpath image validates this pattern.

3.3.2 Similar Radial Plots

Similar scanpath sub-sequences can also be identified by radial plots. Table 5 shows absolute and relative angle radial histo-grams that contain saccade count data. In each of these scan-paths, the absolute angle plots had significant bins at 0 o, 180 o,

208

Page 7: Goldberg Visual Scanpath Representation

and 270o. Corresponding histograms based on relative angle were dominated by data at 90 o, indicating left turns relative to current direction. The three scanpaths each travel left, down, right, up, and down in a similar manner.

Table 4. Similar scanpaths, identified from similar x-component

time expansions.

Table 5. Similar scanpaths, identified from absolute and relative

angle radial histograms. Each scanpath was dominated by find-

ing the left table column, scanning down to the bottom row,

locating the ‘Status’ column header, then ending at the desired

cell.

Other scanning tendencies were also noted. The two scanpaths shown in Table 6 exhibited remarkably similar absolute and relative histograms. The relative mean shifts were at an angle of about 120 o, indicating lots of leftward turning. Each scanpath made a partial CCW circle starting at the center of the table.

Radial histograms with many filled angular bins and mean shifts near the origin can indicate circular closed loop scanning. This was observed in Table 7, where both scanpaths made a CCW circle before moving rightward in the table.

Table 6. Similar scanpaths, from similar radial histograms.

Table 7. Similar scanpaths, from similar relative angle radial

histograms.

4 Discussion

The scanpath representations introduced in this paper provide eye tracking and usability specialists with a new tool for identi-fying scanpaths and scanning strategies. We have described them, generated examples using prototypical scanpath shapes, and showed how they can be used to diagnose scanning tenden-cies.

Scanpaths are fascinating records of human visual attention in space and time. Scanpaths capture fixation start-time, duration, and position. Because the samples are time-stamped, they are ordered and form a sequence.

Each type of representation carries its advantages, much like providing multiple windows into a dataset. The scaled traces provide relative comparisons of scanpath fixation density and complexity. The time expansions help to visually unwind scan-paths of circular scanning or backtracking. The radial plots and mean shifts can show individual, aggregated, and multiple scan-path data. Like heatmaps, they display tendencies within and across groups and conditions. Unlike heatmaps, however, they provide directional scanning strategy information. Whether or not these plots are exposed to the end user, they can serve as a basis for metrics that can describe aspects of scanning behavior. It is possible, for example, to count the number of reversals in the time expansions or the dominant bins in the radial plots. These metrics could be applied to a set of prototype strategies and used to train a classification analysis to help tease apart similarities and differences among scanpaths.

The compact representations discussed here can be applied to any path or sequence-related problem with a spatial reference, not just scanpaths. For example, routing and logistics domains could benefit from new ways to compare transportation com-plexities. Radial plots could convey the number, length, and direction of delivery routes. More abstract spatial analogies,

209

Page 8: Goldberg Visual Scanpath Representation

such as comparing imaginary numbers plotted in polar coordi-nates to compare filter response, are also possible.

We believe that higher-level cognitive strategies can be exposed by analyzing individual scanpath sequences, then showing ag-gregated strategies. As a first step, we have developed visual representations of scanpaths based on data derived from their sequences: saccade lengths, saccade angles, and time expan-sions. We have also explored how radial plots that aggregate saccades by their inter-fixation angles can be used to character-ize actual scanpaths.

Future work will attempt to further understand how compact scanpath representations can help identify scanning strategies that are exhibited in and across scanpaths. This information, in turn, will help eye tracking and usability specialists understand how people scan images and software user interfaces.

References AULA, A., MAJARANTA, P., AND RAIHA, K. 2005. Eye-tracking

Reveals the Personal Styles for Search Result Evaluation, Proc. INTERACT 2005, Int. Fed. Info Proc, 1058-1061.

FEUSNER, M., AND LUKOFF, B. 2008. Testing for Statistically

Significant Differences Between Groups of Scan Patterns.

Proc. ETRA 2008, ACM Press, 43-46.

GOLDBERG, J.H., AND KOTVAL, X.P. 1999. Computer Interface

Evaluation using Eye Movements: Methods and Constructs. Int. J. Industrial Ergonomics, 14, 631-645.

GOLDBERG, J.H., AND KOTVAL, X.P. 1998. Eye Movement-Based

Evaluation of the Computer Interface, in Kumar, J. (Ed.),

Advances in Occupational Ergonomics and Safety, IOS

Press, 529-532.

HEMBROOKE, H., FEUSNER, M., AND GAY, G. 2006. Averaging

Scan Patterns and What They Can Tell Us. Proc. ETRA 2006, ACM Press, 41.

HEMINGHOUS, J., AND DUCHOWSKI, A.T. 2008. iComp: A Tool

for Scanpath Visualization and Comparison. PROC APGV

2008, Symposium on Applied Perception in Graphics and

Visualization, ACM Press, 152.

HORNOF, A.J. AND HALVERSON, T. 2003. Cognitive Strategies

and Eye Movements for Searching Hierarchical Computer Displays, Proc. CHI 2003, ACM Press, 5, 249-256.

JOSEPHSON, S. AND HOLMES, M.E. 2002. Visual Attention to

Repeated Internet Images: Testing the Scanpath Theory on

the World Wide Web. Proc. ETRA 2002, ACM Press, 43-49.

MARSHALL, S.P. 2007. Identifying Cognitive State from Eye

Metrics. Aviation, Space, and Environmental Medicine,

78(5), B165-B185.

NOTON, D. AND STARK, L.W. 1971. Scanpaths in Saccadic Eye

Movements While Viewing and Recognizing Patterns, Vi-sion Research, 11, 929-942.

RAIHA, K., AULA, A., MAJARANTA, P., RANTALA, H., AND KOI-

VUNEN, K. 2005. Static Visualization of Temporal Eye-

Tracking Data. Proc. INTERACT 2005, Int. Fed. Info Proc,

946-949.

RANTALA, H. 2008. Eye2i: Coordinated Multiple Views for

Gaze Data. Proc. ETRA 2008, ACM Press, 147.

SANTELLA, A., AND DECARLO, D. 2004. Robust Clustering of

Eye Movement Recordings for Quantification of Visual In-terest. Proc. ETRA 2004, ACM Press, 27-34.

TSAI, C. 2001. A C-program for Scanpath Compression,

http://research.chtsai.org/papers/scanpath-compression.html.

TUFTE, E. 1983. The Visual Display of Quantitative Information.

Graphics Press, Cheshire, CT., 170-175.

TUFTE, E. 2006. Beautiful Evidence. Graphics Press, Cheshire, CT.

UWANO, H., NAKUMURA, M., MONDEN, A., AND MATSUMOTO, K.

2006. Analyzing Individual Performance of Source Code

Review Using Reviewers’ Eye Movement. Proc. ETRA 2006, ACM Press, 133-140.

WEST, J.M., HAAKE, A.R., ROZANSKI, E.P., AND KARN, K.S.

2006. eyePatterns: Software for Identifying Patterns and

Similarities Across Fixation Sequences. Proc. ETRA 2006, ACM Press, 149-154.

YARBUS, A.L. 1967. Eye Movements and Vision. New York, Plenum Press.

210


Recommended