Spatial information AR.pdf

8/9/2019 Spatial information AR.pdf

1/23

Journal of Management Information Systems / Spring 2007, Vol. 23, No. 4, pp. 163184.

2007 M.E. Sharpe, Inc.07421222 / 2007 $9.50 + 0.00.

DOI 10.2753/MIS0742-1222230408

Attention Issues in Spatial InformationSystems: Directing Mobile Users VisualAttention Using Augmented Reality

FRANK BIOCCA, CHARLES OWEN, ARTHUR TANG, ANDCOREY BOHIL

F RANK B IOCCA is AT&T Chaired Professor of Telecommunication, Information Stud-ies, and Media at Michigan State University and at the Center for Knowledge and

Innovation Research, Helsinki School of Economics. His research interests focuson humancomputer interaction, specically interfaces that augment individual andgroup cognition. He is the founder and director of the M.I.N.D. Labs, a collaborativenetwork of 11 labs in seven countries.

C HARLES O WEN is an Associate Professor in the Department of Computer Scienceand Engineering at Michigan State University. He is the Director of the Media andEntertainment Technologies Laboratory. Dr. Owen conducts research in augmentedreality, computer graphics, and multimedia.

A RTHUR T ANG is an Assistant Professor in the Department of Industrial Engineeringand Management Systems at the University of Central Florida. He is the associatedirector of the M.I.N.D. Lab at the University of Central Florida. His research interestsinclude human factors in augmented reality and virtual reality, cognitive psychologyin computer interface, experimental evaluation of computer interfaces, and computer-mediated communication.

C OREY B OHIL is a Postdoctoral Fellow and Lab Manager at Michigan State UniversitysM.I.N.D. Lab. He is a cognitive psychologist with interests in humancomputer inter-action, perceptual classication, perception and action, and cognitive modeling.

A BSTRACT : Knowledge of objects, situations, or locations in the environment can be pro-ductive, useful, or even life-critical for mobile augmented reality (AR) users. Users mayneed assistance with (1) dangers, obstacles, or situations requiring attention; (2) visualsearch; (3) task sequencing; and (4) spatial navigation. The omnidirectional attention

funnel is a general purpose AR interface technique that rapidly guides attention to anytracked object, person, or place in the space. The attention funnel dynamically directsuser attention with strong bottom-up spatial attention cues. In a study comparing theattention funnel to other attentional techniques such as highlighting and audio cueing,search speed increased by over 50 percent, and perceived cognitive load decreasedby 18 percent. The technique is a general three-dimensional cursor in a wide array ofapplications requiring visual search, emergency warning, and alerts to specic objectsor obstacles, or for three-dimensional navigation to objects in space.

K EY WORDS AND PHASES : augmented reality, geospatial information system, location-based services, mobile computing, spatial information systems, visual attention.


2/23

164 BIOCCA, OWEN, TANG,AND BOHIL

The Use of Mobile Systems in the Management ofInformation and Objects

W ITH THE EVOLUTION OF MOBILE COMPUTER SYSTEMS , there is a tighter and more ubiquitousintegration of the virtual information space with physical space. For example, the use ofdatabases marked by geospatial data or radio frequency identication (RFID) taggingand mobile displays enable potential integration of virtual information and physicalassetsthe two are dynamically linked. Locations, such as buildings or rooms, andobjects, such as packages, vehicles, or tools, are often linked to arrays of informationin databases. But interfaces are still emerging that allow mobile users to efcientlyand fully use this information on-site for navigation, team coordination, object loca-tion, and object retrieval. Of current interfaces, the most suited to mobile geospatialinformation display is augmented reality (AR). AR systems allow users to be aware

of perfectly spatial registered information from simple two-dimensional (2D) labelsto three-dimensional (3D) labels or virtual markers.AR techniques allow users to see buildings, objects, and tools superimposed with

computer-generated virtual annotations. Unlike its cousin virtual reality (VR), AR en-hances the real environment rather than replacing it with computer-generated imagery.Graphics are superimposed on the users view of the real environment.

Early adoptions of AR interfaces can be found in information systems wherespatially registered 3D information can improve the performance of users. Currentapplication areas that incorporate AR interfaces include industrial training [5, 34, 35,36], computer-aided surgery [1], homeland security and military information systems[4, 14, 18, 19, 20], computer visualization, engineering design, interior design and

modeling [8, 16], computer-assisted instruction (CAI) [7, 34, 35, 36], and entertain-ment [2, 13, 26].

One of the most promising applications of AR is the display of computer-generatedinformation to guide the work of a user to specic spatial locations such as buildings,tools, packages, and other assets tracked by database systems. The ability to overlay andregister any type of information on the working environment in a spatially meaningfulway allows AR to be a more effective medium for information display.

Studies of user performance in AR-based information systems indicate that they canprovide unique human factors benetsas compared to approaches using traditionalprinted manuals or other computer-based approachessuch as improved task perfor-mance, decreased error rates, and decreased mental workload [34, 35, 36]. Informationobjects such as labels, overlays, 3D objects, and other information are integrated intothe physical environment. Objects, tasks, and locations can be cued when appropriateto support navigation and mobile active user tasks.

The pervasiveness, mode of delivery, and degree of control over information sys-tems in organizations have been evolving continually [37, p. 6]. Increased networkaccess via heterogeneous wireless network topologies enables mobile users to haveanytime, anywhere access of information for work and personal communication[6]. The rapid proliferation of mobile information services such as cellular phones,short message services (SMS), and global positioning systems (GPS) have created an


3/23

ATTENTION ISSUES IN SPATIAL INFORMATION SYSTEMS 165

array of new mobile location-based services. For example, users real-time geospatialinformation can be incorporated into mobile permission marketing [15] to create anew location-based mobile marketing service.

Mobile AR are the most compatible systems for geospatial data as the systems aredesigned to register virtual information to locations in space far more precisely thanthe typical geographic information system (GIS). An example is the use of AR totightly integrate medical 3D data (e.g., CAT scans, MRI images) with the patientsbody during surgery [1, 29]. This capability creates the potential for location-basedservices that provide an additional dimension to existing information systems andservicesthe guidance of user mobile attention to any spatial location for guidance,alerts, navigation, or object retrieval.

At the user level, mobile interfaces that can continuously guide users place demandson user attention. However, despite the rapid growth of mobile telephony and the

mobile Internet, research concerning m-commerce interfaces is still in the early stages[17, p. 98]. Mobile information-rich applications of AR systems begin to push upagainst a fundamental human factors limitation, the limited attention capacities ofthe human cognitive system. For example, cell phones split attention between virtualinformation (i.e., a caller talking about a different spatial context) and the demands ofthe users physical environment. These attention demands of mobile interfaces suchas cellular phones appear to contribute to automobile accidents [28, 33].

If AR interfaces are to guide user attention in real time, then a fundamental interfaceissue needs to be addressed: How can an AR system successfully manage and guidevisual attention to places in the environment where critical information or objectsare present, even when they are not within the visual eld? To describe the problemanother way: What does a 3D omnidirectional cursor look like? This question is partof a larger set of issues that we refer to as attention management and augmentationin mobile AR and VR interfaces.

Example Scenarios Where Visuospatial Cueing CanSupport User Search and Navigation

To illustrate the benets of managing visuospatial attention using a mobile AR infor-mation system, consider the following common scenarios.

Telecollaborative Spatial Cueing

An emergency paramedic wears a head-mounted camera and an AR head-mounteddisplay (HMD) while collaborating with a remote physician during a medical emer-gency. The remote physician is viewing the scene through the camera and needs topoint to a piece of equipment that the technician must use next. What is the quickestway to direct the technicians attention to the correct tool among a large and clutteredset of alternatives, especially if the tool tray is outside the technicians visual eldand he or she does not know the subtle difference between a Schroeder and a Pozzitenaculum forcep?


4/23


Object Search

A warehouse worker uses a mobile AR information system to manage inventory, and

is searching for a specic box in an aisle stocked with dozens of virtually identicalboxes. Based on inventory records of the information systems integrated into thewarehouse, the box is stored on a shelf behind the user. What is the most efcientway to signal the location to the user?

Procedural Cueing During Training

A trainee repair technician uses an AR system to learn a sequence of procedural stepswhere parts and tools are used to repair complex manufacturing equipment. Howcan the computer best indicate which tool and part to select next in the proceduralsequence, especially when the parts and tools may be distributed throughout a largeworkspace?

Spatial Navigation

A service repair technician with a personal digital assistant (PDA) equipped with theGPS is looking for a specic building and piece of equipment in a large ofce complexwith many similar buildings. The building is around the corner down the street. Whatis the fastest way to signal a walking path to the front door of the building?

Attention ManagementA TTENTION IS ONE OF THE MOST LIMITED MENTAL RESOURCES [30]. Attention is used tofocus the human cognitive capacity on a certain sensory input so that the brain canconcentrate on processing information of interest. Attention is primarily directedinternally, from the top down according to the current goals, tasks, and larger dis-positions of the user. Attention, especially visual attention, can also be cued by theenvironment. For example, attention can be user driven, that is, nd the screwdriver,collaborator driven, use this scalpel now, or system driven, please use this toolfor the next step.

Attention management is a central humancomputer interaction issue in the designof interfaces and devices [12, 24]. For example, the attention demands of current in-terfaces such as cellular phones and PDAs may play a signicant role in automobileaccidents [28, 33]. The scenarios from the previous section illustrate various caseswhere attention must be guided, augmented, or managed by the AR system or by aremotely communicating user.

Attention Cueing in Existing Information Interfaces

Users and interface designers have evolved various ways to direct visual attention ininterpersonal interaction, architectural settings, and standard interfaces.


5/23


Attention Cueing During Interpersonal Interaction

In interpersonal interaction, there are various sets of cues that are labeled indexical

cues. The phrase comes from the most obvious cue to visual attention, the pointingof an index nger directing the eyes to look there. Similarly, we learn early in lifeto monitor movement of other peoples gaze, drawing a mental vector to the spa-tial location of the persons visual attention. These virtual vectors create an implicitcue of look there. Gestures, eye movement, and various other linguistic cues helpdisambiguate otherwise confusing spatial terms in languages such as this, that,over there, and vague descriptive references to objects or locations in space.

Spatial linguistic cues can be the most ambiguous spatial cues. The meaning ofspatial language (e.g., left, here, in front of) varies with respect to the spatialreference frame of the speaker, listener, and the environment. For areas that needaccuracy (e.g., boating, theater), conventions are used (e.g., stage left, dolly in, port,starboard) to partially resolve this ambiguity problem, but the language in commonusage does not include this level of specialization.

The ambiguity of spatial language creates major communication problems when aninformation system needs to communicate spatial content to a user, or when anotherperson communicates to the user remotely through an AR or other collaborativesystem. Neither natural language nor nonverbal interactions in current interfaces aresufcient for complex and remote interactions.

Spatial Cueing in Windows Interfaces

WIMP (window, icon, menu, and pointer) interfaces benet from the assumption thatthe users visual attention is directed to the limited real estate of the screen. Visualcues such as ashing cursors, pointers, radiating circles, jumping centered windows,color contrast, or content cues are used to direct visual attention to spatial locationson the screen surface. The integration of audio with visual cues helps draw attentioneven when vision is not directed to the screen.

Of course, these systems work within the connes of a very limited physical area,an area so small that most users can scan it quickly. These techniques cannot easilycue objects in the 3D environment around a mobile user, for example, pointing at atool, building, or team member located behind a user equipped with a PDA.

Spatial cueing techniques used in interpersonal communication, WIMP interfaces,

and architectural environments are not easily transferred to mobile systems, be theyPDAs, tablet PCs, or mobile AR systems.

In mobile AR environments, attention is shared and spread across many tasksin the physical and virtual environment. Tasks in the virtual space may not be theprimary user task. This is very different from typical computer tasks such as wordprocessing in standard WIMP interfaces. For example, individuals may be walkingfreely in the environment, working with physical tools and objects, and interactingwith others while processing virtual information. The user may not be at the correctlocation in the scene, or looking at the correct spatial location or information neededto accomplish a task.


6/23


When communicating with remote users, the indexical cues of interpersonal com-munication are not available or are presented in a decreased modality, so nger-pointingand eye gazing are useless and linguistic references to this, that, and over thereare even more ambiguous than in direct communication.

Spatial Cursors and Cueing Techniques inAugmented Reality Systems

Currently, there are few, if any, general mobile interface paradigms to quickly directspatial attention to information or locations anywhere in the environment. In mobileAR environments, the volume of information is potentially vast and omnidirectional. AR environments have the capacity to display large amounts of informational cues to

physical objects in the environment.Responsiveness is important for mobile multitasking computing environments. In

a mobile multitasking setting, a users ability to detect specic virtual or physicalinformation at the appropriate time is limited. Visual attention is even more limited,because the system may have information about objects anywhere in an omnidirec-tional working environment around the user. Visual attention is limited to the eld ofview of human eyes (< 200 degrees), and this limitation is often further narrowed bythe eld of view of HMDs (< 80 degrees).

Alternative Interface Approaches

We are introducing the omnidirectional attention funnel, a unique, generalizableinterface design for mobile information search. To place the development of the at-tention funnel in context, we provide a review of alternative approaches to the samecommon problem.

Simple and Spatial Audio Cueing

In collaborative applications of mobile phones, the simplest and most common tech-nique for cueing the location of objects is languagethat is, The red box shouldbe on our left. The ambiguity and limitations of this method have been discussed,and are especially limiting when response time is a factor or the language cannotbe presented in an interrogatory setting, where users can ask questions that help toresolve ambiguities.

An alternative audio cueing method for mobile systems is the use of stereo spatialaudio to produce directional audio cues. These have been used for guidance for theblind and sighted [21, 23]. Spatial audio and the human auditory systems do not havethe spatial resolution to inform spatial location precisely [31] and localization can beslow, especially in a noisy auditory eld [25].


7/23


WIMP Cursor and Highlighting Techniques

Many AR systems adopt WIMP cursor techniques or visual highlighting to direct

users attention to an object (e.g., [7, 22]). Pointers in space appear over the objectof attention or the object is outlined as a wire diagram. These techniques may notbe effective for mobile AR systems. Highlighting techniques, such as highlighting awhole building, assumes that a detailed virtual model of the object, building, or toolis known. AR systems often need to direct attention to real-world objects, and virtualmodels generally do not exist even if a GPS or RFID location is known. Also, cuessuch as highlighting or cursors assume that the user is looking in the direction of thecued object (i.e., that it is on the screen or in the display). The cued objects may beoff to the side or behind the user.

Maps

In mobile systems, maps are sometimes used to cue the GPS or spatial location ofbuildings, and so on. Maps may be adequate for very large objects such as buildings,but become ambiguous when cueing the location of small objects such as tools (forexample, one of several emergency medical tools such as a scalpel). When maps areutilized, users must spatially correlate the map image with the surroundings, mentallytransferring the marked location to the real world, a sometimes daunting task.

Omnidirectional Attention Funnel: A Cursor Paradigm for

Mobile 3D InteractionTHE LIMITED IMPLEMENTATION OF A GENERAL TECHNIQUE for directing visual attention in3D space suggests that interface design in a mobile AR system presents three basicchallenges in managing and augmenting the attention of the user:

1. Omnidirectional cueing. How to quickly and successfully cue visual attentionto any location of physical or virtual information when there is an immediateneed.

2. Minimal attention demands. How to keep virtual information from consum-ing or interfering with attention to tasks, objects, or navigation in the physicalenvironment.

3. General applicability. How to provide a general technique that helps users ndand interact with physical or virtual objects at various distances while the useris mobile.

To meet these challenges, we have designed a new spatial interface concept, called theOmnidirectional Attention Funnel, as part of the Mobile Infospaces project, a multiyearcollaborative effort that examines human factors issues in the design of high volume,mobile AR systems. The attention funnel interface techniques are designed as a generalpurpose interface paradigm that addresses the broad range of attention management


8/23


challenges of mobile AR systems implemented on various platforms from high-end

head-mounted wearable systems to tablet PCs, PDAs, or smart phones.The omnidirectional attention funnel is an AR display technique for rapidly guid-

ing visual attention to any location in physical or virtual space. The fundamentalcomponents of the attention funnel are illustrated in Figures 1 and 2. The most vis-ible component is the set of dynamic 3D virtual objects linking the view of the userdirectly to the virtual or physical object. In spatial cognitive terms, the attentionfunnel visually links a head-centered coordinate space directly to an object centeredcoordinate space, funneling focal spatial attention of the user to the cued object. Theattention funnel takes advantage of spatial cueing techniques impossible in the realworld, along with ARs ability to dynamically overlay 3D virtual information ontothe physical environment.

Like many AR components, the AR funnel paradigm consists of (1) a display tech-nique, the attention funnel, combined with (2) methods for tracking and detecting thelocation of objects to be cued.

Components of the Attention Funnel

To test and demonstrate the concept, the attention funnel interface component wasimplemented as a user interface widget designed for mobile AR applications inthe ImageTclAR development environment [27]. This interface widget provides a

Figure 1. Illustration of the Attention Funnel Note: The attention funnel links the head of the viewer directly to an object anywhere aroundthe body.


9/23


mechanism for drawing visual attention to locations, information, or paths in an ARenvironment.

The basic components of the attention funnel, as illustrated in Figure 2, are

1. a view plane with a virtual bore-sight in the center and a pointer arrowabove;

2. a dynamic set of increasingly smaller funnel planes;

3. 3D crosshairs targeting the object location; and 4. a curved, dynamic path (see Figures 1 and 3) linking the head or viewpoint of

the user and all the elements directly to the object.

Along the curved dynamic path, the funnel planes are repeated in space and normalto the line. We refer to this line and the repeated patterns as an attention funnel. Thepath drawn for near objects is dened by a Hermite curve [10]. A Hermite curve is acubic curve segment dened by a start location, end location, and derivative vectorsat each end. The curve follows a path from the starting point in the direction of thestarting end derivative vector. It ends at the end point with the curve approaching theend point in the direction of the derivative vector. As a cubic curve segment, the curve

presents a smoothly changing path from the start point (i.e., the users view plane) to theend point (i.e., the 3D crosshairs target) with curvature controlled by the magnitudeof the derivative vectors. Hermite curves are a standard cubic curve method. Figure 3clearly illustrates the curvature of the funnel from a birds-eye view.

The start point for the Hermite curve is located at a specied distance in front ofthe origin in a frame dened to be the viewpoint of the user (the center of projectionfor a single viewpoint or average of two viewpoints for stereo viewers). The curveterminates at the target. The curve is a cubic interpolating curve that creates a smoothlyvarying path from start to target. The derivative vectors that specify the end curvaturesof the curve are selected so as to emit an attention funnel in the view direction that

Figure 2. Basic Component of an Attention Funnel Notes: Three basic patterns are used to construct a funnel: (A) the head-centered planeincludes a bore-sight to mark the center of the pattern from the users viewpoint; (B) funnel

planes, added in a xed pattern (approximately every 0.2 meters) between the user and theobject; and (C) the object marker pattern, which includes crosshairs marking the approximatecenter of the object.


10/23


approaches the target from the viewers direction. The curvatures of the starting andending points are specied in the application.

The orientation of each pattern along the visual path is obtained by spherical linear

interpolation of the up direction of the source frame and the up direction of the targetframe, so as to transition from an alignment with the view frame to an upright align-ment with the target. Spherical linear interpolation was introduced to the computergraphics society by Shoemake [32], and it is different from linear interpolation in thatthe angle between each interval is constantthat is, the changes of orientations of thepatterns are smooth. The formula used is:

Figure 3. Illustration of the Attention Funnel from a Birds-Eye View Notes: As the head and body move, the attention funnel dynamically provides continuous

feedback. Affordances from the perspective cues automatically guide the user toward thecued location or object. Dynamic head movement cues are provided by the skew (e.g., left,right, up, down) of the attention funnel. The level of alignment (skew) of the funnel providesan immediate intuitive sense of how much the body or head must turn to see the object.

t

t t ( ) =

( )( )( )

+( )( )1 2

1sin

sin

sin

sin.


11/23


In this equation, t [0,1], and is the angle between

1 and

2 computed as

The computational cost of this method is very small, involving the solution of the

cubic curve equation (three cubic polynomials), the spherical interpolation equation,and a rotation matrix for each pattern display location.

The purpose of the attention funnel is to draw visual attention to a target physical orvirtual object when it is not properly directed. When the user is looking in the desireddirection, the attention funnel becomes superuous and can cause visual clutter anddistraction. The solution to this case is to fade the funnel planes to only the view planeand target 3D crosshairs as the dot product of the source and target derivative vectorapproaches 1, indicating the direction to the target is close to the view direction.

Affordances in the Attention Funnel that GuideNavigation and Body Rotation

The attention funnel uses various overlapping visual cues that guide body rotation,head rotation, and gaze direction of the user.

Building on an attention sink pattern introduced by Hochberg [11], the attentionfunnel uses strong perspective cues as shown in Figure 4. Each attention funnel planehas diagonal vertical lines that provide depth cueing toward the center of the pattern.Each succeeding funnel plane is placed so that it ts within the preceding plane whenthe planes are aligned in a straight line. Increasing degrees of alignment cause theinterlocking patterns to draw visual attention toward the center. Three basic patternsare used to construct a funnel: (1) the head-centered plane includes a bore-sight tomark the center of the pattern from the users viewpoint; (2) funnel planes, added ina xed pattern (currently every 12 centimeters) between the user and the object; and(3) the object marker pattern, which includes a bounding box marking the approximatecenter of the object. Patterns 1 and 3 are used for dynamically cueing the user thatthey have locked onto the object (see below).

As the head and body move, the attention funnel provides continuous feedback thatindicates to the user how to turn his or her body or head toward the cued locationor object. Continuous dynamic head movement cues are provided by the skew (e.g.,

left or right) of the attention funnel. The pattern of the funnel provides an immediateintuitive sense of the location of the object relative to the head. For example, if thefunnel skews to the right, then the user knows to move his or her head to the right(e.g., more skewing suggests that more body rotation is needed to see it). The funnelcontinuously changes, providing a dynamic cue that one is getting closer to beingin sync and locked onto the cued object. When looking directly at the object, thefunnel fades so as to minimize visual clutter. A target behind the user is indicated bya funnel that moves forward for visibility, then turns and heads behind the user, aclear visual cue.

= ( )

cos .1

1 2

.


12/23


Methods for Sensing or Marking Target Objects or Locations

Attention funnels are applicable to any augmented vision display technology capableof presenting 3D graphics including HMDs and video see-through devices such astablet PCs or handheld computers. The location of target objects or locations in theenvironment may be known to the system because they are (1) virtual objects intracked 3D space, (2) tagged with sensors such as visible markers or RFID tags, or(3) predened spatial locations as in GPS coordinates. Virtual objects in tracked 3Dspace are the most straightforward case, as the attention funnel can link the user to thelocation of the target virtual object dynamically. Objects tagged with RFID tags arenot necessarily detectable at a distance, but local sensing in a facility may be sufcientto indicate a position that can be utilized for attention direction.

In some cases, the location of the object is detected by sensors and is not knownahead of time. An implementation we are currently exploring involves the detection ofvisible markers with omnidirectional cameras, which can be implemented in a videosee-through or optical see-through system. (Note that this implementation is differentfrom the traditional video see-through system, where the only camera used representsthe viewpoint of the user.) The head-mounted omnidirectional camera detects markersin a 360-degree environment around the user. The relation of the camera to the usersviewpoint is known. Detected objects can be cued for the user based on task needs orsearch requests by the user (e.g., nd the tool box).

Figure 4. Example of the Attentional Funnel Drawing Attention of the User to an Object onthe Shelfthe Box


13/23


User Evaluation in a Visual Search andRetrieval Task D OES THE ATTENTION FUNNEL TRULY DIRECT user attention more efficiently than the mostcommon techniques used in current AR interfaces? We conducted a study to evaluatethe effectiveness of the attention funnel in guiding attention around the immediatespace of the user [3].

A common task for an AR cursor system in a mobile setting is to guide a user toan object that the user needs to retrieve in the immediate environment. The attentionfunnel paradigm was tested against two alternative techniques: (1) a commonly usedAR highlighting technique, where the target object is cued by a surrounding greenbounding box, and (2) a control condition mimicking interpersonal interaction, wherethe object to be found is indicated only by its name (e.g., pick up the screwdriver).

A 360-degree omnidirectional workspace was created using four tables as shown inFigure 5. Forty-eight objects were distributed over the four tables (12 objects each).Half of these objects were primitive geometric objects of different colors and the otherhalf recognizable tools (e.g., screwdriver, stapler, and notebook).

Methodology

A within-subjects experiment was conducted to test the performance of the atten-tion funnel design against other conventional attention direction techniquesvisualhighlighting and verbal cues. The experiment had one independent variable, the methodused for directing attention, with three alternatives: (1) the attention funnel, (2) visualhighlight techniques, and (3) a control condition consisting of a simple linguistic cuecommon in current mobile phones (i.e., look for the red box.)

Participants

Fourteen paid participants drawn from a university student population participatedin the study.

Stimulus Materials and Test Environment

Three interface metaphors for directing visuospatial attention were designed andimplemented: (1) the attention funnel, (2) visual highlighting of the spatial locationof the object, and (3) an audio instruction interface using a verbal description of anobject.

Attention Funnel Condition

In the attention funnel interface, a series of linked rectangles dynamically links thevisual eld to the spatial location of the target object.


14/23


Visual Highlight Condition

For the visual highlight interface, a 3D bounding box was placed so as to appearspatially registered at the location of the target object.

Audio Instruction Condition

For the audio instruction condition, visual search was directed by playing a prerecordedaudio description of the target object for the user via a pair of headphones (e.g., Please

grab the [item]). Each audio cue took approximately 1.52 seconds to play.

Apparatus and Test Environment

A 360-degree omnidirectional workspace was created using four tables as shown inFigure 5. Twelve objects were placed on each table: six primitive objects of differentcolors (e.g., red box, black sphere) on a shelf, and six general objects (e.g. stapler,notebook) on the tabletop.

Visual cues were displayed in stereo with the Sony Glasstron LDI-100B HMD,and audio stimulus materials were presented with a pair of headphones. Head motion

Figure 5. Test Environment Note: The user sat in the middle of the test environment for the visual search task. Itconsisted of an omnidirectional workspace assembled from four tables, each with 12 objects(six primitive shapes and six general ofce objects), for a total of 48 target search objects.


15/23


was tracked by an Intersense IS-900 ultrasonic/inertia hybrid tracking system. Stereographics were rendered in real time based on the data from the tracker. A pressuresensor was attached to the thumb of a glove to capture the reaction time when thesubject grasped the target object.

Presentation of stimulus materials, audio instructions for participants, experimentalprocedure sequencing, and data collection for the experiment were automated so thatthe experimenter did not need to manually record the experimental results. The experi-ment was developed in the ImageTclAR AR development environment [27].

Measurements

Search Time, Error, and Variability

Search time in milliseconds was measured as the time it took for participants to graba target object from among the 48 objects following the onset of an audio cue tone.The end of the search time was triggered by the pressure sensor on the thumb of theglove when the user touched the target object. An error was logged for cases whenparticipants selected the wrong object.

Mental Workload

Participants perceived task workload in each condition was measured using the NASATask Load Index after each experimental condition [9].

Procedure

Participants entered a training environment where they were introduced and trainedto use each interface (audio, visual highlight, attention funnel). They then began theexperiment. Each subject experienced the interface treatment conditions (audio, visualhighlight, and attention funnel) and each object search trial in a randomized order. Foreach condition, participants were cued to nd and touch one of the 48 objects in theenvironment as quickly and accurately as possible. Participants participated in 24 trialsbalanced such that 12 trials involved searching for a random selection of primitiveobjects and 12 trials involved randomly selected general everyday objects.

Results

A general linear model repeated measure analysis of variance (ANOVA) was con-ducted to test the effect of metaphors on the different performance indicators. Therewas a signicant effect of interface type on search time, F (2,13) = 10.031, p < 0.001,and on search time consistency (i.e., smallest standard deviation), F (2,13) = 23.066,

p < 0.000. The attention funnel interface clearly allows subjects to nd objects in theleast amount of time and with the most consistency (mean [M] = 4473.75 milliseconds[ms], standard deviation [SD] = 1064.48) compared to the visual highlight interface


16/23


(M = 6553.12, SD = 2421.10) and the audio only interface (M = 4991.94 ms, SD =3882.11), which had the largest standard deviation. See Figure 6.

There was a signicant effect of interface type on the participants perceived mentalworkload, F (2,14) = 4.178, p < 0.05. The results indicate that the attention funnelinterface has the lowest mental workload (M = 44.64, SD = 16.96), comparing tothe visual highlight interface (M = 54.57, SD = 18.26) and the audio interface (M =55.57, SD = 12.43). See Figure 7.

There was no signicant effect of interface type on error, F (2,13) = 1.507, p < 0.05(attention funnel, M = 1.14, SD = 0.77; visual highlight, M = 1.43, SD = 1.56; audio,M = 0.86, SD = 1.03).

Discussion

When compared to standard cueing techniques such as visual highlighting and audiocueing, we found that the attention funnel decreased the visual search time by 22

percent overall, or approximately 28 percent for the visual search phase alone, and 14percent over its next fastest, as shown in Figure 6. While increased speed is valuablein some applications of AR, such as medical emergency and other high-risk applica-tions, it may be critical that the system support the users consistent performance. The

Figure 6. Search Time and Consistency by Experimental Condition Note: Attentional funnel decreased search time by 22 percent on average (28 percent whenreach time is subtracted) and increased search consistency (decreased variability) by 65percent.


17/23


attention funnel had a very robust effect on making the user search consistently, withsignicantly lower standard deviation comparing with the other two cueing techniques.The interface increased users consistency by an average of 65 percent and 56 percentover the next best interface.

A key criterion for a mobile interface is the need for minimal attention demand. Incases where AR environments are used for emergency services, repair work, othertime-critical and attention-demanding applications, search time may require costlymental effort. The effects of interface type of mental workload are illustrative, asshown in Figure 7. Cueing users with only audio, which involved holding the objectin memory, required additional mental workload. But visual highlighting techniques,which demand less memory, demanded additional mental workload, possibly becauseof the uncertainty of where to search. The attention funnel, which placed limited de-mand on memory and which directed search immediately and continuously, providedan 18 percent decrease in mental workload.

In summary, the attention funnel led to faster search and retrieval times, greaterconsistency of performance, and decreased mental workload when compared to verbalcueing and visual highlighting techniques.

LimitationsT HE ATTENTION FUNNEL WAS DESIGNED as a unique interface technique for directing andguiding users attention to any location in 4 steradians. The approach is unique and

Figure 7. Mental Workload Measured by NASA TLX [9] for Each Experimental Condition


18/23


patent pending. As indicated above, current techniques used in 3D games and simu-lations, such as the highlighting of 3D objects, are not feasible in real-world scenes.No virtual 3D model will preexist for most real-world objects such as buildings,packages, tools, and so on, even if the location is known using global positioning orRFID tags.

As there is no standard, we tested the attention funnel against the most commonlyused AR techniques [3]. This presents a limitation to the current study, as the logicalcomparison is a set of possible or unknown techniques, which have not been imple-mented. We are currently implementing and exploring other possible cueing techniquessuch as 3D arrows, lines, and so on.

Furthermore, an ideal test of the attention funnel would take place in complex,outdoor environments with fully mobile individuals cued to nd objects within andfar outside of reach. This would add ecological validity to the ndings.

Application of the Attention Funnel to Various Mobile and3D InterfacesT HE ATTENTION FUNNEL PARADIGM INVOLVES basic techniques that have potentially broadapplicability in AR and VR interfaces: a users attention has to be directed to objectsor locations in order to accomplish tasks.

Broadly, the attention funnel techniques can support user performance in the fol-lowing generic classes of fundamental AR tasks:

Physical object selection. Situations in which a user may be looking for a physi-cal object in space; for example, a tool on a workbench, a box in a warehouse, adoor in space, the next part to assemble during object assembly, and so on. Thesystem can direct the user to the correct object.

Virtual object selection. An AR system may insert labels or 3D objects insidethe environment. These may be within or outside the current view of the user.Attention funnels can cue them to look at the spatially registered label, tool, orcue.

Visual search in a cluttered space. The user may be searching in a highly clutterednatural or articial environment. An attention funnel can be used to cue them tothe correct location to view, even if they are not looking in the right place.

Navigation in near space. The system might also need to direct the walking pathof the individual through near space (e.g., through aisles, etc.). A directionalfunnel path (a slightly different implementation than the attention funnel above)can be used to indicate and cue the users direction, and provide dynamic cuesas to path accuracy.

Navigation in far space. An attention funnel can direct users to distant landmarks.As an example, someone walking toward an ofce several blocks away mustmaintain a link to the landmark as they navigate through an urban environment,even when landmarks are obscured.


19/23


With the success of AR systems, designers will seek to add potentially rich, evenunlimited, layers of virtual information onto physical space. As AR systems are usedin various real, demanding, mobile applications such as manufacturing assembly,warehousing, tourism, navigation, training, and distant collaboration, interface tech-niques appropriate to the AR medium will be needed to manage the mobile userslimited attention, improve user performance, and limit cognitive demands for optimalspatial performance. The AR attention funnel paradigm represents an example of cog-

nitive engineering interface techniques for which there is no real-world equivalent,and which is specically adapted for users of AR systems navigating and working ininformation- and object-rich environments.

Future Work W E ARE CURRENTLY IMPLEMENTING the attention funnel technique on other mobile de-vices, including handheld devices such as PDAs and cell phones. The attention funnelcan be overlaid on a live video stream captured by a handheld camera, while spatiallocation of the user can be determined using GPS, digital compass, or triangulation

of cellular or RFID signals. Figure 8 illustrates the implementation of the attentionfunnel technique on a tablet PC. The attention funnel technique has some importantimplications to usability of location-based consumer information systems. As an ex-ample, the attention funnel can be used to display navigation information generatedby commercial GISs (e.g., Microsoft Mappoint, Google Maps) from a rst-personperspective, as illustrated in Figure 9. The attention funnel technique can also be usedto display location-based touring alert information to a mobile user via a PDA or cellphone (e.g., the location of a shop or restaurant can be cued by an attention funneldisplayed on the screen of a PDA).

Figure 8. Implementation of the Attention Funnel Technique on a Tablet PC Note: The attention funnel can be used with a tracker-enabled tablet PC. In thisimplementation, the tablet PC (or smart phone) acts as a magic window upon scenesannotated with information.


20/23


Acknowledgments: The authors acknowledge the assistance of Betsy McKeon, Amanda Hart,and Mark Rosen in the preparation of this paper. They also appreciate the suggestions and rec-

ommendations provided by the three anonymous reviewers on an earlier version of this paper.This project is one element of the Mobile Infospaces project and supported in part by a grantfrom the National Science Foundation CISE 0222831. Any opinions, ndings, and conclusionsor recommendations expressed in this material are those of the authors and do not necessarilyreect the views of the National Science Foundation.

R EFERENCES

1. Bajura, M.; Fuchs, H.; and Ohbuchi, R. Merging virtual objects with real world: Seeingultrasound imagery within the patient. Computer Graphics, 26, 2 (1992), 203210.

2. Bimber, O. Video see-through AR on consumer cell phones. In Proceedings of the Third IEEE and ACM International Symposium on Mixed and Augmented Reality. Los Alamitos, CA:IEEE Computer Society Press, 2004, pp. 252253.

3. Biocca, F.; Tang, A.; Owen, C.; and F., Xiao. Attention funnel: Omnidirectional 3D cursorfor mobile augmented reality platforms. In R. Grinter, T. Rodden, P. Aoki, E. Cutrell, R. Jef-fries, and G. Olson (eds.), Proceedings of the ACM CHI 2006, Conference on Human Factorsin Computer Systems. New York: ACM Press, 2006, pp. 11151122.

4. Brown, D.; Stripling, R.; and Coyne, J. Augmented reality for urban skills training. InProceedings of IEEE Virtual Reality Conference 2006. Los Alamitos, CA: IEEE ComputerSociety Press, 2006, pp. 249252.

5. Caudell, T., and Mizell, D. Augmented reality: An application of heads-up display tech-nology to manual manufacturing processes. In Ralph H. Sprague Jr. (ed.), Proceedings of theTwenty-Fifth Annual Hawaii International Conference on System Sciences. Los Alamitos, CA:IEEE Computer Society Press, 1992, pp. 659669.

6. Fang, X.; Chan, S.; Brzezinski, J.; and Xu, S. Moderating effects of task type on wirelesstechnology acceptance. Journal of Management Information Systems, 22, 3 (Winter 20056),123157.

Figure 9. An Illustration of a Navigation Funnel, Drawn on the Real-World Scene to Guidean Individual to Distant Objects or Destinations


21/23


7. Feiner, S.; MacIntyre, B.; and Seligmann, D. Knowledge-based augmented reality. Com-munications of the ACM, 36, 7 (1993), 5262.

8. Feiner, S.; Webster, A.; Krueger, T.; MacIntyre, B.; and Keller, E. Architectural anatomy.

Presence: Teleoperators and Virtual Environments, 4, 3 (1995), 318325.9. Hart, S. Development of NASA-TLX (task load index): Results of empirical and theo-retical research. In P. Hancock and N. Meshkati (eds.), Human Mental Workload. Amsterdam:North-Holland, 1988, pp. 239250.

10. Hearn, D., and Baker, M.P. Computer Graphics, C Version. Upper Saddle River, NJ:Prentice Hall, 1996.

11. Hochberg, J. Representation of motion and space in video and cinematic displays. In K.Boff, L. Kaufman, and J. Thomas (eds.), Handbook of Perception and Human Performance,vol. 1. New York: Wiley, 1986, pp. 22.122.64.

12. Horvitz, E.; Kadie, C.; Paek, T.; and Hovel, D. Models of attention in computing andcommunication: From principles to applications. Communications of the ACM, 46, 3 (2003),5259.

13. Jebara, T.; Eyster, C.; Weaver, J.; Starner, T.; and Pentland, A. Stochasticks: Augmentingthe billiards experience with probabilistic vision and wearable computers. In Proceedings of the

First International Symposium on Wearable Computers. Los Alamitos, CA: IEEE ComputerSociety Press, 1997, pp. 138145.

14. Julier, S.; Baillot, Y.; Lanzagorta, M.; Brown, D.; and Rosenblum, L. BARS: Battleeldaugmented reality system. Paper presented at the NATO Symposium on Information ProcessingTechniques for Military Systems, Istanbul, Turkey, October 2000.

15. Kavassalis, P.; Spyropoulou, N.; Drossos, D.; Mitrokostas, E.; Gikas, G.; and Hatzista-matiou, A. Mobile permission marketing: Framing the market inquiry. International Journalof Electronic Commerce, 8, 1 (Fall 2003), 5579.

16. Klinker, G.; Stricker, D.; and Reiners, D. Augmented reality for exterior constructionapplications. In W. Bareld and T. Caudell (eds.), Fundamentals of Wearable Computers and

Augmented Reality. Mahwah, NJ: Lawrence Erlbaum, 2001, pp. 379427.17. Lee, Y., and Benbasat, I. A framework for the study of customer interface design for mobile

commerce. International Journal of Electronic Commerce, 8, 3 (Spring 2004), 79102.18. Livingston, M.; Brown, D.; Julier, S.; and Schmidt, G. Military applications of augmented

reality. Paper presented at the NATO Human Factors and Medicine Panel Workshop on VirtualMedia for Military Applications, West Point, June 2006.

19. Livingston, M.; Rosenblum, L.; Julier, S.; Brown, D.; Baillot, Y.; Swan, E., II; Gabbard,J.; and Hix, D. An augmented reality system for military operations in urban terrain. Paperpresented at the Interservice/Industry Training, Simulation and Education Conference, Orlando,FL, December 2002.

20. Livingston, M.; Swan, E., II; Julier, S.; Baillot, Y.; Brown, D.; Rosenblum, L.; Gabbard,J.; and Hllerer, T. Evaluating system capabilities and user performance in the battleeldaugmented reality system. Paper presented at the Performance Metrics for Intelligent SystemsWorkshop, Gaithersburg, MD, August 2004.

21. Loomis, J.; Golledge, R.; and Klatzky, R. Navigation system for the blind: Auditorydisplay modes and guidance. Presence: Teleoperators and Virtual Environments, 7, 2 (1998),193203.

22. Mann, S. Telepointer: Hands-free completely self contained wearable visual augmentedreality without headwear and without any infrastructural reliance. In Proceedings of Fourth

International Symposium on Wearable Computers. Los Alamitos, CA: IEEE Computer SocietyPress, 2000, pp. 177178.

23. Marston, J.; Loomis, J.; Klatzky, R.; Golledge, R.; and Smith, E. Evaluation of spatialdisplays for navigation without sight. ACM Transactions on Applied Perception, 3, 2 (2006),110124.

24. McCrickard, D., and Chewar, C. Attentive user interface: Attuning notication design touser goals and attention costs. Communications of the ACM, 46, 3 (2003), 6772.

25. Middlebrooks, J., and Green, D. Sound localization by human listeners. Annual Reviewof Psychology, 42 (1991), 135159.

26. Ohshima, T.; Satoh, K.; Yamamoto, H.; and Tamura, H. AR2 hockey system: A collab-orative mixed reality system. Transactions of the Virtual Reality Society of Japan, 3, 2 (1998),5560.


22/23


27. Owen, C.; Tang, A.; and Xiao, F. ImageTclAR: A blended script and compiled codedevelopment system for augmented reality. Paper presented at STARS2003: The InternationalWorkshop on Software Technology for Augmented Reality Systems, Tokyo, Japan, 2003.

28. Redelmeier, D.A., and Tibshirani, R.J. Association between cellular telephone calls andmotor vehicle collisions. New England Journal of Medicine, 336, 7 (1997), 453458.29. Rolland, J.; Wright, D.; and Kancherla, A. Towards a novel augmented-reality tool to

visualize dynamic 3D anatomy. In K. Morgan, H. Hoffman, D. Stredney, and S. Weghorst (eds.),Proceedings of Medicine Meets Virtual Reality 5. 1997, pp. 337348.

30. Shiffrin, R. Visual processing capacity and attentional control. Journal of ExperimentalPsychology: Human Perception and Performance, 5, 1 (1979), 522526.

31. Shinn-Cunningham, B. Localizing sounds in rooms. Paper presented at the ACM SIG-GRAPH and EUROGRAPHICS Campre: Acoustic Rendering for Virtual Environments,Snowbird, UT, May 2001.

32. Shoemake, K. Animating rotation with quaternion curves. Computer Graphics, 19, 3(1985), 245254.

33. Strayer, D.L., and Johnston, W. Driven to distraction: Dual-task studies of simulated driv-ing and conversing on a cellular phone. Psychological Science, 12, 6 (2001), 462466.

34. Tang, A.; Owen, C.; Biocca, F.; and Mou, W. Experimental evaluation of augmentedreality in object assembly task. In Proceedings of the First IEEE and ACM International Sym-

posium on Mixed and Augmented Reality. Los Alamitos, CA: IEEE Computer Society Press,2002, pp. 265266.

35. Tang, A.; Owen, C.; Biocca, F.; and Mou, W. Comparative effectiveness of augmentedreality in object assembly. In V. Bellotti, T. Erickson, G. Cockton, and P. Korhonen (eds.),Proceedings of ACM CHI 2003, Conference on Human Factors in Computing Systems. NewYork: ACM Press, 2003, pp. 7380.

36. Tang, A.; Owen, C.; Biocca, F.; and Mou, W. Performance evaluation of augmented real-ity for directed assembly. In A. Nee and S. Ong (eds.), Virtual Reality and Augmented Reality

Applications in Manufacturing. London: Springer-Verlag, 2004, pp. 301322.37. Zwass, V. Management information systemsBeyond the current paradigm. Journal of

Management Information Systems, 1, 1 (Summer 1984), 310.


23/23

Date post:	01-Jun-2018
Category:	Documents
Upload:	sara-namdarian
View:	227 times
Download:	0 times

Spatial information AR.pdf

Documents