Dow

nloa

ded

By:

[dbk

aber

@eo

s.nc

su.e

du] A

t: 19

:30

11 J

une

2007

Situation awareness and driving performancein a simulated navigation task

R. MA{ and D. B. KABER*{

{AREVA NP Inc., Charlotte, North Carolina, USA

{Edward P. Fitts Department of Industrial & Systems Engineering,

North Carolina State University, Raleigh, North Carolina, USA

The objective of this study was to identify task and vehicle factors that may

affect driver situation awareness (SA) and its relationship to performance,

particularly in strategic (navigation) tasks. An experiment was conducted to

assess the effects of in-vehicle navigation aids and reliability on driver SA and

performance in a simulated navigation task. A total of 20 participants drove

a virtual car and navigated a large virtual suburb. They were required to

follow traffic signs and navigation directions from either a human aid via a

mobile phone or an automated aid presented on a laptop. The navigation aids

operated under three different levels of information reliability (100%, 80%

and 60%). A control condition was used in which each aid presented a

telemarketing survey and participants navigated using a map. Results

revealed perfect navigation information generally improved driver SA and

performance compared to unreliable navigation information and the control

condition (task-irrelevant information). In-vehicle automation appears to

mediate the relationship of driver SA to performance in terms of operational

and strategic (navigation) behaviours. The findings of this work support

consideration of driver SA in the design of future vehicle automation for

navigation tasks.

Keywords: In-vehicle automation; Automation reliability; Situation aware-

ness; Driving performance; Navigation aiding

1. Introduction

Driving is considered to be a complex task requiring perception, comprehension and

projection of states of the roadway environment, as well as decision making on courses

of action and execution of driving behaviours. Situation awareness (SA) encompasses

three of these aspects of driving performance and has been found to be critical to

decision making in complex tasks (Endsley 1995a). Endsley (1995a) defined SA as: ‘the

*Corresponding author. Email: dbkaber@eos.ncsu.edu

Ergonomics

Vol. 50, No. 8, August 2007, 1351–1364

ErgonomicsISSN 0014-0139 print/ISSN 1366-5847 online ª 2007 Taylor & Francis

http://www.tandf.co.uk/journalsDOI: 10.1080/00140130701318913

Dow

nloa

ded

By:

[dbk

aber

@eo

s.nc

su.e

du] A

t: 19

:30

11 J

une

2007

perception of the elements in the environment within a volume of time and space, the

comprehension of their meaning, and the projection of their status in the near future’.

Ma and Kaber (2005) summarized underlying factors in driver SA, including navigation

knowledge, environment and interaction knowledge, spatial orientation knowledge and

vehicle status knowledge. They suggested an integration of these forms of knowledge in

a model of driver information processing to achieve accurate SA. However, there have

been few empirical studies of how drivers may form these types of knowledge for SA

and what task, environment or system features, such as in-vehicle automation, play a

role.

Ward (2000) and Matthews et al. (2001) related levels of SA defined by Endsley,

including perception (Level 1 SA), comprehension (Level 2 SA) and projection (Level 3

SA), to specific driving tasks or behaviours, including operational, tactical and strategic

behaviour. For example, in operational driving tasks, drivers are engaged in actions upon

vehicle actuators in order to maintain stable control. Ward (2000) and Matthews et al.

(2001) said operational driving tasks, including steering and braking responses, primarily

require Level 1 SA. Matthews et al. (2001) said: ‘Level 2 SA may (also) be involved, if

driving processes generate error messages’. For example, a driver may comprehend the

rate of vehicle deceleration to be insufficient (Level 2 SA) for a particular perceived

stopping distance (Level 1 SA). In tactical driving tasks, Matthews et al. observed that

there is a high requirement for Levels 1 and 2 SA to, for example, facilitate safe

manoeuvring of a vehicle in traffic by judging and comparing lane positions. Level 3 SA is

also relevant to such interactive driving situations where there is a high requirement for

near-term projection of changes in the driving course and traffic patterns. In strategic

driving tasks, when navigation plans are being formulated, there is also a high

requirement for Level 3 SA.

At this point in research on driver SA, there is a need to identify operator, task,

environment and system factors that may mediate the role of the various levels of SA in

specific driving behaviours, including operational, tactical and strategic behaviours, in

following tasks, navigation tasks, etc. As an example, Ma and Kaber (2005) investigated

the effects of adaptive cruise control (ACC–see also Stanton and Young 2005) and mobile

phone use on driver SA and performance in a lead-car following task primarily involving

operational behaviours. Their results revealed that SA improved with the use of ACC and

decreased with use of the mobile phone under normal driving conditions. They also found

significant positive correlations between SA and aspects of driving performance. They

observed that all levels of SA (perception, comprehension and projection) appear to have

an impact on operational driving behaviours in following tasks and, consequently,

performance. In general, the use of ACC and mobile phones may be mediating factors in

linkages of SA to specific driving behaviours and performance.

Additional empirical work is needed to identify other in-vehicle system factors that

may be influential in driver SA and to describe the role of each level of SA in performance

of tactical and strategic driving behaviours or more complicated driving tasks (Walker

et al. 2006). Links between Levels 1 and 2 SA and tactical behaviour are supported by

Matthews et al.’s work, as well as a link between Level 3 SA and strategic behaviour. For

example, the use of in-vehicle automated navigation aids has become prevalent for

driving in new environments, or to unknown destinations, to support driver decision

making. However, like the use of mobile phones while driving, navigation aid use

represents a secondary task that has the potential to decrease driver attention to the

roadway and SA, particularly Level 1 (perception). This may in turn lead to performance

problems. That is, the aid may facilitate achieving high-level SA for navigation but may

1352 R. Ma and D. B. Kaber

Dow

nloa

ded

By:

[dbk

aber

@eo

s.nc

su.e

du] A

t: 19

:30

11 J

une

2007

also undermine attention to operational or tactical driving behaviours (e.g. braking,

passing, etc.) at certain points in the driving task.

In order to prevent automation-induced performance problems (e.g. Parasuraman and

Riley 1997) in lower-level (operational and tactical) driving behaviours, the reliability and

manner of delivery of navigation information is also critical in the design of in-vehicle

systems. There are few studies that have systematically varied automation reliability

levels in investigating human performance in complex systems control and results are

mixed. Dzindolet et al. (2001) required participants to view slides of battlefield terrain

and to indicate the presence or absence of a camouflaged soldier through the assistance of

an automated decision aid. Their results suggested that operators were insensitive to

differences in automation reliability. Wiegmann et al. (2001) examined the effect of

different levels of, and changes in, automation reliability on user performance with

automated diagnostic aids and found sensitivity to the different levels of aid reliability.

The objectives of the current research were to investigate the potential mediating effect

of in-vehicle navigation aids and aid reliability on the relationship of driver SA (at

multiple levels) with performance in a strategic driving and navigation task.

2. Experiment

2.1. Task

This study used a ‘home-grown’ virtual reality-based driving simulator to present a suburb

navigation and driving task in which drivers were only required to deal with following

traffic (i.e. there was no traffic in front of the user car). The simulation ran on a high-

performance graphics workstation and was presented to users through a stereoscopic

display. Participants viewed the 3-D display of the virtual driving environment and

manually controlled a virtual car (sport truck) to perform the navigation task. User inputs

to the simulation occurred through realistic automobile control interfaces, including a

physical steering wheel and acceleration and brake pedals (see figure 1). The simulation

was limited in fidelity in that it did not provide kinesthetic cues to drivers, as is possible in

full-motion simulators. This is important because it may have reduced driver sense of

vehicle feedback, which has been found to be relevant to speed and turning control

(Walker et al. 2006).

Participants were required to drive from a virtual freeway exit through the suburb and

to arrive at a pre-identified destination. The destination was located four (suburban)

blocks away from the freeway exit. There were three construction sites on the suburb

roadways and participants were required to make five or more turns to navigate the

sites and reach the destination. Figure 2 presents a map of the suburb area with street

names and the destination labelled. There were different types of traffic signs along

the streets, including ‘speed limit’ (30 or 45 mph), ‘stop’ signs and street name signs.

Participants were to obey all traffic signs and follow navigation aiding information

provided by either a human or automation aid without deviating from the directions in

their driving.

In another study, Dzindolet et al. (2002) found that humans exhibited a significant bias

towards the use of automated aids in terms of complex systems control over other human

aids. The automated aids were generally perceived as being more reliable for performance

than a human aid. With this in mind, we also wanted to see if the source of the navigation

aiding information mediated driver achievement of SA and strategic task/navigation

performance. Half of the participants (n¼ 10) received navigation instructions from a

Situation awareness and driving performance 1353

Dow

nloa

ded

By:

[dbk

aber

@eo

s.nc

su.e

du] A

t: 19

:30

11 J

une

2007

Figure 1. Experiment equipment setup.

Figure 2. Paper map of the suburban area with street names and destination.

1354 R. Ma and D. B. Kaber

Dow

nloa

ded

By:

[dbk

aber

@eo

s.nc

su.e

du] A

t: 19

:30

11 J

une

2007

remote experimenter via a Motorola T720 mobile phone (Motorola Inc.). The other half

(n¼ 10) received navigation information from an automated aid presented on an IBM

laptop placed directly adjacent to the primary virtual reality simulation display (see figure

1). In all navigation trials, participants had a paper map, which showed the entire suburb

navigation area. The participants could use the map to assess the reliability of the

navigation aid information or the accuracy of the route information during trials.

2.2. Variables

The independent variables in the experiment included the navigation aid type (i.e. human

or automation), which was manipulated between-subjects, and the level of navigation aid

reliability manipulated as a within-subjects variable, including 100%, 80% and 60%

reliable conditions, as well as a control condition involving a telemarketing survey

delivered through the navigation aids. The navigation aiding provided drivers with

turning information (street names) and speed limit information. All of the information

was correct according to the street signage; however, the overall accuracy of the route

information varied across aid reliability conditions in terms of the number of turns and

the total elapsed driving time. For example, if the minimum number of turns in an

optimal route from the freeway exit to the destination was five, and an aid provided

correct directions for only three of these turns, at least two corrective turns would need to

be added to the optimal route (for a total of seven turns) to reach the destination.

Consequently, the aid would only be 60% reliable in presenting correct turns on the

optimal route. In the experiment, the 100% reliable navigation condition required drivers

to make five turns and they spent an average of 11.7 min in reaching the destination from

the freeway exit. The 80% reliable navigation condition required drivers to make six

turns and average navigation time was 14.2 min. The 60% reliable navigation condition

required seven turns and an average driving time of 15.1 min.

Under the telemarketing survey condition, the paper copy of the map was marked with

an optimal route for navigation. The telemarketing survey was communicated by the

human through the mobile phone or by the automation aid through the laptop display.

Participants were required to answer the survey verbally (questions about shopping in the

Raleigh, North Carolina area) while an experimenter recorded their responses in writing.

The participants assigned to human or automation aiding followed the same trial orders

in terms of exposure to the reliability conditions. They experienced the levels of aid

reliability in descending order, followed by the telemarketing survey condition. This was

necessary to determine if any systematic change occurred in driver SA and performance

as the level of aid reliability degraded. Based on Dzindolet et al.’s (2002) finding, the

authors also wanted to determine if changes in SA with decreasing reliability might be

more pronounced for human aids vs. automated aids.

The dependent variables included driving performance, assessed in terms of the

number of errors drivers made in following the directional advice of a navigation aid and

their consistency in speed control in the suburb area (relative to posted limits). During the

navigation task, drivers were instructed to adhere to the directions of the aids regardless

of whether they knew the aid was unreliable; that is, the route information was

inaccurate. If, for example, the automated aid advised a driver to turn left at an

intersection and the driver turned right, an error count of one was recorded. The mean

number of errors was calculated across all trials for each navigation aid by reliability

condition. This approach was necessary, in part, to ensure that the performance measures

were sensitive. If participants were permitted to drive freely in the virtual suburb, it would

Situation awareness and driving performance 1355

Dow

nloa

ded

By:

[dbk

aber

@eo

s.nc

su.e

du] A

t: 19

:30

11 J

une

2007

not have been possible to assess the accuracy with which they perceived and executed

instructions from the aids, nor would it have been possible to completely assess the effect

of aid reliability on driver SA and performance.

Driver SA was assessed at the end of each trial using an adaptation of the SA global

assessment technique (SAGAT; Endsley 1995b). Nine queries were selected from a larger

database of queries and three questions targeting each level of driver SA (perception,

comprehension and projection) were presented. Participants were asked to recall, for

example, signs they passed during driving, when they passed certain signs or streets, etc.

They were also asked to project the drive time to the destination and possible optimal

routes to reach the destination. (Appendix 1 presents an example set of SA queries posed

to test drivers.) All queries were presented on paper and participants responded in

writing. Their responses were graded on the basis of the ‘ground truth’ of the simulation

recorded by the virtual reality system during each trial. Driver percent correct responses

to all queries were calculated as a measure of overall SA. The percentage of correct

responses to Levels 1, 2 and 3 SA queries were also determined in order to associate

changes in any of the specific levels with navigation performance changes.

One limitation of posing SA queries at the close of trials is that participants may need

to recall specific information on the driving environment from their earlier performance.

Therefore, the measure may be biased by subject recall ability. However, if SAGAT

freezes were used during the navigation trials, as in the original SAGAT methodology

developed by Endsley (1995b), there may have been disruptions to driver navigation

performance, influencing the other response measures. These types of trade-offs are

typical in the use of direct, objective measures or SA, such as SAGAT, and are often

evaluated based on the dynamics of the task under study (e.g. Kaber et al. 2005).

2.3. Participants and procedures

A total of 20 students were recruited for the study from the North Carolina State

University student population and they were required to have at least 3 years of driving

experience to participate. The average age of participants was 28.1 years and the sample

had an average of 8.5 years of driving experience.

Before participants began the test trials, they were provided with training on how to

control the virtual car in the simulator and how to maintain the vehicle on the virtual

roadway using the physical steering wheel, accelerator and brake pedals. In general, the

responsiveness of the simulator was very much like that of a real vehicle. The training

simulation environment presented no traffic signs or street name signs. Subsequently,

participants were provided with a practice session in which the simulation environment

remained the same. Drivers were shown the paper map of the suburb in advance and they

were informed of the location of the driving destination. They had access to the paper map

throughout the practice session. At the end of the suburb navigation, participants were

required to answer example SA questions. The practice session was intended to account

for potential roadway learning effects across the navigation aid reliability conditions (the

within-subjects variable settings). However, it is possible that some additional learning of

the simulated roadways may have occurred during the actual experiment.

Participants completed four test trials. They were instructed that the task was to

involve navigation under normal driving circumstances and had no reason to expect

hazardous conditions or otherwise. During the navigation task, the aids presented

information to drivers as they approached each street intersection or decision point.

A ‘wizard-of-oz’ technique was used, in which an experimenter, observing participant

1356 R. Ma and D. B. Kaber

Dow

nloa

ded

By:

[dbk

aber

@eo

s.nc

su.e

du] A

t: 19

:30

11 J

une

2007

driving performance on a separate remote monitor, called the participant on the mobile

phone, or the experimenter controlled the automated navigation aid display to deliver

specific driving direction information at the ‘right’ time. Figure 3 presents an example

script read by the human aid and an example automated aid display. There was a total of

six to eight mobile phone calls or six to eight automated aid display changes (depending

upon the navigation aid reliability) during each test trial. Participants were informed in

advance of the trials that the aids might not be completely reliable in the information

presented, but they were not informed of the specific reliability levels.

2.4. Detailed hypotheses

Higher reliability navigation aiding was expected to facilitate better driving performance.

It was also expected that presentation of the telemarketing survey (task-irrelevant

information) would significantly degrade driver performance, as compared to driving

with the human or automation navigation aiding. It was expected that there would be

higher SA scores with perfectly reliable guidance from the human advisor or automation

aid than with 80% and 60% reliable information, particularly for higher-level SA (e.g.

projection of routes).

Opposite to the above expectations for performance and SA improvements, the mobile

phone conversation with the human aid was expected to distract driver attention from the

driving environment and to degrade lower-level SA and performance. However, the

accessibility and persistence of visual information in the automated aid condition was

expected to support lower-level driver SA and navigation and speed control.

3. Results

3.1. Driving performance

Multivariate ANOVA (MANOVA) results revealed a significant effect (F(6,106)¼ 3.99,

p5 0.05) of the navigation aid reliability on the collection of performance measures.

There was no significant main effect of the navigation aid type and no interaction effect of

aid type and reliability level.

Figure 3. Example navigation instructions by human aid and automation aid display.

Situation awareness and driving performance 1357

Dow

nloa

ded

By:

[dbk

aber

@eo

s.nc

su.e

du] A

t: 19

:30

11 J

une

2007

ANOVA results revealed a significant effect of aid reliability on driving errors

(F(3,79)¼ 5.00, p5 0.01). Figure 4 presents the mean driving errors across the various

reliability conditions. (It is important to note that in many trials, participants committed

no navigation errors and, therefore, the condition means were less than 1.) In agreement

with the hypothesis, Tukey’s test revealed significantly lower errors (p5 0.05) when

participants received 100% reliable aiding, as compared to all other conditions.

ANOVA results revealed a significant effect of navigation aid reliability on variations

in driver speed control (F(3,79)¼ 3.49, p¼ 0.0217). Figure 5 presents the root mean

square error of speed control for the settings of reliability. Tukey’s test revealed

significantly greater deviations in speed (p5 0.05) when participants performed under the

control condition (i.e. telemarketing survey), as compared to the 80% and 60% reliable

aiding conditions.

3.2. Driver situation awareness

MANOVA results revealed a significant effect (F(12,135)¼ 5.01, p5 0.01) of the

navigation aid reliability on SA measures. There was no significant main effect of the

Figure 4. Driving errors across various navigation aid reliability conditions.

Figure 5. Root mean square error (RMSE) of speed control across various navigation aid

reliability conditions.

1358 R. Ma and D. B. Kaber

Dow

nloa

ded

By:

[dbk

aber

@eo

s.nc

su.e

du] A

t: 19

:30

11 J

une

2007

navigation aid type and no interaction effect. Figure 6 presents the mean Level 1, Level 2,

Level 3 and overall SA scores for the various navigation aid reliability conditions. As was

expected, the plot reveals that, on average, drivers exhibited better overall SA when aid

information was 100% reliable and the worst SA when participants were required to

address the telemarketing survey. Exceptions included Level 1 SA for the 100% reliable

condition and Level 2 SA for the 60% reliable condition. (This is addressed below.)

ANOVA results revealed a significant effect of navigation aid reliability (F(3,79)¼7.19, p5 0.01) on the percentage of correct responses to Level 1 SA queries. As was

hypothesized, Tukey’s tests revealed significantly higher (p5 0.05) perceptual knowledge

of the driving environment with the 60% and 80% reliable aiding compared to the

control condition; however, the worst perceptual knowledge occurred with the 100%

reliable aiding condition.

ANOVA results also revealed a significant effect of navigation aid reliability

(F(3,79)¼ 16.78, p5 0.01) on Level 2 SA. According to Tukey’s tests, the 100% reliable

navigation aiding produced significantly greater (p5 0.05) percent correct responses to

level 2 SA queries in comparison to all other conditions. The 80% navigation aid

reliability also produced significantly greater (p5 0.05) Level 2 SA compared to the

control condition and the 60% aid reliability condition.

Finally, ANOVA results also revealed a significant effect of navigation aid reliability

(F(3,79)¼ 3.17, p5 0.05) on Level 3 SA (projection of states of the driving environment).

Tukey’s test revealed significantly higher (p5 0.05) Level 3 SA for all navigation aid

reliability levels, as compared to the control condition. Any navigation aiding appeared

to support route planning and strategic behaviour better than none.

3.3. Correlation analyses

Simple correlation analyses were also conducted to identify any significant relationships

among driving performance and SA. A Pearson coefficient revealed a significant negative

linear association between Level 3 SA scores and driving navigation errors (r¼70.25,

p5 0.05) (i.e. as Level 3 SA increased, errors decreased). This finding was in line with

Matthews et al. (2001) theoretical linkage of Level 3 SA with strategic driving behaviour.

Figure 6. Mean percentage of correct responses to situation awareness (SA) queries for

navigation aid reliability conditions.

Situation awareness and driving performance 1359

Dow

nloa

ded

By:

[dbk

aber

@eo

s.nc

su.e

du] A

t: 19

:30

11 J

une

2007

4. Discussion

The automation and human aids appeared to cause comparable levels of distraction on

driver navigation (i.e. there were no significant differences among aid types in terms of

performance). This observation was counter to the findings of Dzindolet et al. (2002);

however, their focus was primarily on human trust in automated aids and not resulting

SA. The automation aid in this study demanded driver visual attention (Ho et al. 2006) for

perception of navigation information. The human aid may have required driver working

memory for storing verbal stimuli (the route instructions and speed limits). Driver

distraction due to mobile phone use and visual attention allocation to the laptop display

also appeared to have comparable influences on driver ability in terms of achieving SA.

Again, there were no significant differences in SA among the aid types; however, SA

accuracy across all reliability levels was far from perfect (32% at the worst and 74% at the

best). Since driving is a visual and motor control process, it is possible that the visual

search demands associated with retrieving information from the automation-aid display

also degraded driver SA and performance comparable to the impact of human use of the

mobile phone. Relevant to this, some other research has provided evidence that mobile

phone use, while driving, significantly degrades SA, particularly Levels 2 and 3 (Gugerty

et al. 2003, Ma 2006, Ma and Kaber 2005). The study by Gugerty et al. found that SA was

significantly degraded when performing a driving task while talking with a partner as

compared to only driving a car. SA was assessed in the study by using location-recall

probes requiring participants to indicate the locations of surrounding traffic.

Like Wiegmann et al. (2001), this study found that human performance in the driving

domain was generally sensitive to different levels of automation aid reliability. The

systematic decrease in the reliability of in-vehicle navigation aiding appeared to subtract

from driver attention to the roadway. This had a negative influence on driver SA and,

possibly, driving performance in terms of strategic behaviours. Driver Level 1 SA was

particularly low under the perfect aiding condition. It is possible that drivers realized the

human advisor or automated aid was highly accurate in directions under this condition

and, as a result, they did not pay as much attention to observing aspects of the roadway.

This potential behaviour was also evidenced in comparison to the 80% and 60% reliable

aiding results. Participants appeared to pay close attention to the driving environment

when they were provided with imperfect aiding. In general, these results revealed that

higher aid reliability produced higher driver SA (Levels 2, 3 and overall SA). For example,

under perfect aiding, of the information in the driving environment that participants

perceived, their relation of this information to driving goals was relatively accurate (74%).

Most importantly, this study provided empirical evidence of linkages of specific

(operational and strategic) driving behaviours to the three levels of SA. From the

experiment and correlation results, it appears that driver SA was significantly affected by

aid reliability, and driver performance was significantly correlated with changes in high-

level SA. That is, the characteristics of in-vehicle navigation aids have a mediating effect

on driver SA, which appears relevant to types of driving behaviour. It is important to

note that the results summarized here apply to idealized driving situations with limited

interactive traffic (no on-coming or overtaking vehicles).The correlation analyses for the

current study, and from Ma and Kaber (2005), revealed positive associations between SA

with one or more dimensions of driving performance. These additional findings

demonstrate the importance of the cognitive construct to driving. Additional research

is needed to further indicate exactly how each mediating factor (i.e. form of in-vehicle

automation) influences driver SA. For example, these findings suggest that perfect

1360 R. Ma and D. B. Kaber

Dow

nloa

ded

By:

[dbk

aber

@eo

s.nc

su.e

du] A

t: 19

:30

11 J

une

2007

navigation aid reliability could negatively influence Level 1 SA in navigation driving tasks

by motivating driver complacency with regard to perceiving changing states of the

roadway environment. Of course, it would not be desirable to present incorrect

navigation information to drivers in real-world systems, but methods for ensuring driver

attention to the roadway with reliable aids should be incorporated in navigation aid

design, such as periodic text or verbal message reminders.

5. Conclusions

The effect of in-vehicle navigation aid use and reliability in driving tasks on driver SA and

performance has not been previously assessed. This experimental study provided some

explanation of the mediating influence of in-vehicle automation (e.g. ACC and navigation

aids) and devices (e.g. mobile phones) on driving task performance. This said, the results

of the study should be applied with caution for making decisions about the design or use

of in-vehicle automation. The experiment was conducted with a ‘home-grown’ driving

simulation, including a fixed-base driving platform providing no kinesthetic motion cues

and there was no interactive traffic represented in the simulation runs. Drivers may (and

likely do) behave differently in actual operational settings because of the serious

consequences of having an accident. Related to this limitation, there is a need to use

highly realistic and complex driving tasks to further investigate the relationship of driver

SA with performance as a basis for future systems design. Another caveat of the study is

related to the adaptation of the SAGAT measure. Based on the training session, all

participants knew SA queries were to occur at the end of the navigation trials and

they may have taken advantage of this knowledge to prepare for quizzes. The experiment

also presented the aiding reliability conditions in the order 100%, 80% and 60%,

followed by the control condition (telemarketing survey). Although participants were

thoroughly trained on the virtual driving environment, this systematic manipulation of

aid reliability may have led to a trial order effect for the dependent variables.

Unfortunately, there were insufficient degrees of freedom provided by the datasets to

make statistical assessment of this potential issue with a full ANOVA model in aid type

and aid reliability.

The authors believe there remains a need to provide empirical evidence of the role of

driver SA in tactical driving behaviour (e.g. changing lanes, overtaking, negotiating

intersections, etc.) and to identify other mediating individual, technological and driving

system factors. Unfortunately, other recent investigations of the effects of in-vehicle

automation (e.g. ACC) on driver SA, such as the study by Stanton and Young (2005),

have not been able to confirm potential relationships between levels of driver SA

(perception, comprehension and projection) and tactical driver behaviour/action, because

they have used indirect (performance-based) measures of SA, or subjective rating

techniques, which do not assess each level of the theoretical construct, as the SAGAT

measure does. Finally, from a design perspective, the results of the present experiment

provide support for the use of in-vehicle navigation aiding under normal driving

conditions for facilitating driver SA. It is possible that the use of in-vehicle automation in

complex driving tasks, involving unexpected or hazardous conditions, may have different

effects on driver performance and SA. Additional mental resources may be required of

drivers under non-normal or hazardous conditions. This factor and other individual

variables, such as age and perceptuo-cognitive abilities, might lead to a different set of

results. Future work should investigate the effects of navigation aids under non-normal

or critical/hazardous driving conditions on driver SA and performance.

Situation awareness and driving performance 1361

Dow

nloa

ded

By:

[dbk

aber

@eo

s.nc

su.e

du] A

t: 19

:30

11 J

une

2007

Acknowledgements

The first author worked as a research assistant in the Edward P. Fitts Department of

Industrial & Systems Engineering at North Carolina State University during the

completion of this research. The authors would like to thank Xuezhong Wang for his

assistance in building all the driving navigation simulation models. We thank Sang-Hwan

Kim, who assisted with the experimental data collection.

References

DZINDOLET, M.T., PIERCE, L.G., BECK, H.P. and DAWE, L.A., 2002, The perceived utility of human and

automated aids in a visual detection task. Human Factors, 44, 79–94.

DZINDOLET, M.T., PIERCE, L.G., BECK, H.P., DAWE, L.A. and ANDERSON, B.W., 2001, Predicting misuse and

disuse of combat identification systems. Military Psychology, 13, 147–164.

ENDSLEY, M.R., 1995a, Toward a theory of situation awareness in dynamic systems. Human Factors, 37, 32–64.

ENDSLEY, M.R., 1995b, Measurement of situation awareness in dynamic systems. Human Factors, 37, 65–84.

GUGERTY, L.J., RANDO, C., RAKAUSKAS, M., BROOKS, J. and OLSON, H., 2003, Differences in remote versus in-

person communications while performing a driving task. Ergonomics Society of Kora: Seoul. In Proceedings

of XVth Triennial Congress of the International Ergonomics Association Conference (CD-ROM), 24–29 August

2003, Seoul, Korea.

HO, C., HONG, Z., TAN, H.Z. and SPENCE, C., 2006, The differential effect of vibrotactile and auditory cues on

visual spatial attention. Ergonomics, 49, 724–738.

KABER, D.B., RILEY, J.M., LAMPTON, D. and ENDSLEY, M.R., 2005, Measuring situation awareness in a virtual

urban environment for dismounted infantry training. In Proceedings of the 11th International Conference on

Human-computer Interaction, vol. 9 (Mahwah, NJ: Lawrence Erlbaum & Assoc).

MA, R., 2006, The effect of in-vehicle automation and reliability on driver situation awareness and trust.

ProQuest UMI.

MA, R. and KABER, D.B., 2005, Situation awareness and workload in driving while using adaptive cruise control

and a cell phone. International Journal of Industrial Ergonomics, 35, 939–953.

MATTHEWS, M.L., BRYANT, D.J., WEBB, R.D. and HARBLUK, J.L., 2001, Model for situation awareness and

driving. Transportation Research Record, 1779, 26–32.

PARASURAMAN, P. and RILEY, V., 1997, Humans and automation: use, misuse, disuse, abuse. Human Factors, 39,

230–253.

STANTON, N.A. and YOUNG, M.S., 2005, Driver behavior with adaptive cruise control. Ergonomics, 48, 1294–

1313.

WALKER, G.H., STANTON, N.A. and YOUNG, M.S., 2006, The ironies of vehicle feedback in car design.

Ergonomics, 49, 161–179.

WARD, N.J., 2000, Automation of task processed: an example of intelligent transportation systems. Human

Factors and Ergonomics in Manufacturing, 10, 395–408.

WIEGMANN, D.A., RICH, A. and ZHANG, H., 2001, Automated diagnostic aids: The effects of aid reliability on

user’s trust and reliance. Theoretical Issues in Ergonomics Science, 2, 352–367.

Appendix 1. An example set of situation awareness queries posed to drivers

at the close of navigation

1. What was the last road sign you saw? (L1)

Pedestrian crossing

Deer crossing

Railroad

Speed limit

Slow sign

Stop sign

I don’t know

1362 R. Ma and D. B. Kaber

Dow

nloa

ded

By:

[dbk

aber

@eo

s.nc

su.e

du] A

t: 19

:30

11 J

une

2007

2. What was the colour of the vehicle behind your car when the simulation stopped?

(L1)

No car

Grey

White

I don’t know

3. What was your vehicle speed (mph) at the time the simulation stopped? (L1)

Less than 25

25–30

30–35

35–40

40–45

45–50

More than 50

4. How long has it been since you passed the last turn in navigating the city suburb? (L2)

Less than 30 s

30 s–1 min

1–1.5 min

1.5–2 min

2–2.5 min

2.5–3 min

More than 3 min

5. How long has it been since you passed the last road sign? (L2)

1–5 s

5–10 s

10–15 s

15–20 s

20–25 s

More than 25 s

6. In which direction from your vehicle was your destination (building) located when

you passed the last turn in the city suburb? (L2)

On the left

On the right

Right in front of me

Behind me

I don’t know

7. How long did you think it would take for your vehicle to reach the destination in the

simulation after you passed the last road sign? (L3)

No sign in sight

1–5 s

5–10 s

10–15 s

15–20 s

More than 20 s

Situation awareness and driving performance 1363

Dow

nloa

ded

By:

[dbk

aber

@eo

s.nc

su.e

du] A

t: 19

:30

11 J

une

2007

8. What was the optimal navigation route to reach your destination when you passed

the intersection of Kaber St. and Ma St. (what route would have generated the

shortest drive time)? (L3)

Go to Ma St., then Riley Rd, then McDowell St

Go to Kaber St., then Noa Dr., then Riley Rd., then McDowell St.

Go to Ma St., then Avent Ferry St.

I don’t know

9. How long would the time have been from the intersection of Kaber St. and Ma St.

until you finished driving through the suburban area on the ‘optimal’ route? (L3)

Less than 1 min

1–2 min

2–3 min

3–4 min

4–5 min

More than 5 min

1364 R. Ma and D. B. Kaber