Page 1
- 1 -
ABSTRACT
Ubiquitous computing environments continuously infer our context and proactively offer
us context aware services and information, suggested by notifications on our mobile
devices. However, notifications come with a cost. They may interrupt the user in the
current task and be annoying in the wrong context. The challenge becomes how to notify
the user about the availability of relevant services while minimizing the level of
disruptiveness. Thus, an understanding of what affects the subjective perception of the
disruptiveness of the notification in a mobile context is needed. As yet, most of the
research on disruptiveness of notifications focused on stationary, task-oriented
environments. In this study, we examine the effect of notifications in a museum visit
scenario. In two user studies conducted in a museum setting, participants used a context-
aware mobile museum guide to receive information on various museum exhibits while
periodically receiving notifications. We examined how the user’s activity, the modality of
the notification, and the message content affected the perceived level of disruption that
the notifications created. Results indicate that the perceived level of disruption was linked
to the user's activity at the time of interruption and the type of information delivered, and
was also affected by the way the notification was presented.
Page 2
- 2 -
CONTENTS
1. INTRODUCTION
2. BACKGROUND
2.1. Factors that influence interruptability in a desktop environment
2.2. Subjective characteristics of the notification message
2.3. Mobile notifications
2.4. Notifications in the museum environment
3. THE CURRENT STUDY
3.1. The museum environment
4. STUDY 1 - THE EFFECT OF USER ACTIVITY AND NOTIFICATION TYPE ON
THE PERCEIVED LEVEL OF INTERRUPTION
4.1. Study design
4.2. Hypotheses
4.3. Participants
4.4. Procedure
4.5. Results
4.5.1. Visual notifications
4.5.2. Modality of the notification
4.5.3. User activity
5. STUDY 2 - THE EFFECT OF THE MESSAGE CONTENT ON THE PERCEIVED
LEVEL OF INTERRUPTION
5.1. Study design
5.2. Hypotheses
5.3. Participants
5.4. Procedure
5.5. Results
5.5.1. Relevance and importance
5.5.2. Message types
5.5.3. User activity
6. DISCUSSION
6.1. Guidelines for museum curators and mobile guide designers
6.2. Generalizing the results
7. CONCLUSIONS
8. REFERENCES
Page 3
- 3 -
1. INTRODUCTION
Context-aware mobile systems react to their environment and use contextual aspects such
as location to suggest relevant information or services to the user (Dey, 2001). The
services and information are often provided proactively where the system anticipates the
user’s needs and accordingly initiates the information presentation. However, proactively
suggesting information requires the system to interrupt the user in his or her current
activity. This interruption might be annoying to the user who may not wish to be
disrupted at the current time. Thus, intelligent context-aware systems aim at delivering
the right notifications at the right time and in the right way in order to maximize the
possible benefits while minimizing the possible cost of the notification to the user.
An intelligent mobile museum visitors’ guide is a specific case of a context-aware mobile
system. Museum visitors move in the museum, looking for interesting exhibits, and wish
to acquire information to deepen their knowledge and satisfy their interests. A smart
context-aware mobile guide may provide the visitor with personalized relevant
information from the vast amount of content available at the museum, adapted for his or
her personal needs. Furthermore, the system may notify museum visitors of events,
provide recommendations, location-relevant information, or deliver messages from other
visitors. For example, a smart museum visitors’ guide system might anticipate the
visitor’s interests and availability and suggest attending a lecture that is just about to start
on the other side of the museum. However, to offer such services, the system must
interrupt the visitor to notify him or her of the service or information, hence, this
potential benefit may also have a cost.
We examine notifications in a museum context for users wandering around the museum
while using a mobile museum guide. According to McCrickard and Chewar (2003),
notification systems are “designed interfaces that are typically used in a divided-attention,
multi-tasking situation, attempting to deliver current, valued information through a
variety of platforms and modes in an efficient and effective manner". We use Bailey,
Konstan, and Carlis’s (2001) definition of interruption as providing information that is
useful or of interest to the user, but not necessarily related to the user's current task. Thus,
in a museum context, the notification system delivers information to the visitor through
interruptions on the mobile museum guide. In the context of this paper, we use the terms
notification and interruption to describe a text message arriving to the user while the user
is engaged in something else. We use the term disruption as a negative implication of
interruption.
In this work, we examine the factors that affect the disruptiveness of notifications in a
museum visit scenario with the main goal of better understanding how to design museum
notification systems. Many factors may influence the possible cost of the notification.
These factors include the current activity of the user, the emotional state of the user, the
modality of interruption, the utility of the notification and more (Ho and Intille, 2005). To
date, most of the works examining these factors were done in a static, desktop
environment. Monitoring the user’s current activity was used to determine when to
interrupt the user in order to defer the notification to a preferred time (Czerwinski, Cutrell
Page 4
- 4 -
& Horvitz, 2000a; Iqbal & Bailey, 2005). It has been suggested that scheduling
notifications at subtasks’ boundaries or breakpoints would reduce the cost of interruption
(Bailey & Constan, 2006; Iqbal & Bailey, 2006). This is based on the notion that a
notification would be less disruptive if delivered when there is a lower mental workload
(Miyata and Norman, 1986; Bailey & Iqbal, 2008). As to how to interrupt the user, there
is a tradeoff between increasing a notification’s noticeability and lowering its
disruptiveness level (McCrickard et al., 2003). The effect of different modalities on the
disruptiveness of the notification has been measured (Arroyo, Selker & Stouffs, 2002). It
has been shown that the interruption presentation format and modality affects the way it
is perceived by the users (Latorella, 1998). Still, it mostly remains unclear which
modality is less disruptive under which circumstance (Warnock, McGee-Lennon &
Brewster, 2011; Arroyo, Selker & Stouffs, 2002). As to what content is delivered in the
notification, it has been shown that the perceived utility of the message has an influence
on the way the interruption is perceived by users (Gluck, Bunt & McGrenere, 2007). For
example, a look at Instant Messages and how they interrupt users working on typical
desktop tasks revealed that relevant messages were perceived as less disruptive than
irrelevant ones (Czerwinski, Cutrell & Horvitz, 2000a).
These research results and others have been used as input for devising models that
determine when and how to apply notifications (Iqbal & Bailey, 2007; Iqbal & Horvitz,
2007; Horvitz et al., 2003). However, most of the studies that examined the factors, costs,
and effects of notifications as mentioned above, assumed goal-directed, structured tasks
(e.g., video editing, route planning, word processing , etc.) on a desktop environment.
Furthermore, most of these studies were conducted in a controlled laboratory
environment (i.e., participants sat in front of a desktop computer, and performed a
primary task, which was then interrupted). A museum offers a different environment with
different user goals, needs, and reactions. In a mobile museum scenario, the users are
moving around (navigating), paying attention to exhibits, and interacting with the
surroundings and with their companions (Falk, 2009). They also interact with the
computing device, but it acts more as a periodical assistant than the continuous focus of
interaction. Assumptions and conclusions made by looking at a work-oriented desktop
setting may not necessarily apply in such a setting.
In the current study setting, participants were wandering around a museum, receiving
contextual information on a smart mobile museum guide. In two user studies, we
examined factors affecting the perceived level of interruption for users’ visiting a
museum and receiving notifications, varying when, how, and what type of notification
message was received. The contribution of this work is first and foremost a framework
for the use of notification with a mobile guide at a museum scenario. We apply
understanding gained in the desktop environment and examine whether and how they
apply in a mobile museum scenario. Second, we discuss the possibility to generalize our
understanding to other context-aware mobile environments.
The rest of this work is organized as follows. In Section 2, the background and related
work are presented. In Section 3, we describe the current study setting including the
museum environment in which the user studies were conducted. In Section 4, we describe
the first user study examining how the visitor’ activity, as well as how the modality of the
Page 5
- 5 -
notification, affects its perceived level of interruption. In Section 5, we describe the
second user study, which examined more closely how the content of the message affects
the way it disrupts the visitor. In Section 6, we discuss the results, including the
implications of these two user studies for museum curators and mobile guide developers,
generalization of the results and limitations. Finally, in Section 7, we present our
conclusions and directions for future research.
2. BACKGROUND
This section details the background for our work. To get an understanding of how to
design interruptions in a mobile guide context, we first take a look at the more established
area of desktop computing interruptions. Thus, we first examine the common strategies
and understandings that were used in desktops to reduce the disruptiveness of
notifications, focusing on when to interrupt the user and how (through which modality) to
interrupt the user. Next we look at works that examined how the content of the
notification message (what) affects its acceptability in the desktop environment. Next, we
look at studies that tried to extend these works and examined notifications in a mobile,
rather than a desktop environment. Finally, we first describe existing work on
notifications with mobile museum guides. We show that while notifications are
commonly used in many mobile museum guide systems, no systematic approach has
been taken to examine the implications of how these notifications affect museum visitors.
2.1 Factors that influence interruptability in a desktop environment
While very few studies explicitly examined the interruptability of notifications with a
mobile guide, there is a wealth of information on notifications and interruptions in
desktop computing. In this section we survey studies that examined the various factors
that influence the disruption of a notification in a desktop environment. Our purpose is to
apply the knowledge gained on disruption within desktop tasks, in a mobile museum
environment and to examine where and how the disruption of the notification differs in a
mobile environment.
Many studies looked at the conflict between the usefulness and potential benefits of
notifications and their disruptiveness in a desktop environment (Hudson et al., 2002). On
the benefits side, notifications may provide significant value to the users, and are often
essential in communication in the workspace and other areas (O’Conaill & Frohlich,
1995). People are willing to accept some distraction in exchange for valued information.
For example, Managers reported the need to receive immediate unplanned items and
important news to accomplish higher level goals (Hudson et al., 2002). Other studies
showed that users can make better decisions and accomplish more when interrupted with
helpful information (McFarlane, 1999). Furthermore, in proactive systems, users often
agree to receive interrupting messages, given that they provide them with useful
information or services (Billsus, Hilbert & Maynes-Aminzade, 2005). On the flip side,
notifications do come with a cost. Studies have shown that notification of an incoming
message, even when the message is ignored, disrupts the current task performance
(Cutrell, Czerwinsky & Horvitz, 2001; Gillie & Broadbent, 1989; Latorella 1998). This is
reflected in slower task completion times (Bailey, Konstan & Carlis, 2001; Czerwinski,
Page 6
- 6 -
Cutrell & Horvitz, 2000b; McFarlane, 1999), more errors (Kreifeldt & McCarthy 1981),
and worse decision making (Speier, Valacich & Vessey, 1999). Interruptions may also
affect users’ emotional state (Bailey, Konstan & Carlis, 2001), causing people to
experience more frustration, annoyance, and anxiety (Adamczyk & Bailey, 2004; Bailey,
Konstan & Carlis, 2001; Zijlstra et al., 1999). For example, Bailey, Konstan and Carlis
(2001) conducted an experiment to measure the effects of interruptions on users’ task
performance, and users’ emotional state. Their findings indicated that interruptions
caused users to perform the interrupted tasks more slowly, which was also reflected in the
user’s perception of the interrupted task being more difficult. Furthermore, they found
that the user's level of annoyance depended on the category of the current task and the
time when the interruption occurred. Finally, they found that users experienced a greater
increase in anxiety during interruptions.
As shown above, the costs resulting from interruptions are a major concern for the
research community. Thus, factors that influence the interruptability of notifications are
investigated in order to devise strategies to reduce their level of disruption. One of the
major factors that influence the perceived level of interruption, and one that can be fairly
well controlled, is the timing of the notification (Adamczyk & Bailey, 2004; Ho & Intille,
2005; Iqbal et al., 2005; Iqbal & Bailey, 2006; Fogarty, Hudson & Lai, 2004). Some
studies tried to define moments for interruptions, suggesting that interruptions should
appear between instances of repetitive action sequences, or at break points in a task
sequence (Fischer, Greenhalgh & Benford, 2011; Iqbal & Bailey, 2007; Monk, Boehm-
Davis & Trafton, 2002). Czerwinski, Cutrell & Horvitz (2000b) described task execution
in three phases: planning, execution, and evaluation. They suggested placing moments for
interruption toward the beginning, middle, or end of a task. Similarly, Bailey, Konstan
and Carlis (2001) suggested that task boundaries are an opportune moment, since these
are times of low memory-load within a user’s task sequence. This is in line with the
suggestion of Miyata and Norman (1986), which noted that moments of lower mental
workload occur between the evaluation of one subtask and the beginning of the next, like
a subtask boundary. Other studies also examined the cognitive load of users as a way to
find an opportune moment for an interruption (Adamczyk & Bailey, 2004; Bailey,
Konstan & Carlis, 2001; Cutrell, Czerwinsky & Horvitz, 2001; Czerwinski, Cutrell &
Horvitz, 2000a; Czerwinski, Cutrell & Horvitz, 2000b; McFarlane, 1999; Speier,
Valacich &Vessey, 1999; Gillie & Broadbent, 1989).
The perceived level of interruption may also be affected by the way the notification is
presented to the user. Researchers looked at using novel visual strategies (Van Dantzich
et al., 2002), as well as using multimodal notification methods (Brewster Wright &
Edwards, 1994; Bailey, Konstan, & Carlis, 2000; Maglio & Campbell, 2000; Somervell,
Chewar & McCrickard, 2002; Arroyo Selker & Stouffs, 2002) to reduce the perceived
disruption levels. The way the notification is presented affects the effectiveness of the
notification as can be measured using response time. A large, blinking icon will be more
noticeable than a small icon on the side of the screen. Warnock, McGee-Lennon and
Brewster (2011) compared different modalities of interruption (visual, audio, tactile, and
olfactory) and found that olfactory notifications showed the longest response time (and
thus were least effective) followed by tactile notifications. Visual and audio notifications
took around the same time and were fastest. In the same study, no difference was found
Page 7
- 7 -
for the modality in terms of the amount of disruptiveness of the notification. Arroyo,
Selker and Stouffs (2002) examined and compared five different modalities (heat, smell,
sound, vibration, and light) as a way to deliver interruptions. They examined which
modality is most effective and least disruptive. They found large individual differences
between participants with no significant differences between the modalities. Other studies
tried to compare interruption modalities in home care, office, and flight deck
environments (Warnock, McGee-Lennon & Brewster, 2011; Latorella, 1998; Arroyo &
Selker, 2003). Again, most of the studies did not find significant differences between the
modalities with respect to the levels of disruption. Thus, no guidelines or models exist
that can help developers choose the appropriate notification modality for a given situation
(Warnock, McGee-Lennon & Brewster, 2011).
2.2 Subjective characteristics of the notification message
In the current research, we focus on interruptions in the form of text notifications that are
simply read, after which the previous task is resumed. Assuming the interruption’s
purpose is to convey a message, the perceived utility of the message has an influence on
the perceived level of its disruptiveness. The perceived utility of a message can be
defined as how important, relevant, and urgent the content is to the recipient (Gluck, Bunt
& McGrenere, 2007). Dabbish and Baker (2003) claimed that the importance of a
message should be considered when deciding when and how to show the notification.
Based on interviews, they developed a model that considers the identity of the interrupter
together with the importance of the message to decide whether to allow or disallow an
interruption. Obermayer and Nugent (2000) recommended that systems present messages
that are more important using notification signals with a high attentional draw (i.e., that
are noticed immediately), while presenting less important messages more subtly so that
they will be noticed only during a natural break. Gluck, Bunt and McGrenere (2007)
examined this guideline and measured the attentional draw of different visual interruption
methods. They suited the level of the attentional draw of a method to the importance and
utility of the interruption and showed that this decreased the users' perceived annoyance
and increased their perception of benefit. Looking at the relevance of a message,
Czerwinski, Cutrell and Horvitz (2000a) found that Instant Messages that are relevant to
the users’ current task were perceived to be less disruptive than irrelevant messages.
Irrelevant messages took longer to process and it was more difficult to reestablish task
context following the interruption. They concluded that interruptions that are relevant to
the ongoing tasks are less disruptive than those that are irrelevant. Finally, the urgency of
the message may also affect the way it is perceived (Szóstek & Markopoulos, 2006;
Obermayer & Nugent, 2000). For example, a study conducted in a home setting found
that the perceived message urgency is a primary indicator of acceptability of
notifications. If people thought the message was urgent, they wanted to receive the
message immediately (Vastenburg, Keyson & de-Ridder, 2008).
2.3 Mobile notifications
Very few studies have looked at what factors affect notifications’ perceived
disruptiveness in a mobile setting, and those that did, focused mostly on the timing of
delivering the notifications. Fischer, Greenhalgh and Benford (2011) looked at opportune
Page 8
- 8 -
moments to deliver notifications for mobile guides. They found that interruptions were
attended to more quickly when the user had finished an episode of a mobile interaction
(they examined voice calls and text messaging) compared to a random time. However,
the timing of the interruption (random or at the end of a mobile interaction episode) did
not affect the perceived level of disruptiveness or how appropriate participants rated the
timing of the notification. Similarly, Ho and Intille (2005) performed a study to examine
interruptions in a context-aware mobile setting. Their assumption was that notifications
arriving when the user is transitioning between physical activities will be less disruptive,
because physical transitions correlate with task switches. Using wireless accelerometers
to measure the amount of activity of the users, they showed that users were more
receptive to interruptions during physical transitions (between sitting and standing and
between sitting and walking) than at random times. However, most of their study
participants were office workers who sat most of the day in front of a computer. In such a
setting, it makes sense that a notification received when the person gets up and walks for
a few seconds is less disruptive than when it is received during work at the desk. In
another field study, this time in the home setting, Vastenburg, Keyson and de Ridder
(2008) presented various notifications at random times to participants in their own living
room. Upon receiving the notification, participants were asked to state their current
activity as well as other factors, including the acceptability of the notification. Unlike Ho
and Intille (2005), the researchers in this case found no effect of physical activity on the
acceptability of notifications. This might be because here, physical activity was not
correlated with task switches. Finally, a study by Picard and Liu (2004) examined the
connection between interruption and stress in a mobile everyday setting. They used
inputs from an accelerometer, a heart rate monitor, and a pedometer to trigger
interruptions on a mobile device at “non-stressful” moments. Subjects in the study were
more receptive to interruptions when the system was emotionally friendly and was
triggered during non-stressful activities.
2.4 Notifications in the museum environment
People usually come to the museum for a social, leisure activity that can provide a
meaningful learning experience (Falk, 2001). In such situations, they are open to explore
and engage in new experiences and thus are usually more susceptible to using novel
technologies (Gammon and Burch, 2008). Museum visitors’ behavior has been widely
researched, including also the impact of mobile technology on the visit experience
(Bowen et al., 2008; Lanir et al, 2013). Falk (2009) suggested the Identity-related
Museum Visit Experience Model, which contains a typology of five visitor-identity
prototypes: (i) Explorer, whose visit to the museum is motivated by curiosity or general
interest in discovering more about the subject matter introduced by the museum; (ii)
Experience Seeker, often a tourist, whose visit is typically motivated by the main
attraction the museum is known to offer; (iii) Professional/Hobbyist, who is interested in
specific topics out of the full collection of the museum; (iv) Recharger, who comes to the
museum to reflect and rejuvenate, or to relax and absorb the atmosphere; and finally, (v)
Facilitator, who visits the museum to satisfy the needs and desires of someone else.
Furthermore, Falk (2009) suggests a museum visit model that has a social, personal and
environment context. All in all, unlike other leisure activities such as a sports event or
watching a movie or theatre, where there is specific focus of attention, visitors in the
Page 9
- 9 -
museum are engaged in various aspects of the visit and are possibly more open to
receiving information and services via notifications.
There has been a vast amount of research conducted so far on applying novel
technologies, specifically a mobile museum guide, at the museum as can be seen in
several surveys (Ardissono, Kuflik, & Petrelli, 2012; Kenteris, Gavalas, & Economou,
2011; Kray & Baus, 2003). Much of the research focused on information delivery,
navigation, personalization, novel interaction modalities as well as context-aware issues.
Because there is usually an abundance of information available while visitor’s attention
span in a museum visit is limited, a smart mobile guide can provide the visitor with
personalized, contextualized information that is given at the right place at the right time.
Proactive mobile museum guides focus on understanding the user’s context and
accordingly providing relevant information to the visitor (Lanir et al., 2011, Cheverst et
al., 2000). However, in order to provide this information, these systems need to notify the
visitor on the existence of new information or services. In addition, many mobile guide
developers today realize that it is important to support inter-visitor and museum-visitor
communication. Thus, mobile notifications do not only relate to nearby artifacts, but may
also be in the form of a message from a friend, or a notification from the museum
advertising a service (e.g., a message on a museum event that will start soon).
Many mobile museum guide systems implemented some sort of notification mechanism.
Looking at when to deliver the information, mobile guide designers and researchers have
often looked at the user’s location as a mean to decide whether to deliver information
(Abowd et al., 1997; Cheverst et al., 2000; Kuflik et al., 2011; Opperman and Specht,
2000; Stock et al., 2007). When the system detects that the user arrives at a new exhibit,
information related to the exhibit is presented. Researchers have also looked at other
contextual factors such as previous user behavior (Abowd et al., 1997; Petrelli and Not,
2005), and visitor’s known interests (Cheverst et al., 2000). However, researchers rarely
looked at the user’s broader activity range such as general movements or engagement
with exhibits. In the current study, we examine the user activities in the museum more
deeply to see how the user’s activity, and not only the user’s location, can determine
when it is best to deliver notifications.
Looking at how to deliver the information, visitors can be notified of existing information
or services in several ways. Systems such as Cyberguide (Abowd et al., 1997), GUIDE
(Cheverst et al, 2000), and others use minor changes in the user interface of the mobile
guide to indicate existing information. The user then interacts with the interface to
request information specific to the location he or she is at. For example, Hippie
(Oppermann and Specht, 2000) signals the availability of new information by displaying
a blinking icon and playing a short indicator sound alert. Information is then delivered by
pressing the icon. Other systems take a more proactive approach. This is based on the
assumption that in the museum environment, most of the visitor’s attention is devoted to
the exhibit and not to the guide system. A guide that automatically decides what to do is
therefore expected to have a greater appeal than one that asks for user assistance (Petrelli
and Not, 2005). In HyperAudio ( Petrelli and Not, 2005), PEACH (Stock et al 2007),
Ubicicero (Ghiani et al, 2009) and in Lanir et al., (2011) a general audio commentary was
Page 10
- 10 -
proactively played when a visitor entered a new location or exhibit, with more
information available through the guide’s interface afterwards. LISTEN (Zimmermann &
Lorenz, 2008) takes the proactive approach to the extreme. In order for technology to be
invisible to the user, LISTEN users do not directly interact with a device and carry only
headphones. It provides museum visitors with audio presentations, adapted to the user's
context. The visitor moves freely in the museum and once the user reaches and faces an
exhibit, the attached acoustic information is proactively activated. The only way the
visitor can (implicitly) control the audio presentations is by moving or changing head
orientation. Finally, the PIL system (Kuflik et al., 2011) uses a hybrid approach. When
the visitor reaches a new exhibit, the display is automatically changed to a picture of the
current exhibit and a beep sound is played. The user can then interact with the system to
receive information about the exhibit. When considering the disruptiveness of the
notification, it is obvious that a proactive notification will be more disruptive than a more
subtle one. However, no studies that we are aware of have examined the modality of a
notification: sound vs. tactile vibration. In this study, we will compare between the
perceived disruption of a sound and a vibro-tactile notification.
Looking at what information to deliver to visitors, most mobile guides focus on providing
interpretive information on the museum exhibits. However, many guides also provide
other types of services such as navigation, means of exploration, and communication
services between visitors. Many visitors come to the museum in small groups, mostly
with family or friends (Aoki et al. 2002). And so, sharing the experience with
companions is an important consideration. In PIL (Kuflik, 2011), small groups of visitors
arriving together can send messages between themselves using the mobile guide.
Message notifications are presented with a small icon that the user can press in order to
read the message. Thus, the PIL guide made a distinction between location-based
information which was presented proactively using a sound and a salient change of the
interface, and social-based information which was left for user control. In CyberGuide
(Abowd et al, 1997) users can send reports about their location to some central service
that others can access. Furthermore, the guide enables users to send general messages to a
set of users such as: “the bus will leave from the departure point in 15 minutes”. Hippie
(Oppermann and Specht, 1999) allows its users to send SMS-like messages that can be
directed to a dedicated addressee, such as family or group members in the museum or to a
full e-mail address of a remote user. While these systems all show the need to send
personal or system notification, none of them examine what messages are perceived as
more disruptive. In the current work, we will analyze different type of messages in the
museum and examine what makes one type of message more disruptive than another.
While all of the works mentioned above implemented information delivery and
notifications within a mobile museum guide system, each study had different assumptions
regarding the way notifications should be handled at the museum environment.
Moreover, none of these studies have empirically looked at the effect that notifications
have on visitors or evaluated the way notifications should be implemented in a museum
environment. In the current study, we systematically compare the various options of
notification design at an actual museum environment, examining when, how and what
notifications influence the subjective disruptiveness of visitors.
Page 11
- 11 -
3. THE CURRENT STUDY The current study examines what affects the disruptiveness of notifications in a museum
visit scenario. We use knowledge gained in the desktop environment and examine
whether it applies in the mobile museum context. Considering when to send notifications,
desktop studies suggest looking at task switches or breakpoints as a time for interruptions
(Iqbal & Bailey, 2007). Ho and Intille (2005) correlated task switches in a desktop
environment with physical activities in a mobile setting, finding that opportune moments
for interruptions are when the system detects that the user is in transition between two
physical activities. On the other hand, in a different mobile setting, Vastenburg, Keyson
and de Ridder (2008), did not find physical transitions to affect notifications’
acceptability. In the first user study, we first analyze the tasks that visitors perform in a
museum. We examine physical transitions as well as other visitor activities to see when it
is best to send a notification in the museum environment.
When considering how to convey a notification in the museum environment, various
methods come to mind. The most common notification modalities include a combination
of audio alert, vibrotactile feedback, and visual indication on the display. Hinckley and
Horvitz (2001) proposed different ways of using these combinations to render mobile
phones less intrusive to their users. They suggested inferring the reaction of the user to
the incoming call based on information received from sensors and the context, and
switching the modality accordingly. Still, it remains mostly unclear which modality is
perceived to be more disruptive in our setting. We also examine this in the first user
study.
Finally, as listed above in Section 2.3, when examining how the message content might
affect the disruptiveness of the notification, evidence from studies done in the desktop
environment propose that important and relevant messages will be perceived as less
disruptive than less important or relevant ones. In the second user study, we will examine
this in the museum, also looking at the specific types of messages that are typical to the
museum setting.
3.1 The museum environment
We conducted the experiments in a museum environment in which participants used a
mobile museum guide to receive information about their surroundings. The studies were
conducted at the ***1 museum, a small to medium size archeological museum located on
the campus of ***. The museum includes several exhibition rooms showing exhibits
related to archeology. The museum is instrumented with a radio frequency (RF) based
indoor positioning system based on a wireless sensor network (WSN). The system
architecture (including both positioning and other services) is illustrated in Figure 1.
Forty five Fixed Beacons are placed in points of interest in the museum and the visitors
are carrying mobile devices called “Blinds”. When a Blind is in proximity of a Beacon
(determining location) that Blind reports this information to the server through the nearest
Gateway. The server parses, filters and enhances the information, determining the
visitor’s position.
1 Removed for anonymization purposes
Page 12
- 12 -
Figure 1. Museum guide’s system architecture.
A research prototype multimedia mobile guide has been developed to provide visitors
with location-aware information about various exhibits in the museum (anonymous).
Visitors use a handheld device (in the current study participants used a Samsung Galaxy
S with a 4” display running an Android OS) to receive information about the various
exhibits in the form of multimedia presentations. When a visitor arrives at a point of
interest (the Blind detects the Beacon and reports this event to the server), he or she is
automatically prompted with an image of the location on which the system presents the
user with a selection of nearby objects (Figure 2, left). The user then selects a specific
object of interest (among those marked by yellow rectangles), which prompts a list of
questions (Figure 2, middle). The questions aim at initiating a dialogue with the visitor
and trigger the visitor to think and select the most interesting one to be answered. Once
the visitor selects a question of interest, a one-minute multimedia presentation is played,
providing an answer to that question (Figure 2, right). Presentations consist of a narrated
text over a set of selected images enhanced with illustrative graphics using different
cinematographic techniques such as pan, zoom and fade. In addition, other services such
as a map screen for each exhibition room, a general map of the museum, navigation to a
specific location and specific tours in the form of a list of locations are available using the
bottom bar on the mobile guide.
The mobile system was successfully deployed at the museum and has since then been
available to the museum visitors free of charge. The system has been evaluated (and
continues to be) by more than 400 actual visitors. We gathered visitors’ log records and
handed out questionnaires, enabling us to conclude that the use of a mobile guide
significantly changed the way visitors behave in the museum (anonymous). We
concluded that overall, visitors enjoyed to use and feel of the system and that it
contributes to the complete visit experience (for further details see anonymous).
Events / Requests
User Data
Sensors and Positioning data
Proximity
Presentations and Information
PIL Museum Visitor’s Guide System Architecture
Fixed BeaconsCommunication
Gateway
Positioning
server
Museum guide
server
Visitor with Blind
Page 13
- 13 -
Figure 2. The mobile museum guide interface. Left - position with highlighted items, middle - list of questions
question, right - presentation running
For the current study, we added to the system the notion of textual notifications. When a
visitor receives a notification, the message icon on the bottom bar changes, indicating
that an incoming message has arrived (Figure 3). When the visitor presses the message
button, a screen simulating the reception of a text message is displayed (see Figure 5).
Figure 3. Text message notification
Page 14
- 14 -
4. STUDY 1: THE EFFECT OF USER ACTIVITY AND
NOTIFICATION TYPE ON THE PERCEIVED LEVEL OF
INTERRUPTION
As stated earlier, the timing of the notification as well as the visual presentation and
modality of the notification may affect its perceived level of disruptiveness. This study
first examined when it is best to interrupt the user. That is, given different user activities
such as standing, walking, engaged with an exhibit, etc., are there situations in which the
user perceives the notification as less interruptive? Second, we examined the perceived
level of disruption of a notification for different types and modalities of notification
methods (For example, it is likely that a sound notification will be more disruptive than a
visual-only notification; however, a sound notification creates more awareness of the
notification than only a visual icon).
4.1 Study design
We employed a three-way (2×3×3) within-subject design with visual indicator (icon,
pop-up), interruption modality (visual-only, sound, vibration), and user activity
(interacting with object, interacting with interface, moving) as independent variables. The
dependent variable was the perceived level of disruption.
For the first variable, visual indicator, we wanted to compare subtle and abrupt visual
notifications. We used an icon indicator as a subtle notification, and one that takes control
of the entire screen and simply appears as a more abrupt notification. Thus, the visual
indicator variable had two values:
Icon – An icon appears on the screen, indicating there is a message waiting. To read
the message, the icon is pressed.
Pop-up – When the message arrives, it immediately appears on the screen.
Interruption modality included the following values:
Visual-only – only a visual indicator appeared
Sound – a beep sound accompanied the visual indicator
Vibration – a vibrotactile notification accompanied the visual indicator
A standard notification sound was used, and the length of the vibration was determined to
be one second, similar to the default length of a vibrotactile notification on a mobile
device.
The third variable, user activity, defined the visitor’s activity during the museum visit
when receiving the notification. To define the values of this variable, in a pilot study, we
conducted observations of real visitors visiting the museum while using the mobile guide.
We came up with the following user activities:
Interacting with the object – the visitor is examining the exhibit, getting a general
impression of the exhibit items and may or may not be reading the exhibit label. The
visitor is not using the mobile device.
Page 15
- 15 -
Interacting with interface – the visitor is using the mobile guide and navigating
through the guide’s screens.
Moving – The visitor is moving around the museum. The visitor is not using the
mobile device.
Another activity that appeared in our observations was that of a visitor interacting with
another visitor (usually a friend or family member). However, a museum visit is a social
activity and it is ill-advised to interrupt the museum visitor during a conversation
(Leindhardt & Knutson, 2004). Other studies have shown that conversations are an
inappropriate time to disturb users (Hudson et al., 2003; Szóstek & Markopoulos, 2006).
Furthermore, to test this situation, we would need to create conversations for the
participants in an artificial way. This is difficult to simulate, and artificial conversations
would have differed from one participant to another. Therefore, we decided not to
examine this situation in our study.
When moving or interacting with an object, the user is not looking at the device’s screen.
Thus, for these situations, we excluded the visual-only modality in which the user is
assumed to be looking at the screen. Except for this, we tested all combinations of visual
notifications, interruption modalities, and user activities.
In order to measure the costs of interruption, the resumption lag (the time it takes from
when the notification window is closed till the user resumes the task) is often used (Iqbal
& Bailey, 2005; Iqbal & Bailey, 2006; Horvitz et al., 2003). However, resumption lag
can only be measured in structured computerized tasks in which the user is fully engaged.
In a mobile scenario such as that used in our study, there is often no primary task to
resume, and even if there is, it is not always measurable. Thus, as in other studies on the
effect of notifications in a mobile environment with unstructured tasks (Ho & Intille,
2005; Vastenburg, Keyson & De Ridder, 2008), we use subjective self-reported measures
to indicate how much the user was interrupted.
4.2 Hypotheses
Based on our assumptions and previous works, we came up with the following
hypotheses:
H1. When looking at the screen, users would experience less disruption with the icon
visual indicator than the pop-up visual indicator. The visual indicator provides the
user to control when to respond to the interruption, while the pop-up method forces the
user to react to the interruption when it appears. While proactively delivering information
can be useful, it usually comes with a feeling of a loss of control (Lanir et al., 2011). We
therefore assume that pop-up will cause a higher level of perceived interruption to the
user than icon changing.
H2. When not looking at the screen, users would experience less disruption with the
vibration method than with the sound method. Hopp et al. (2005) examined the
effectiveness of using tactile cues, such as vibration, to mitigate the negative effects of
interruptions. They found that using such cues led to more frequent and faster responses
Page 16
- 16 -
to the secondary task without any significant associated decrement in the central task.
Based on their discussion, we hypothesize that tactile cues would be perceived as less
interruptive than audible cues.
H3. The users’ current activity would affect the way users perceive interruptions. As
detailed in Section 2.2, the level of attention is related to the level of mental workload,
and it is not recommended to create interruptions when the user has a high mental
workload (Adamcyzk et al., 2005). We expect these results to be applicable in a mobile
environment as well. Thus, when the user’s current task requires low-level attention,
such as moving from one place to another, the level of the perceived interruption will be
lower than when the user’s current task requires high-level attention, such as when
interacting with the mobile device or an exhibit.
4.3 Participants
Twenty four participants took part in the study (12 males, 12 females) of ages ranking
between 20 and 37 (M=23.95, SD=3.94). All were students from our university. Most
mentioned that they had good computer skills (22), used computers on a daily basis (24),
had an advanced control of mobile phones (12), used them on a daily basis (18), and in
the past used mobile guides a couple of times (13).
4.4 Procedure
Participants first received a short explanation about how to use the mobile guide system,
followed by a short demonstration of the guide and the way notifications are presented.
Participants were then asked to visit the museum at their own pace using the mobile
guide. During the experiment, participants moved freely from one exhibit to another,
examining the exhibits and using the mobile guide to receive information about the
museum's exhibits (Figure 4). At various times during the visit, participants received a
notification of a text message waiting to be read. The participants were told that this
simulates a message with information from the museum, the mobile guide system, or a
friend who came to the museum with them. When the participant accessed the message
content, instead of text, a rating screen appeared (Figure 5). Participants were instructed
that upon receiving the notification they should rate their perceived level of interruption
on a 5-point Likert scale ranging from 1 (not disruptive) to 5 (very disruptive).
Figure 4. A participant receiving information on an exhibit using the mobile guide during the experiment
Page 17
- 17 -
Figure 5. Notification rating screen
In order to create the interruptions at the right time (according to the participants’
activity), we used the Wizard of Oz technique. An experimenter followed the participants
during the visit at a reasonable distance and, using a handheld device that he carried with
him, triggered interruption events according to the participant’s activity at that particular
time. The order of the interruption method and user activity were randomized for each
participant and determined in advance. Each participant experienced 14 interruptions
(2×3×3, excluding both the visual-only notifications when interacting with an object or
when moving) with a 3-4 minute break between the interruptions to avoid the effects that
sequential interruptions would have on each other. This interruption frequency falls
within the range used in previous studies (Bailey, Konstan & Carlis, 2001; Cutrell,
Czerwinsky & Horvitz, 2001; Czerwinski, Cutrell & Horvitz, 2000b; Fogarty, Hudson &
Lai, 2004; Iqbal et al., 2005; McFarlane, 2002), in which frequency varied from 3
seconds to 5 minutes, but was usually between 2-5 minutes. The entire session took a
little more than one hour, and participants were compensated with cash for their time.
4.5 Results
With ranked ordinal data and a relatively small sample size, a non-parametric statistical
test is recommended (Huck et al., 1974). We therefore used the Friedman analysis of
variance of ranks test to examine differences in ranking between the groups, followed by
post hoc Wilcoxon Signed Ranked-test for pair-wise comparisons if needed.
4.5.1 Visual notifications
We compared participants’ ratings of perceived interruption (on a Likert scale between 1,
indicating a low-level interruption, and 5 indicating a high-level interruption). Results
indicate that the Icon notification (M = 2.8) was perceived to be overall less interruptive
than the pop-up notification (M = 3.44). This difference was significant, (1) = 4.20, p <
Page 18
- 18 -
.001, supporting H1. Figure 6 presents the average ratings for both types of notifications
for the overall results as well as in each situation independently.
User activity Methods
p-value icon Pop-up
Overall 08.2 (1.19) 4833 (1.19) 0.000
Interacting with interface 0822 (1.26) 4823 (1.23) 0.001
Interacting with object 4832 (1.18) 4833 (1.09) 0.098
Moving from one place to another 0823 (1.06) 4832 (1.19) 0.036
Figure 6. Mean (and standard deviation) of the perceived level of disruption for the icon and pop-up visual
notifications. The Wilcoxon signed ranked test was used to measure significant differences (bold marks
significant difference). N=24.
4.5.2 Modality of the notification
We compared participants’ ratings of perceived interruption between the different
interruption modalities. Because the visual-only modality was applied only in the
“interacting with interface” activity (as this is the only one where it is meaningful), we
first examined the differences between the three modalities when “interacting with
interface.” Unsurprisingly, results indicated that the visual-only modality (“none”) was
less interruptive than the sound or vibration modality (unsurprisingly, since the sound and
vibration notifications also included the visual component). The Friedman test indicated
significant differences between the three modalities (2) = 22.30, p < .001. The results
of a Wilcoxon signed ranked test for pair-wise comparison indicated a significant
difference between the visual-only modality and both the vibration (p<.001) and the
sound (p<.001) modalities.
Next, we examined the overall differences between the sound and the vibration
modalities. Results of the Wilcoxon Signed Ranks Test indicate an overall significant
difference (p = .003) with vibration being less disturbing (M = 3.00) than sound (M =
3.46), thus supporting H2. Figure 7 summarizes the results of the comparison between the
sound and vibration modalities overall and in each situation.
User activity Methods
p-value Sound Vibration
Overall 3.46 3.00 0.003 Interacting with interface 482. 4843 28003
Interacting with object 48.0 4822 28230 Moving from one place to another 483. 0820 020.0
Figure 7. Mean ratings of the perceived level of disruption for vibration and sound methods and the significance
of the difference between them using the Wilcoxon signed ranks test. A higher rating number denotes a higher
level of disruption (bold marks significant differences). N = 24.
4.5.3 User activity
Figure 8 shows the differences in perceived level of interruption between the three user
activities. The results indicate that when moving from one place to another participants
Page 19
- 19 -
perceived a lower level of interruption (M = 2.91, SD = 0.42) than when interacting with
an object (M = 3.29, SD = 0.67), which in turn, was perceived as having a lower level of
interruption than when interacting with the interface (M =3.50, SD = 0.56). The Friedman
test provided evidence for significant differences among the three groups, (2) = 18.6, p
< .001, supporting H3. When examining the pair-wise comparisons using the Wilcoxon
test, we found that receiving a notification while moving was perceived as less
interruptive than when interacting with an object, Z = -277, p < 0.05, or when interacting
with the interface, Z = -3.84, p < 0.001. The difference between interacting with an object
and interacting with an interface was not significant, Z = -1.49, p = ns.
Figure 8. Mean of perceived level of disruption of participants in the different user activities. Error bars display
95% confidence interval.
5. STUDY 2 - THE EFFECT OF THE MESSAGE CONTENT ON
THE PERCEIVED LEVEL OF INTERRUPTION
The second study looked at how the content of the notification, specifically the perceived
importance and relevance of the received message, affects the perceived level of
disruption.
5.1 Study Design
We employed a within-subject design with Message type (4) and user activity (3) as
independent variables. As in study 1, the dependent variable was the perceived level of
interruption. In addition, we asked participants to rate each message's perceived level of
importance and relevance. Importance and relevance are subjective: a message can be
important to some people and not to others, and the same applies to relevance. According
to these subjective results, we tested whether there was a correlation between
importance/relevance and the perceived level of disruption.
We expect a high variability in perceived level of importance and relevance of a message.
Different participants may rate messages differently according to the message structure,
content, and phrasing, and according to individual subjective traits. Thus, similarly to
1.00
1.50
2.00
2.50
3.00
3.50
4.00
Interacting withInterface
Interacting withobject
Moving fromone place to
another
Page 20
- 20 -
Vastenburg , Keyson and de Ridder (2007; 2008), we analyze our results according to
perceived importance and relevance rather than trying to induce importance or relevance
as independent variables. In order to elicit a wide range of perceived importance and
relevance ratings, we devised four kinds of message categories that are relevant to the
museum context.
Recommendation. Messages that provide the museum visitors with information about
what to see and what to do in the museum.
Social. Messages that support the visit in its social context by creating
communication between the visitors during the visit.
Urgent. Messages informing the visitors about events concerning the museum. These
are time sensitive information sent by the system, with an expiration time.
Regular (control). As a control message type, regular, irrelevant messages were also
sent to the users. These contained general information that was related to the museum
visit scenario but not related to any other type of message.
Figure 9 shows the messages used in the study according to the four message categories.
Message Category Message Text
Recommendation
For your information, most visitors selected the second
question/presentation from the list.
For your information, this exhibit is one of the five most observed exhibits
among the museum visitors.
For your information, the exhibit on your right is highly recommended as
it is related to the current exhibit.
Regular (Control)
For your information, you can come to the museum with your own smart
phone and use the system with it.
For your information, you can watch the presentations that are in the
guide from your home as well through the museum website.
For your information, next week the museum will be closed on Wednesday
for replacement of exhibits.
Social
You received a message from your friend John: You should visit the
exhibit of the ancient ship from Ma'agan Michael.
You received a message from your friend John: I can see your position
from the device. I will join you in a few minutes.
You received a message from your friend John: The visit is very
interesting. I'll be happy to come back to this museum.
Urgent
System message: the museum will close in ten minutes.
System message: In the exhibit you are currently in, a guided tour will
start for a group of visitors. You are welcome to stay and listen to the
guide or move elsewhere because of the expected noise
System message: A movie that describes the removal of the ship from the
sea will be broadcast in a five minutes at the shipyard
Figure 9. Notification messages according to message category
Page 21
- 21 -
The user activity, included the same activities as in study 1:
Interacting with the object – the visitor is examining the exhibit, getting a general
impression of the exhibit items. The visitor is not using the mobile device.
Interacting with interface – the visitor is using the mobile device and navigating
through the screens.
Moving – the visitor is moving around the site. The visitor is not using the mobile
device.
5.2 Hypotheses
H4. Relevant messages will be perceived as less disrupting than irrelevant messages.
In a desktop environment, irrelevant messages were found to take longer to process and it
was more difficult to reestablish task context following the interruption, and thus they
were determined to be more disruptive (Czerwinski, Cutrell & Horvitz, 2000a). We
hypothesize that in a mobile environment they will also be more disruptive. We assume
this disruptiveness will also be reflected in the perceived ratings.
H5. Visitors will be more tolerant toward messages that are interesting and/or
important to them. Several researchers, looking at the office setting, claimed that the
subjective importance of the message should be considered when deciding when and how
to interrupt the user (Dabbish & Baker, 2003; Szóstek & Markopoulos, 2006). We thus
hypothesize that also in a leisure mobile setting important messages will be perceived as
causing a lower level of interruption than non-important messages.
H6. There will be differences in the perceived level of disruption between different
types of messages. Specifically, urgent, social, and recommendation types of
messages will be less disruptive than regular messages. In a museum scenario, visitors
may receive different types of messages. Messages may contain recommendations about
what to do or where to go, have a social aspect when they are sent from other visitors, or
can have a time barrier that makes them urgent. A study by Vastenburg , Keyson and de
Ridder, (2007) showed that urgency and the delivery method may affect the acceptability
of a notification. A different study (Nagel, Hudson & Abowd, 2004) showed that the
user’s activity and social context are also important factors in determining the
acceptability of an interruption. According to Bowen and Filippini-Fantoni (2004),
personalized recommendations help visitors deal with the "information overload" of the
museum environment by presenting information based on their interests and background.
Thus, we hypothesized that these types of messages would be more relevant to the user
and thus less interruptive.
5.3 Participants
Twenty four paid participants (3 males, 21 females) of ages ranging between 20 and 27
(M = 22.87, SD = 1.65) participated in the study; none of them participated in study 1. All
were students from our university. Most mentioned that they had advanced computer
skills (23), used computers on a daily basis (19), had a basic and advanced control of
mobile phones (10 for each option), and in the past had used mobile guides a couple of
times (12).
Page 22
- 22 -
5.4 Procedure
As in the previous study, participants were requested to go around the museum using a
handheld mobile museum visitors’ guide, stopping at various exhibits and listening to
information on the exhibits using the mobile guide. While touring the museum,
participants received notifications. Unlike in the previous experiment, here the
notifications were accompanied by a text message, which the participants were requested
to first read, and only then rate the perceived level of interruption for that message.
Participants were asked to imagine that these were real messages that related to their visit
(i.e., the social message was actually from a friend that came with them to the museum).
After rating the perceived level of interruption, participants were presented with a form
asking them to rate the subjective level of importance and relevance of the message on a
5-point Likert scale, with 1 denoting a low and 5 denoting a high level. Participants were
asked: “How important was the message to you”, and “How relevant was the message to
what you are currently doing?” Participants were informed of an incoming notification by
a vibration plus an icon notification. The vibration method was chosen since it was the
preferred method from the first study. Pressing the message button (see Figure 2) brings
the participants to the message and rating screen.
As in the first study, we used the Wizard of Oz technique in which the experimenter
followed the visitor during the visit at a reasonable distance and triggered the
interruptions according to the observed situations. The visitor was instructed to ignore the
experimenter. Each participant received 12 messages during the visit (4 message types
3 situations), resulting in a total of 288 total notifications in the entire experiment.
Messages were sent within 4-5 minutes of each other in order to avoid effects that one
message may have on the other. The order of the messages for each participant was
randomized.
5.5 Results
5.5.1 Relevance and importance
To examine the effect of perceived importance on perceived interruption, we performed a
random effects model (mixed) with message as a repeated measure, perceived
interruption as the dependent variable, perceived importance as the fixed variable and
subject as a random variable. Results indicate an estimate of -0.23 for perceived
importance indicating a negative correlation of perceived importance on perceived
interruption. These results were significant (p<0.001). This relation suggests that the
higher the importance of a message, the lower the perceived interruption, supporting H5.
We performed the same analysis, this time with the perceived relevance as the fixed
variable. Results indicate an estimate of -0.17 for perceived relevance indicating a
negative correlation of perceived relevance on perceived interruption. These results were
also significant (p<0.01). This relation suggests that a higher relevance of a message is
tied to a lower interruption level, supporting H6.
Page 23
- 23 -
To visualize these results we show the cross tabulation rating counts of all results. While
not taking into account personal repeated measures, this gives an indication of the general
correlation of both importance and relevance with the perceived interruption.
Importance Relevance
rating 1 2 3 4 5 1 2 3 4 5
Per
ceiv
ed
dis
rupti
on
1 6 7 8 21 26 9 7 10 12 27
2 5 9 20 27 8 8 12 11 21 15
3 4 19 18 9 10 10 16 19 5 8
4 9 16 12 8 4 20 12 10 7 6
5 19 9 3 5 6 18 11 2 4 8
Figure 10. Cross distribution count of perceived disruption vs. importance and perceived disruption vs.
relevance. Ratings are on a 5-point Likert scale with 1 denoting low and 5 denoting high. N = 24, total of 288
notifications
5.5.2 Message types
We compared participants’ ratings of perceived interruption between the different
message types. The mean results of the four categories are presented in Figure 11. Results
of the Friedman one-way ANOVA by ranks showed that the differences between the
groups were significant (2) = 8.59, p < .035, supporting H6. Examining pair-wise
comparisons using the Wilcoxon signed-ranked test with the Bonferroni correction found
that social messages (M = 2.53, SD = 0.81) were perceived to be less interruptive than the
control messages (M = 2.90, SD = 1.34), Z = -2.14, p = .031, while the difference
between urgent (M= 2.55, SD = 1.2) and control messages was marginally significant (Z
= -1.80, p = 0.064). No other pair-wise comparison was significant.
1
2
3
4
recommendation social urgent control
Page 24
- 24 -
Figure 11 – Mean ranking of perceived interruption according to different message types. Ranking is on a 5-
point Likert scale with 1 denoting low interruption and 5 denoting high interruption. Error bars display 95%
confidence intervals.
To examine the difference between message types further, we compared the four message
types according to the perceived relevance and perceived importance ratings given to
messages of each type. The results are presented in Figure 12. For the analysis, we used
pair-wise comparisons with the Wilcoxon signed-ranked test with the Bonferroni
correction. Comparing the perceived importance of the messages, results indicate that
urgent messages were perceived to be significantly more important than social messages
and recommendation messages, which in turn were perceived to be significantly more
important than control messages. Looking at the perceived relevance of the messages, we
found that urgent and recommendation messages were perceived to be significantly more
relevant than control and social messages.
Figure 12 – Mean ranking of perceived importance and relevance according to different message types. Ranking
is on a 5-point Likert scale with 1 denoting high importance or relevance and 5 denoting low importance or
relevance. Error bars displaying 95% confidence intervals.
5.5.3 User activity
To corroborate our results from study 1 (Section 4.6.3), we again looked at the difference
in the perceived interruption level between the three user activities. Similar to the results
in study 1, when receiving messages while moving, participants were least interrupted (M
= 2.22, SD = .79), followed by when participants interacted with an object (M=2.78, SD
= 1.02). Finally, participants were most interrupted when receiving a notification while
interacting with the interface (M=3.10, SD = .90). Results of the Friedman one-way
ANOVA by ranks showed these differences were significant, (2) = 16.4, p < .001.
Results of the Wilcoxon signed-ranked test of the pair-wise comparisons showed
significant differences between all pairs (moving-interacting with interface: Z=-3.84,
p<0.01; moving-interacting with object: Z=-2.85, p<0.01; interacting with interface-
interacting with object: Z=-2.21, p=.026). These results further support H3.
1
2
3
4
5
Importance
Relevance
Page 25
- 25 -
6. DISCUSSION
Many factors may influence the disruptiveness of a notification. These may include the
activity of the user, the modality and utility of the message, the emotional state and the
social engagement of the user and more. Furthermore, different factors may be more or
less influential in different contexts (Ho and Intille, 2005). Our main purpose in this study
was to examine the main factors that influence interruptability in the context of a museum
visit. This can help mobile guide designers in building models and strategies that would
reduce disruptiveness by deciding on when and how to send the notification, similar to
what has been done at the desktop environment (Iqbal & Bailey, 2007; Iqbal & Horvitz,
2007; Horvitz et al., 2003). Out of the possible factors, we chose to closely examine the
activity of the user (when), the modality of the message (how) and the utility of the
message (what) since these factors are commonly used in interruption models and are
most easily detectable in a context-aware mobile system.
In order to explore when to send the notification, we looked at how the activity of the
visitor at the museum environment affects the subjective disruptiveness of the
notification. Our results support the idea that the perceived level of disruption depends on
the current activity of the museum visitor. Participants felt that the notifications were less
disruptive when they were walking between exhibits than when they were standing in
front of an exhibit or when they were interacting with the mobile guide. This supports the
results of Ho and Intille (2005), which showed that interruptions were better accepted
during times of physical transition in a mobile computing environment. The results for
how the notification was presented showed that visitors experienced less disruption with
an icon visual indicator than with a pop-up (full screen) visual indicator. This is not
surprising. A pop-up visual indicator is much more abrupt and naturally more disruptive
than an icon indicator. We note that both of these options are mostly relevant in situations
where the visitor is looking at the screen of a mobile device when receiving the message.
In other situations, the indicator has to be combined with other modalities of notification
(such as sound or vibration) to be effective. We also found that tactile notifications in the
form of vibration were perceived to be less interruptive than a sound notification. This
builds upon and extends the work of Hopp et al. (2005). Finally, focusing on what
content is delivered, we examined different aspects of the notification message and how
the content might affect the perceived level of the disruption. Our results showed a
negative correlation between the relevance of a notification and its perceived level of
disruption. That is, messages that are highly relevant to the user’s current task were
perceived as less disruptive and vice versa. These findings extends the results of
Czerwinski, Cutrell and Horvitz (2000a) about the effects of relevance in Instant
Messaging. Regarding the importance of a message, results showed a negative correlation
between the perceived level of importance and the perceived level of disruption,
indicating that important messages were also perceived as less disruptive. These results
support the literature research in the desktop environment that claimed that the
importance of a message is also a factor to consider when modeling when to show
notifications (Dabbish & Baker, 2003; Szóstek & Markopoulos, 2006) and extends them
to the museum area.
6.1 Guidelines for museum curators and mobile guide designers
Page 26
- 26 -
In order to enhance the museum visitor’s experience, museum curators and mobile guide
designers are interested in offering the visitor options from a wide range of information
and services that are available at the museum. However, they need to be cautious not to
overload the visitor with messages, which may eventually detract rather than enhance
from the visitor experience. Thus, it is critical for mobile guide designers to understand
how to minimize the level of disruption that the notifications cause.
Looking for the opportune moments to send notifications, we first examined the main
visitor activities at the museum. For a typical exhibit-centered museum such as the one
used in the current study, the main activities we observed were a visitor interacting with
an exhibit, interacting with the mobile guide, transitioning from one exhibit to another,
and talking to a friend. Our results indicated that out of the first three activities, it is best
to send notifications when transitioning from one exhibit to another. In previous studies
in the desktop environment, the level of mental workload was shown to affect the
perceived level of interruption (Adamcyzk et al., 2005). Thus, breakpoints between tasks,
when mental workload is low, were recommended as opportune moments for interruption
(Iqbal & Bailey, 2007). When moving around the museum, the museum visitor, being in
transition between exhibits and searching for the next interesting item, most probably
uses less cognitive effort than in the other situations. This is of course also true when the
visitor is resting. When interacting with an exhibit, on the other hand, the visitor is
focusing on the object, often trying to understand or learn about it. This demands much
more attention. When interacting with interface, the visitor is trying to navigate through
the mobile device’s screens until he or she reaches the service they want. This also
requires a high level of attention, this time focused on the device. Finally, it is
recommended not to disrupt the user when they are having a conversation (Leindhardt &
Knutson, 2004, Hudson et al., 2003; Szóstek & Markopoulos, 2006). Thus, museum
systems should aim at recognizing the visitor activity and send the notifications when the
visitor is doing an activity with a low level of mental workload such as when
transitioning between exhibits or resting. Still, it might be difficult to recognize when the
visitor is in transition and is not focusing on exhibits. One way may be to look for
physical movements of visitors. Another is to try to recognize when a visitor is engaged
with an exhibit, which is not a good time to send notifications. Several works have started
exploring how to recognize visitor engagement using information such as eye gaze
(Yamazaki et al., 2009, Qvarfordt, and Zhai, 2005).
Visitors experienced less disruption with an icon visual indicator than with a pop-up (full
screen) visual indicator. The difference between these methods is in their noticeability
and control. There is a tradeoff here. The icon visual indicator is subtle and keeps the user
in control – the user may decide to take action or not, while the pop-up notification is
more noticeable, but does not leave the user much choice. As was seen in Lanir et al.
(2011) visitors usually prefer control over enhanced proactiveness in mobile guides.
McFarlane (2002) distinguishes between immediate and negotiated interruption
strategies. In an immediate strategy, the user is immediately interrupted regardless of his
or her current actions. In the negotiated strategy, the agents announce their need to
interrupt, and then support a negotiation with the user. This allows the user to control
how to deal with the interruption. While the pop-up visual indicator fits the immediate
strategy in which the interruption is forced upon the user, the icon visual indicator fits the
Page 27
- 27 -
negotiated strategy in which the user decides when to address the interruption. Although
participants were instructed to rate the interruption as soon as they were aware of it, and
thus immediately responded to the interruption, participants still preferred the icon visual
indicator, which is associated with the “friendlier” negotiation strategy that leaves control
in the hand of the user. Participants also preferred the vibration over the sound
notification. This can be explained by the experiment’s environment, which was a quiet
archeological museum during its opening hours. In such an environment, a sound
notification may be perceived as more disturbing to other museum visitors and the
general museum atmosphere than a tactile notification. Still, there might be a tradeoff
here between the disruptiveness and noticeability of the interruption. Thus, in a museum
we recommend to use subtle visual notifications rather than abrupt ones and tactile
vibration notifications rather than audio alerts.
Finally, our results point out that designers should carefully consider the content of the
notification. Important notifications and ones that are relevant to the visitor’s current task
are more acceptable. Moreover, we examined three message types typical to the museum
scenario: recommendation, social and urgent (time related) messages and compared them
to baseline, control messages. Our Results showed that social messages caused a lower
perceived level of disruption than regular (control) messages (see Figure 11). The
museum is a social place, with most visits occurring in small groups of family and friends
(Falk, 2009). Museum researchers have shown that communication between visitors is
essential and important for the visit experience (Leindhardt & Knutson, 2004; Aoki et al.,
2002). However, the use of a mobile guide is mostly personal and often isolates and
detaches the visitor from his or her group (Lanir et al., 2013). Thus, we recommend that
mobile guide systems support small group messaging services, and give preference for
social, visitor-to-visitor messages over other types of messages.
Looking at the other categories of message types, the difference in perceived disruption
between urgent and regular messages was only partly significant, even though urgent
messages were perceived to be both more important and relevant. Since other studies
showed that the urgency of a message affects its acceptability (Vastenburg , Keyson & de
Ridder, 2008), we believe that there is an actual difference, and the articulation and
possibly the context of our current messages caused them to be only partly significant.
Further research will examine recommendation, social, and urgent messages in other
mobile contexts.
6.2 Generalizing the results
One of the purposes of this work was to take a close look at notifications in a mobile
setting. As mentioned, while many studies examined the disruption of notifications in the
desktop environment, very few have looked at mobile contexts. A location-aware mobile
guide is a canonical context-aware application and is a perfect medium for investigating
issues of context-aware applications (Dey, 2001). Examples of other context-aware
scenarios include a shopper walking in a shopping center, occasionally using his or her
mobile device to get directions or information on certain stores, a tourist walking the
streets of a new city looking for tourist attractions, or a person strolling in a library. In
Page 28
- 28 -
such settings, a smart system might be able to assist the user by offering context-aware
services or information via notifications.
As any study conducted in a specific environment, we should be cautious in generalizing
our results. However, we believe that while some results are museum specific, others can
generalize to other mobile contexts. For example, while the activities we examined are
specific to the museum, the guideline of looking for times with a lower mental workload
is more general and stands in other contexts. Looking at an exhibit at the museum may be
similar to looking at a shop window in a mall, where in both situations the user is visually
and mentally engaged. Thus, our results that an opportune time for a notification is when
the user is transitioning from one exhibit to another seems to generalize to other mobile
contexts whenever moving demands less mental effort than sitting or standing. Regarding
the modality of the message, the tactile notification may have been perceived as less
disruptive than the auditory notification because of the quiet museum environment.
Future research should examine if a tactile notification is also perceived as less disruptive
in other context-aware environments, such as a shopping center or a sports event, and
whether tactile notifications are less noticeable than auditory ones.
The contents of the messages were highly related to the museum visit. Still, our results
regarding the perception of relevant and important notifications were in line with desktop
studies that found that relevant notifications disrupted productivity tasks less
(Czerwinski, Cutrell and Horvitz, 2000) and that important notifications were considered
less disruptive (Dabbish & Baker, 2003). Thus, we extend these claims to the perceived
level of disruption caused by notifications in a mobile museum environment. While the
notions of relevancy as well as importance are very different in a goal-directed, structured
desktop environment, we believe that similar to how we showed these to affect
notifications in the museum environment, these results will hold in other mobile contexts.
Still, future work would validate this and examine different message types in other
contexts.
6.3 Limitations
The current study has several limitations. First, while conducted in an actual museum at
opening hours, this was still a structured user study in which participants were instructed
what to do. Participants were asked to wander around the museum looking at exhibits as
in a regular visit. Still, their behavior and motivation was probably affected by the user
study setting. Future studies will look at actual museum visitor behavior. Second, as
mentioned above in Section 6.2, the study was conducted in one specific environment: an
archeology museum. It might be difficult to generalize the results to other, different type
of context-aware environments such as shopping malls or sport events. Finally, in the
current study we isolated the variables (user activity + visual indicator, and user activity
+ message content) to create two separate studies. This was done mostly because the
experimental setup was too complex to examine all variable combinations in one
experiment. Thus, there might have been an interaction effect between the modality and
the content of the interruption that we did not meausre. Future research will examine the
connection between the urgency or relevance of a notification and its attentional draw.
Page 29
- 29 -
7. CONCLUSIONS
With the vast increase in smart mobile devices and various mobile services, context-
aware computing has emerged as a new and important research topic. However, for
applications to suggest services and information to users proactively, systems need to
control the amount and rate of notifications arriving at the user. Thus, mobile notification
management is crucial for the success and acceptability of such services. In this research,
we examined the factors that affect disruptiveness in a context-aware mobile museum
environment, looking at when, how, and what type of notifications are less disruptive.
Our results indicate that the user activity affected the perceived level of disruption. It
seems that the perceived level of disruption is correlated to the level of attention that is
required from the user in his or her current activity. In a museum environment,
transitioning between exhibits demands the least amount of attention and therefore is the
best time for sending notifications. We also found that in the museum environment,
tactile notifications provided a lower perceived level of disruption than audible
notifications. Finally, the characteristics of the notification itself, such as the relevancy
and importance of the message, also affected its perceived level of disruption.
These results can be specifically applied to a context-aware museum environment. Many
museums are equipped today with various sensors that are placed in the environment or
that already exist in the mobile device (Kuflik et al., 2011; Cheverst et al., 2000; Stock et
al., 2007). These sensors may help the system to infer the user context and react
accordingly. The system can thus infer the preferred time to present notifications by
recognizing when the visitor is transitioning between activities, specifically when the user
moves from one exhibit to another. Since the museum visit scenario is composed of
repetitive sessions of examining exhibits and pausing, finding an appropriate moment can
be possible without delaying the notification too much. When the visitor is recognized to
be engaged in other activities (such as interacting with an exhibit), it is recommended to
notify him or her only of highly relevant or important information. Finally, in the
museum environment, a negotiation strategy (McFarlane, 2002) is recommended along
with a tactile notification.
While this research contributed specifically in providing guidelines for optimizing the use
of notifications in the museum environment, we believe that most of these results are
generalizable for understanding notification acceptability in other leisure-oriented,
context-aware mobile computing environments. For example, it is likely that transitioning
from one activity to another poses an opportune moment for interruption, as has also been
suggested by other studies. However, other results, such as preferring tactile over audio
notifications, should be examined in other types of mobile environments that may include
other types of activities. Further research will apply these guidelines in museum and other
mobile environments, using sensors to infer the user’s current activities, while modeling
the users’ information needs in order to provide the users with the right notifications in
the right way at the right time.
Page 30
- 30 -
References
Abowd, G. D., Atkeson, C. G., Hong, J. et al. (1997), Cyberguide: A Mobile Context-
Aware Tour Guide, Wireless Networks, 3. 421-433.
Adamczyk, P. D., & Bailey, B. P. (2004). If not now, when?: The effects of interruption
at different moments within task execution. Proceedings of the SIGCHI
Conference on Human Factors in Computing Systems, 271-278. New York:
ACM.
Aoki, P. M., Grinter, R. E., Hurst, A., Szymanski, M. H., Thornton, J. D., & Woodruff,
A. (2002). Sotto voce: exploring the interplay of conversation and mobile audio
spaces. Proceedings of the SIGCHI conference on Human factors in computing
systems, 431-438. New York: ACM.
Ardissono, L., Kuflik, T., Petrelli, D. (2012). Personalization in Cultural Heritage: The
Road Travelled and the One Ahead. User Modeling and User Adapted
Interaction, 22, 1-2, 73-99.
Arroyo, E., Selker T., & Stouffs, A. (2002). Interruptions as multimodal outputs: Which
are the less disruptive? Proceedings of the 4th IEEE International Conference on
Multimodal Interfaces, 479-482. New York: ACM.
Arroyo, E., & Selker, T. (2003). Self-adaptive multimodal-interruption interfaces.
Proceedings of the 8th International Conference on Intelligent User Interfaces
(IUI'03), 6-11. New York: ACM.
Bailey, B. P., Konstan, J. A., & Carlis, J. V. (2000). Adjusting windows: Balancing
information awareness with intrusion. Proceedings of the 6th Conference on
Human Factors and the Web. 39-83
Bailey, B. P., Konstan, J. A., & Carlis, J. V. (2001). The effects of interruptions on task
performance, annoyance, and anxiety in the user interface. Proceedings of
INTERACT, 593-601. IOS Press.
Bailey, B. P., & Konstan, J. A. (2006). On the need for attention-aware systems:
Measuring effects of interruption on task performance, error rate, and affective
state. Computers in Human Behavior, 22(4), 685-708.
Bailey, B. P., & Iqbal, S. T. (2008). Understanding changes in mental workload during
execution of goal-directed tasks and its application for interruption
management. ACM Transactions on Computer-Human Interaction (TOCHI),
14(4), 21.
Billsus, D., Hilbert, D. M., & Maynes-Aminzade, D. (2005). Improving proactive
information systems. Proceedings of the 10th International Conference on
Intelligent User Interfaces, 159-166. New York: ACM.
Page 31
- 31 -
Bowen, J. P., & Filippini-Fantoni, S. (2004). Personalization and the Web from a
museum perspective. The 8th International Museums and the Web Conference,
63-78.
Brewster, S. A., Wright, P. C., & Edwards, A. D. N. (1994). The design and evaluation of
an auditory-enhanced scrollbar. Proceedings of the SIGCHI Conference on
Human Factors in Computing Systems: Celebrating Interdependence, 173-179.
New York: ACM.
Cheverst, K., Davies, N., Mitchell, K., Friday, A., & Efstratiou, C. (2000). Developing a
context-aware electronic tourist guide: Some issues and experiences. Proceedings
of the SIGCHI Conference on Human Factors in Computing Systems, 17-24. New
York: ACM.
Cutrell, E., Czerwinski, M., & Horvitz, E. (2001). Notification, disruption, and memory:
Effects of messaging interruptions on memory and performance. Proceedings of
Interact 2001. IFIP. Conference on Human-Computer Interaction, Tokyo, Japan,
July, 263-269
Czerwinski, M., Cutrell, E., & Horvitz, E. (2000a). Instant messaging: Effects of
relevance and timing. People and Computers XIV: Proceedings of HCI 2000, 71-
76.
Czerwinski, M., Cutrell, E., & Horvitz, E. (2000b). Instant messaging and interruption:
Influence of task type on performance. OZCHI 2000 Conference Proceedings,
356-361.
Dabbish, L. A., & Baker, R. S. (2003). Administrative assistants as interruption
mediators. CHI'03 Extended Abstracts on Human Factors in Computing Systems,
1020-1021. New York: ACM.
Dey, A.K. (2001). Understanding and using context. Personal Ubiquitous Computing, 5,
4–7.
Falk, J. H. (2009). Identity and the museum visitor experience. Walnut Creek CA. Left
Coast Press.
Fischer, J. E., Greenhalgh, C., & Benford, S. (2011). Investigating episodes of mobile
phone activity as indicators of opportune moments to deliver notifications.
Proceedings of the 13th International Conference on Human Computer
Interaction with Mobile Devices and Services, 181-190, New York: ACM.
Fogarty, J., Hudson, S. E., & Lai, J. (2004). Examining the robustness of sensor-based
statistical models of human interruptibility. Proceedings of the SIGCHI
Conference on Human Factors in Computing Systems, 207-214. New York:
ACM.
Page 32
- 32 -
Gammon, B., & Burch, A. (2008). Designing mobile digital experiences. Digital
technologies and the museum experience: Handheld guides and other media, 35.
Ghiani, G., Paternò, F., Santoro, C., & Spano, L. D. (2009). UbiCicero: A location-aware,
multi-device museum guide. Interacting with Computers, 21(4), 288-303.
Gillie, T., & Broadbent, D. (1989). What makes interruptions disruptive? A study of
length, similarity, and complexity. Psychological Research, 50(4), 243-250.
Gluck, J., Bunt, A., & McGrenere, J. (2007). Matching attentional draw with utility in
interruption. Proceedings of the SIGCHI Conference on Human Factors in
Computing Systems, 41-50. New York: ACM
Hinckley, K., & Horvitz, E. (2001). Toward more sensitive mobile phones. Proceedings
of the 14th Annual ACM Symposium on User Interface Software and Technology,
191-192. New York, ACM.
Ho, J., & Intille, S. S. (2005). Using context-aware computing to reduce the perceived
burden of interruptions from mobile devices. Proceedings of the SIGCHI
Conference on Human Factors in Computing Systems, 909-918. New York:
ACM.
Hopp, P. J., Smith, C., Clegg, B. A., & Heggestad, E. D. (2005). Interruption
management: The use of attention-directing tactile cues. Human Factors: The
Journal of the Human Factors and Ergonomics Society, 47(1). pp. 1-11.
Horvitz, E., Kadie, C., Paek, T., & Hovel, D. (2003). Models of attention in computing
and communication: from principles to applications. Communications of the
ACM, 46(3), 52-59.
Hudson, J. M., Christensen, J., Kellogg, W. A., & Erickson, T. (2002). I'd be
overwhelmed, but it's just one more thing to do: Availability and interruption in
research management. Proceedings of the SIGCHI Conference on Human Factors
in Computing Systems: Changing our World, Changing Ourselves, 97-104.
Minneapolis, Minnesota, USA, April.
Hudson, S., Fogarty, J., Atkeson, C., Avrahami, D., Forlizzi, J., Kiesler, S., et al. (2003).
Predicting human interruptibility with sensors: A Wizard of Oz feasibility study.
Proceedings of the SIGCHI Conference on Human Factors in Computing
Systems, 257-264. Fort Lauderdale, Florida, USA, April.
Iqbal, S. T., Adamczyk, P. D., Zheng, X. S., & Bailey, B. P. (2005). Towards an index of
opportunity: Understanding changes in mental workload during task execution.
Proceedings of the SIGCHI Conference on Human Factors in Computing
Systems, 311-320. New York: ACM
Page 33
- 33 -
Iqbal, S. T., & Bailey, B. P. (2005). Investigating the effectiveness of mental workload as
a predictor of opportune moments for interruption. CHI'05 Extended Abstracts on
Human Factors in Computing Systems, 1489-1492. ACM.
Iqbal, S. T., & Bailey, B. P. (2006). Leveraging characteristics of task structure to predict
the cost of interruption. Proceedings of the SIGCHI Conference on Human
Factors in Computing Systems, 741-750. ACM.
Iqbal, S. T., & Bailey, B. P. (2007). Understanding and developing models for detecting
and differentiating breakpoints during interactive tasks. Proceedings of the
SIGCHI Conference on Human Factors in Computing Systems, 697-706. New
York: ACM
Iqbal, S. T., & Horvitz, E. (2007). Disruption and recovery of computing tasks: field
study, analysis, and directions. Proceedings of the SIGCHI conference on Human
factors in computing systems, 677-686. ACM.
Kenteris, M., Gavalas, D. and Economou, D. (2011). Electronic mobile guides: a survey.
Personal and Ubiquitous Computing, 15(1), 97-111.
Kray, C., & Baus, J. (2003). A survey of mobile guides. 5th Int. Symposium on HCI with
Mobile Devices and Services. Udine, Italy.
Kreifeldt, J., & McCarthy, M. (1981). Interruption as a test of the user-computer
interface. Proceedings of the 17th Annual Conference on Manual Control, 655-
667, JPL Publication 81-95.
Kuflik, T., Stock, O., Zancanaro, M., Gorfinkel, A., Jbara, S., Kats, S., et al. (2011). A
visitor's guide in an active museum: Presentations, communications, and
reflection. Journal on Computing and Cultural Heritage (JOCCH), 3(3), 11.
Lanir, J., Kuflik, T., Wecker, A. J., Stock, O., & Zancanaro, M. (2011). Examining
proactiveness and choice in a location-aware mobile museum guide. Interacting
with Computers, 23(5), 513-524.
Lanir, J., Kuflik, T., Dim, E., Wecker, A. J., & Stock, O. (2013). The Influence of a
Location-Aware Mobile Guide on Museum Visitors' Behavior. Interacting with
Computers, 25(6), 443-460.
Latorella, K. A. (1998). Effects of modality on interrupted flight deck performance:
Implications for data link. Proceedings of the Human Factors and Ergonomics
Society Annual Meeting, 42(1), 87-91.
Leindhardt, G., & Knutson, K. (2004). Listening in on museum conversations. Rowman
Altamira.
Page 34
- 34 -
Maglio, P. P., & Campbell, C. S. (2000). Tradeoffs in displaying peripheral information.
Proceedings of the SIGCHI Conference on Human Factors in Computing
Systems, 241-248, New York: ACM.
McCrickard, D. S., & Chewar, C. (2003). Attuning notification design to user goals and
attention costs. Communications of the ACM, 46(3). 67-72.
McCrickard, D. S., Catrambone, R., Chewar, C. M., & Stasko, J. T. (2003). Establishing
tradeoffs that leverage attention for utility: empirically evaluating information
display in notification systems. International Journal of Human-Computer
Studies, 58(5), 547-582.
McFarlane, D. C. (1999). Coordinating the interruption of people in human-computer
interaction, Human-computer Interaction-INTERACT'99, 295-303.
McFarlane, D. C. (2002). Comparison of four primary methods for coordinating the
interruption of people in human-computer interaction. Human-Computer
Interaction, 17(1), 63-139.
Miyata, Y., & Norman, D. A. (1986). Psychological issues in support of multiple
activities. User Centered System Design, D.A. Norman and S.W. Draper (Eds).
Lawrence Erlbaum. Hillsdale, N.J., USA. pp. 265-284.
Monk, C. A., Boehm-Davis, D. A., & Trafton, J. G. (2002). The attentional costs of
interrupting task performance at various stages. Proceedings of the Human
Factors and Ergonomics Society Annual Meeting, 46(22), 1824-1828.
Nagel, K. S., Hudson, J. M., & Abowd, G. D. (2004). Predictors of availability in home
life context-mediated communication. Proceedings of the 2004 ACM Conference
on Computer Supported Cooperative Work, 497-506. Chicago, IL, USA,
November.
Obermayer, R. W., & Nugent, W. A. (2000). Human-computer interaction for alert
warning and attention allocation systems of the multimodal watch station.
Proceedings of SPIE 2000, SPIE-The International Society for Optical
Engineering, 14-22. Bellingham, WA, USA.
O'Conaill, B., & Frohlich, D. (1995). Timespace in the workplace: Dealing with
interruptions. Conference Companion on Human Factors in Computing Systems,
262-263. Denver, Colorado, USA, May.
Oppermann, R. and Specht, M. (2000). A context-sensitive nomadic exhibition guide,
Handheld and Ubiquitous Computing. 31-54
Petrelli, D., & Not, E. (2005) User-Centred Design of Flexible Hypermedia for a Mobile
Guide: Reflections on the Hyperaudio Experience, User Modeling and User-
Adapted Interaction, 15, 303-338
Page 35
- 35 -
Picard, R. W., & Liu, K. K. (2007). Relative subjective count and assessment of
interruptive technologies applied to mobile monitoring of stress. International
Journal of Human-Computer Studies, 65(4), 361-375.
Qvarfordt, P., & Zhai, S. (2005). Conversing with the user based on eye-gaze patterns.
In Proceedings of the SIGCHI conference on Human factors in computing
systems (pp. 221-230). ACM.
Somervell, J., Chewar, C., & McCrickard, D. S. (2002). Evaluating graphical vs textual
secondary displays for information notification. Proceedings of the 40th Annual
ACM Southeast Conference, 153-160. Raleigh, NC, USA, April.
Speier, C., Valacich, J. S., & Vessey, I. (1999). The influence of task interruption on
individual decision making: An information overload perspective. Decision
Sciences, 30(2), 337-360.
Stock, O., Zancanaro, M., Busetta, P., Callaway, C., Krüger, A., Kruppa, M., et al.
(2007). Adaptive, intelligent presentation of information for the museum visitor in
PEACH. User Modeling and User-Adapted Interaction, 17(3), 257-304.
Szóstek, A. M., & Markopoulos, P. (2006). Factors defining face-to-face interruptions in
the office environment. CHI'06 Extended Abstracts on Human Factors in
Computing Systems, 1379-1384. Montreal, Canada, April.
Van Dantzich, M., Robbins, D., Horvitz, E., & Czerwinski, M. (2002). Scope: Providing
awareness of multiple notifications at a glance. Proceedings of the Working
Conference on Advanced Visual Interfaces, 267-281. Trento, Italy, May. .
Vastenburg, M. H., Keyson, D. V., & De Ridder, H. (2008). Considerate home
notification systems: A field study of acceptability of notifications in the home.
Personal and Ubiquitous Computing, 12(8), 555-566.
Vastenburg, M. H., Keyson, D. V., & De Ridder, H. (2009). Considerate home
notification systems: A user study of acceptability of notifications in a living-
room laboratory. International Journal of Human-Computer Studies, 67(9), 814-
826.
Warnock, D., McGee-Lennon, M., & Brewster, S. (2011). The role of modality in
notification performance. Human-Computer Interaction–INTERACT 2011, 6947,
572-588.
Yamazaki, K., Yamazaki, A., Okada, M., Kuno, Y., Kobayashi, Y., Hoshi, Y., ... &
Heath, C. (2009). Revealing Gauguin: engaging visitors in robot guide's
explanation in an art museum. In Proceedings of the SIGCHI Conference on
Human Factors in Computing Systems, 1437-1446. ACM Press.
Page 36
- 36 -
Zijlstra, F. R. H., Roe, R. A., Leonora, A. B., & Krediet, I. (1999). Temporal factors in
mental work: Effects of interrupted activities. Journal of Occupational and
Organizational Psychology, 72(2), 163-185.
Zimmermann, A., & Lorenz, A. (2008). LISTEN: A User-Adaptive Audio-Augmented
Museum Guide, User modeling and user-adapted interaction, 18, 389-416
