Date post: | 01-Jan-2016 |
Category: |
Documents |
Upload: | erin-hudson |
View: | 43 times |
Download: | 2 times |
Principles of Language Use for User Centered Design
of Software User Interfaces
Derek BrockNavy Center for Applied Research in
Artificial IntelligenceNaval Research Laboratory
230 July 2002Derek Brock, Navy Center for Applied Research in Artificial Intelligence
A different design perspective
• Designing a software user interface is like writing a reference book with many chapters and subsections– The Designer is the author – Users are the readers
• Interaction starting points in the user interface—menu choices, documents, tool bar icons, etc.—are like chapters and subsections
• The author’s goal in each chapter or subsection is to provide readers with:– Clear and useful expositions– Pointers to dependent and related references
330 July 2002Derek Brock, Navy Center for Applied Research in Artificial Intelligence
What’s the point?
• Software is designed by people to be used by people– The purpose of a user interface is to coordinate what
the designer thinks users need to understand—and do—to use the software
– But few users can read a designer’s mind
• Any form of coordination between people requires some form of communication
• Successful communication requires—– Collaboration (a willingness to work together)– Meaning and understanding– The use of various kinds of signals– A shared basis for the use of those signals
430 July 2002Derek Brock, Navy Center for Applied Research in Artificial Intelligence
What goes on when we read a book?
• A successful communication takes place—– The author and the reader carry out a kind of
collaboration—both have to work at it– The author’s meaning (ideally) becomes the reader’s
understanding– A variety of linguistic and nonlinguistic signals are
used—text, diagrams, illustrations, etc.– The author and the reader begin with a communally
shared basis for the signals used…• …which becomes a personally shared basis when the
communication is complete
530 July 2002Derek Brock, Navy Center for Applied Research in Artificial Intelligence
The same ideas apply to software use
• When software use is successful—– The designer and the user carry out a kind of
collaboration—again, both have to work at it– The designer’s meaning (ideally) becomes the user’s
understanding– A variety of linguistic and nonlinguistic signals are
used—labels, icons, actions, sounds, messages, etc.• The user also makes use of input signals—more on this
later
– The designer and the user begin with a communally shared basis for the signals used…• …which becomes a personally shared basis when the
communication is complete
630 July 2002Derek Brock, Navy Center for Applied Research in Artificial Intelligence
The larger picture
• Signal use is really language use• People use language to do things with each other • People’s language use skills are their greatest
resource whenever meaning and understanding are involved
• The design of human-computer interaction is a representation problem that primarily involves the designer’s meaning and the user’s understanding
• The principles of language use are a foundational framework for this design problem
730 July 2002Derek Brock, Navy Center for Applied Research in Artificial Intelligence
Introducing the study of language use
• This is not the study of linguistics– Linguistics is the study of the development and
structure of languages, particularly spoken and written languages
• Instead, it’s the study of what’s involved in how people use signals to do things together– How do people convey and grasp each other’s
intentions?– How do people succeed in coordinating this?– How does this work when people aren’t face-to-face?– What are the cognitive and social processes
involved?– How is language used in different conceptual
domains of action?
830 July 2002Derek Brock, Navy Center for Applied Research in Artificial Intelligence
First concepts and terms of art
• Language use requires participants and raises the notions of speaker and addressee
• In addition to signals, language use always involves speaker’s meaning and addressee’s understanding
• Joint actions are atomic instances of language use– They are not composed of disjoint actions that are simply
carried out by speaker and addressee independently– But, instead, are the coordinated result of actions that
speaker and addressee carry out together, as a duet—even when this must happen across time and space
• Joint activities emerge from sequences of joint actions– Interactive joint activities can never be fully planned in
advance– This is the proper level for specifying roles, social purposes,
goal hierarchies, etc.• Language use always takes place in a setting
– Settings shape how language is used
930 July 2002Derek Brock, Navy Center for Applied Research in Artificial Intelligence
Settings
• The notion of a setting in language use combines notions of scene and medium– Scene—where the language is used– Medium—the manner of language use (e.g., spoken,
written, recorded, gestural, a mixture, etc.)
• Settings range in type, in their properties, and in what they require of people and the roles each person plays
• The fundamental paradigms are:– Face-to-face settings—conversation, lecture, wedding…– Written settings—books, letters, movies, recorded music…
• A joint activity’s setting determines how joint actions are coordinated and what skills are needed– Face-to-face settings are the most basic (or natural)– Written settings are the most demanding (or contrived)
1030 July 2002Derek Brock, Navy Center for Applied Research in Artificial Intelligence
Joint actions
• Complete joint actions involve the coordination of a causally related ordering of cognitive, physical, and perceptual subactions (speakers addressees)– Proposing an intention considering it (engagement) – Conceptually signaling the intention (meaning) recognizing
it (understanding)– Presenting the signal identifying it– Executing the presentation attending to it
• All coordinated joint actions between people require these subactions in some form; some examples—– A handshake, paddling a canoe, dancing, making music, etc.– Asking for directions, posting a notice, driving in traffic, etc.
• To coordinate these subactions as parallel parts of the whole, people depend on their intuitions of downward evidence and upward completion
1130 July 2002Derek Brock, Navy Center for Applied Research in Artificial Intelligence
But wait…
• How are signals used to convey meaning and achieve understanding?– How can a speaker be confident a particular signal
will convey his or her intended meaning– How does an addressee converge on the intended
meaning represented by the signal
• It turns out this is a deep problem for computer science, but something that nature has solved rather well
• Meaning and understanding depend on the notion of common ground
1230 July 2002Derek Brock, Navy Center for Applied Research in Artificial Intelligence
Characterizing common ground informally
• Common ground is knowledge people intuitively assume they can use with each other on a presumed basis of shared information– The idea is that when Alan and Zoe go to a movie…– …afterwards, they can presume they had mostly
similar experiences of its content…– …and can assume that each will now understand the
other’s use of this newly shared information
• Shared experience naturally becomes a shared basis of information– In fact, this is exactly how people introduce and then
use new ideas with each other
• But even more is implied by this notion of a shared basis…
1330 July 2002Derek Brock, Navy Center for Applied Research in Artificial Intelligence
Awareness of a shared basis is the essence of common ground
• Verification or inference of similar experiences also builds shared bases– Alan and Zoe don’t even need to see a movie
together to use their knowledge of it with each other—they can simply find out if they have both seen it
– Common ground acquired in this way often contains many discrepancies—but all common ground contains discrepancies
– And note: Alan and Zoe’s separate knowledge of a movie isn’t common ground at all until they have evidence that confirms they’ve both seen it—that their knowledge is actually shared
• In general, people can usually account for what they take to be common ground
1430 July 2002Derek Brock, Navy Center for Applied Research in Artificial Intelligence
Building common ground
• Common ground is built in joint activities– People expect each other to reason similarly and to
maintain a similar awareness of what they are doing together
– People expect each other to be engaged• Common ground in joint activities can be divided into at
least three parts:1. Knowledge initially taken to be held in common2. Knowledge of the activity’s current state3. Knowledge of what has conspicuously taken place so far
• When common ground is missing, meaning and understanding immediately break down– But people are very good at repairing misunderstandings
and filling in missing common ground• Building common ground is a lifelong social and
personal process
1530 July 2002Derek Brock, Navy Center for Applied Research in Artificial Intelligence
Using common ground in joint actions: posing coordination problems
• People implicitly justify their common ground by using what they take to be shared references to it– Shared references are simply signals that refer to elements of
common ground; they function as coordination devices
• Joint actions involving meaning and understanding are really coordination problems people pose and agree to solve together
• People use coordination devices to pose coordination problems, focus their intent, and to indicate their solution
• Most coordination problems in language use are familiar and most are readily solved
• To make a coordination problem easy to solve—– It should have a straightforward solution when it is posed
– Its coordination devices should fully indicate the solution
– The solution should be obvious from what is common ground
1630 July 2002Derek Brock, Navy Center for Applied Research in Artificial Intelligence
For example…
• What does Zoe do if she and Alan are sharing popcorn at the movies and she wants a sip from Alan’s drink?– She can signal Alan either by asking him for a sip or by simply
pointing to his drink in a way that draws his attention– By asking or gesturing, Zoe makes a clear and obvious shared
reference to her common ground with Alan – Either ploy is an excellent coordination device because—
• Zoe has a straightforward solution in mind that Alan can fulfill• Either device explicitly indicates her solution • Her solution is obvious given her common ground with Alan
– In agreeing to Zoe’s coordination problem, Alan correspondingly expects that—• Zoe has an intended solution in mind (what he’s supposed to do)• Her signal contains enough information to solve the problem• He can figure out the solution on the basis of what is obvious in their
common ground
1730 July 2002Derek Brock, Navy Center for Applied Research in Artificial Intelligence
More about coordination problems
• Codifying the joint premises for solving coordination problems:– Solvability (a straightforward solution exists)– Sufficiency (the solution is fully indicated)– Salience (the solution is obvious from common ground)
• Working out coordination problems is a dominant part of people’s language use skills.– These skills have their origins in the perceptual richness of
face-to-face settings • In choosing the right shared basis to use as a
coordination device, joint salience is usually the most important consideration. – Appealing to this aspect of people’s attention in context
greatly increases the likelihood of their converging on the intended solution.
1830 July 2002Derek Brock, Navy Center for Applied Research in Artificial Intelligence
Salient signals
• Signals that make salient coordination devices with respect to common ground include:– Actions (including sounds and gestures)
• These command perceptual attention
– External representations• These are physical aspects of a language use setting that have
conceptual meaning(s) for a joint activity; for example—– Playing cards, the elements of a board game, ceremonial objects, etc.
• External representations enjoy perceptual immediacy
– Conventions of use• Conventions are representations of standing solutions to
frequently occurring coordination problems• When a convention is used as a coordination device, addressees
usually know how to proceed immediately• Languages are full blown systems of conventions• Conventions of use give written settings their power
1930 July 2002Derek Brock, Navy Center for Applied Research in Artificial Intelligence
Layers—conceptual domains of action
• Layers are shifts in the conceptual domain of action (and/or its setting) that shape the focus of common ground for purposes of solving particular kinds of coordination problems in joint activities
• Layers depend on people’s abilities to change perspective• Storytelling illustrates the basic paradigm—
– Primary layer• A storyteller and an audience use language in a real world setting
as themselves– Secondary layer
• The characters of the story use language in the story world setting
• Note that the audience has no trouble shifting its common ground into the story world
– Further layers within the story may arise• Layers can involve play acting, removes in space, time,
and reality, persons not present, changes in setting, etc.
2030 July 2002Derek Brock, Navy Center for Applied Research in Artificial Intelligence
Summary of language use principles
• Language use takes place in settings (e.g., face-to-face, written) that shape its coordination and its required skills
• Language use is a joint activity; it is participatory as opposed to an individual activity
• Joint actions are the unit of language use– They involve cognitive, physical, and perceptual levels of
subactions– They are coordination problems
• Coordination devices are signals people use to pose, focus, and solve coordination problems– The joint premises for solving coordination problems are
solvability, sufficiency, and salience– Salient sources for coordination devices include actions,
external representations, and conventions of use• Layers are conceptual shifts of perspective in joint
activities that shape the focus of common ground
2130 July 2002Derek Brock, Navy Center for Applied Research in Artificial Intelligence
Language use in human-computer interaction
• Speaker and addressee in human-computer interaction are designer and user– Software use is their joint activity– This joint activity fully involves designer’s meaning
and user’s understanding– Software use takes place in a written setting and
makes use of an important secondary layer
• Joint actions are the unit of software use– An entire class of joint actions in software use are
called user interactions– User interactions and other joint actions in software
use are simply coordination problems– Designing an effective system of coordination devices
highlights the problem of building common ground
2230 July 2002Derek Brock, Navy Center for Applied Research in Artificial Intelligence
Thinking about software as awritten setting
• As with any written setting, a software user interface is designed and produced in advance of when it is taken up by its user– It is useful to think of a written setting as a type of pre-
coordinated joint activity
• Conventional written settings build common ground through a static, serial narrative whose domain typically defines a secondary layer of action
• The written setting of software use differs from this paradigm in three important ways– It has a non-serial narrative– Its presentation is dynamic (non-static)– Its secondary layer of action places the user in a type of
face-to-face interactive setting
2330 July 2002Derek Brock, Navy Center for Applied Research in Artificial Intelligence
Characterizing the first two layers of software use
• In the primary layer of software use—– The designer and the user pursue their joint activity
(software use) as themselves in a written setting– In effect the user “reads” the designer’s presentation
—the display portion of the user interface
• In the secondary layer of software use—– The user participates in a face-to-face interaction
with the computer (as opposed to the designer)
• Particularly because of the change in setting, each layer makes different demands of the user’s language use skills
2430 July 2002Derek Brock, Navy Center for Applied Research in Artificial Intelligence
Characterizing the primary layer of software use in language use terms• The design poses a fixed set of coordination problems • Most software uses a cafeteria paradigm for user interaction
– This paradigm implies a non-serial narrative– The design pre-coordinates the joint activity, but the user has
opportunistic control of how the software use proceeds– The user interface generalizes to a huge menu of starting points
• The presentation uses a language of coordination devices
made up of a variety of linguistic and nonlinguistic signals, – These include elements of natural language, visual artifacts,
and behaviors (i.e., actions and procedures)– Many elements of this language are (rightly) presumed to be
conventions• The presentation serves, in part, as an external
representation of the joint activity – Manipulable display elements are generally given specific
conceptual meanings in the context of the software use
2530 July 2002Derek Brock, Navy Center for Applied Research in Artificial Intelligence
Building common ground in the primary layer of software use
In a cafeteria-style presentation:• Both the designer and the user strive to find and build
common ground, but…• The designer’s control over the process of building a
complete body of common ground with the user in the planned manner of a serial narrative is sacrificed
• The user’s common ground with the designer builds contingently (irregularly) on the basis of—– Opportunistically solving coordination problems posed in
the interaction design – And, more generally, opportunistically becoming familiar
with the presentation language • Despite the presentation’s non-serial narrative—
– Over time, users generally develop a sense of where common ground is missing and will look for it unless the effort proves to be too costly
2630 July 2002Derek Brock, Navy Center for Applied Research in Artificial Intelligence
Designing coordination problems in the primary layer of software use
• Coordination problems in a software design deliberately anticipate the user’s domain goals – But addressees’ knowledge in written settings can never fully
anticipated– Users would benefit from an account of what the software can
and cannot do• All coordination problems posed by a design should honor
users’ expectations of salience, sufficiency, and solvability– From a practical standpoint, this is not always possible
• Nontrivial designs generally involve multidimensional conceptual dependencies
• Opportunistic user control pre-empts the designer’s control over the introduction of critical material
• When users are unfamiliar with relevant supporting concepts, they may not know what to do through no fault of their own
– Working solution: identify where common ground may be missing and provide users with immediate, context dependent access to it
2730 July 2002Derek Brock, Navy Center for Applied Research in Artificial Intelligence
Other primary layer issues in software use
• Serial narratives naturally correspond to the third part of common ground—people’s record of what has openly occurred so far in their joint activity– Serial narratives in hard copy are—
• Intuitively indexed; viewed as external representations, they easily represent their current state in this way
• Readily open to review when misunderstandings, questions, or lapses of memory arise
• These advantages are de-emphasized in the written setting of software use– The presentation may not always represent the current state
of the joint activity– External representations in the presentation are generally
dynamic (non-static) and actions are evanescent– Cafeteria organization of entry points obfuscates the user’s
intuitive ability to index the presentation and/or locate particular functions
– Little or no narrative record is kept for the user to consult
2830 July 2002Derek Brock, Navy Center for Applied Research in Artificial Intelligence
Characterizing the secondary layer of software use in language use terms• What we call “human-computer interaction” takes place in
the secondary layer of software use– In this layer, the user participates in a direct interaction with
the computer (as opposed to the designer)• The interaction is designed to resemble a joint activity in a
face-to-face setting…– But this setting is not equivalent with a face-to-face setting
between people– The user and the computer use different communication
languages…– …and their perceptors are mismatched– External representations in the presentation are technically
available to the computer but are generally unused by the system to justify common ground beyond, at most, small numbers of interactions
• Joint actions between the user and the computer are designed to closely resemble joint actions between people
2930 July 2002Derek Brock, Navy Center for Applied Research in Artificial Intelligence
Digression—knowledge people accumulate about activities in general
Claim: In any interactive activity, the knowledge people accumulate and are able to justify corresponds structurally to the three parts of common ground:1. Knowledge initially held about the activity2. Knowledge of the activity’s current state3. Knowledge of what has conspicuously taken place so far in
the activity
• This accumulated knowledge of an activity— – Is common ground when the interaction takes place
between people and shared informational bases exist to justify it as common ground
– Is only an individual’s conception of the activity when the interaction takes place with elements of the environment that accrue no such knowledge
– Is part common ground and part individual conception in human-computer interaction
3030 July 2002Derek Brock, Navy Center for Applied Research in Artificial Intelligence
Building common ground in the secondary layer of software use
When the user engages the computer through a software user interface:
• The interaction is designed to resemble a joint activity in a face-to-face setting
• The user actively keeps track of contingent interaction knowledge corresponding to three parts of common ground as well as informational bases to justify it
• At most, software keeps track of some of the same knowledge and shares informational bases in limited, ad hoc ways, such as undo mechanisms and other kinds of interaction histories
• Contingent interaction knowledge the software fails to be able to justify and use cannot be construed as common ground in the secondary layer’s joint activity
• Consequently, some of the user’s most valuable face-to-face language use skills go largely unused
3130 July 2002Derek Brock, Navy Center for Applied Research in Artificial Intelligence
Key points about language use for the design of software user interfaces• The written setting of software use differs from
conventional, serially narrated written settings in several critical ways—– Its non-serial narrative structure builds common ground
irregularly and introduces problems associated with conceptual dependencies
– Dynamic and evanescent aspects of the presentation and its cafeteria organization reduce its hard-copy utility for the user for purposes of reviewing the software use
• User interactions and other joint actions in software use are simply coordination problems– Almost all display elements of a user interface should be
understood as coordination devices– Users expect all coordination problems posed by a design to
honor the joint premises of salience, sufficiency, and solvability
• Strategies for the accumulation of common ground with the user in the secondary layer remain ad hoc, at best
3230 July 2002Derek Brock, Navy Center for Applied Research in Artificial Intelligence
Practical language use advice for user centered design
• Focus on coordination problems from a primary layer perspective
• Identify coordination problems where common ground may be missing for the user– Provide the user with immediate links to context sensitive help;
make it easy for the user to return from help to the task– Provide explanatory drop-down labels for icons (“tool tips”) and
other external representations; do this even for cases where a terse standing label may be initially insufficient
– Give users a simple, point-of-interaction facility for annotating their software use
• Provide a hyperlinked index to the software’s set of functions (this can be part of a help mechanism)
• Provide a deep undo mechanism• Be aware of the disorienting effect of interface clutter, the
value of meaningful organization, the harm of unexplained evanescent behaviors, and the inevitable wrath of the user
3330 July 2002Derek Brock, Navy Center for Applied Research in Artificial Intelligence
Take home messages
When the normal function of common ground in a language use setting is impaired, people are forced to work harder cognitively to accomplish their goals
Designers of software user interfaces must strive to compensate users for a paradigm that, in primary layer of software use, inherently builds irregular common ground
Building true common ground between the user and the computer in the secondary layer of software use awaits the future
3430 July 2002Derek Brock, Navy Center for Applied Research in Artificial Intelligence
Language useful references
Brock, D. (2002). A language use perspective on the design of human-computer interaction. Proceedings: Workshop on Cognitive Elements of Effective Collaboration. Arlington, VA: Office of Naval Research, Code 342.
Alterman, R. and Garland, A. (2001). Convention in joint activity. Cognitive Science (25)4,2001.
Clark, H. H. (1996). Using language. New York, NY: Cambridge University Press. (read this if nothing else!)
Hutchins, E. (1996). Cognition in the wild. Cambridge, MA: MIT Press.
Norman, D. A. (1992). Turn signals are the facial expressions of automobiles. Reading, MA: Addison-Wesley.
Norman, D. A. and Draper, S. W., eds. (1986). User centered system design. Hillsdale, NJ: Lawrence Erlbaum.
Winograd, T. and Flores, F. (1986). Understanding computers and cognition. Reading, MA: Addison-Wesley.