Date post: | 13-Apr-2018 |
Category: |
Documents |
Upload: | jojo-kaway |
View: | 219 times |
Download: | 0 times |
of 6
7/26/2019 Collaborative Ethnography, An Approach to the Elicitation of Cognitive Requirements of Teams
1/6
Proceedings the 2009 13th International Conference on Computer Supported Cooperative Work in Design
Collaborative Ethnography: An approach to the elicitation of cognitive
requirements teams
Luiz Carlos L Silva Junior , Marcos
R
S Borges , Paulo Victor
R
de Carvalho
Programa de P6s-Graduac;ao em Informatica Universidade Federal do Rio de Janeiro Brasil.
{luizcarlos mborges}@nce. ufrj.r
Abstract
The paper presents a collaborative ethnography
approach
or
cognitive requirements elicitation
o
work
teams in complex environments. It discusses the
concepts
o
cognitive systems and their requirements
andpresents a review
o
methods commonly used in the
elicitation
o
requirements both in the case
o
traditional systems and complex systems. hen it
points to some advantages
o
a collaborative approach
in comparison to other approaches. An evaluation plan
o
the approach based on experimentation and the
development
o
a groupware
to
support the proposed
methodology is also presented. he groupware aims to
stimulate collaboration and an organization in the
elicitation process
o
cognitive requirements.
Keywords: Ethnography, CSCW, Requirements,
Cognitive systems.
1. Introduction
The human interactions in group work activities are
important objects in the study human problem
solving and decision-making processes, and give
important clues to investigate the tacit knowledge that
teams uses during their work activity. Behind these
interactions we can find important mechanisms
reasoning that will drive people, according to their
experience, to select and manage their actions in
context-dependent work situations. The analysis
these interactions can lead to the discovery basic
requirements for the construction artifacts that can
efficiently support the process
team members
decision-making.
The aim
this paper is to show how collaborative
ethnography (i.e. an ethnography performed by many
agents who can interact with each other) can be used to
elicit the knowledge that shape team members
interactions. We also show in which way the
knowledge obtained using the collaborative
ethnography can be very important to elicit useful
cognitive requirements for the implementation
information support systems that will give adequate
support to people tasks.
We begin the paper with cognItIve systems
definitions and its requirements through some examples
978-1-4244-3535-7/09/ 25.00 2009 IEEE
these systems in real life. Then we present some
cognitive task analysis (CTA) methods used in
knowledge elicitation process describing their strong
and weak points. Later, the collaborative ethnography
approach for cognitive requirements elicitation will be
presented and compared with the other methods
normally used. Finally, we show the requirements a
groupware to support the collaborative ethnography
approach to improve some weak points
the method,
enabling its use in an effective way.
Cognitive systems design requirements
A cognitive system is a self regulated and adaptable
system that functions using knowledge about itself and
the environment to planning and modify their actions
[9]. adequate design a cognitive system depends
basically on: The existence a common vocabulary
between their parts; the study
the agents' cognition
during their actual activity, analyzing the man
technology interactions in real environments [10] rather
than studies that analyze each system component apart
(i.e. man, machine and interface) in more controlled
settings. Our research focus on cognitive systems
design because their ever growing sociotechnical
complexity due new technologies (automation,
communication, information) and new system
structures (more regulation levels, consumer feedback
and control) claims to a change the traditional
(linear) models and old paradigms
human cognition
in system design to they take in consideration the actual
human-system interactions.
Nowadays we can observe the technology
facilitating the execution
tasks and operations with
high degree automation. Cognitive systems are
present in energy power plants, air traffic management
systems, vehicles, control
emergencies, and in work
environments that demand high degree coordination.
To produce effective cognitive systems, i.e. cognitive
systems that can be adequately handled by the human
agents, there is a need to consider actual cognitive
requirements in the system design phase.
However, the increase use
technology results in
an increase the complexity cognitive systems. The
digital control rooms with large video display units,
computers with ever growing processing capacity, and
other technological devices to support cognitive tasks
167
7/26/2019 Collaborative Ethnography, An Approach to the Elicitation of Cognitive Requirements of Teams
2/6
generate another problems, as the search for high
efficiency, making that the systems operate closer to
their maximum capacity and safety boundaries, an
increasing performance dependence among the various
tasks, and a significant growth o the amount o data
that have to be processed by the human being [10].
Such situations can generate a considerable
degradation in these systems. To cope with complexity,
human agents adapt their activities generating
variability in the tasks execution, which may lead to
systems malfunction the drift to failure mechanism). It
is important to note that in the last few decades serious
accidents have happened in sociotechnical complex
systems, indicating the need to change the paradigm on
how human cognition is treated in complex systems
design [18].
Many o these system failures and accidents emerge
in situations, in which operators did not understand the
system actual situation, i.e., the information systems do
not provide the right information in the right moment
to the operators who were not able to adapt their
behaviors according to the actual situation demands and
make their decisions in a safe way. To deal with this
problem is necessary to emphasize design processes
that facilitate the adaptation o the human cognition to
the system functionality.
The cognitive requirements, in the scope
o
this
work, can be defined as the functional requirements o
the system that will give critical support to the cognitive
activities o the operator throughout the execution o
his/her work. This means that these requirements will
be paramount for the construction
o
information
systems capable to enable people to achieve adequate
situation awareness enhance perception possibilities,
decision-making and action planning support).
These cognitive requirements must be incorporated
in the design o displays, man-machine interfaces, and
are present in the collaborative characteristics o many
systems. We can observe some examples
in
control
rooms [5], [19]; military decision-making systems [3];
air traffic control [2], [8].
To elicit knowledge for complex system design, we
argue that is necessary to carry out cognitive task
analysis in actual work situations, understanding how
and why operators make their actions, and the intrinsic
human-system relations human-organization, human
technology, and human-human actions). Therefore, we
claim that collaborative ethnography is one o the most
adequate approach to elicit cognitive requirements. In
the next section, we will describe some methods
traditionally used to elicit knowledge for information
systems design, in order to compare them to the
collaborative ethnography approach we propose in this
paper.
3. Knowledge elicitation approaches for
cognitive
t sk
analysis
In this section, some CTA methods already used for
in information system design in several environments
will be described. Some o these methods, like
interviews and direct observation are often used in the
elicitation o requirements for information system
design and business-oriented processes. Ethnographic
approaches are not so used in information systems
design, but they are widely used in the elicitation
o
experts knowledge and social factors.
3.1. Interviews
Interview is one
o
the most common methods for
requirements elicitation, however nor always does this
technique capture all the information needed to a
complete requirements definition. To a large extent, the
data generated from interviews are field notes, meeting
reports, which are difficult to analyze later on, and can
be complemented with the existing documentation in
the organization.
In many situations, the information elicited from
interviews was incomplete, due the existence o
difficult questions to be answered, i.e., questions related
to activities where tacit knowledge is used. Bell [ ]
noted that is not rare the situation in which people tell
the idealized information, i.e., people say how they
should do their tasks rather than how they actually do
them.
There are research to refine the interview technique
[4], helping the organization and integration o the data
using argument structures or domain models, as the
Joint Application Development JAD) approach that
provide a way to requirement elicitation allowing the
identification o different points o views, and means to
solve conflicts and achieve consensuses.
3.2. Observation
The observation o people actual work activities in
the organizations is a useful way to understand the
interactions, practical skills, and tacit knowledge
developed by the work teams, as well as the
organization culture. According to Bell [1], we do not
have to ask people what they do. Rather, we must
observe how people carry out their activities, because
from observation findings it is possible to discover the
reasons and motivations behind systems use, as well as
the problems or difficulties related with system
usability.
The use o observation appears in some domains for
knowledge elicitation and system improvement. In
ergonomics, direct observations are the most used
method to understand the actual work conditions, the
operators activities and their influence on the people
health and in the system production [6].
168
7/26/2019 Collaborative Ethnography, An Approach to the Elicitation of Cognitive Requirements of Teams
3/6
The observation can be participative or non
participative according to the behavior
o
the observers
in the observation process [17]. In the participative
observation, observers interact with the workers
observed to achieve a better understanding o the work
activities. The non-participative observations can be
direct or indirect. In the direct observations the people
are observed individually, the behaviors are written
down and registered field notes , whereas in the
indirect observations there is the use o technology to
support the register process audio and video
equipment .
However, observations techniques are not efficient
for the capture
o
cognitive skills in many cases. In
emergence situations where the observer cannot
intervene directly or even be present during the activity,
it is easy to catch the tacit knowledge and the cognitive
skills using only non-participative observations.
Therefore, the experiments, simulations, interviews or
tell histories techniques may support the capture tacit
knowledge in these situations [3].
3.3. Ethnography
The ethnography is a methodology that comes from
the Social Anthropology, which consists o studying an
object living the same reality o this object. This
approach aims to understand and to describe, a nation,
its people or culture, using natural or participative
observation for long periods.
Using ethnography, the observed group or culture
issues are understood by living in the same
environment, being present, and making the things that
the people make and as they make [1]. It is possible to
understand how and, mainly, why the activities are
done in one determined way, because the phenomenon
is studied inside the social, cultural and organizational
context [16].
It is important to note that the ethnography contains
traces o the two previous techniques, because the
information collected are complemented with data from
observations and from informal interviews. However,
the difference in the ethnography is that all individuals
observers and observed are situated in the same
context. Therefore, the method has many possible
advantages [13]: Bigger familiarization with the domain
o the organization; discovery o the informal activities,
issues related to cognitive and collaborative
mechanisms; better understanding
o
the actions,
decisions, strategies, behaviors and interactions
o
the
agents in the context where they occur; perception on
how the interaction with the resources or devices
occurs; and more visibility about usability problems o
the existing technologies.
Moreover, the ethnography shows how the physical
disposal
o
the work environment and the layout
o
the
devices and equipment have influence, positive or
negative, on the efficiency o the activities as well as in
the difficulties, and the respective adaptations to face
work problems.
Using the ethnography for the elicitation
o
requirements, we are able to get more details and a
more complete description o these requirements,
because the activities are observed instead o described
by the users. These factors are especially important for
the analysis and design
o
complex systems.
One
o
the ethnographic techniques we will use
ahead in the Collaborative Ethnography we propose in
this research is the Fast Ethnography [11] see figure
that recognizes the time and resource limitations to
carry out detailed and intensive ethnographic studies,
and uses short term observations together with the
debriefmeetings feedback.
Figure Processo de etnografia rapida
The ethnographic approach presents also some
problems. Normally, its execution is complex, due the
impossibility o one person alone to capture and to
analyze all the information
o
the social environment.
The method demands extensive planning and
coordination issues
the ethnography will be carried
out by teams, because people may have different
perceptions and points o view about the observed
activities. Another limitation is the need o a long time
to capture and analyze what occurs in the field, to get
rich and detailed information [15]. In some cases, is
necessary a familiarization with the domain to be
studied. Moreover, it is difficult the use o the
technique in large work settings, complex and
distributed, due the huge number o variables to be
observed.
The technique also can present risks for the
researchers, or to be impracticable the presence o the
researches/observers jeopardize the work activities
observed. In some cases, there is the need o getting the
permission for entrance in the work settings and for
registering the information, and also the acceptance and
the assent from those who will be observed [6].
In the scope o software development process the
ethnographic technique also presents some difficulties.
One o them is absence o formalism and objectivity
required for information systems designers, because it
presents the results in a descriptive and textual format
[20], making difficult their transformation in software
requirements.
69
7/26/2019 Collaborative Ethnography, An Approach to the Elicitation of Cognitive Requirements of Teams
4/6
4. The collaborative ethnography approach
for cognitive requirements elicitation
In this section we will discuss how the
collaborative approach can address the elicitation
of
cognitive requirements elicitation
of
wor k teams. We
first stress the importance
of
the combination
of
collaboration and ethnography particularly how this
combination can help the study
of
complex activities.
Then we present a succinct review
of
previous work
which have used this combined approach to elicitate
of
interactions and the social dimension
of
teamwork.
groups and their interaction being performed by a team
of
ethnographers is a very challenging goal. The huge
amount
of
information collected that needs to be
organized and processed requires not only an organized
process bu t also an appropriate supporting tool. The
complexity
of
this alternative is a consequence
of
combined circumstances originated by the multiple and
perhaps conflicting perspectives from the observers
p ar t an d the potentially high nu mb er
of
interactions
among the members
of
the observed group. We will try
to avoid the loss
of
relevant information intrinsic to
scenario B.
4.1. hycollaborative ethnography?
4.2. Applications of collaborative ethnography
The collaborative ethnography is a technique that
has been used extensively in other domains for the
social analysis
of
work activities. May and Pattillo
McCoy [14] reported a work reproducing the way the
ethnographers discuss their findings in a collaborative
way generating a single document with the information
collected and their different viewpoints. They reported
how different the ethnographers perceptions about the
environment are. They claim that variety is due to
several cognitive factors that also help the reach
of
consensus in the discussion.
It is also important to note the value of
the
collaborative ethnography to complement the details
of
the object
of
study. The inconsistencies and ambiguities
of
observations are more easily identified and solved. In
other words the ethnographers
playa
positive influence
on the results
of
the process.
On the other hand it becomes clear that the
collaborative ethnography has some drawbacks. First it
shares or perhaps increases the time spent to collect
analyze and conclude about the collected information.
The need to join to compare and to discuss the
information collected requires time. The collaboration
required needs extra effort and is time consuming. The
field notes needs to be combined in order to reflect the
different perspectives
of
observers.
According to May and Patillo-McCoy [14] the
observation itself influences the results
of
the
ethnography. Thus the use
of
several observers opens
the possibility
of
analyzing this impact in the process
and at the same time enriching the information
collected. We expect that the approach contributes to
the expose
of
tacit information specialized knowledge
and the decision rationale mostly due to the interaction
between the ethnographer with the task the
environment and especially with other ethnographers.
Machado [13] also carried out a collaborative
ethnography study aimed to elicitate software
requirements. The study reports an experience with
field observations and with the use
of
a tool that
facilitates the consolidation
of
data from multiple
sources into a shared repository. In comparison with
traditional techniques based on interviews the
h t
the om Ina o e
nograp Iy s U les
Observer/ Observed Individual Teams
Individual A B
Teams C
T bl 1 C r
First it is pertinent to clarify why to add the
collaboration to the ethnography method. A
straightforward combination between observers and
observed subjects is presented in Table
We ca n have
a single pe rso n observing another pers on o r a group
and a team observing a single person or a group.
Scenario A is the most common case: An
individual observing another individual carrying out
some task. This situation also refers to a number
of
observer/observed pairs being done in parallel.
f
the
observation refers to the same task some variation may
occur between different observers/observed. The results
are quite dependent on the background and on the
previous experience
of
the observer. There may also be
some information loss during the observation. In this
scenario the aggregation
of
information from different
sources is not an easy task and may show some
inconsistencies.
Scenario B is possible but not common. In this
scenario the loss
of
information is potentially high. It is
very difficult to a single observer to capture and to
understand all tasks and interactions particularly if the
activities are complex. A way to overcome these
difficulties is to do the observation in several sessions.
The scenario C is normally used when the
observation requires multiple perspectives over the
same observer o r task. In this scenario it is expected
that the information gathered and processed would be
richer that that collect by a single observer. This can be
considered a particular case
of
scenario D assuming
that the observers would meet and discuss about their
findings before during and after the ethnography. This
is not easy to realize without the support
of
an
appropriate process and a supporting tool.
Finally the scenario D is the circumstance that
will be dealt with in this work. The observation
of
170
7/26/2019 Collaborative Ethnography, An Approach to the Elicitation of Cognitive Requirements of Teams
5/6
observation approach have shown more efficient and
generated more complete information.
The experiments carried out in that study has
shown that a groupware plays an important role in the
coordinating of activities and on the organization of the
data videos, photos, comments, notes, etc. reducing
the burden generated by the collaborative ethnography.
5. roupw re requirements to support
collaborative ethnography
According to Guerlain [7], the evaluation
of
activities from a coordinated team involves the
independent observation of multiples individuals. The
analysis of these independent observations requires they
should be coordinated, codified and correlated before a
subjective evaluation can be performed.
Thus, similar to other works [7]
[ 2]
[13], the
support
of
a groupware tool aimed to assist the
information exchange and the discussion among
ethnographers is desired. The groupware tool that
supports the collaborative ethnography should facilitate
collaboration and interaction after the gathering
of
field
data in order to identify the cognitive requirements with
less effort.
The basic functions for the groupware to support
the collaborative ethnography are:
register, update and closing
of
ethnography
projects;
record profiles of observer and observed persons;
assign of observers to observed activities and
persons;
record of notes and maintenance ofhistorical data;
document creation and its association with
problems, comments, ideas and hypotheses raised
by ethnographers;
support the discussion and negotiation about the
documents and the requirements generated;
support awareness about the level of participation
and contribution;
support multimedia elements;
support awareness mechanisms for both
synchronopus and asynchronous interaction,
informing them about contributions new elements,
comments, etc. added by other members.
The groupware tool will be used after the
information capture in the field. The team members will
be able to individually insert all type of information
they believe are important to the requirements
elicitation. Then, the tool will support the discussion,
the consensus information, and also how the former led
to the latter. It will also allow members to group their
notes using categories actor, setting, process, etc. . The
notes can be associated with problems, cognitive
requirements and possible solutions to these
requirements. See Figure 2 for a schematic view of the
data managed by the groupware.
Figure Products and their relationships
Collaborative Ethnography
The tool will provide means to coordinate the
ethnography activities in both asynchronous and
distributed way. It should help analysts to reach
consensus, besides inducing the reflexive thinking. It
will also help the planning
of
further studies and
observations on specific parts
of
the work, which
demand a deeper analysis. The tool should be designed
using usability principles that should be evaluated
during the study. However, the main goal of the study
and the tool, which is to contribute to the efficiency and
efficacy
of
the process, will be pursued. A set
of
questionnaires will try to verify the conformity
of
the
approach and the tool to these goals.
6
Conclusions nd future work
In this work we reviewed some important concepts
in the area of cognitive systems. We also revised the
main approaches for the knowledge elicitation:
interviews, observation and ethnography. We
confirmed that these approaches, though extensively
adopted, still show some inadequacies and inefficiency
in the requirements elicitation process, particularly in
relation to cognitive requirements in complex
environments.
To deal with these problems, we proposed a
collaborative ethnography approach as a more
appropriate alternative for the cognitive requirements
elicitation. We claim that the ethnography approach has
many advantages, such as a detailed analysis of the
environment where the operator performs his/her task.
When ethnography is combined with a collaborative
approach that supports different viewpoints as well as
interaction and discussion among the observers, we
believe we have an approach that produces a more
efficient and consequential results.
Part of the tuning of our approach is dependent on
the application domain. Thus the groupware tool has to
be highly parameterized. The goal is to facilitate the
identification
of
cognitive requirements from the mass
of information gathered in the process and the reaching
of consensus over the main points to address.
7
7/26/2019 Collaborative Ethnography, An Approach to the Elicitation of Cognitive Requirements of Teams
6/6
For the experiments we will conduct two types
of
techniques for the elicitation
of
cognitive requirements:
the first based on interviews and the second based on
the approach proposed in this paper. We intend to carry
out a qualitative and quantitative comparison of the
results. We aim to evaluate not only the method itself
but also the support
of
the groupware tool. Our purpose
is to check
if
our hypothesis about the efficiency and
efficacy
of
our approach hold true in a variety
of
situations.
eferen es
[1]
Bell, G., (2004). Insights into Asia: 9 Cities, 7
Countries, 2 Years - What people really want from
technology. Technology@lntel Magazine.
[2] Bentley, R., Hughes, J., Randall, D., Rodden, T.,
Sawyer, P., Shapiro, D., Sommerville, I., (1992).
Ethnographically-informed systems design for air traffic
control, In Proc. Computer-Supported Coooperative
Work Conference, ACM Press, New York, 123-129.
[3]
Crandall, B., Klein, G Hoffman R.R., (2006). Working
minds: a practitioner s guide to cognitive task analysis
Cambridge, Mass.: MIT Press.
[4] Christel,
M
G.; Kang,
K
C., (1992). Issues in
Requirements Elicitation, Technical Report, CMU/SEI
92-TR-012, ESC-TR-92-012, September.
[5] dos Santos, I.1.A.L., Teixeira, D.V., Ferraz, F.T.,
Carvalho, P.V.R. (2007). The use of a simulator to
include human factors issues in the interface design
of
a
nuclear power plant control room. Journal of Loss
Prevention in the Process Industries 21(3), 227-238.
[6] Guerin, F., Laville, A., Daniellou, F., Duraffourg, 1.,
Kerguelen, A., (2004). Compreender 0 trabalho para
transforma-Io: a pratica da ergonomia. Edgard Blucher:
u n d ~ o
Vanzolini, Sao Paulo.
[7] Guerlain S., Shin T., Guo H., Calland 1.F., (2002). A
team performance data collection and analysis system.
In: Proc. of the human factors and ergonomics society
46th annual meeting, Baltimore, MD, 30 September-4
October.
[8] Harper, R., Hughes, J.A. and Shapiro, D., (1990).
Harmonious Working and CSCW: Computer
Technology and Air Traffic Control, In: Studies in
Computer Supported Cooperative Work: Theory,
Practice and Design, ed. Bowers and
S
Benford,
Amsterdam, North- Holland.
[9]
Hollnagel, E Woods, D D., (1999). Cognitive
Systems Engineering: New wine in new bottles.
International Journal of Human-Computer Studies,
Volume 51, Number 2, August 1999 p p 339-356(18)
[10] Hollnagel,
E
Woods,
D
D., (2005). Joint cognitive
systems: Foundations of cognitive systems engineering.
CRC Press, Inc., Boca Raton, FL, 2005.
[11] Hughes, J., King, V., Rodden, T., and Andersen, H.,
(1994). Moving out from the control room: ethnography
in system design. Proc. of the ACM conference on
Computer supported cooperative work, Chapel Hill,
North Carolina, October 1994, p 429-439.
[12] Hughes, 1., O Brien, 1., Rodden, T., Rouncefield, M
(1997). Ethnography, Communication and Support for
Design. Technical Report Ref. CSEG/24/1997, Lancaster
University, Lancaster, UK, 1997. Available at:
http://www.comp.lancs.ac.uk/computing/research/cseg/9
7_rep.html
[13]
Machado, R G., (2008). Developing of a Collaborative
Ethnography Method for Software Requirements
Elicitation. Rio de Janeiro, 2008. M.Sc. Dissertation,
Graduate Program in Informatics, Federal University of
Rio de Janeiro, Brazil (In Portuguese).
[14]
May, R A B., Pattillo-McCoy, M., (2000). Do You See
What I See? Examining a Collaborative Ethnography.
Qualitative Inquiry, Vol. 6,No 1 65-87.
[15] Millen, D., (2000). Rapid ethnography: time deepening
strategies for HCI field. Proc. ACM Symposium on
Designing Interactive Systems, ACM Press, New York,
280-286.
[16] Myers, M D., (1999). Investigating information systems
with etnographic research. Communications
of
the
Association for Information Systems, Vol. 2, Article 23,
December.
[17] Nardi, B., (1997). The use of ethnographic methods in
design and evaluation. In: Helander, M.G., Landauer,
T.K., Prabhu, P (Eds), Handbook of Human-Computer
Interaction. Elsevier Science, North-Holland, Holanda,
361-366.
[18] Reason,
T., (1990). Human Error. Cambridge, U.K.:
Cambridg University Press.
[19] Roth, E M., Christian, C.K., Gustafson, M., Sheridan,
T.B., Dwyer, K., Gandhi, T.K., Zinner, M.1., Dierks,
M.M., (2004). Using field observations as a tool for
discovery: analysing cognitive and collaborative
demands in the operating room. Cognition, Technology
and Work 6, 148-157.
[20] Sommerville, I., Rodden, T., Sawyer, P., Bentley, R.,
Twidale, M., (1993). Integrating ethnography into the
requirements engineering process. Proc. of IEEE
International Symposium on Requirements Engineering,
San Diego, CA, 165-173.
172