+ All Categories
Home > Documents > [IEEE 2012 IEEE 3rd International Conference on Cognitive Infocommunications (CogInfoCom) - Kosice,...

[IEEE 2012 IEEE 3rd International Conference on Cognitive Infocommunications (CogInfoCom) - Kosice,...

Date post: 08-Dec-2016
Category:
Upload: agnes
View: 213 times
Download: 1 times
Share this document with a friend
6
The Role of Discourse Markers in the Generation and Interpretation of Discourse Structure and Coherence Ágnes Abuczki Department of General and Applied Linguistics University of Debrecen Debrecen, Hungary [email protected] AbstractThe present paper studies the notions of turn management and topic orientation, and more specifically, a group of pragmatic elements, so-called discourse markers (DMs) which indicate the act of next speaker selection, turn-keeping, topic elaboration and digression. After definitions of discourse marker, turn, floor control types/turn segments, topical units and actions are provided, a list of verbal and nonverbal discourse markers will be given, and they will be grouped into subclasses on the basis of the meaning relations between the linked discourse segments and the type of floor control and thematic control affected. The fundamental question is if discourse markers are key indicators of discourse structure and can be used in the automatic interpretation of utterance meaning and rhetoric relations. The goal of the paper is twofold: firstly, to identify how nonverbal behavior (gestures, posture, gaze) may help disambiguate the actual function(s) of a discourse marker in a given context; secondly, to make explicit how different modalities work together in a synchronized manner in turn regulation and topic control during cooperative interaction. It is examined in twenty spontaneous dialogues of the Hungarian HuComTech multimodal corpus what roles and functions verbal and nonverbal discourse markers play in indicating discourse structure and coherence relations. Concordances and corpus queries presented address both the verbal and nonverbal features of different turn management behavior and topical actions accompanied or supported by discourse markers. Taking a semasiological approach, some of the controversial defining features of discourse markers generally listed in the literature will be tested on a few Hungarian discourse markers: hát ('well'), tehát ('so'), mondjuk ('say'), amúgy ('otherwise'), egyébként ('by the way'), szerintem ('I think'). The features in question regard their position (turn-initiality, topic unit-initiality) as well as the genre-specificity and gender-specificity of their use. Keywordscomputational pragmatics; discourse analysis; turn management; topic orientation; discourse structure markers I. INTRODUCTION: MOTIVATION, RESEARCH QUESTIONS It has been shown in a number of previous studies ([1], [2]) that discourse structure and coherence are maintained and expressed by various verbal and nonverbal markers. Coherence relations establish the link between the discourse segments, and this relation is frequently expressed by verbal discourse markers (DMs), such as and, but, so, well, you know, I mean or by the way. Besides marking boundaries, transitions and transition relevant places between discourse segments, DMs also signal the communicative function(s) of their host units. As in [1], a major goal of studies of discourse structure and computational pragmatics generally is “to explore possible meanings and functions of discourse markers in dialogues as reflected in observable utterance features (prosodic, syntactic, lexical), to enable their successful recognition and classification”. The present study aims at uncovering the functions of a few Hungarian DMs, their verbal and nonverbal contextual cues as well as their role in turn regulation and topic orientation. Previous research on segmentation has found that discourse markers, particles, cue phrases and nonverbal cues all give indispensable clues to discourse structure interpretation [3]. As a result, the disambiguation of the function of DMs would be useful for the automatic segmentation and understanding of dialogues, too. Weydt asked the question why we use discourse particles (Abtönungspartikeln in his terminology) and carried out an experiment to find a satisfactory answer. The informants in his experiment were asked to judge two almost identical dialogues. The difference between the two dialogues was that dialogue A contained rather a lot of discourse markers (Abtönungspartikeln); on the contrary, all DMs had been removed from dialogue B. The respondents‟ task was to score the dialogues for the following features: natural, rejecting, warm, wooden, smooth, authentic, difficult to make contact with, and friendly. His results received clearly suggest that the use of DMs makes our utterances sound more authentic, natural, cooperative and friendly, as well as easier to follow and understand [4]. Another important question that needs to be answered in the field of HCI technologies is how speakers apportion the floor among themselves. This paper discusses if the detection of discourse markers can help us identify the so-called transition relevance places (TRPs) where a natural speaker change may occur (e.g. during a pause for breath). One of the objectives of this study is to collect both the verbal and nonverbal markers of TRPs as well as various topical actions. The present study is supported by the TÁMOP-4.2.2/B-10/1-2010-0024 project. 531 978-1-4673-5188-1/12/$31.00 ©2012 IEEE CogInfoCom 2012 • 3rd IEEE International Conference on Cognitive Infocommunications • December 2-5, 2012, Kosice, Slovakia
Transcript
Page 1: [IEEE 2012 IEEE 3rd International Conference on Cognitive Infocommunications (CogInfoCom) - Kosice, Slovakia (2012.12.2-2012.12.5)] 2012 IEEE 3rd International Conference on Cognitive

The Role of Discourse Markers

in the Generation and Interpretation

of Discourse Structure and Coherence

Ágnes Abuczki

Department of General and Applied Linguistics

University of Debrecen

Debrecen, Hungary

[email protected]

Abstract—The present paper studies the notions of turn

management and topic orientation, and more specifically, a

group of pragmatic elements, so-called discourse markers (DMs)

which indicate the act of next speaker selection, turn-keeping,

topic elaboration and digression. After definitions of discourse

marker, turn, floor control types/turn segments, topical units and

actions are provided, a list of verbal and nonverbal discourse

markers will be given, and they will be grouped into subclasses

on the basis of the meaning relations between the linked

discourse segments and the type of floor control and thematic

control affected. The fundamental question is if discourse

markers are key indicators of discourse structure and can be

used in the automatic interpretation of utterance meaning and

rhetoric relations. The goal of the paper is twofold: firstly, to

identify how nonverbal behavior (gestures, posture, gaze) may

help disambiguate the actual function(s) of a discourse marker in

a given context; secondly, to make explicit how different

modalities work together in a synchronized manner in turn

regulation and topic control during cooperative interaction. It is

examined in twenty spontaneous dialogues of the Hungarian

HuComTech multimodal corpus what roles and functions verbal

and nonverbal discourse markers play in indicating discourse

structure and coherence relations. Concordances and corpus

queries presented address both the verbal and nonverbal features

of different turn management behavior and topical actions

accompanied or supported by discourse markers. Taking a

semasiological approach, some of the controversial defining

features of discourse markers generally listed in the literature

will be tested on a few Hungarian discourse markers: hát ('well'),

tehát ('so'), mondjuk ('say'), amúgy ('otherwise'), egyébként ('by

the way'), szerintem ('I think'). The features in question regard

their position (turn-initiality, topic unit-initiality) as well as the

genre-specificity and gender-specificity of their use.

Keywords—computational pragmatics; discourse analysis; turn

management; topic orientation; discourse structure markers

I. INTRODUCTION: MOTIVATION, RESEARCH QUESTIONS

It has been shown in a number of previous studies ([1], [2]) that discourse structure and coherence are maintained and expressed by various verbal and nonverbal markers. Coherence relations establish the link between the discourse segments, and this relation is frequently expressed by verbal discourse

markers (DMs), such as and, but, so, well, you know, I mean or by the way. Besides marking boundaries, transitions and transition relevant places between discourse segments, DMs also signal the communicative function(s) of their host units.

As in [1], a major goal of studies of discourse structure and computational pragmatics generally is “to explore possible meanings and functions of discourse markers in dialogues as reflected in observable utterance features (prosodic, syntactic, lexical), to enable their successful recognition and classification”. The present study aims at uncovering the functions of a few Hungarian DMs, their verbal and nonverbal contextual cues as well as their role in turn regulation and topic orientation. Previous research on segmentation has found that discourse markers, particles, cue phrases and nonverbal cues all give indispensable clues to discourse structure interpretation [3]. As a result, the disambiguation of the function of DMs would be useful for the automatic segmentation and understanding of dialogues, too.

Weydt asked the question why we use discourse particles (Abtönungspartikeln in his terminology) and carried out an experiment to find a satisfactory answer. The informants in his experiment were asked to judge two almost identical dialogues. The difference between the two dialogues was that dialogue A contained rather a lot of discourse markers (Abtönungspartikeln); on the contrary, all DMs had been removed from dialogue B. The respondents‟ task was to score the dialogues for the following features: natural, rejecting, warm, wooden, smooth, authentic, difficult to make contact with, and friendly. His results received clearly suggest that the use of DMs makes our utterances sound more authentic, natural, cooperative and friendly, as well as easier to follow and understand [4].

Another important question that needs to be answered in the field of HCI technologies is how speakers apportion the floor among themselves. This paper discusses if the detection of discourse markers can help us identify the so-called transition relevance places (TRPs) where a natural speaker change may occur (e.g. during a pause for breath). One of the objectives of this study is to collect both the verbal and nonverbal markers of TRPs as well as various topical actions.

The present study is supported by the TÁMOP-4.2.2/B-10/1-2010-0024 project.

531978-1-4673-5188-1/12/$31.00 ©2012 IEEE

CogInfoCom 2012 • 3rd IEEE International Conference on Cognitive Infocommunications • December 2-5, 2012, Kosice, Slovakia

Page 2: [IEEE 2012 IEEE 3rd International Conference on Cognitive Infocommunications (CogInfoCom) - Kosice, Slovakia (2012.12.2-2012.12.5)] 2012 IEEE 3rd International Conference on Cognitive

Consequently, if the goal would be to model and enable natural or naturally-sounding HCI, then a generated dialogue model should be built on the principle of a universal feature of conversation, i.e. the turn-taking system; and should involve the use of these structuring devices, discourse markers as well; besides, the modeled user should be able to appropriately use and interpret them. Furthermore, generating discourses evidently requires strategic planning. Some of the approaches to dialogue generation emphasize text structuring above sentence level. Rhetorical structure theory (RST) is one of the most influential and widely applied approaches in dynamic text structuring [5]. RST assumes that coherent discourses can be attained if the discourse segments are hierarchically structured by rhetorical relations. Fundamental types of relations within a discourse involve for instance sequence, alternative, and purpose. The relevance of this theory to the study of DMs is that the above mentioned rhetorical relations are often signaled by DMs, such as in addition or last indicating sequence; either and or marking alternative relation, and so as to indicating purpose. Applying these relations, discourse generating systems can build longer stretches of text or speech, such as paragraphs or complex utterances [6].

This paper proposes a somewhat similar alternative taxonomy of meaning relations. The difference is that this study classifies the types of relations at two higher layers of interaction: based on the mechanisms of speaker change (i.e. turn management) and thematic control, and takes into account nonverbal markers of meaning as well. Therefore, this approach is both speaker- and hearer-oriented, and it might contribute to multimodal dialogue planning as well.

II. DEFINITIONS OF DISCOURSE MARKERS

Schiffrin, one of the most quoted authors in DM studies defines DMs as “sequentially dependent elements which bracket units of talk” and describes their role as “providing contextual coordinates for ongoing talk” that indicate for the hearer how an utterance is to be interpreted [7]. In the view of Relevance Theory [8], DMs contribute to relevance understanding by reducing the processing effort needed by the hearer to reach the intended interpretation [9]. Redeker highlights that DMs are “markers of discourse transitions” [10], i.e. they tend to occur at boundaries. Moreover, Fraser adds that they even “signal a sequential relationship” between discourse units [11], which means that they give instructions to the hearer how to interpret the role and function of the DM and its host unit in the actual discourse context. Therefore, it is assumed that DMs can be useful devices to be employed in HCI (human-computer interaction) theories and technologies, especially in discourse modeling, dialogue generation and discourse interpretation.

Although there is general agreement among researchers dealing with discourse markers about the primary function of discourse markers (i.e. discourse structuring), they disagree about which elements/cues should be involved and excluded from the group of discourse markers. For instance, Fraser [11] excludes non-turn-initial (i.e. turn-internal and turn-final) items as well as items with truth-conditional meaning.

To sum up the properties DMs display, Furkó [12] has reviewed the literature on discourse markers and pragmatic markers, and proposed the following list of features: (1) non-propositionality, (2) optionality, (3) context-dependence, (4) multifunctionality, (5) sequentiality, (6) weak clause association, (7) variable scope, (8) procedural meaning – non-compositionality, (9) high frequency, (10) orality.

Since I follow a theoretical framework emphasizing the multimodal nature of communication [13], I agree with Schiffrin‟s broad view of DMs who includes nonverbal devices (manual and facial gestures, posture changes, gaze direction, etc.) in addition to verbal markers (with both non-truth-conditional and truth-conditional meaning).

III. CORPUS DATA AND METHODOLOGY

Twenty dialogues of the annotated multimodal HuComTech corpus were used for the empirical analysis of discourse markers. Among these, ten dialogues are randomly selected semi-guided informal conversations about casual topics (24 minutes on average), while the other ten randomly selected dialogues are simulated job interviews (13 minutes on average) with asymmetrical power relations. In both kinds of dialogues the participants are equally participating, however, in the second type they have clearly different goals. These differences enable the cross-genre comparison of the frequency of use of discourse markers between informal and formal dialogues.

My data contain approximately 6 hours of conversation altogether between a constant agent and twenty different young speakers. This subcorpus of the HuComTech corpus consists of 43478 words (tokens) along with discourse-level and video annotation. At discourse level, the transcription is segmented into floor control (also called turn segment) types (turn-take, turn-keep, turn-give, backchannel). An average informal conversation in this subcorpus contains 89 turn-takings (i.e. speaker changes) (3.7 occurrences/minute) and 14 backchannel phenomena (0.6 occurrences/minute), while a formal job interview contains 37.7 turn-takings (2.9 occurrences/minute) and 2.6 backchannels (0.2 occurrences/minute) on average. The video annotation of the HuComTech corpus involves the labeling of facial expressions, gaze, eyebrows, head movement, hand shape, posture, touch motion, deictic gestures and emblems. Luckily, multimodal annotation has enabled me to systematically research the co-occurrences of DMs with the use of manual gestures and gaze behavior in spontaneous interaction data.

Taking a semasiological approach, the point of departure in this research was a group of verbal discourse markers: hát ('well'), tehát ('so'), mondjuk ('say'), amúgy ('otherwise'), egyébként ('by the way'), szerintem ('I think'). After looking at their typical co-occurrences with floor control (turn segment) types, topic control segments (topic initiation, topic elaboration, topic change) and their accompanying visual features (especially gaze direction and manual gestures as labeled in the gaze direction and hand shape-type levels of the video annotation), I investigated the meaning relations between the discourse segments linked as well as the range of textual and interpersonal functions these forms may fulfill. This paper does not describe all DMs occurring in the HuComTech

Á. Abuczki • The role of discourse markers in the generation and interpretation of discourse structure and coherence

532

Page 3: [IEEE 2012 IEEE 3rd International Conference on Cognitive Infocommunications (CogInfoCom) - Kosice, Slovakia (2012.12.2-2012.12.5)] 2012 IEEE 3rd International Conference on Cognitive

corpus; instead, the most frequent and multifunctional DMs are discussed.

This subcorpus contains 958 tokens of hát, 86 tokens of tehát, 252 tokens of mondjuk, 145 tokens of szerintem, 67 tokens of egyébként, and 39 tokens of amúgy. These relatively large numbers of occurrences show that these “small words” along with their frequently accompanying nonverbal-visual cues form an integral part of our speech production and communication. Weydt‟s experiment outlined above has also proven that not only the content of our utterances matter, but also the way we structure them by verbal and nonverbal DMs [4]. By concordance searches, I will test the initiality, gender-specificity and genre-specificity (frequency of use in formal versus informal contexts) of the Hungarian DMs in question.

In short, my methodology combines the qualitative microanalysis and the quantitative concordance search and corpus query of ten formal and ten informal dialogues of the multimodal HuComTech corpus. My approach is inductive, interpretive and hermeneutic, which means that the description of DMs and turn management behavior and their theories are still evolving in the light of new data.

IV. VERBAL DISCOURSE MARKERS IN TURN MANAGEMENT

AND TOPIC ORIENTATION

As described above, DMs are useful elements for segmenting discourse into meaningful units and indentifying relations between these units [14]. However, discourse can be segmented into units at various different layers. Frank-Job identifies three layers of interaction: (1) turn-taking system, (2) macrostructure, i.e. thematic sequential system, and (3) superstructure, i.e. opening and closing a conversation [15]. This study concerns the first two layers of interaction. On the structural layer of turns, DMs are used in order to regulate turn management and guarantee the smooth, cooperative functioning of conversation. Speakers use DMs and nonverbal cues in order to avoid turn-taking problems and smoothly shift the right of speakership. On the second structural layer, conversation participants use DMs in order to inform their partner(s) in advance about their cognitive orientation and attitudes as well as the intention that a new topical action is to be performed, such as topic closing and topic change.

A. Positions of Discourse Markers in Dialogues

Conflicting theoretical assumptions can be found in the literature of DMs concerning their position. Their position is often defined at sentence level, and they are typically claimed to appear sentence-initially. Instead of sentence-level distinction, their position should rather be defined in terms of their position (1) within a turn (turn-initial, -medial or –final), (2) within a clause (initial or non-initial), (3) within an adjacency pair (first or second pair part), and (4) within topical units (topic initiation, elaboration or topic change). In the subcorpus of the HuComTech corpus I have found that the majority of DMs appear turn-internally, however, there is large variation among individual DMs depending on the actual function performed. 61% of the occurrences of hát, 64% of the occurrences of szerintem and 52% of egyébként occur turn-initially (i.e. in turn-takings); while 89 % of the occurrences of

tehát, 71% of mondjuk, and 68% of amúgy occur turn-internally. Gender and genre differences (between formal and informal dialogues) have not been found in the frequency of DM use.

B. Typical Discourse Markers in Different Floor Control

Types (Turn Segments)in our Annotation Scheme

(1) DMs in the turn-take unit (T): szerintem, hát, dehát tényleg, egyébként, úgy értem;

(2) DMs in the turn-keep unit (K): tehát, mondjuk, amúgy, mivel, tudod.

C. Marking Transition Relevant Places (TRPs)

Marked interactional behaviors, for instance, taking the floor (grabbing a turn) by uttering a dispreferred second pair part or shifting the discourse topic have to be announced before they happen [16]. Marked behaviors are coined as dispreferred because the speakers are required to give an account for their acts in order to inform the listeners about the circumstances of the unexpected response. Dispreferred answers are usually of „No-plus‟ form [17] since they elaborate on the reasons for the negative reply (e.g. Actually, …; or Igazából …). In particular, ends-of-turn (TRPs) are also often expressed by DMs, such as ennyi (‘that’s it’, ‘okay’) or meg ilyenek (‘and so on’) in Hungarian.

D. Verbally marked topic orientation, reorientation and

digression

Before discussing the classification of topic orientation markers, it is first necessary to define the concept of discourse topic. Fraser provides a general definition of discourse topic as “what the discourse is currently about, what the participants recognize they are talking about from what has been contributed to this point” [18]. Chafe labels larger coherences of semiactive information discourse topic, and adds that the fact that speakers use discourse markers before introducing a new topic suggests their awareness that consciousness has a need for this kind of information [19].

A common (verbally and/or nonverbally) marked behavior is when a speaker introduces a new topic in the conversation. When a participant wishes to digress from the topic of the prior utterances, they must provoke attention to their utterance, very often by using discourse markers (e.g. “otherwise”, “you know”, “by the way”).

Fraser [18] classifies topic orientation markers into four categories, depending on the speaker‟s intention to signal: (1) return to prior topic, e.g. back to my point; (2) add to or continue with the present topic, e.g. as I was saying; (3) digress from the present topic, e.g. by the way; and (4) introduce a new topic, e.g. but, on a different note.

The multimodal pragmatic annotation scheme of the HuComtech corpus divides the thematic control of conversations into three topical segments: (1) topic initiation, (2) topic elaboration, and (3) topic change.

Table I. below attributes typical DMs to the topical actions and communicative functions expressed or performed by them.

533

CogInfoCom 2012 • 3rd IEEE International Conference on Cognitive Infocommunications • December 2-5, 2012, Kosice, Slovakia

Page 4: [IEEE 2012 IEEE 3rd International Conference on Cognitive Infocommunications (CogInfoCom) - Kosice, Slovakia (2012.12.2-2012.12.5)] 2012 IEEE 3rd International Conference on Cognitive

TABLE I. COMMUNICATIVE FUNCTIONS AND THEIR SUPPORTING

DISCOURSE MARKERS

Function

Discourse Marker (Supporting the Function)

Position of the

DMa

Hungariann

DMs

DMs’

Equivalents in

English

topic initiation mostly turn-initial szerintem;

tényleg

as I see it, as

for me, I think; actually

topic

elaboration (turn-keeping)

mostly turn-

internal

tehát;

mondjuk, mellesleg

so;

(let‟s) say; by the way

closing a topic mostly turn-final

ennyi; meg

ilyenek; na mindegy

okay, that‟s all, that‟s it;

and so on;

anyway

topic change mostly turn-initial egyébként otherwise

turn-taking mostly turn-initial szerintem;

hát

as I see it, as

for me, I think;

well

giving example,

approximation

mostly turn-

internal mondjuk (let‟s) say

face

management,

softener

mostly turn-initial mondjuk

at the same

time, kind of,

sort of

correction,

clarification mostly turn-initial úgy értem I mean

giving

background

information

mostly turn-internal

mivel; tudod

because; you know

hesitation,

delay, lexical search

mostly turn-initial hát;

mondjuk

well;

(let‟s) say

a. as in the annotation scheme of the HuComTech corpus

Taking a semasiological approach, Table II. sums up one of the major features of DMs, that is multifunctionality, from the perspective of individual DMs. As outlined in both Table I. and Table II:, different functions can be simultaneously related to the individual DMs due to their ability to operate at different levels of discourse, both at textual/sequential level (regulating floor and thematic control) and interpersonal level (involvement in face management, face saving, pragmatic force modifying and the act of agreement). For example, in my corpus data, the same DM, mondjuk fulfills the interpersonal function of softening and saving face, and also marks connections between different topical units: between (1) interrupted, (2) disrupted, (3) related, and (4) unrelated topics.

TABLE II. MULTIFUNCTIONAL DISCOURSE MARKERS SUPPORTING

VARIOUS COMMUNICATIVE FUNCTIONS

Hungarian

Discourse

Marker

English Equivalents

of the Hungarian

Discourse Marker

Common Crommunicative

Functions Served

hát well hesitation, softener

tehát so connective, conclusion

egyébként otherwise digression, topic management

mondjuk say, let‟s say, sort of, giving example, contrast

amúgy by the way topic shift

szerintem as I see it, as for me, I

think change of perspective

V. NONVERBAL DISCOURSE STRUCTURE MARKERS

Besides verbal markers, conversation participants both consciously and subconsciously provide nonverbal cues about their cognitive state, orientation and attitudes towards the other participant(s), the flow of the topic and its management. Therefore, in order to discuss the phenomena of discourse structuring in a multimodal model, the list of verbal indicators of discourse structure must be complemented with the description of their accompanying nonverbal features and gestures as well.

A. A Definition of Gestures

In Kendon‟s definition, a gesture is „a form of human expression, an activity that is significant for the understanding of a speaker‟s expression”. Moreover, he adds that “the gestural component and the spoken component interact with one another to create a precise and vivid understanding”. This means that nonverbal behaviour and verbal content are orchestrated together, gesture completes (although sometimes precedes) the utterance‟s meaning, i.e. meaning is a composite notion [20]. Gesture families then might as well be seen as to form a nonverbal sign and code system, which implies that it might be relatively easy to teach their system to machines. Among other phenomena, the mechanism by which conversation partners take turns talking are spoken and nonverbal, as well as open and subconscious [21]. In other words, discourse structure is expressed by both verbal and nonverbal means described below.

B. Nonverbal Markers of Floor Control Types

The results of the qualitative microanalysis below outline the typical nonverbal features of DMs used in the four different turn segments annotated in the HuComTech corpus: (1) turn-take, (2) turn-keep, (3) turn-give, and (4) backchannel. Among the nonverbal cues, manual gesturing and gaze direction will be described in detail.

(1) To take the conversation turn smoothly, the next floor holder frequently tends to look at the present floor holder in order to make eye contact with him/her [22]. Similarly, when the hearer performs manual gestures, such as hand raising, he/she may request the control of the floor. Duncan coined this phenomenon the ”traffic signal” approach [16].

(2) During holding the floor the direction of gaze is moving relatively fast. When speakers try to recall something, they tend to look upwards, which be interpreted as a sign of cognitive activity, such as cognitive (pre-) planning. “Looking at the conversational partner or looking away from the partner can provide indirect cues of the speaker's willingness to continue interaction, and gazing at particular elements in the vision field can tell where the speaker's focus of attention is” [22]. The largest amount of gesturing is performed during turn-keep and topic elaboration. As mentioned above, when a speaker is still gesturing at the end of his/her utterance, he/she probably intends to continue talking and control the floor even after the end of the utterance [22], [23].

Á. Abuczki • The role of discourse markers in the generation and interpretation of discourse structure and coherence

534

Page 5: [IEEE 2012 IEEE 3rd International Conference on Cognitive Infocommunications (CogInfoCom) - Kosice, Slovakia (2012.12.2-2012.12.5)] 2012 IEEE 3rd International Conference on Cognitive

(3) The most reliable nonverbal cue indicating turn-giving intention of the present speaker is his/her gaze behaviour which is usually a long glance. The current floor holder continuously makes (or tries to make) eye contact with the other conversation partner in order to get a reply, provoke a future action or any reaction. In general, “gaze shifts toward the listener frequently coincide with a shift in conversational turn and can be seen as a signal that the floor is available” [22]. Regarding gesturing, hand gestures are very rarely performed during this turn type in our corpus, too. Duncan has observed that “the cessation of manual gesturing is especially typical of turn-gives“ [16].

(4) The nonverbal behaviour of backchanneling typically includes nodding without performing any manual gestures. Besides nodding, short iterative phrases indicate that listener pays attention to and understands what the speaker is saying. “It can be seen as a response to the speaker's nonverbal request for feedback” [22]. Nodding as an indicator of paying attention is very frequent in the HuComTech corpus. As proposed by McClave [24], it has been frequently observed in our corpus as well that listener nods precede their vocalizations. In Yngve‟s understanding [25], vocal or gestural expressions of the listener‟s backchannels indicate that he/she does not want to grab the floor.

Most importantly, interactional tasks, such as TRPs (i.e. transition relevant places) are especially frequent is gesturing. For instance, open hand gestures with the palms up are generally described in the literature as turn-completive gestures, indicating the handing over of the floor [26]. However, their occurrence with this function is quite rare in the Hungarian corpus, especially in the formal dialogues. This fact might be explained by the fact that the participants were a bit stressed in the job interview scenario and tried to control their nonverbal behavior. On the contrary, open hands were rather observed before telling a longer stretch of narrative, disclosing the intention of the speaker to hold the floor for a longer period of time. Open hands are components of shrugs as well, which might also be performed at turn-closings, indicating that “no particular fashion in which the talk should be taken is proposed” [26].

C. Nonverbal Features of Thematic Control Types

Novel things, new lexical-semantic information (new in comparison to what has just been uttered previously in the context of the conversation) are most often brought into the domain of discourse during the turn-keep and topic elaboration discourse segments. Pieces of novel information are almost always marked, usually not verbally, but by nonverbal means, very often accompanied by heavy gesturing, especially bilateral open hand manual gesturing.

Moving the hand(s) aside and outward by a lateral movement usually marks the closing of a peripheral or unrelated topic (i.e. closing a digression) and speaker intention to turn back to a prior topic. This hand movement is often accompanied by uttering a topic closing DM, such as hát ennyi ('that's all'), meg ilyenek ('and so on'), or na mindegy („anyway’).

Shrugs are also indicators of the thematic control of a conversation. Shrugs are usually performed together with raising paired body parts, especially shoulders and eyebrows. In the HuComTech corpus they either express disengagement and distancing or mark the beginning of dispreferred second pair parts [26].

VI. RESEARCH CONCLUSIONS

In conclusion, Table III. below sums up the defining and

non-defining multimodal properties of turn segments as they

can typically be observed in the annotation files of the

HuComTech subcorpus. Major differences have not been

found among the formal and informal interviews, and the turn

management behavior and DM use of males and females.

It can be concluded that the study of discourse markers

may contribute both to dialogue interpretation and generation.

On the one hand, it has been argued that discourse markers

guide the listener(s) in the interpretation of the discourse since

they express the cognitive orientation of the speaker. On the

other hand, it has been suggested that the use of discourse

markers in dialogue modeling and generation should be

facilitated in order to enable natural human-computer

interaction. However, it has been shown that the detection of

verbal discourse markers alone - due to their highly

multifunctional nature - is not enough for making reliable

assumptions about discourse structure. In order to compensate

their multifunctionality and disambiguate their actual function

in the situation, the analysis of nonverbal markers must also be

involved in their functional interpretation.

TABLE III. MULTIMODAL FEATURES OF TURN SEGMENTS

Floor

Control

Types (Turn

Segments) a

Multimodal Features

Verbal

Utterances and

DMs Involved

Nonverbal-

Acoustic,

Suprasegmental

Features

Nonverbal-

Visual Features

turn-take szerintem, hát

(„I guess‟,

„well‟)

acoustic

realizations of

turn grabbing or interruption

posture shift, gaze sideways,

hand raise

turn-keep

tehát, vagyis, akkor,

mondjuk („so‟,

„or‟, „then‟, „say‟)

F0 increase, higher intensity

gaze forward (at

the other speaker/s), gaze

upward

(recalling), manual

gesturing

turn-give

na mindegy,

szerinted? („anyway‟,

„how about

you?‟)

raising intonation

(in questions), followed by

pause (550 msec

on average)

long gaze at the

next speaker

selected, cessation of

manual

gesturing, eyebrow raise

backchannel

jajaja, aha

(„uhum‟,

„right‟)

acoustic

realizations of iteration and

humming

default case of nodding

a. as in the annotation scheme of the HuComTech corpus

535

CogInfoCom 2012 • 3rd IEEE International Conference on Cognitive Infocommunications • December 2-5, 2012, Kosice, Slovakia

Page 6: [IEEE 2012 IEEE 3rd International Conference on Cognitive Infocommunications (CogInfoCom) - Kosice, Slovakia (2012.12.2-2012.12.5)] 2012 IEEE 3rd International Conference on Cognitive

My future research plans involve (1) tagging the (various

textual and interpersonal) functions of DMs; (2) analysis of

further textual features (DMs‟ position within clause and

adjacency pair, sequential properties, statistical analysis of co-

occurrence patterns); (3) measuring the segment duration of

the host unit, its energy (RMS) and speaking rate

(syllables/sec); as well as (4) collecting the synchronously

performed gestures and other contextual cues.

VII. GENERAL CONCLUSIONS FOR COGNITIVE

INFOCOMMUNICATIONS

Cognitive infocommunications investigates the link between the research areas of infocommunications and cognitive sciences with the goal to provide an integrated view of how cognitive processes can co-evolve with infocommunications devices [27]. Evidently, producing and understanding human behavior, involving language and nonverbal behavior, requires the mobilization of a great variety of cognitive processes. Our research group (HuComTech) investigates intra-cognitive communication where information transfer occurs between two cognitive beings with equivalent cognitive capabilities, that is, between two humans. We attempt to model human-computer interaction (HCI) based on human-human interaction (HHI), since we believe that HCI should be similar to natural, spontaneous HHI. It has been proposed that nonverbal and timing aspects in HHI should be transferred to HCI and human-media communication as well, and virtual humans should adapt their behavior to the behavior of their human interaction partners in order to maintain a natural and continuous interaction in which the turn-takings and topic changes of the participants are temporally synchronized [28]. It is still a challenge in language technology to predict transition relevant places, speaker changes and topic changes in dialogues, although these transitions form the basic units of interaction, therefore, they should be predicted in HCI. My previous work contributes to the development of dialogue management systems with a decision tree automatically distinguishing two basic discourse segments, turn-keep and turn-give (speaker change) based on nonverbal-acoustic (duration of pauses), and nonverbal-visual (gaze direction, hand gestures, posture) cues. This paper is a step forward in this field with involving the analysis of the relevance and role of verbal markers, that is, discourse markers as well in the detection of disocurse dynamics. Since turn-taking and topic-shifting are universal features of all interactions, regardless of language or medium, these theoretical results on the role of discourse markers indicating dialogue structure might be applied in dialogue modeling.

REFERENCES

[1] V. Petukhova, H. Bunt, “Towards a multidimensional semantics of disocurse markers in spoken dialogue,” Proceedings of the 8th International Conference on Computational Semantics. Tilburg, January 2009, pp. 157–168.

[2] J. Hirschberg, D. Litman, “Empirical studies on the disambiguation of cue phrases,” Computational Linguistics, vol. 25 (4), 1993, pp. 501–530.

[3] B. Grosz, C. Sidner, “Attention, intentions, and the structure of discourse,” Computational Linguistics, vol. 12, 1986, pp. 175–204.

[4] H. Weydt, “What are particles good for?,” Approaches to Discourse Particles, K. Fischer, Ed. Amsterdam: Elsevier, 2006, pp. 205–217.

[5] W. Mann, S. Thompson, “Rhetorical structure theory,” Text, 8, 1988, pp. 243–281.

[6] S. Gillis, W. Dealemans, K. DeSmedt, “Artificial intelligence,” Cognition and Pragmatics (Handbook of Pragmatics Highlights), D. Sandra, J. Östman, J. Verschueren, Eds. Amsterdam/Philadelphia: John Benjamins, 2009, pp. 16–40.

[7] D. Schiffrin, Discourse Markers. Cambridge: Cambridge University Press, 1987.

[8] D. Sperber, D. Wilson, Relevance: Communication and cognition. Oxford: Blackwell, 1986.

[9] D. Blakemore, Understanding Utterances. Oxford: Blackwell, 1992.

[10] G. Redeker, “Discourse markers as attentional cues at discourse transitions,” Approaches to Discourse Particles, K. Fischer, Ed. Amsterdam: Elsevier, 2006, pp. 339–358.

[11] B. Fraser, “An approach to discourse markers,” Journal of Pragmatics, 14, 1990, pp. 383–395.

[12] P. B. Furkó, The pragmatic marker-discourse marker dichotomy reconsidered – the case of well and of course. Debrecen: Kossuth Egyetemi Kiadó, 2007.

[13] L. Hunyadi, “Multimodal human-computer interaction technologies,” Argumentum, 7, Debrecen: Debreceni Egyetemi Kiadó, 2011, pp. 240–260.

[14] H. C. Bunt, W. J. Black, Abduction, Belief and Context in Dialogue, Amsterdam: John Benjamins, 2000.

[15] B. Frank-Job, “A dynamic-interactional approach to discourse markers,” Approaches to Discourse Particles, K. Fischer, Ed. Amsterdam: Elsevier, 2006, pp. 359–374.

[16] S. Duncan, “Some signals and rules for taking speaking turns in conversations,” Journal of Personality and Social Psychology 23.2, 1972, pp. 283–292.

[17] H. Sacks, Lectures on Conversation, G. Jefferson, Ed. Oxford: Blackwell, 1992.

[18] B. Fraser, “Topic orientation markers,” Journal of Pragmatics, 41, 2009, pp. 892–898.

[19] W. Chafe, “Consciousness and language,” Cognition and Pragmatics (Handbook of Pragmatics Highlights), D. Sandra, J. Östman, J. Verschueren, Eds. Amsterdam/Philadelphia: John Benjamins, 2009, pp. 135–145.

[20] A. Kendon, Gesture. Visible Action as Utterance. Cambridge: Cambridge University Press, 2004.

[21] J. M. Wiemann, M. L. Knapp, “Turn-taking in Conversation,” Journal of Communication, Spring, 1975, pp. 75–92.

[22] Á. Abuczki, “Multimodal Annotation and Analysis of Turn Management Strategies,” Proceedings of HUSSE 10 Conference. Linguistics Volume, K. Balogné Bérces, K. Földváry, R. Mészárosné Kóris, Eds. Debrecen: Hungarian Society for the Study of English, 2011, pp. 107–115.

[23] Á. Abuczki, “A multimodal analysis of the sequential organization of verbal and nonverbal interaction,” Argumentum, 7. Debrecen: Debreceni Egyetemi Kiadó, 2011, pp. 261–279.

[24] E. McClave, “Cognitive and Interactional Functions of Head Movements in Conversation.” In Oralité et Gestualité, S. Santi et al., Eds., 1998, pp. 365–369.

[25] V. Yngve, “On getting a word in edgewise,” Papers from the Sixth Regional Meeting, Chicago: Chicago Linguistic Society, 1970, p. 568.

[26] J. Streeck, Gesturecraft: The Manu-facture of Meaning. Amsterdam: John Benjamins, 2009.

[27] P. Baranyi, A. Csapo, “Cognitive Infocommunications: CogInfoCom”, 11th IEEE International Symposium on Computational Intelligence and Informatics, Budapest, Hungary, 2010.

[28] B. Reeves, C. Nass, The media equation: how people treat computers, television, and new media like real people and places. New York, NY, USA: Cambridge University Press, 1996.

Á. Abuczki • The role of discourse markers in the generation and interpretation of discourse structure and coherence

536


Recommended