1
Main Question: Are experimental tasks a good measure of next-mention likelihood? 1) Metacognitive judgments 2) Story continuations Results Sandra A. Zerkle & Jennifer E. Arnold Discourse constraints on referential predictability: Is the subject predictable? Background • The grammatical subject of a sentence is a topical position (Ariel, 1990; Givón, 1983; Grosz et al., 1995), and this constrains how we produce pronouns (Fukumura & van Gompel, 2010; Kehler et al., 2008; Stevenson et al. 1994) and interpret pronouns (Gernsbacher, 1990; Gordon et al., 1993). • Can topicality be defined in terms of predictability? Do people have an expectation for the subject to be mentioned again in natural discourses and experimental tasks? Support for predictability – measures of frequency in real discourse show that subjects are mentioned more often than objects/obliques: Evidence against predictability – experimental tasks measuring next-mention likelihood: • People tend to talk about the cause . • People do not tend to talk about the subject if it isn’t the cause. Ariel, M. (1990). Accessing Noun-Phrase Antecedents. v Arnold, J. E. (1998). Reference form and discourse patterns (Doctoral dissertation, Stanford University).v Arnold, J. E. (2001). The effect of thematic role on pronoun use and frequency of reference continuation. Discourse Processes. v Brocher, A., Chiriacescu, S., & von Heusinger, K. (2016). Effects of information status... Discourse Processes. v Frank, M., & Goodman, N. (2012). Predicting pragmatic reasoning in language games. Science. v Fukumura, K., & Van Gompel, R. (2010). Choosing anaphoric expressions: Do people take into account likelihood of reference? JML. v Gernsbacher, M. A. (2013). Language comprehension as structure building. v Givón, T. (1983). Topic continuity in discourse: A quantitative cross-language study (Vol. 3). v Gordon, P., Grosz, B., & Gilliom, L. (1993). Pronouns, names, and the centering of attention in discourse. Cognitive Science. v Grosz, B., Weinstein, S., & Joshi, A. (1995). Centering: A framework for modeling the local coherence of discourse. Comp. linguistics. v Kehler, A., Kertz, L., Rohde, H., & Elman, J. (2008). Coherence and coreference revisited. Journal of semantics. v Kehler, A., & Rohde, H. (2013). A probabilistic reconciliation of coherence-driven & centering-driven theories... Theoretical Linguistics. v Nappa, R., & Arnold, J. E. (2014). The road to understanding is paved with the speaker’s intentions... Cognitive psychology. v Rohde, H., & Kehler, A. (2014). Grammatical and information-structural influences on pronoun production. LCN. v Stevenson, R. J., Crawley, R. A., & Kleinman, D. (1994). Thematic roles, focus and the representation of events. LCP. Funded by NSF Grant 1651000 to J. Arnold Contact: [email protected] 65% 74% 0% 20% 40% 60% 80% 100% 1-sentence 3-sentence % subject mentioned first Exp. 4: past tense/PP phrase Subjects ARE predictable 81% 35% 10% 0% 20% 40% 60% 80% 100% Gaze to Subject Gaze Neutral Gaze to Non-Sub % subject chosen next Exp. 1: metacognitive judgments Subjects NOT predictable 65% 50% 26% 66% 61% 44% 0% 20% 40% 60% 80% 100% Gaze to Subject Gaze Neutral Gaze to Non-Sub % subject mentioned first Exp. 2: story continuation Subjects NOT predictable 1-sentence 3-sentence Methods : all participants native English M-Turk workers Experiments 1 & 2 (N=67, N= 91) • Fragments of video stimuli (Nappa & Arnold, 2014), ‘X with Y’ construction • Narrator: “This story is about Panda Bear and Puppy. This is Panda Bear (places animal), and this is Puppy (places animal). Panda Bear is having lunch with Puppy. ” • Narrator gazed when uttering pronoun: Subject (A), Neutral (B), Non-Subject (C) • Here, video ends after gaze but before pronoun • Exp. 1: next mention preference: “Who do you think she will talk about next? ” • Exp. 2: story continuation: “Type in a continuation to the story ” • 2A: one sentence long; 2B: three sentences long Experiments 3 & 4 (N=22, N=27) • Written stimuli (no manipulation of gaze); both experiments tested story continuation, with length (1- vs. 3-sentence) manipulated by block • Exp. 3: present tense: “Ana is cleaning up with Liz.” • Exp. 4: past tense, PP phrase: “Ana was cleaning up with Liz in the kitchen.” 32% 55% 0% 20% 40% 60% 80% 100% 1-sentence 3-sentence % subject mentioned first Exp. 3: present tense Subjects NOT predictable 42% 10% 0% 20% 40% 60% 80% 100% subject referents obj-of-PP referents % of all first references Arnold (2001): corpus analysis 73% 27% 0% 20% 40% 60% 80% 100% Subject Object/Oblique % total references Arnold (1998): text analyses Q2: Does gaze affect predictability judgments? Q3: Does the experimental task change performance? Conclusions • Subject-mention in experiments only matches corpus frequency in past- tense narratives • è Experimental tasks are NOT a good measure of next-mention predictability • In metacognitive & story continuation tasks, there may be an expectation of “fairness”: Panda Bear was the topic of the last sentence, so now Puppy should be mentioned. • Task demands modulate performance: • More subjects mentioned in longer discourse (Brocher et al., 2016) • More subjects mentioned in past tense/PP phrase constructions • Raises questions about the use of simple experimental tasks to estimate the probability of reference (e.g., Frank & Goodman, 2012) Experiments 1 & 2: Subject and gaze predictability Experiments 3 & 4: Subject predictability in short vs. long continuations Are subject characters judged to be more predictable? NO Are gazed-at characters judged to be more predictable? YES p<.0001 Are subject characters more likely to be mentioned next? NO Are gazed-at characters more likely to be mentioned next? YES p=0.002 Are subject characters more likely to be mentioned in longer continuations? YES p=0.0068 Are subject characters more likely to be mentioned in longer continuations? YES p=0.028 Numerical trend: subjects more likely in longer narratives Are subject characters more likely to be mentioned next? NO Are subject characters more likely to be mentioned next? YES 73% 26% 54% 0% 20% 40% 60% 80% 100% Subj-biased IC Obj-biased IC Non-IC % subject continuations Rohde & Kehler (2014): sentence continuations (no pronoun prompt)
