1
S SBAR S VP SBAR S VP VP PP NP NN telescope 1 DT a 2 IN like 2 PRT RP up 2 VB shut 2 MD could 2 NP PRP I 4 VBP wish 2 NP PRP I 3 WHADVP WRB how 4 , , INTJ UH Oh 3 CP CP C’ TP T’ vP v’ VP VP CeP TP T’ ModalP vP v’ VP p likeRP PlikeP Plike’ DP(1) Plike DP(1) NumP NP telescope 4 Num D a 5 p likeR p likeR Plike like 3 VP PupP up 7 V v v V shut 11 DP(2) Modal T T Modal could 3 DP(2) I 21 Ce V degP(3) v DP(4) T T v v V wish 4 DP(4) I 5 C degP(3) how 2 exclamP oh 2 5 more empty categories are nodes, too! X-bar There is a growing consensus that the left anterior temporal lobe (blue arrows, right) implements some sort of basic syntactic processing (Friederici and Gierhan 2013; see e.g. Dronkers et al. 2004; Humphries et al. 2005; Bemis and Pylkkänen 2013). But what is this processing? No consensus exists, so we consider four alternative formalizations using a correlational method. We hypothesize that basic syntactic processing reflects the traversal of syntactic structures, and thus higher node counts should correlate with stronger BOLD signals in the left anterior temporal lobe. I. What parsing mechanism correlates best with observed BOLD signals? We derived time series predictors from the first chapter of Alice in Wonderland, and fit them to preprocessed fMRI data from 11 right-handed, college-age participants. One of these predictors, speech rate, marks the offset of words. This predictor localized participant- specific regions in the temporal lobe. We then evaluated the ability of different syntactic predictors to account for mean BOLD signal in a 10mm spherical region around these participant-specific peaks. Predictors derived from the text and audio were convolved with a canonical hemodynamic response function and summed together to yield predictions about activation in anterior temporal regions of interest (cf. Just and Varma 2007, fig. 11). II. Correlational method Simple formalization of parser effort in terms of node count predicts BOLD time series in left anterior temporal region. X-bar node count correlates better than Penn Treebank node count. The result persists even with prosodic breaks as a co-predictor. The syntactic predictors idealize incremental parsing. Node counts define the height of point events. Total predicted hemodynamic response is the sum of each word's response. All node counts were orthogonalized against the hemodynamic response to speech rate. Each event yields a predicted hemodynamic response based on a canonical response function (Friston et al. 1998). Predictors came from the text of the narrative as well as the audio that participants heard. 0 0.5 1 1.5 2 2.5 3 −0.5 0 0.5 Oh how I wish I could shut up like a telescope Time (sec) Amplitude 0 0.5 1 1.5 2 2.5 3 0 10 20 30 Time (sec) 0 0.5 1 1.5 2 2.5 3 −5 0 5 10 15 x 10 −4 Time (sec x10) Amplitude 0 0.5 1 1.5 2 2.5 3 −5 0 5 10 15 x 10 −4 Time (sec x10) Amplitude These dots correspond to coefficients on syntactic predictors. The best-fitting predictor is based on X-bar structures and top-down parsing (e.g. blue numbers in the trees above). Node count is a significant predictor of anterior temporal lobe activation. Prosodic break strength and unigram (log) frequency are not. This is also the case for the right anterior temporal lobe. For each of the four combinations of syntactic analysis and parsing strategy, we fit a regression model with syntactic node count as a predictor. Each model also included three other predictors: speech rate, unigram log frequency, and prosodic break index (ToBI; Beckman et al. 2005). Modeling Neural Correlates of Syntactic Structure Building John Hale, Frederick Callaway, Elana Feldman, Jaclyn Jeffrey-Wilensky, David Lutz, Adam Mahar (Cornell University) Jonathan Brennan (University of Michigan), Sarah Van Wagenen (UCLA) [email protected] We calculated node counts based on bottom-up and top-down parsing (cf. Hale 2014, ch 3) for both grammar types. Number of new nodes at each word using top-down parsing strategy. Penn Treebank IIIa. Alternative syntactic structures IIIb. Alternative parsing strategies V. Analysis and Results IV. Deriving predictors We followed Van Wagenen et al. (2012) in calculating node counts using two different types of syntactic structure. One type adheres to the Penn Treebank (Marcus et al. 1993) annotation guidelines. A second type is based on X-bar theory in the sense of Chomsky (1970) and Jackendoff (1974,1977). We used Minimalist Grammars (e.g. Stabler 2013) to generate these X-bar structures. Beckman, M. E., Hirschberg, J., & Shattuck-Hufnagel, S. (2005). The original ToBI system and the evolution of the ToBI framework. In S.-A. Jun (ed.) Prosodic Typology —The Phonology of Intonation and Phrasing. Bemis, D. K., & Pylkkänen, L. (2013). Flexible composition: MEG evidence for the deployment of basic combinatorial linguistic mechanisms in response to task demands. PloS One, 8(9), e73949. doi:10.1371/journal.pone.0073949 Brennan, J., Nir, Y., Hasson, U., Malach, R., Heeger, D. J., & Pylkkänen, L. (2012). Syntactic structure building in the anterior temporal lobe during natural story listening. Brain and Language, 120(2), 163–73. doi:10.1016/j.bandl.2010.04.002 Chomsky, Noam (1970). Remarks on nominalization. In: R. Jacobs and P. Rosenbaum (eds.)Reading in English Transformational Grammar, 184-221. Waltham: Ginn. Dronkers, N. F., Wilkins, D. P., Van Valin, R. D., Redfern, B. B., & Jaeger, J. J. (2004). Lesion analysis of the brain areas involved in language comprehension. Cognition, 92(1-2), 145–77. doi:10.1016/j.cognition.2003.11.002 Friederici and Gierhan (2013) The language network. Current Opinion in Neurobiology. Friston, K. J., Fletcher, P., Josephs, O., Holmes, A., Rugg, M. D., and Turner, R. (1998). Event-related fMRI: characterising differential responses. NeuroImage 7, 30–40. Hale, John T. (2014). Automaton Theories of Human Sentence Comprehension. CSLI. Humphries, C., Love, T., Swinney, D., & Hickok, G. (2005). Response of anterior temporal cortex to syntactic and prosodic manipulations during sentence processing. Human Brain Mapping, 26(2), 128–38. doi:10.1002/hbm.20148 Jackendoff, R. (1974). Introduction to the X" Convention. Mimeo, distributed by the Indiana University Linguistics Club, Bloomington, IN. Jackendoff, R. (1977). X" Syntax: A Study of Phrase Structure. Cambridge, MA: MIT Press. Just, M. A., & Varma, S. (2007). The organization of thinking: what functional brain imaging reveals about the neuroarchitecture of complex cognition. Cognitive, Affective & Behavioral Neuroscience, 7(3), 153–91. Marcus, Marcinkiewicz, and Santorini (1993). Building a large annotated corpus of English: The Penn Treebank. Computational Linguistics. Stabler (2013). Two models of Minimalist, incremental syntactic analysis. TopICS in Cognitive Science. 3(5). 611-633 Stowe, L. A, Broere, C. A, Paans, A. M., Wijers, A. A., Mulder, G., Vaalburg, W., & Zwarts, F. (1998). Localizing components of a complex task: sentence processing and working memory. Neuroreport, 9(13), 2995–9. Van Wagenen, Brennan, and Stabler (2012). Quantifying parsing complexity as a function of grammar complexity. Talk presented at the 25th Annual CUNY Conference on Human Sentence Processing. Conclusions Node count, though correlated with prosodic break strength (r 2 = .25), significantly improves all models when compared with models that exclude it. Penn bottom-up p = 0.04 Penn top-down p = 0.02 X-bar top-down p < 0.0001 X-bar bottom-up p < 0.001 Log likelihood tests were evaluated against a χ 2 distribution with df = 1, comparing models which differ only in whether node count is included as a predictor. Left Anterior Temporal Lobe -0.05 0.00 0.05 0.10 penn bottom-up x-bar bottom-up penn top-down x-bar top-down Est. % signal change Left Anterior Temporal Lobe 0.00 0.05 0.10 rate logfreq break x-bar top-down nodecount Est. % signal change Regions of interest X-bar top-down nodecount
