1
R e f e r e n c e s [1] Frank van der Velde, Marc de Kamps. Neural blackboard architectures of combinatorial structures in cognition. Behavioral and Brain Sciences 29 (2006) [2] Ray Jackendoff. Foundations of Language. (2002) [3] Frank Velde, Marc Kamps. Ambiguity resolution in a Neural Blackboard Architecture for sentence structure. (2015) [4] Marc de Kamps. A Generic Approach to Solving Jump Diffusion Equations with Applications to Neural Populations. arXiv preprint arXiv:1309.1654 (2013) [5] Marc de Kamps, Volker Baier, Johannes Drever, Melanie Dietz, Lorenz Mösenlechner, Frank van der Velde. The state of MIIND. Neural Networks 21, 1164–1181 (2008) [6] Ahmet Omurtag, Bruce W. Knight, Lawrence Sirovich. On the simulation of large populations of neurons. Journal of computational neuroscience 8, 51–63 (2000) [7] Adrien Wohrer, Mark D. Humphries, Christian K. Machens. Population-wide distributions of neural activity during perceptual decision-making. Progress in Neurobiology 103, 156–193 (2013) [8] Farran Briggs, George R. Mangun, W. Martin Usrey. Attention enhances synaptic efficacy and the signal-to-noise ratio in neural circuits. Nature 499, 476–480 (2013) [9] Nelson, M. J., El Karoui, I., Giber, K., Yang, X., Cohen, L., Koopman, H., Cash, S., Naccache, L., Hale, J., Pallier, C., & Dehaene, S. Neurophysiological dynamics of phrase structure building during sentence processing. (Under review) [10] C. Pallier, A.-D. Devauchelle, S. Dehaene. Cortical representation of the constituent structure of sentences. Proceedings of the National Academy of Sciences 108, 2522–2527 (2011) Acknowledgments: The work was supported by the Ecole Doctorale Cerveau, Cognition et Comportament with PHD funding from Universite Paris Descartes. Also by the lab UNICOG INSERM-U992 located at NeuroSpin. N e u r o i m a g i n g p r e d i c t i o n s MA SA G GK Ctl G GK Ctl WM GK GK G G WM Excitatory (4, 0.36) Inhibitory (60, -0.36) Artificial (1, 1) driven by word event inp can drive other compartment circuit MA Type (n connections, synaptic efficacy) WM Ctl max rate (Hz) 10.0 inp driven by control event 10.0 4.5 kickoff rate (Hz) 2300 rate slope (Hz/ms) 0.1 rate duration (ms) 100 0.1 - B 1.0 - 0.1 - B MA SA GK G Baseline activity drives all LIF populations with excitatory connections (16, 0.36) A. Compartment Circuit Implementation B. Artificial Populations time (ms) rate (Hz) rate slope rate duration max MA SA WM Ctl G GK Main Sub Working Memory Control Gate Gate Keeper B inp Baseline Event input rate Time (ms) Normalized Firing rate (Hz) Normalized Firing rate (Hz) Time (ms) Binding related neural activity W 1 W 2 W 3 W 4 PN 1 PN 2 PN 3 assembly summary total activity CC 1 CC 2 CC 3 change rate (Hz) Time (ms) Right branched tree sentence for a phrase grammar with bottom-up parsing Hemodynamic Response ECOG predictions from increasing size right branching trees fMRI predictions simulating varying constituent size as in Pallier et al. 2011 Example size 4 constituents What next? Binding related drops Increasing activity level Increasing activity level Binding related drops Nelson et al 2016 (in review) Neural time courses in aSTS We plan to extend the simulation to at least three parsing schemes: bottom-up, top-down and left corner. Then we will fit the nodes activity in the circuit and the timing of working memory nodes and controls to ECOG data in specific regions. The models fitted to specific regions will allow quantitive predictions on diverse fMRI dataset In a study of Pallier et al [10], about manipulation of the size of constituents in sequences of 12 words presented visually to participants, they observed a sublinear increase of the amplitude of hemodynamic responses in language related regions as a function of constituent size. We reproduce the pattern under a phrase grammar theory with a simple bottom-up parsing scheme. In case both MAs are active and Ctl activates to allow flow from MAs to their SAs, then binding will take place by activation of WM. Moreover other interesting non trivial dynamics take place. During the process of binding the reverberating activity of WM that inhibits the GKs connecting SAs creates a sudden burst of activity leading to a pronounced spike, this is due to the fact that SAs create a self excitatory loop, also elevating the activity of Gs and GKs between them. This burst of activity quickly drops back to a steady state that leaves the inner circuit in a level of activity far greater than in its original resting state, facilitating possible future communication between MAs. Tree structure hypothesized for a given 4 words phrase. It is shown how compartment circuits correspond to sections of the tree structure and how the nodes corresponding to grammatical categories of words processed or phrase nodes are instantiated in time under a bottom-up parsing scheme. The activity of the LIF populations of each compartment circuit are shown separately, followed by their summary and total activity. Each node in the compartment circuit (A), represents a LIF neural population, except for Working Memory, control and baseline that represent activity in reverberating artificial neural populations described by the function plotted in (B). The LIF model parameters were taken from Omurtag et al [6], baseline activity was determined from Wohrer et al [7] and synaptic efficacy was taken from Briggs et al [8]. Nelson et al. (under review) [9], shows that Local Field Potentials increase with phrase constituent size and drop after binding of words into constituents. We observe both qualitative properties with a simulation of right branched phrases with increasing number of words. Hemodynamic Amplitude Constituents size Time (ms) Time (ms) High Gamma power Time (ms) P r e d i c t i n g n e u r a l a c t i v i t y b y m o d e l l i n g t h e n u t s a n d b o l t s o f l a n g u a g e t r e e s A b s t r a c t 1. UNICOG INSERM/CEA/SAC/DSV/DRM/Neurospin center. Bât 145, Point Courier 156. F-91191 Gif-sur-Yvette Cedex FRANCE 2. Institute for Artificial Intelligence and Biological Systems. School of Computing. University of Leeds. LS2 9JT Leeds. UK N e u r a l B l a c k b o a r d A r c h i t e c t u r e I m p l e m e n t a t i o n Martin Perez-Guevara 1 , Marc De Kamps 2 and Christophe Pallier 1 Few attempts have been made to model language in biological neural networks. The Neural Blackboard Architecture (NBA), proposed by Van der Velde and De Kamps [1] is one of them. It was designed to answer many challenges in the neural modeling of sentence processing, including the ones detailed by Jackendoff [2]. Here we expand on previous simulations of the Blackboard Architecture [3] on leaky-integrage-and-fire (LIF) populations with population density techniques [4] implemented in MIIND [5], to compare simulated time courses of neural activity associated to sentence representation and parsing with functional magnetic resonance data (fMRI) and intracranial recordings (electro-corticography; ECOG).
