The hippocampal subregion CA1 and CA3 in noveltydetection in a visually rich virtual reality system

Jhoseph Shin, Seung-Woo Jin, and Inah LeeDepartment of Brain and Cognitive Sciences

Seoul National UniversitySeoul, South Korea 08826lee.inah@gmail.com

Abstract

It is widely acknowledged that the hippocampus plays critical roles in memoryencoding and retrieval. However, explaining how the hippocampal circuit canreliably process and store memories under different learning conditions remains amajor question. To optimize the efficiency of the memory system, redundancy of theinformation should be minimized to reduce the interference between pre-existinginformation and new incoming information. It has been suggested that CA1 mightact as a comparator based on the information from CA3 (i.e., prior experience) andentorhinal cortex (i.e., present experience). So far, several studies had demonstratedthat the activity of CA1 is elevated while subjects are experiencing an unexpectedstimulus. However, neural evidence of the model at the single-unit level is lacking,and most importantly, the exact role of hippocampal subregion CA1, which isthe major cortical output of the hippocampal circuit, has been largely unknownup to date. To examine the functional role of the hippocampal subregions in thenovelty detection, we recorded single units of CA1 and CA3 simultaneously in ahead-restrained rat navigating in a virtual reality (VR) environment system. As therat foraged in two distinct contextual environments, robust place fields were presenton the track and different visual environments were represented by place cells inboth CA1 and CA3. However, when foggy conditions in the familiar environmentwere introduced after the rat finished normal laps as a baseline, nearly half of theCA1 population are expected to form a new field representation while the most ofthe CA3 population are expected to retain its firing field. These data will serve asdirect evidence to the hypothesis that the CA3 might provide the memory of a priorenvironment by maintaining its field representation while subsets of CA1 mightgenerate the novelty signals by forming a new field representation.

1 Introduction

For the decades, the hippocampus has been known to play crucial roles in the memory system. Theseinclude episodic memory based on clinical cases [7, 28, 32] or spatial memory based on cognitivemap theory [25] with the discoveries of place cells in the rodent hippocampus [24]. In our daily life,even if similar routines are repeated, they are remembered separately. In this process, there has been ahypothesis [4, 9, 10, 19, 20, 23, 26, 33] that the hippocampus requires computation that ‘retrieve’ theexisting experience and ‘compare’ it with the current experience. In addition, the unique anatomicalstructure [2, 34, 35] of the hippocampus further emphasizes the hypothesis. The entorhinal cortex(EC) sends sensory information to the hippocampus via the perforant pathway/temporoammonicpathway, while recurrent collateral in the CA3 facilitates information retrieval [22, 30, 31]. It istempting to speculate that a match/mismatch comparator will occur in CA1 by interpreting theseanatomical features as memory components from CA3 and sensory components from EC. That

NeurIPS 2020 Workshop on BabyMind, Vancouver, Canada.

is, if recall matches to the sensory stimulus, the hippocampal state will stay in the ‘recall’ mode.In contrast, if recall does not match the sensory input, the hippocampal state will be available foracquiring novel information. Several experiments have reported significantly modulated activity inthe CA1 [8, 12-14, 21] or increased expression of the immediate-early gene (IEG) in the CA1 [11]under novelty conditions. However, causal evidence of the hypothesis is still lacking. For example,in a situation where there are incongruencies between past experiences (i.e., memory) and presentexperiences (i.e., sensory reality), what is expected to empirically happen in CA1 and CA3? It is notclear how the relationship between CA1 and CA3 will be. Our goal is to examine the interactionswithin the subregions in the hippocampus when there are incongruencies between the memory andthe sensory input that have occurred in the VR system.

2 Methods

2.1 Virtual reality (VR) system

The VR system was designed for head-restrained rats (n=8) on a circular cylinder using game engineUnreal 4.0 (Epic Software). The custom-made virtual environment was created with visually richcontexts and auditory contexts. Three 24-in LG monitors were placed at a 135 ° angle to display thevirtual environments generated by the VR software. The optical sensor was positioned on the center-axis platform to detect the rotation of the Styrofoam cylinder. The licking tube was positioned directlyin front of the animal to deliver water as a reward. The delivery of the reward was controlled byevents from the Unreal engine via ARDUINO connected to the solenoid valve. For each frame of theUnreal engine, a TTL signal was triggered from the ARDUINO to Neuralynx for the synchronizationof neural data and the virtual reality system.

Figure 1: (A): Histology verification of the tetrodes (B): Virtual reality (VR) system setting (left);Contexts used in the experiment (right);

2.2 Data collection and pre-processing

Single units were recorded simultaneously from the CA1, CA3, and visual cortex by implantinghyperdrive as the head-fixed rat navigated in a virtual space by running in a cylindrical treadmillsurrounded by three LCD monitors. In each environment, rats were trained to move forward along alinear track. Once the rats were trained to criterion (>100 meters for two consecutive days), singleunits were recorded with tetrodes as the rat randomly foraged in two distinct contextual environments.After four days of recording in the familiar environments, we manipulated the following environmentalcomponents across days: a) auditory contextual reversal, b) sensory-motor gain manipulation, c)visibility (controlled by fog density). Single units were isolated manually using custom software(WinCluster) using multiple parameters such as peak, energy, and waveform [6, 15]. Only neuronswith the following set of criteria were used in the further analysis: (1) total numbers of spikes during

2

the behavior session > 100, (2) average firing rate during the behavior session > 0.5Hz, (3) Proportionof spikes within a 1ms refractory period 0.5.

3 Experimental results

3.1 Robust place cells were observed in the VR system

To confirm whether place fields are activated in our VR system, single units were sampled from CA1,CA3, and visual cortex (Fig.1A) as the rats run through the linear track in the virtual environments(i.e., Context A : forest, Context B : city) to get water rewards in the random location (Fig.1B).Each context was separated by inter-trial-zone in which there was no visual stimulus. As a result,place-specific firing patterns were observed (CA1: 25.41%; 154/606, CA3: 19.38%; 126/650 fromthe sampled population; Fig.2A) on the VR track. Each context was represented orthogonally bysubsets of population in CA1 and CA3 (Fig. 2C). In addition, Fields were formed as a uniformdistribution over the entire track (CA1: p=0.7483 / CA3: p= 0.1029, Kolmogorov–Smirnov test), andno significant differences in the field distribution or numbers of fields between CA1 and CA3 wereobserved (p = 0.4247, Kolmogorov–Smirnov test; Fig 2B).

Figure 2: (A): Raw example of a hippocampal place cell. The trajectory of the animal in the virtualtrack (gray-dotted). Each red dot indicates the spike on the track. Rate maps were generated by rawspike maps (right). Red areas indicate the most active position of the place cell (B): Total numberof fields in each hippocampal subregion (top); Field distribution over entire track per each context(bottom) (C): Rate maps are aligned by peak position. This data suggests the uniform distribution ofthe visually induced place fields in the VR system

3.2 Different population dynamics are expected to be observed in the hippocampalsubregions CA1 and CA3

To investigate the interaction of CA1 and CA3 in detecting the novelty of the environment, we haveadded visual noise in the familiar environment. To quantitatively manipulate the visual scene, thestructural similarity index (SSIM) was calculated from the original scene (i.e., the scene used inthe standard session). As rats finished the normal laps, foggy conditions (e.g., 0%, 15%, and 30%)in the familiar environments were introduced. As previously reported [8, 13], firing rates in CA1was significantly increased (p < 0.001, Kruskal-Wallis test; Data not shown). To further analyzethe heterogeneous activity of CA1, the population will be classified into the stable population vsremapped population to see whether there will be subregion differences in the heterogeneity. Also,principal component analysis (PCA) will be conducted to test the temporal impact of the novelty

3

induced in the environment on the hippocampal system. These data will serve as direct evidence ofthe novelty detection model where CA1 acts as a comparator by maintaining the pre-existence mapwhile signaling novelty induced in the environment.

Figure 3: Working model of the hippocampal system in detecting novelty (A): Heterogenous modulesin the CA1. Each module consists of sets of pyramidal neurons and inhibitory neurons. Theseheterogeneous modules are in a position to play a role in providing feed-forward inhibition to eachother. (B): It is assumed in the model that there will be a recall module and acquisition modulein CA1 depending on the difference in synapse strength from CA3 or EC. If the expected sensorystimulus processed in CA3 and the sensory stimulus coming from the EC match, the recall module isdominantly activated. (C) On the other hand, if the expected sensory stimulus and sensory stimulusdo not match, the degree of activation of the recall module decreases. This leads to disinhibition ofthe acquisition module.

4 Discussion

In our virtual reality system, we could confirm that the activity of the cell firing was mainly affectedby the visual contexts. Previous studies using VR systems have also shown that the activity of placefields is sustained by visual stimulus [1, 3, 5]. Also, the proportion of the active place cells wereequivalent to the previous studies [3, 27]. The main findings of this study are expected to comefrom where the visual novelty was induced in the familiar environment: Heterogeneous populationdynamics are observed in the CA1 (Fig 3A). Although several studies up to date have producedthe discrepancy in the familiar environments to examine the roles within hippocampal subregions[16-18], there has been a limitation. It has not been easy to dissociate the novelty factor from theplace components. For example, Lee, et al. (2004b) examined the activities of CA1 and CA3 inthe double rotation paradigm but interpreting the results of CA1 was not easy due to the mixedresponses of the local cue, distal cue, and combined. However, in our study, modification in theenvironment was added in a uniformly distributed manner to dissociate the place factor. In thisway, we will be able to observe a coherent, but the heterogeneous activity of CA1. The populationthat maintained the field representation will be likely to have a similar state dynamic to the CA3,while the population that newly formed the field representation will be likely to signal the noveltyin the environment (Fig 3C) to facilitates the new learning. Overall, to our knowledge, this resultwill serve as the direct physiological evidence of hippocampal match/mismatch computations in thecontext of CA1 receiving two anatomical inputs (i.e., perforant pathway/ temporoammonic pathway)adopted as the basis for the novelty detection model (Fig 3B, C). In addition, discriminating whatis remembered from what is experienced is important for efficient learning. To be successfullyrecognized as an artificial intelligence machine imitating the intelligence of the baby, it must beable to explore and learn the unknown world on its own, while the surrounding circumstances aredynamically changing. Although this study lacks the computational modeling or the developmentalrelevance, understanding the role of the hippocampal subfields should suggest the flexibility in themodel (i.e., adaptive machine learning).

4

Broader Impact

The experimental evidence sought in the current research is expected to find some breakthroughsin the current systems neuroscience field. This is largely due to discrepancies found between thepredictions made from the theory and the existing empirical data. In addition, episodic memory isclosely related to our everyday life and is gradually affected by aging and dementia. The resultsfrom the current study will contribute to the early diagnosis and treatment of these debilitating braindisease-related cognitive dysfunctions related to the hippocampal memory system. The techniquesused in the current study and the results from the application of those techniques will contribute toimplementing virtual reality and artificial intelligence in the industrial fields as well.

Acknowledgments and Disclosure of Funding

The authors declare no competing financial interests. This study was supported by grants fromthe BK21+ program (5286-2014100), Basic Research Laboratory program (2017M3C7A1029661,2018R1A4A1025616, 2019R1A2C2088799), Brain Research Program (2019R1A2C2088799,2017M3C7A1029661), AI Institute of Seoul National University (AIIS) through its AI FrontierResearch Grant (0670-20200011), 10-10 Project of SNU, and Institute for Information & Commu-nications Technology Planning & Evaluation(IITP) grant funded by the Korea government(MSIT)(No.2019-0-01367, Infant-Mimic Neurocognitive Developmental Machine Learning from InteractionExperience with Real World (BabyMind)).

References

[1] Acharya, L., Aghajan, Z. M., Vuong, C., Moore, J. J., & Mehta, M. R. (2016). Causal Influence of VisualCues on Hippocampal Directional Selectivity. Cell, 164(1-2), 197-207. doi:10.1016/j.cell.2015.12.015

[2] Amaral, D. G., & Witter, M. P. (1989). The three-dimensional organization of the hippocampalformation: a review of anatomical data. Neuroscience, 31(3), 571-591. doi:10.1016/0306-4522(89)90424-7

[3] Aronov, D., & Tank, D. W. (2014). Engagement of neural circuits underlying 2D spatial naviga-tion in a rodent virtual reality system. Neuron, 84(2), 442-456. doi:10.1016/j.neuron.2014.08.042

[4] Berteau, S., & Bullock, D. (2020). Simulations reveal how M-currents and memory-based inputsfrom CA3 enable single neuron mismatch detection for EC3 inputs to the CA1 subfield of hippocampus. JNeurophysiol, 124(2), 544-556. doi:10.1152/jn.00238.2019

[5] Chen, G., King, J. A., Burgess, N., & O’Keefe, J. (2013). How vision and movement combine inthe hippocampal place code. Proc Natl Acad Sci U S A, 110(1), 378-383. doi:10.1073/pnas.1215834110

[6] Delcasso, S., Huh, N., Byeon, J. S., Lee, J., Jung, M. W., & Lee, I. (2014). Functional relation-ships between the hippocampus and dorsomedial striatum in learning a visual scene-based memory task in rats.J Neurosci, 34(47), 15534-15547. doi:10.1523/JNEUROSCI.0622-14.2014

[7] Eichenbaum, H. (2000). A cortical-hippocampal system for declarative memory. Nat Rev Neu-rosci, 1(1), 41-50. doi:10.1038/35036213

[8] Fyhn, M., Molden, S., Hollup, S., Moser, M. B., & Moser, E. (2002). Hippocampal neurons responding tofirst-time dislocation of a target object. Neuron, 35(3), 555-566. doi:10.1016/s0896-6273(02)00784-5

[9] Hasselmo, M. E., & Schnell, E. (1994). Laminar selectivity of the cholinergic suppression ofsynaptic transmission in rat hippocampal region CA1: computational modeling and brain slice physiology. JNeurosci, 14(6), 3898-3914.

[10] Hasselmo, M. E., Schnell, E., & Barkai, E. (1995). Dynamics of learning and recall at excita-tory recurrent synapses and cholinergic modulation in rat hippocampal region CA3. J Neurosci, 15(7 Pt 2),5249-5262.

[11] Jenkins, T. A., Amin, E., Pearce, J. M., Brown, M. W., & Aggleton, J. P. (2004). Novel spatial arrangementsof familiar visual stimuli promote activity in the rat hippocampal formation but not the parahippocampal cortices:a c-fos expression study. Neuroscience, 124(1), 43-52. doi:10.1016/j.neuroscience.2003.11.024

5

[12] Karlsson, M. P., & Frank, L. M. (2008). Network dynamics underlying the formation of sparse, informativerepresentations in the hippocampus. J Neurosci, 28(52), 14271-14281. doi:10.1523/JNEUROSCI.4261-08.2008

[13] Kumaran, D., & Maguire, E. A. (2006). An unexpected sequence of events: mismatch detectionin the human hippocampus. PLoS Biol, 4(12), e424. doi:10.1371/journal.pbio.0040424

[14] Larkin, M. C., Lykken, C., Tye, L. D., Wickelgren, J. G., & Frank, L. M. (2014). Hippocampaloutput area CA1 broadcasts a generalized novelty signal during an object-place recognition task. Hippocampus,24(7), 773-783. doi:10.1002/hipo.22268

[15] Lee, I., & Kim, J. (2010). The shift from a response strategy to object-in-place strategy duringlearning is accompanied by a matching shift in neural firing correlates in the hippocampus. Learn Mem, 17(8),381-393. doi:10.1101/lm.1829110

[16] Lee, I., Rao, G., & Knierim, J. J. (2004). A double dissociation between hippocampal subfields:differential time course of CA3 and CA1 place cells for processing changed environments. Neuron, 42(5),803-815. doi:10.1016/j.neuron.2004.05.010

[17] Lee, I., Yoganarasimha, D., Rao, G., & Knierim, J. J. (2004). Comparison of population coherence of placecells in hippocampal subfields CA1 and CA3. Nature, 430(6998), 456-459. doi:10.1038/nature02739

[18] Leutgeb, J. K., Leutgeb, S., Treves, A., Meyer, R., Barnes, C. A., McNaughton, B. L., Moser, E.I. (2005). Progressive transformation of hippocampal neuronal representations in "morphed" environments.Neuron, 48(2), 345-358. doi:10.1016/j.neuron.2005.09.007

[19] Lisman, J. E., & Grace, A. A. (2005). The hippocampal-VTA loop: controlling the entry of in-formation into long-term memory. Neuron, 46(5), 703-713. doi:10.1016/j.neuron.2005.05.002

[20] Lisman, J. E., & Otmakhova, N. A. (2001). Storage, recall, and novelty detection of sequencesby the hippocampus: elaborating on the SOCRATIC model to account for normal and aberrant effects ofdopamine. Hippocampus, 11(5), 551-568. doi:10.1002/hipo.1071

[21] Maass, A., Schutze, H., Speck, O., Yonelinas, A., Tempelmann, C., Heinze, H. J., Duzel, E.(2014). Laminar activity in the hippocampus and entorhinal cortex related to novelty and episodic encoding. NatCommun, 5, 5547. doi:10.1038/ncomms6547

[22] Marr, D. (1971). Simple memory: a theory for archicortex. Philos Trans R Soc Lond B BiolSci, 262(841), 23-81. doi:10.1098/rstb.1971.0078

[23] Meeter, M., Murre, J. M., & Talamini, L. M. (2004). Mode shifting between storage and recall based onnovelty detection in oscillating hippocampal circuits. Hippocampus, 14(6), 722-741. doi:10.1002/hipo.10214

[24] O’Keefe, J., & Dostrovsky, J. (1971). The hippocampus as a spatial map. Preliminary evidencefrom unit activity in the freely-moving rat. Brain Res, 34(1), 171-175. doi:10.1016/0006-8993(71)90358-1

[25] O’Keefe J., & Nadel L (1978). The hippocampus as a cognitive map. Oxford, UK: Claren-don.

[26] Ranganath, C., & Rainer, G. (2003). Neural mechanisms for detecting and remembering novelevents. Nat Rev Neurosci, 4(3), 193-202. doi:10.1038/nrn1052

[27] Ravassard, P., Kees, A., Willers, B., Ho, D., Aharoni, D. A., Cushman, J., Mehta, M. R. (2013). Multisensorycontrol of hippocampal spatiotemporal selectivity. Science, 340(6138), 1342-1346. doi:10.1126/science.1232655

[28] Scoville, W. B., & Milner, B. (2000). Loss of recent memory after bilateral hippocampal le-sions. 1957. J Neuropsychiatry Clin Neurosci, 12(1), 103-113. doi:10.1176/jnp.12.1.103

[29] Soltesz, I., & Losonczy, A. (2018). CA1 pyramidal cell diversity enabling parallel informationprocessing in the hippocampus. Nat Neurosci, 21(4), 484-493. doi:10.1038/s41593-018-0118-0

[30] Treves, A., & Rolls, E. T. (1992). Computational constraints suggest the need for two distinctinput systems to the hippocampal CA3 network. Hippocampus, 2(2), 189-199. doi:10.1002/hipo.450020209

[31] Treves, A., & Rolls, E. T. (1994). Computational analysis of the role of the hippocampus in

6

memory. Hippocampus, 4(3), 374-391. doi:10.1002/hipo.450040319

[32] Vargha-Khadem, F., Gadian, D. G., Watkins, K. E., Connelly, A., Van Paesschen, W., & Mishkin, M. (1997).Differential effects of early hippocampal pathology on episodic and semantic memory. Science, 277(5324),376-380. doi:10.1126/science.277.5324.376

[33] Vinogradova, O. S. (2001). Hippocampus as comparator: role of the two input and two outputsystems of the hippocampus in selection and registration of information. Hippocampus, 11(5), 578-598.doi:10.1002/hipo.1073

[34] Witter, M. P., Canto, C. B., Couey, J. J., Koganezawa, N., & O’Reilly, K. C. (2014). Architec-ture of spatial circuits in the hippocampal region. Philos Trans R Soc Lond B Biol Sci, 369(1635), 20120515.doi:10.1098/rstb.2012.0515

[35] Witter, M. P., Wouterlood, F. G., Naber, P. A., & Van Haeften, T. (2000). Anatomical organization of theparahippocampal-hippocampal network. Ann N Y Acad Sci, 911, 1-24. doi:10.1111/j.1749-6632.2000.tb06716.x

7