BIOMEDICAL ENGINEERING
SPIKE TRAIN ANALYSIS OF SPATIAL DISCRIMINATIONS AND
FUNCTIONAL CONNECTIVITY OF PAIRS OF NEURONS
IN CAT STRIATE CORTEX
JASON MICHAEL SAMONDS
Thesis under the direction of Professor A. B. Bonds
We studied changes in ensemble responses of striate cortical pairs for small
(<10deg, 0.1c/deg) and large (>10deg, 0.1c/deg) differences in orientation and spatial
frequency. Examination of temporal resolution and discharge history revealed
advantages in discrimination from both dependent (connectivity) and independent
(bursting) interspike interval properties. We found the average synergy (information
greater than that summed from the individual neurons) was 50% for fine discrimination
of orientation and 25% for spatial frequency and <10% for gross discrimination of both
orientation and spatial frequency.
Dependency (Kullback-Leibler "distance" between the actual responses and two
wholly independent responses) was measured between pairs of neurons while varying
orientation, spatial frequency, and contrast. In general, dependency was more selective to
spatial parameters than was firing rate. Variation of dependence against spatial
frequency corresponded to variation of burst rate, and was even narrower than burst rate
tuning for orientation. We also found a gradual decline (adaptation) of dependency over
time that is faster for lower contrasts and which is likely a result of the decrease in
isolated (non-burst) spikes.
The results suggest that salient information is more strongly represented in bursts,
but that isolated spikes also have a role in transferring this information between neurons.
The dramatic influence of burst length modulation on both synaptic efficacy and
dependency around the peak orientation leads to substantial cooperation that can improve
discrimination in this region.
Approved___________________________________________ Date________________
SPIKE TRAIN ANALYSIS OF SPATIAL DISCRIMINATIONS AND
FUNCTIONAL CONNECTIVITY OF PAIRS OF NEURONS
IN CAT STRIATE CORTEX
By
Jason Michael Samonds
Thesis
Submitted to the Faculty of the
Graduate School of Vanderbilt University
in partial fulfillment of the requirements
for the degree of
MASTER OF SCIENCE
in
Biomedical Engineering
May, 2002
Nashville, Tennessee
Approved: Date:
ACKNOWLEDGEMENTS
I express my gratitude to Professor A. B. Bonds for his guidance and
encouragement throughout this project, as well as his support during my time at
Vanderbilt University. I would also like to extend my gratitude to Professor Don
Johnson for his assistance in explaining the finer details of type analysis, and to Professor
Jonathan Victor for sharing his knowledge and experience with spike train analysis. I am
very grateful to Professor Ross Snider for working with me in order to use his spike
sorting and cross-correlation software to contribute to my results. And lastly, I would
like to thank Dr. John Allison and Heather Brown for their help in collecting the data.
Although this project would not be possible without their assistance, all the ideas, type
analysis software, writing, and conclusions are my own work.
I would also like to thank the Graduate School and the National Institute of Health
(Grant RO1 EY03778) for providing financial support during my time in graduate school.
And it goes without saying that I am always grateful for the support from friends and
family that has always been there throughout my educational pursuits.
ii
TABLE OF CONTENTS
Page
ACKNOWLEDGEMENTS............................................................................................... ii
LIST OF FIGURES .......................................................................................................... iv Chapter I. NEURAL POPULATION ANALYSIS REVIEW........................................................ 1
Introduction............................................................................................................ 1 Theoretical Background............................................................................. 2 Single-unit Research .................................................................................. 6 Multi-unit Research ................................................................................. 10 Functional Imaging .................................................................................. 14
Correlation and Connectivity............................................................................... 15 Point Process and Cross-correlation ........................................................ 15 Partialization ............................................................................................ 19 Gravitational Clustering........................................................................... 19 Information Theory: Dependency and Complexity ................................. 21 Causality .................................................................................................. 22 Nonlinear Methods................................................................................... 23
Neural Code Theory............................................................................................. 24 Average Spike Rate Code ........................................................................ 25
Temporal Code......................................................................................... 26 Bursting....................................................................................... 27 Latency........................................................................................ 28 Spatiotemporal Patterns .............................................................. 30 Oscillations ................................................................................. 31 Chaos Theory ........................................................................................... 33
Information Theory............................................................................................. 35 The Future........................................................................................................... 39
II. COOPERATION BETWEEN AREA 17 NEURON PAIRS THAT ENHANCES FINE DISCRIMINATION OF ORIENTATION................................................................. 41
Introduction......................................................................................................... 41 Methods............................................................................................................... 44
Preparation ............................................................................................... 44 Stimuli...................................................................................................... 45 Data Acquisition and Spike Classification .............................................. 47
Type Analysis .......................................................................................... 48
iii
Results................................................................................................................. 53 Latency..................................................................................................... 55 Temporal Resolution................................................................................ 59 Discharge History .................................................................................... 63 Synergy, Independence, and Redundancy ............................................... 68 Confidence in Distance Estimations ........................................................ 73 Functional Connectivity........................................................................... 76
Discussion........................................................................................................... 87 Latency..................................................................................................... 87 Independent ISI Characteristics ............................................................... 89
Bursts and Connectivity........................................................................... 90 Functional Connectivity and Synergy...................................................... 92 Orientation Discrimination ...................................................................... 94
III. FUTURE EXPLORATIONS.................................................................................... 95
Introduction.......................................................................................................... 95 Cortical Function Theory..................................................................................... 96
Multidimensional Data....................................................................................... 100 Cortical Clustering ............................................................................................. 101 Spatiotemporal Connectivity ............................................................................. 103 REFERENCES .............................................................................................................. 105
iv
LIST OF FIGURES Figure Page 1. An example of firing rate tuning and fine and gross discriminations..................... 55 2. Enhanced discrimination with latency differences ................................................. 58 3. Distance rates versus temporal resolution............................................................... 62 4. Determination of Markov order of analysis............................................................ 64 5. Discharge history contribution to orientation discrimination ................................. 66 6. Discharge history contribution to spatial frequency discrimination ....................... 67 7. Ensemble distance versus individual neuron distances .......................................... 72 8. Distances and synergy versus random sample size................................................. 75 9. Another example of sample size functions ............................................................. 77 10. Temporal dynamics of dependnecy between neurons ............................................ 81 11. Difference between dependency and firing rate during contrast modulation ......... 83 12. Temporal dynamics of dependency adaptation....................................................... 84 13. Dependency tuning for orientation and spatial frequency modulation................... 86
v
CHAPTER I
NEURAL POPULATION ANALYSIS REVIEW
Introduction
One of the most elusive questions in biological sciences has been how the brain is
able to encode and decode the multi-dimensional features of sensory signals and to
formulate perceptions or perform actions on the order of hundreds of milliseconds. The
majority of neurophysiological studies have relied on counting the number of spikes
recorded from a single neuron and demonstrating how this spike count varies according
to sensory input features. Although these data represent the average response of only one
neuron to specific stimulus features, neurophysiologists have at the same time
acknowledged that the brain could only perform their complex operations with
populations of neurons. The brain cannot be thought of as simply as a passive screen
receiving a projected image of the outside world. The brain is able to separate figure
from background, perform invariant recognition, and make accurate and generalized
predictions. Understanding the representation of sensory information both on local and
global levels is equally important, and neither of these tasks presents a clear approach to
finding a solution. It cannot even be posed as a simple encoding and decoding problem
because there is not always a clear input and output.
In this introduction, we describe how the theories of brain function have evolved
over the last two centuries and how anatomical and physiological studies of individual
neurons and small groups of neurons have contributed to these theories. Then we
1
describe the methods that have been developed to analyze interactions between nerve
cells and when and how they are applicable to providing insight to cortical functions.
One of the difficulties in understanding brain function is deciding where to start the
analysis. We describe some of the theories of what aspects of the individual neuron's
signal carry the sensory information. Since we are looking at how information is
transmitted and manipulated in the brain, information theory has become an important
analytical tool. Lastly, we discuss where the theories of neural population codes might be
heading and how some of the conflicting arguments might be resolved.
Theoretical background
In a recent review, Doetsch (2000) points out that the idea that sensory
information was encoded in patterns of populations of neurons was proposed as early as
1802 by Young and elaborated by von Helmholtz in 1860 for explaining color vision.
Sherrington (1941) wrote on the importance of understanding the cooperation of groups
of nerve cells beyond their individual properties even with the understanding that the
brain does utilize localization for many of its functions. In 1949, Hebb suggested that
groups of cells form regional circuits and are activated by the appropriate spatiotemporal
firing pattern and then produce some appropriate spatiotemporal output pattern. Hebb’s
theory was proposed to explain many of the phenomena observed in psychophysical
studies. The main idea is that neurons that ‘fire’ together will ‘wire’ together, which is
the foundation behind learning in the brain. Learning itself is a slow and tedious process
to create the ‘wiring’ that later on leads to the fast and generalized perceptions.
2
In 1972, Barlow proposed a contradictory theory where the individual cells
represent their information independently. The theory was based on the data that was
available at the time from single neuron (single-unit) recordings. The idea is that the
individual nerve cells are very specialized feature detectors that are activated with the
appropriate stimulus in the appropriate location. The theory is known as the ‘cardinal’
cell theory because the information converges as it is passed on to more specialized cells
higher up in the hierarchy of the brain’s perceptual regions. However, the convergence is
not so drastic as to end up activating a single ‘grandmother’ cell (the notion that every
perception has its own cell—i.e., your grandmother activates a particular nerve cell).
Von der Malsburg (1981) introduced a correlation theory based on some of the
Hebbian principles, along with theories on pattern recognition and neural networks. The
purpose of this theory was to address the deficiencies of the previous brain function
theories and propose solutions to these problems. In general, von der Malsburg’s
correlation theory is based on the synaptic strength modulations on short- and long-term
time scales (also a theory behind short- and long-term memories). The long-term
modulations are based on anatomical and physiological modifications of synaptic
connections, while the short-term modulations might be induced within the temporal
structure of cellular signals. The key is that synaptic strengths are dynamic and lead to
competition and the creation of subnets within a larger network. Uncorrelated subnets
can coexist without interference and it is correlation (or synaptic coupling) that ties
information together and determines the activity patterns. Rather than requiring ‘hard-
wired’ specialized detectors, von der Malsburg’s theory predicts that only simple feature
3
detectors would be required and that more complex features are extracted through the
activation of synaptic subnets.
Another theory derived from the combination of theoretical neural networks and
neurophysiological data is the synfire chain model of the cortex (Abeles 1991). The
synfire chain model is a network of converging and diverging connections where
synchronization is fundamental to the processing and transmission of neural information.
An individual neuron basically acts as a coincidence detector (Abeles, 1982) that passes
on spikes to the postsynaptic neuron that are synchronized with sub-millisecond
precision. One of the motivating aspects behind the theory is that the neuron is typically
much more sensitive to synchronous inputs over integrating (asynchronous) inputs. An
argument against this theory is the unreliability of synaptic transmission (Shadlen and
Newsome, 1994), but theoretical models have demonstrated the possibility of sub-
millisecond precision with a synfire chain model incorporating on the order of 100
neurons (Deismann et al., 1999).
Shadlen and Newsome (1994) believe the unreliable synapse can only be used in
a model that incorporates integration. Each neuron receives thousands of inputs that
create a balance between excitation and inhibition (chaos) to reach threshold in the
postsynaptic neuron faster than with the resting membrane potential, but still avoid
saturation. The signals of individual neurons are very noisy and highly redundant. They
believe the sensory information is represented by the form of an average firing rate
pooled across populations of 50-100 neurons. There is an asymptote reached for signal-
to-noise with populations of this size as averaging cannot remove correlated noise. The
irregularity of the interspike interval (ISI) would seem to be an argument for its role in
4
carrying neural information, but Shadlen and Newsome (1998) believe that this
irregularity is a result of inhibition in balancing the chaos. They believe that redundancy
is not a problem with the massive number of neurons in the cortex and reasonably
accurate firing rates can be transmitted with integration times as fast as 10 ms.
Hopfield (1995) views the problem of understanding brain function by
considering the problem of pattern recognition and the capabilities of neurophysiology.
Although it might not be as efficient from an information-theoretic point of view, a code
based on timing rather than rate makes more sense from both the pattern recognition and
biological point of view. A network based on timing allows for scale-invariant
recognition. Delays can easily be caused by synaptic, axonal, or cellular mechanisms and
decoding is provided by a coincident-detection scheme.
The physiological and psychophysical studies of the last 20 years have continued
to demonstrate the nonlinearities of the visual system (Wilson and Wilkinson, 1997).
The nonlinear pooling mechanisms and interactions that are found in the neural
processing suggest a model similar to winner-take-all (WTA) networks and make it clear
that the visual system cannot be broken down into independent spatial channels. These
mechanisms have been demonstrated for texture perception, stereopsis, motion
perception, and form perception. Wilson and Wilkinson (1997) point out that one of the
reasons nonlinear mechanisms would make sense for visual processing is that no matter
how many linear calculations are performed, they can always be reduced down to one
single linear calculation.
Lastly, an alternative to neural network-based or spatiotemporal-based coding
models is a population coding theory where individual neurons can be thought of as
5
vectors (Pouget et al., 2000). Because individual neurons are tuned to a feature, the
magnitude of the responses of a population of neurons can be added in vector space based
on their tuning properties. The theory has been derived from populations in the middle
temporal visual area (area MT) and motor cortical studies to determine direction. The
vector approach allows for nonlinear mappings and is most efficient when applying
Bayesian classification principles. However, recent evidence against the population
vector hypothesis has been shown in the neural activity of the motor cortex in predicting
hand movement (Scott et al. 2001).
Single-unit research
The brain averages responses across populations whereas in the laboratory it is
averaged over time. This practice is based on the assumption that the average firing rate
is the primary component in the representation of sensory information and that the single
unit is sufficiently representative of the population. Still, regardless of the theory of brain
function, single-unit recordings can be used to reveal certain properties of the network
circuitry with careful selection of the stimulus, iontophoretic application of
neurotransmitters, and anatomical studies of the cell types and synaptic connections at the
recording sites.
Hubel and Weisel (1962) were the first to study the functional architecture of the
visual cortex through an extensive analysis of the individual neurons. The anatomy of
the visual cortex reveals 6 distinct layers and diversity and organization of cells, but
Hubel and Weisel demonstrated organization and cell types by measuring the responses
of single units. They discovered visual cortical cells responded to bars of light at a
6
preferred orientation, sometimes at a preferred direction, and across a continuum of
monocular to binocular stimulation. Their research demonstrated an organization of
orientation columns and ocular dominance hypercolumns across the cortical layers, along
with a retinotopic mapping across the surface of the cortex. The analysis of the receptive
field properties of the cells revealed two primary classifications of cells (simple and
complex). They proposed a feed-forward linear model of the visual cortex based on the
properties of lateral geniculate nucleus (LGN) and cortical cells. The model proposes a
hierarchy from LGN to simple to complex cells that explains the origin of orientation
tuning, along with the other receptive field properties they discovered.
This same experimental approach was used more recently to examine functional
organization of additional properties of cortical cells (DeAngelis et al., 1999). DeAngelis
et al.'s results showed that although almost all properties (spatial frequency, orientation,
temporal frequency, and latency) were organized in clusters and columns, there was
diversity in the organization in the spatiotemporal receptive fields. The difference in
spatial phase of the receptive fields of nearby cells prevents overlap and redundancy
among the clusters and columns. Their results provide an explanation for the relative
lack of redundancy found in nearby cortical neurons when the tuning characteristics
would suggest otherwise.
Sillito (1972) studied the role inhibition had in receptive field properties by
iontophoretically applying bicuculline to suppress the inhibitory neurotransmitter
gamma-aminobutyric acid (GABA). By comparing tuning functions of individual cells
with and without inhibition, Sillito was able to demonstrate that inhibitory mechanisms
play a role in simple and complex cell orientation tuning. Without inhibition, the simple
7
cells had much broader tuning, lost the linear “on” and “off” receptive field distinctions,
and had a loss or reduction in directionality specificity. Bicuculline resulted in less
dramatic broadening or no change in orientation tuning and a less significant effect in
directionality specificity for complex cells. The results provide evidence that the visual
cortex cannot be thought of as a simple linear feed-forward model (Hubel and Weisel,
1962). They do not rule out the role feed-forward mechanisms might have in receptive
field properties such as orientation tuning, but simply demonstrate that inhibitory
mechanisms also play a role and a more complicated model of cortical organization is
required.
Toyoma et al. (1974) examined the organization of the visual cortex by using a
stimulating electrode along with a recording electrode. This procedure allowed them to
examine axonal projections and synaptic connections within and across cortical layers.
After determining the organization of the projections and identifying excitatory and
inhibitory connections, they were able to come up with a rough model of the circuitry of
the cortex. Even their simple model once again demonstrated the influence of inhibition
and the complexity of the cortical network with the inclusion of inhibitory interneurons.
Creutzfield et al. (1974a,b) examined the vertical organization of the visual cortex
with intracellular recordings and analysis of the peristimulus time histogram (PSTH).
They found that inhibition usually followed an excitatory response and that orientation
tuning did not appear to be a simple result of precise spatial arrangement from afferent
neurons. They also did not find a lot of shared input or any excitatory connections within
orientation columns suggesting there is not a lot of convergence. The inhibition they
found was not also very localized, but was almost always apparent in a diffuse form.
8
Their results suggest that many of the large number of synaptic connections within the
cortex are inhibitory.
Another method used to derive network properties from the response of a single
unit is to use sub-threshold stimulation. Sub-threshold stimulation is when a stimulus
does not evoke a response when presented alone, but causes a change in the response to
another stimuli that does evoke a response. Because the sub-threshold stimulation is
below threshold it does not result in a response on its own, but it does still induce a
postsynaptic potential, which can lead to changes in the network interactions when
stimuli are shown that do result in a response.
One example of this protocol is the cross-orientation stimulus (Morrone et al.,
1982; Bonds, 1989) used to study the role of inhibition in orientation tuning. The
stimulus consists of two rapidly interleaved sine wave gratings with one grating at the
optimal orientation and the other one varied to reduce the response. The results of these
two studies suggested that the inhibition was a result of pools of cells and not a property
of the recording cell. The results also suggest that inhibition is intracortical (from other
simple or complex cells and not from LGN cells).
Another example of a sub-threshold stimulus is stimulation outside of the classic
receptive field. The term classic receptive field is used because the receptive field was
traditional referred to as the region in the visual field which when stimulated produced an
excitatory response. Sillito and Jones (1996) used both discrete stimuli and an annulus
outside of the classic receptive field while stimulating the classic receptive field and find
facilitation many times with cross-oriented stimulation in the periphery, suggesting a
possible "discontinuity detector" and at least demonstrating further complexities of
9
neurons when not considered in the context of the network. Vinje and Gallant (2000)
have also used stimulation outside of the class receptive field to verify their natural
stimulus results that suggest the cortex employs a sparsely distributed representation.
Single-unit responses can also be analyzed across time to provide some clues into
the population dynamics. Volgushev et al. (1995) studied the postsysnaptic potential
(PSP) responses and found that excitatory orientation tuning becomes tighter over a 20-
60 ms period. Their results suggested that a 15-25 ms delayed (likely feedback)
inhibition played a role in the narrower tuning. Ringach et al. (1997) found similar
results using a reverse correlation method. However, they found that the delayed
suppression had broader tuning than the excitation and that the overall sharpened tuning
occurred within 6-10 ms.
Rolls et al. (1997a) analyzed a population of 14 neurons individually recorded in
the inferior temporal cortex (IT) for responses to 20 visual stimuli. Because the
recordings were not simultaneous, the analysis ignores any temporal dependencies
between neurons. The cells they recorded are involved in face recognition and they used
information-theoretic approaches to determine the redundancy or independence of the
neurons. Even though they could not document any interactions between cells, they still
found the neurons to be relatively independent and that the representation of faces was
distributed in IT.
Multi-unit research
As early as 1981, a population of 19 neurons was recorded simultaneously in the
monkey visual cortex using a 30-electrode microelectrode array (Kruger and Bach,
10
1981). In the last two decades, there have been advances in the areas of microelectrode
arrays and tetrode arrays, but it is still difficult to obtain high resolution simultaneous
recordings of greater than 100 neurons (Nadasdy, 2000). With the improvements and
availability of this technology more research has moved into the area of population
analysis and has stimulated many recent reviews (Pouget et al., 2000; Milton and
Mackey, 2000; Doetsch, 2000; Nadasdy, 2000). Studies are beginning to show the ability
to understand the neural code from simultaneous population recordings (greater than
pairs) in the aplysia abdominal ganglion (Wu et al., 1994), the rat motor cortex (Laubach
et al., 2000), the primate motor cortex (Maynard et al., 1999; Wessberg et al., 2000), the
moth olfactory lobe (Christensen et al., 2000), the somatosensory cortex (Doetsch, 2000;
Nicolelis et al., 1997), the rat hippocampus (Nadasdy, 2000), the retina, (Warland et al.,
1997), the auditory cortex (Eggermont, 1998), the LGN (Mehta et al., 2000), and the
visual cortex (Gray et al., 1995; Nordhausen et al., 1996; Reich et al., 2001).
Rolls et al. (1997) point out that even a sparsely distributed representation would
have drastic advantages in efficiency of encoding. If encoding were done on a single
neuron level, the number of representations would be equal to the number of neurons. If
the encoding were fully distributed, the number of possible representations would be
equal to 2 raised to the number of neurons (2#neurons). Results do appear to support the
idea that responses are in some way distributed across cortical regions.
Even without the advances of multi-neuronal (multi-unit) recording technology,
studies have been done on small populations of neurons (usually on pairs of neurons) to
reveal properties of the cortex as a network. This is possible with the use of a single
electrode and spike sorting algorithms (Abeles and Goldstein, 1977; Snider and Bonds,
11
1998) or two electrodes recording simultaneously. The recordings of small populations
within a small region are then analyzed for correlation and functional connectivity.
Cross-correlation analysis has been used in the visual cortex to study the
connections and organization within and across cortical layers (Toyoma et al., 1981a,b;
Michalski et al., 1983; Alonso and Martinez, 1998). Toyoma et al. (1981a,b) used the
neurotransmitter glutamate to enhance their responses and found that half the pairs of
cells they recorded shared common input and only 10% of the pairs showed any direct
excitatory or inhibitory interaction. They found common excitatory input connections
into layer III to V (likely from LGN), intracortical direct excitatory connections from
layer III-IV to layer II-III, and intracortical inhibitory direct connections from the deeper
part of layer IV up to the middle layers. Inhibition was found to be between simple cells
or from simple cells to complex cells, and only excitation was found between complex
cells. They also did not find many direct connections across orientation columns.
Michalski et al. (1983) found similar results with rare connections across columns and
found twice as many direct excitatory over direct inhibitory connections within columns.
Alonso and Martinez (1998) were able to find more direct excitatory connections
between layer IV simple cells and layer II/III complex cells, but also reported a
continuum of shared input to direct connections from layer IV to layer II/III
demonstrating that the LGN does not only project into layer IV and providing further
evidence against the feed-forward hierarchical model.
Cross-correlation has also been used to verify long-range connections (>1mm) in
the cortex (Ts’o et al., 1986). Ts’o et al. found excitatory interactions across several
millimeters using two electrodes. The correlation was most apparent when the two cells
12
had similar orientation preferences and facilitation was found when the cells had similar
eye preferences. Gray et al. (1989) have also examined long-range interactions between
cells and discovered that cells that had oscillatory responses (40-60 Hz) that were
precisely synchronized. The synchronization was strongest when stimuli had similar
orientations and in the same direction, and even stronger with a single object to stimulate
both cells. Singer and Gray (1995) have proposed that the oscillations are a mechanism
for long-range synchronization and that it might have a role in either synchronizing cell
assemblies or binding features of an object (because it is strongest with coherent and
connected stimuli).
Information-theoretic analysis of small populations of neurons has also provided
evidence on the redundancy, independence, or cooperation between neurons. The results
have been used to provide support for or against brain function theories. Warland et al.
(1997) analyzed populations of retinal ganglion cells and found the information to be
redundant unless cell types differed and even then the maximum advantage of
information as a population was reached at 4 cells. Nirenburg et al. (2001) also studied
retinal ganglion cells in pairs and found that ignoring the correlation between the cells
still provided over 90% of the possible information suggesting that cells for the most part
act independently. Dan et al. (1998) studied pairs of cells in the LGN and found that the
precise synchronizations provided on average an additional 20% more information.
Gawne et al. (1996a) showed that on average 20% of the information in nearby visual
cortical cells was redundant, and Reich et al. (2001c) found the information to be
independent in the visual cortex unless the responses were summed (where useful
information may be discarded).
13
Functional imaging
Alternatives to electrophysiological recordings, such as functional magnetic
resonance imaging (fMRI), positron emission tomography (PET), and optical imaging,
can also be used to reveal population activity, although it is not able to reveal anything
about the underlying code. Functional imaging is able to provide localization of brain
activity by measuring changes in the hemodynamic response, but is unable to provide
accurate temporal information and information about the individual cellular responses.
As fMRI and functional PET studies continue to grow and a better understanding
of how the signal relates to cellular activity is achieved, the results can continue to aid in
the understanding of neural processing (Raichle, 1998). Very recently fMRI responses
have been compared directly to neural spiking responses (Logothetis et al., 2001), where
the results showed that the hemodynamic response may underestimate neural output
because of the lower signal to noise ratio found in fMRI. However, it may also
overestimate activity because it was found that responses are linked to incoming input
and local responses and not the output activity (i.e., high synaptic activity does not
necessarily result in high output activity). One of the most appealing aspects of
functional imaging is that it provides a non-invasive measurement of neural activity that
can be used to compare human responses with animal neurophysiological data.
Optical imaging has provided better spatial resolution than fMRI or functional
PET, but does have limitations because it records only surface hemodynamics. It has
been successful in displaying the functional architecture of the upper layers of the visual
cortex by showing the orientation columns clearly and discovering their ‘pinwheel’
14
organization (Grinvald, 1992). Optical imaging has been drastically improved over the
last decade with the temporal precision of recording to go along with the spatial precision
by using voltage-sensitive dyes (Fitzpatrick, 2000).
Correlation and Connectivity
Point process and cross-correlation
In (1967a), Perkel et al. introduced the study of neuronal spike trains in terms of
stochastic point processes. When looking at the information-bearing aspect of neuronal
spike trains, the importance is in the times at which discharges occur and not in the
precise voltage measurements or the variations in the action potential waveforms. A
stochastic point process consists of a series of point events that are considered
instantaneous and indistinguishable. By analyzing spike trains as stochastic point
processes, it allows the investigator to implement many computational techniques that
will allow them to extract information about the function and the mechanisms of the
nervous system. Careful study of the temporal relationships in an observed cell can
reveal how the cell produces spikes and how a presynaptic input is transformed into a
postsynaptic output. More importantly, looking at multiple spike trains simultaneously
recorded provides the information necessary to understand the interconnections and
functional interactions between cells. The statistical analysis of pairs of neuronal spike
trains was the genesis of the study of brain function in terms of groups of neurons.
Extending the approach of expressing neuronal spike trains as stochastic point
processes, Perkel et al. (1967b) introduced a method of statistical analysis for two
15
simultaneously recorded spike trains. Measuring the backward and forward recurrence
times of spikes from one neuron relative to each spike in the other neuron creates a cross-
correlation function. The cross-interval histogram takes spikes from one train and is a
histogram of the times to the nearest spikes in the other train. The cross-correlation
histogram is a histogram of all the spikes in one train to each spike in the other train. The
cross-interval histogram is used to corroborate independence indicated by the cross-
correlation histogram or to explore suspected short-latency interactions.
The cross-correlation histogram is used to detect possible dependencies between a
pair of neurons. This dependence can result from either (or both) the functional
interaction between the two neurons or from a common input. The interaction can be a
result of direct synaptic connection or mediated through interneurons. One difficulty
discovered with the cross-correlation histogram is the ability to determine independence
when looking at pacemaker cells. The cyclic action of the cells can lead to false
designation of dependence between cells when in fact they are independent. There can
also be false attributions of independence when the dependence is too weak to be noticed
above noise levels. Another problem of applying cross-correlation analysis to
neurophysiological experiments is that cross-correlation histograms will detect changes
in firing rate as dependencies. This can be difficult to avoid because of response changes
that occur naturally during experiments. Moderate degrees of nonstationarity, however,
will not mask out effects when there is neuronal interaction.
Computer simulations were used to produce cross-correlation histograms to be
used as templates or rules for classifying experimental data (Perkel et al., 1967b). The
simulations showed that there are difficulties in discriminating common input
16
dependencies and indirect connection dependencies. Several different arrangements of
functional interaction can lead to the same cross-correlation. It is important to remember
when using cross-correlation analysis that the results provide insight as to possible
connections and interactions and do not represent any information on the actual anatomy
or physiology.
The cross-correlation histogram can be used to distinguish neuronal pairs between
three different functional relationships: (1) no interaction, (2) interaction (either direct or
through interneurons), and (3) stimulus-modulated interaction (the interaction is modified
by the stimulus). The functional relationships are determined from the cross-correlation
histogram and the prediction of the cross-correlation. A prediction of the cross-
correlation can be determined with the mean firing rates of both cells under stimulated
and unstimulated conditions, the cross correlation function with the stimulus off, and the
PSTHs for both cells. The predicted cross-correlation is used to determine whether the
interaction between neurons is stimulus-modulated. The rules for determining functional
relationship are:
• If the cross-correlation histogram is flat, there is no interaction.
• If the cross-correlation histogram does agree with the predicted cross-
correlation, the interaction is not stimulus-modulated.
• If the cross-correlation histogram does not agree with the predicted cross-
correlation, the interaction is modulated by the stimulus.
Gerstein and Perkel added a new dimension to cross-correlation analysis in 1972
by introducing the joint PST scatter diagram. The joint PST scatter diagram is essentially
another method of displaying the correlation between spike trains, but provides greater
17
understanding into the interactions between the neurons. The scatter diagram is created
by plotting the spike train of one neuron versus the spike train of the other neuron. A
point is plotted wherever an occurrence of a spike from one neuron crosses the
occurrence of a spike from the other neuron.
As cross-correlation analysis has been incorporated into neurophysiology studies,
there have been observations made to better describe the properties of the techniques and
changes made to improve the techniques. One property of cross-correlation analysis that
was discovered was an asymmetry in the sensitivity of cross-correlation analysis for
excitatory versus inhibitory interactions. Aertsen and Gerstein (1985) discovered through
an evaluation of neural connectivity and the associated cross-correlation analysis that it is
much more difficult for inhibitory effects to appear in the cross-correlation histogram
than excitatory effects. Unless the inhibitory effects are significant, they will go
unnoticed during the cross-correlation analysis. Because of this asymmetry, there may be
a false indication of more excitatory connections than inhibitory connections occurring in
different studies using cross-correlation analysis.
Enhancements were made by Palm et al. (1988) and Aertsen et al. (1989) to the
cross-correlation analysis techniques to provide a quantitative approach to classifying
neuronal interactions. Formulae for probability distributions of measures were created so
that data could be compared under significance levels. This allows inferences to be
evaluated using a significance test referred to as “surprise”. A quantification procedure
was created for the study of stimulus-locked, time-dependent correlation of firing
between two neurons so that direct and indirect stimulus effects could be described
quantitatively. The changes make it easier to determine “effective connectivity”.
18
Quantitative measures can be used to separate characteristics of diagonal features to
determine whether interaction results from direct interaction or shared input. The
additions to the cross-correlation analysis still only determine “effective connectivity,”
which is not necessarily the actual anatomical description of the connections. It should
be thought of as an equivalent neuronal circuit that can represent any number of actual
physiological circuits that would result in the same output.
Partialization
Because connections between any 2 neurons in the visual cortex are usually weak,
it is usually difficult to detect the coactivation of groups of neurons because of the
complicated circuitry in between the 2 neurons. An alternative to the shift predictor
method (Perkel et al., 1967) is the method of partialization. Partialization is used in
conjunction with cross-correlation. It separates out the independent and common input
contributions between two neurons in the Fourier domain to make an estimate of the
functional connectivity. The method is more effective as the population of assemblies
grows and has been successfully used to analyze changes in assembly strength with
respect to changes in anesthesia (van der Togt et al., 1998). Because the effectiveness of
partialization depends on larger populations of assemblies, the method is not
advantageous over shift predictor methods when analyzing pairs of neurons.
Gravitational clustering
Advances in techniques allowing larger populations of neurons to be
simultaneously recorded have led to a new approach in the analysis of these populations.
19
In 1985, Gerstein and his colleagues describe a method that analyzes groups of neurons
as a whole rather than in pairs. Simultaneous recordings of 10 or more neurons would be
very difficult using cross-correlation analysis because the relationships have to be
examined pairwise. Their new approach maps the activity of neurons into motions of
particles in a multidimensional Euclidean space. Each neuron is thought of as a point
particle in space and each spike results in an increment in a “charge” for that particle.
The particles are then essentially plotted in space and by observing their movements, the
interrelationships between the neurons can be determined. The neurons that are
interconnected tend to move towards each other so after a simulation, groups of neurons
that are connected or receive the same input will cluster together. Relationships of the
neurons can also be seen by plotting the distance between pairs of neurons versus time.
The stronger the connection between neurons, the faster the distance will approach zero.
If the neurons are independent from each other, the distance will remain constant.
There are many variations of this technique that can factor into the effectiveness
of this approach to describing neuronal group characteristics. Modifying the definition of
the charges and force rules are necessary in order to observe inhibitory relationships. The
approach does appear to display successfully the characteristics of a simulated group of
10 neurons. Only 50 spikes from each neuron were necessary to produce the clusters and
display the relationships within the network. Compared to the cross-correlation
techniques that require hundreds to thousands of spikes to demonstrate similar results,
this approach appears to be much more sensitive (Gerstein et al., 1985; Gerstein and
Perkel, 1985; Strangman, 1997). The method has been successfully applied in several
neurophysiological studies (Lindsey et al., 1992a,b; Lindsey et al., 1994; Maldonado and
20
Gerstein, 1996; Lindsey et al., 1997; Morris et al., 2001) and has recently been improved
to detect weak synchrony among neural populations at various spike intervals (Baker and
Gerstein, 2000). Because the results of gravitational clustering are not as clear as cross
correlation, the method is typically only used for studying larger populations of neurons.
Information theory: dependency and complexity
Another method recently developed under information-theoretic principles can be
used to compare the probability distributions of neurons and the temporal dynamics of
their dependence (Johnson et al., 2001). These analysis techniques can be carried out on
larger populations of neurons or on a pair-by-pair basis to determine the neuronal
dependency of a network and how the dependency changes across time and across
stimulus modulations. The method forms probability mass functions (types) for
spatiotemporal patterns and then calculates the probability functions if the neurons were
assumed to be independent (forced-independent type). An accumulated distance between
the two responses is then calculated across time. If the neurons are independent, their
probabilities of firing at given time in a spatiotemporal pattern should be equal to the
product of their individual probabilities of firing. Any variance from this equality means
there is some inhibitory (less than the forced-independent) or excitatory (greater than the
forced-independent) dependency between the individual neurons.
Tononi et al. (1994) also developed an information-theoretic method to measure
connectivity. Their method measures the deviance from independence from entropy and
mutual information calculations. The complexity is then defined as the relative deviance
of a local region with respect to the deviation from the average deviance of the overall
21
network. The complexity of a network is lowest when the units are fully integrated or
when the units are fully independent, and the complexity is highest in between the two
extremes (smaller strongly connected groups sparsely connected). The method can be
used in functional imaging studies where the voxel or pixel represents the individual unit
and the results can characterize complexity changes when the strength of activity does
not vary, which is the case in pathologies such as schizophrenia (Tononi et al., 1997).
The method can be applied to any multi-dimensional data set making it ideal for
neurophysiological studies as well as neuroimaging studies. Beyond identifying the
strength of complexity, the method has been further expanded to characterize the
complexity (Sporns et al., 2000). In other words, the functional clusters can be identified
so that connectivity patterns can be identified and related back to behavioral changes.
Causality
A problem with correlation and coherence measurements is that many times they
do not resolve the directionality of information flow, which becomes very relevant in the
brain with both feed-forward and feedback interactions. Bernosconi and Konig (1999)
developed a technique based on the methods of structural analysis in the field of
econometrics. The basic idea is based on autoregressive modeling and quantitative
measures of linear relationships between multiple time series. The concept is known as
Wiener-Granger causality and the strength and direction of relationships are derived from
the predictability of the models. The simplest description of the principle is that “the past
and present may cause the future, but the future cannot cause the past”.
22
Multivariate time series are analyzed in the time and frequency domain for
causality, but analysis is restricted to stationary responses (autoregressive modeling) and
may be limited by the available amount of data (dimensionality restrictions). In addition
to stationarity requirements, the method would be very ineffective in detecting
instantaneous interactions. Some of the issues of stationarity can be dealt with under the
assumption of piecewise stationarity and modeling each section separately. Overall,
because the method can be applied in both the time and frequency domain, it can be
useful to detect general cortical interactions that will help with other methods that can
analyze the instantaneous interactions.
Pastor et al. (2000) have also used a method for determining causal connectivity
in cerebral activity using regional cerebral blood flow (RCBF) data from PET imaging
studies. Their method is more suited strictly for functional imaging studies because the
approach is both coarse (where regions such as the visual cortex are considered as
elements) and minimalist (minimizing the number of information processors).
In general, causality is better suited for long-range and regional interactions
within the brain rather than local interactions between neurons. The advantage of
causality over correlation is providing directionality information, but cross-correlation
and the shift predictor are able to provide this information when the analysis is performed
on neurons within the range of direct synaptic interactions.
Nonlinear methods
In almost all the methods we describe for determining correlation and
connectivity of neural activity (the information-theoretic approaches being the
23
exceptions), the primary computation is to determine linear relationships between
elements (in time or frequency between neurons or between regions). It should be
expected that these methods would not detect all the possible interactions because we are
dealing with elements that have several nonlinear properties. Neurons and neural
networks cannot be thought of as passive linear elements because they have properties
such as thresholding, intrinsic bursting, and chaos.
Friston and Buchel (2000) developed a nonlinear model to analyze the feedback
influences of attention in the posterier parietal cortex (PPC) on area MT responses. The
model uses the Volterra series to model the nonlinear transformation and the effective
connectivity is determined by solving for the unknown kernels in the convolution of the
time series. The kernels are estimated by a time series expansion of temporal basis
functions. Friston and Buchel (2000) apply the analysis on fMRI data, but it can also be
used on data with higher temporal acuity (i.e., electrophysiological recordings) by simply
expanding the number of temporal basis functions. As is the case with the linear
methods, the nonlinear effective connectivity is only an estimation of the possible
interactions.
Neural Code Theory
To compound the problem of analyzing larger populations of neurons there is still
much controversy over what aspects of the individual neuron’s output are relevant to the
neural code, or representation of information. Whether the theory is that the neurons are
independent feature detectors, elements that form spatiotemporal patterns, or a feature
vector, there must be an element to represent a magnitude. If it is assumed that the neural
24
responses are considered point processes, then this element must be some property of
time. Examples of these properties are impulse rates or counts, ‘Morse code’-type
patterns, precise spike arrival times, and interspike intervals (ISI).
Average spike rate code
Since Adrian and Zotterman (1926) discovered a relationship between the firing
rate of neurons and the magnitude of sensory stimulation (touch and pressure), the rate
code has been the primary property of neurons measured by neurophysiologists. In the
simplest form, the firing rate is determined by listening to neural responses. More
precisely, the firing rate can be measured across time by averaging responses to repeated
sensory stimulations and forming the PSTH. From the PSTHs, tuning curves can be
measured across feature variations to characterize a neuron for different properties of the
sensory stimulation. From these tuning functions, optimal stimulus parameters and their
bandwidths of the response can be determined. From these properties, the functional
organization of the brain has been determined and neurons have been classified (Hubel
and Weisel, 1962). It is from these tuning characteristics that Barlow (1972) formulated
his ‘cardinal’ cell theory.
Even to this day the average firing rate is the simplest and most straightforward
measurement made in neurophysiological studies. One problem with the average firing
rate is that it is highly variable across stimulus repetitions (Gershon et al., 1998). This
leads to the requirement of forming the averaged PST histogram across repeated stimulus
presentations. The rationale was that the brain could average responses across a
population instantaneously (or over a short integration time constant) in the same manner
25
the scientist measures the rate in a single cell across time. There is still both anatomical
and physiological evidence to support this hypothesis (Shadlen and Newsome, 1998)
although there is now just as much evidence to support seemingly contradictory theories.
Temporal code
Principle component analysis of spike trains by Richmond et al. (1987 and 1990)
have revealed that a significant amount of information can be contained in the temporal
structure of the spiking output leading to many studies to look beyond only the spike
count. The information was significant and correlated to the stimulus variations. This
was later verified by Victor and Purpura (1996) using a metric-space information-
theoretic method and a different visual stimulus. De Ruyter van Steveninck et al. (1997)
have also shown that the temporal structure contained information that was much more
efficient in characterizing dynamic stimuli in fly H1 responses. They also found that
variance and ISI variability depend on the stimulus. The dependence on the stimulus was
also shown by Mechler et al. (1998) to explain any discrepancies between the results of
Richmond et al. and Victor and Purpura. They found that the temporal coding was much
more robust in transient stimuli (i.e., bars and Walsh patterns) than steady-state stimuli
(i.e., sine wave gratings).
A temporal code theory has also gained support by studies that found the spike
train containing distinct patterns occurring much more often than what would be expected
by chance in the individual neuron (Strehler and Lestienne, 1986) or across a population
of neurons (Dayhoff and Gerstein, 1983b). The precision of some of these patterns can
be less than a millisecond (Strehler and Lestienne, 1986).
26
Another reason it is believed that the spike trains may contain information beyond
the spike count is the irregularities found in the ISI histograms showing a non-Poisson
distribution (Cattaneo et al., 1981a,b; Gray and McCormick, 1996; DeBusk et al., 1997;
Victor, 2000). If the neural code was just a noisy rate code, it would be expected to have
a random Poisson distribution in the ISI histogram. Victor (2000), over several studies,
has shown how different stimulus features (i.e., contrast, orientation, spatial frequency)
are most informative at different temporal resolutions of ISI statistics. His theory is that
the ISI itself may be where the information lies and that stimulus features are multiplexed
in the spike train.
Recent reviews in the visual cortex (Bair, 1999) and sensory cortices (Grothe and
Klump, 2000) demonstrate the vast amount of data that has been shown to support all the
aspects of temporal coding that are related to sensory input. Temporal coding (arrival
times or interval times) has been broken down into several areas that have been shown to
modulate with changes in sensory stimulation:
Bursting
One proposal of the ISI irregularity is that it occurs because of a bursting behavior
that occurs in neurons that is modulated by stimulus features (Cattaneo et al., 1981a,b;
DeBusk et al., 1997). Bursts were first discovered in pyramidal cells in the hippocampus
and were shown to be an intrinsic property of the cells that might be a form of
amplification (Traub and Miles, 1991). Bursts are also a likely explanation for the 2-5-
spike patterns discovered by Strehler and Lestienne (1986). Bursts themselves
(considered as a single event) have been shown to carry information more precisely than
27
the spike count for certain stimulus features by looking at tuning functions (Cattaneo et
al., 1981; Eggermont and Smith, 1996; DeBusk et al., 1997; Gabbiani and Metzner,
1999) and using information measures (Reinagel et al., 1999; Brenner et al., 2000).
Reich et al. (2000) have also shown improved efficiency in shorter time intervals and that
these spikes contribute disproportionately to the overall receptive field properties of
neurons. Bursts are also more reliable from stimulus trial to trial for receptive field
properties (Victor et al., 1998) and response latencies (Guido and Sherman, 1998).
Bursting has also been shown to overcome synaptic unreliability through facilitation
(Lisman, 1997; Usrey et al., 1998) or temporal integration (Snider et al., 1998) and is
therefore more efficient in passing on information to the next neuron.
There has also been studies that model networks and look at bursts as possibly
distinct patterns of doublets and triplets and how they may be used to create stability and
synchrony in networks (Karbowski and Kopell, 2000). Bowman et al. (1995) also
proposed bursts as an amplifier to signal interneurons for synchronization.
Latency
One aspect in the absolute timing of spikes that was discovered is that the latency
of a response can vary with respect to stimulus changes (Gawne et al., 1996). Gawne and
his colleagues found that the latency varied from orientation and contrast changes in a
visual stimulus. Although both features caused latency differences, they found that only
the contrast caused a correlated modulation of the latency (higher contrast leads to shorter
latencies). They proposed that orientation would be encoded in the spike count while the
contrast was encoded in the latency and that image features of the same contrast would
28
arrive at the same time and this was one way to bind these features. Latency modulation
was examined further by Reich and his colleagues (2001b), who showed that the latency
continues to modulate at high contrasts when the average spike rate no longer modulates
and that the latency allows subtle contrast differences to be detected in this range
therefore increasing the overall dynamic range of contrast encoding. Similar
relationships have been found in the auditory cortex with respect to amplitude changes of
sound independent of the location (Heil, 1997) and in area MT with respect to stimulus
speed (Lisberger and Movshon, 1999).
It is also important to consider whether the cortex would even be able to
determine latency. In the laboratory, latency is measured from stimulus onset, but
whether the brain actually knows the stimulus onset has not been answered. Victor has
hypothesized that saccades could lead to a population of cells in the visual cortex firing to
signal stimulus onset and reset synchronization (Victor, 2000). Indeed this activity does
occur and a recent saccade study has demonstrated that there is significant activity in the
visual cortex that is coupled to saccade offset and therefore stimulus onset (Park and Lee,
2000). Previously, visual cortical activity was thought to be coupled to saccade onset.
Latency becomes much more relevant when considering a coding scheme such as
bursting (Segundo et al., 1963), which is probably the reasoning behind the increased
reliability with bursts found by Guido and Sherman (1998) in measuring LGN latencies.
Another consideration is that latency differences might reveal delay mechanisms
that the cortex uses. By incorporating delays into the cortical network, it would add
another dimension to the neural code making it more robust. An example is time-delay
neural networks (TDNN) that allow for invariant recognition (Hopfield, 1995), which is a
29
distinct property of the visual system. TDNNs are able to perform invariant recognition
by performing operations such as translation, rotation, and scaling on input signals
(Schalkoff, 1997). An example of an application of TDNNs is their use in a system that
is able to recognize objects regardless of the direction they are moving or the velocity at
which they are moving (Wohler and Anlauf, 1999a,b). It is certainly plausible to
consider distinct variable delays occurring in the cortex when considering the axons as
delay lines (Segev and Schneidman, 1999).
Spatiotemporal patterns
When considering the results reviewed in the bursting and latency sections
together with the results initially described by Dayhoff and Gerstein (1983b), it is much
more plausible to start to consider a neural coding scheme of reliable spatiotemporal
patterns. Bursting provides a mechanism of propagating reliable timing information, and
the latency and spatiotemporal pattern data support that precise timing information is
correlated with stimulus modulations. Abeles and Gerstein (1988) improved the Dayhoff
and Gerstein algorithm (1983a) to detect whether patterns occur more often than what
would be expected with a random distribution of spikes. They use the analogy of long
punched-paper to describe their pattern recognition procedure. An individual row
represents the activity of a particular neuron and each hole represents a spike. If a copy
of this paper was placed over the original and the superimposed sheets were observed
against a light source at different displacements, a repeated pattern would appear much
brighter than the average. The method might be considered a bit crude in that it ignores
synaptic mechanisms that essentially filter spikes depending on ISI properties.
30
The method has recently been modified to account for a small amount of jitter that
may occur in the patterns and to perform more rigorous statistical analysis (Tetko and
Villa, 2001a,b). Such complicated patterns might well occur when considering that the
signal might be bursts and not individual spikes leading to a nearly perfect synapse. In
this case it would be very critical to account for the jitter of a burst arrival time. One of
the reasons bursts are so efficient is that the probability of transmission increases
significantly for the second spike in a burst, if the first spike fails to cause a postsynaptic
spike (by synaptic facilitation). If the first spike does cause a postsynaptic spike, than the
probability of the second spike in the burst to cause a postsynaptic spike is much lower
than normal (synaptic depression). The idea is that the probability is very high that at
least one spike in the burst will lead to a postsynaptic spike, but because it can vary for
which spike it is, the precise time can vary leading to a jitter (Lisman, 1997).
Even with the improvements to spatiotemporal pattern detection, Tetko and
Villa's method still focus on patterns of individual spikes. One of the alternative methods
is to consider the spatiotemporal patterns of the firing rate of the neurons (Doetsch,
2000), but this falls on the other end of the spectrum of spatiotemporal theories (i.e., too
coarse for the temporal representation). An ideal method would be to detect either
spatiotemporal patterns of bursts or even better yet, to determine the patterns of synaptic
modulations (considering all properties such as bursts, burst length, and threshold level).
Oscillations
Synchronization across the visual cortex has been found for oscillatory responses
and has been proposed as a mechanism to link feature characteristics (Gray et al., 1989;
31
Singer and Gray, 1995). Although the idea that synchronization binds features has
received criticism (Shadlen and Movshon, 1999; Ghose and Maunsell, 1999; Farid and
Adelson, 2001), it still reveals the presence of organized activity distributed across broad
regions of the cortex. The purpose (if any) of this organization has not been sufficiently
explained. One argument against the linking hypothesis is that more experiments are
necessary to reveal whether the synchronization is really related to binding features and
that it likely does not even occur in the visual cortex (Shadlen and Movshon, 1999).
Another argument is that temporal binding of features is not necessary with the number
of cortical cells available (Ghose and Maunsell, 1999). Farid and Adelson (2001)
provide evidence that the synchronization is purely a result of the temporal structure of
the stimuli used and the temporal filtering characteristics of cortical cells. However, it is
clear that there are cells found in layer II/III of the cortex (which project to higher visual
areas) that intrinsically oscillate in the 20-70 Hz range with bursts of 2-5 spikes (Gray
and McCormick, 1996). Why and how these cells precisely synchronize is the question
and has received much theoretical analysis.
Ernst et al. (1995) studied a model of oscillators to examine how long distance
synchronizations would be possible despite temporal constraints of neural transmission.
They found that inhibitory coupling was best at inducing synchronization and that
excitatory coupling lead to decaying synchronized clusters. Ernst et al. (1998) later
discovered that a delay was necessary for stable excitatory coupling. They also
discovered that networks of more than 2 neurons lead to spontaneous synchronizations
with excitatory coupling and spatiotemporal patterns or clusters with inhibitory coupling.
Karbowski and Kopell (2000) also demonstrate the possibility of long-range
32
synchronization of oscillators, but their model uses both excitatory and inhibitory
oscillators together and requires a small amount of disorder for stability and interneurons
with ‘doublets’. Van Vreeswijk (1996) examined a model of a population of oscillators
and discovered that fast excitatory coupling was both unstable in the synchronous and
asynchronous state, but with inhibitory coupling, the population breaks up in to
synchronized clusters with the number of clusters increasing with faster coupling. The
theoretical studies on a whole demonstrate that oscillations might be a mechanism for (1)
long-range synchronization and (2) forming synchronized assemblies.
Chaos theory
One important aspect of the cortex is that regardless of the sensory input,
individual cells have a maintained discharge. This maintained discharge is especially
apparent in complex cells of the visual cortex (Hubel and Weisel, 1962). At first glance,
this may seem insignificant because when cells are stimulated, the response is much
stronger, leaving the maintained discharge nearly unnoticed. However, there are
thousands of connections between neurons and only hundreds of spikes within a
relatively large time constant are necessary to fire a postsynaptic neuron. The first
question this should raise is how the cortex would maintain stability under these
circumstances. It would appear that with all of these cells firing and connected to each
other, that the system would reach some sort of physiological limitation or saturation.
The solution comes from the fact that a significant portion of these connections are
inhibitory and help to balance out the driving force of the excitatory connections.
33
The large number of both excitatory and inhibitory connections has been
hypothesized as the reason behind much of the variability found in the individual
neuron’s response (Shadlen and Newsome, 1994). What is the purpose of having
maintained discharge in a tug-of-war between excitatory and inhibitory synapses, which
is referred to as chaos? The theory is that the excitation essentially raises the
postsynaptic potential, while the inhibition prevents the potential from reaching the
threshold. In other words, the chaos moves the neuron closer to activation, but prevents it
from saturating (Shadlen and Newsome, 1998; Bell et al., 1995).
Obviously, this allows the neuron to fire more easily, but the real significance
behind the chaos is its ability to ‘speed up’ the synaptic transmission and lead to a more
precise temporal code. The argument that noise or chaos improves precision may seem
counterintuitive, but it is effective because of the low reliability of synapses and their
filter-like properties. Koch et al. (1996) reveal how the increase in random background
noise of a network of cortical neurons leads to a reduction in the time constant of the
postsynaptic neuron. Initial measurements of neuronal time constants were measured in
vitro and estimates were as high as 50 ms in passive neurons. This would make one
skeptical in regard to any sort of precise temporal code, but when as little as 10 spikes per
second (sps) is found in the background activity of a cortical network, the time constant is
less than 2 ms in the active neuron (Koch et al., 1996).
Indeed, Karbowski and Kopell (2000) required a small amount of disorder for
their model of synchronized oscillators to stabilize. Hansel (1996) studied a model of
Hodgkin-Huxley neurons with a massive amount of excitatory and inhibitory connections
and found that the chaos lead to a synchronized state that produces cross-correlations
34
similar to those found in cortical studies. Van Vreeswijk and Sompolinsky (1996) also
tested a similar network and found that although there was chaotic dynamics, the system
as a whole exhibited a linear response, and more importantly, the network responded
faster than the time constant of a neuron.
Information Theory
Information theory has been another analytical tool that in the last decade has
been used to explain the nature of neural code throughout the central nervous system.
Based on the work of Shannon (1948), information theory techniques have been
developed to analyze several scientific and mathematical problems (Cover and Thomas,
1991). Because neural signals are simplified in the form of a spike train, they can further
be simplified into a binary sequence by binning the data at a desired temporal resolution.
This format makes it ideal for information-theoretic analysis.
The fundamental measure in information theory is entropy, which is a logarithmic
measurement of the information carrying capacity of a signal because of the exponential
nature of most communications signals. Essentially, it measures the variability in a
probability distribution of the signal. With the proper decoding mechanisms, this
variability translates into more capacity. The conditional entropy is the amount of
variability that depends on another random variable, and the reduction of the entropy of
the original variable with the conditional entropy of this new variable is the mutual
information. Entropy and mutual information are the starting points for essentially all the
information-theoretic methods formed for the analysis of neural codes. The original
variable is the spiking output of one or more neurons and the second variable that is used
35
to determine the mutual information is usually a variable of the sensory input or stimulus
(i.e., contrast or spatial structure).
Information-theoretic measures, binning procedures, and neural analysis
techniques in general are estimated measures and must be considered in terms of
significance and confidence. In particular, entropy measures are always positive and any
error can only result in a positive bias. The amount of error will depend on the sampling
of neural signals and the accuracy of the estimated probability distributions. In general,
these estimations reach an asymptotic behavior that can be used to determine bias
estimations and confidence intervals. Fagan (1978) developed the jackknife method to
assess the statistical confidence in information measures, and Efron and Tibshirani (1993)
have developed a similar and more popular method (bootstrap) to deal with potential
under-sampling problems that can be used to make bias estimations and provide
statistical information about the estimation. Panzeri and Treves (1996) have also
developed a method to look at the asymptotic behavior of bias and make estimations of
bias for neural information measures in particular.
The initial applications of information theory for the analysis of neural coding
entailed determining either the information rate or information per spike, as well as
determining channel capacities and the efficiency of neural channels. Reike et al. (1997)
developed a method that measured the information of a neuron and then used the
information to reconstruct the visual stimulus. They have applied this method under
several very interesting experiments to study the fly H1 neurons that are able to detect
direction. The results they find range from the surprisingly high capacity of neurons and
the relatively impressive efficiency of the information about the stimulus carried by a
36
single neuron. Their results have also provided clues into the relevance in the neural
code with respect to the firing rate and arrival times of spikes and the temporal windows
that are most efficient in decoding this information.
Victor and Purpura (1996) developed an alternative method that attempted to
overcome many of the sampling difficulties associated with traditional information-
theoretic methods. The high firing rates and increased stability of recording fly H1
neurons versus mammalian cortical neurons makes it very difficult to compare reliably
any properties of the neural codes and information capacities. Victor and Purpura’s
method is based in the metric space whereas the traditional information measures are
based in the vector space and are subject to many dimensionality restrictions that require
sufficient sampling to make confident estimations. The basic idea is that one spike train
is compared to another spike train and a property of the spike trains is measured to see
‘how much’ one must be modified to be indistinguishable from the other according to this
particular property. Obviously, one sacrifice of this method is that an assumption must
me made about the nature of the code to determine what is the significant property. They
choose the properties of spike count, spike time (absolute arrival time), and spike
interval. In addition, the comparison is made at several temporal resolutions to determine
which resolution is the optimal at displaying the greatest amount of difference in the
particular metric (count, time, interval).
The metric-space analysis has been used in primate cortical neurons to
demonstrate that there is information contained in the temporal structure of the spike
trains, and even more interestingly, Victor (2000) and his colleagues have found that
variations in different stimulus features (i.e., contrast, texture, spatial frequency) are
37
optimal at different resolutions with the interval metric. They have also expanded the
measure to test the significance of labeling a particular neuron for population analysis to
test the redundancy and independence of neighboring cortical cells (Reich et al., 2001c).
Johnson et al. (2001) have developed a method based on information and
classification theory to examine populations of neurons (type analysis). Type analysis
extends the binary sequence to include a larger alphabet to include spatial information
(i.e., which neuron fired). Conditional probability distributions and a Markov chain are
used to include the discharge history so that neural codes can be studied at particular
resolutions and still consider delays or lags that are inherent in cortical networks. The
method is particularly powerful in that it does not make any assumptions about the nature
of the code and only determines how different population responses are with respect to an
optimal classifier. The difference is represented across time so that responses can be
separated in transient and sustained portions. The temporal dynamics can reveal any fast
modulations that might occur in dependencies or firing patterns that are significant in
discriminating two responses. One drawback of the method is that it requires substantial
amounts of data and the requirements rise exponentially with respect to spatial (number
of neurons) and temporal dimensionality (discharge history). This becomes a
considerable problem with in vivo recordings that have stability and time constraints.
Some methods have been developed to overcome some of these requirements
(Dimitrov and Miller, 2001), but these methods are still limited to many practical
applications. Dimitrov and Miller incorporate type analysis, stimulus reconstruction (or
neural decoding), and a quantization procedure that uses a distortion function to
overcome sampling deficiencies. The method can reveal the type of information and
38
even the neural code, but can be computationally extensive and still run into
dimensionality problems before revealing meaningful results.
Eguia and his colleagues (2000) point out that information theory may sometimes
not reveal all the information that can be extracted. The fundamental rule in information
theory is that information cannot be gained through processing. This however is not true
when information is passed through active units. Because neurons have been shown to
exhibit bursting and chaotic behavior, they cannot be considered passive units and
therefore “information creation” can actually exist in cortical networks. This becomes
especially relevant in studying the information transfer between neurons or across layers.
Information theory has proven to be a powerful tool in neuroscience, but
sometimes the purpose becomes buried in the computations. It is important to understand
the significance when assumptions are made and that regardless of the technique, it does
not ever ‘prove’ what features the brain does or does not consider relevant. It is merely a
mathematical tool that can reveal characteristics of the spike train and networks of spike
trains that may not be in any other way apparent.
The Future
Overall, there appears to be agreement that the cortex requires populations to
encode the complex information contained in a sensory signal. The actual function being
performed, the level of distribution, and the fundamental signal are still open to debate
and for the most part each theory holds up to the presently available data. To resolve
some of these debates will require analysis of cortical function that more closely reveals
actual cortical processing. Whether the number of neurons simultaneously recorded does
39
not cover enough of the actual circuitry to reveal the nature of the code or is excessive
and highly redundant, it is necessary for a better understanding of neural coding. At this
point, we cannot be sure if tuning functions, receptive fields, and cortical maps embody
how information is encoded and transferred or whether they are just inherent properties
necessary for sensory processing network functions. It would be difficult to incorporate
fault tolerance and generalization if each neuron performs a unique calculation, but at the
same time, it would be biologically inefficient to be highly distributed. The technology
that is currently available now allows careful examination using a variety of analysis
techniques of significantly large populations. These populations need to be tested under a
wide range of stimuli and using the proper protocol to begin to resolve some of the neural
coding debates.
The vast amount of support found for seemingly conflicting theories might just
lead to the simplest explanation—all explanations are correct. It took scientists a long
time to except that light could be thought of as both a wave of energy and a particle with
momentum, but it was the only plausible explanation that could be understood
conceptually. It is quite possible that both a coarse rate code and a precise temporal code
represent sensory information that is passed on and transformed across neural networks,
and not only do they both carry information, but also cooperate, where one does not exist
without the other.
40
CHAPTER II
COOPERATION BETWEEN AREA 17 NEURON PAIRS THAT ENHANCES FINE DISCRIMINATION OF ORIENTATION
Introduction
Hebb (1949) introduced the idea that information could be passed between
regions of the brain as spatiotemporal patterns. The difficulties in supporting this theory
are obtaining a sufficient number of simultaneous recordings and determining exactly
how the pattern is represented, but as laboratories improve multi-unit recording and
population analysis techniques, evidence that supports the theory that sensory
information is represented as spatiotemporal patterns of activity within cortical regions
continues to grow (Doetsch, 2000; Milton and Mackey, 2000; Nadasdy, 2000). Theories
about the representation cover the spectrum from the impulse rate (Doetsch, 2000) to the
precise arrival times of individual action potentials (Dayhoff and Gerstein, 1983).
If the spatiotemporal patterns are irrelevant to the encoding of the sensory signal,
individual units can be considered as independent entities, each producing part of the
information representing a stimulus feature or a region of visual space (Barlow, 1972),
which when aggregated are sufficient to yield the percept. Independence between pairs
of neurons has in fact been shown in the retina (Nirenburg et al., 2001), and in the visual
cortex, the correlation between pairs of neurons has been shown to be slightly redundant
(Gawne et al., 1996a) and independent (Victor, 2000; Reich et al., 2001) under different
information-theoretic methods. There nonetheless remain questions about representation,
as it has been demonstrated that both the average firing rate and the temporal structure of
41
individual spike trains carry sensory information (Richmond and Optican, 1987; Victor
and Pupura, 1996; de Ruyter van Steveninck et al., 1997).
There is also evidence against the cardinal cell theory (Barlow, 1972) showing
that the synchronous activity between LGN pairs can enhance information by as much as
40% (Dan et al., 1998). The distributed representation of faces in the inferior temporal
(IT) cortical cells (Rolls et al., 1997a,b) also contradicts the idea of convergence and
specialization that would be expected with independent units.
Barlow (1972) even points out in the end of his article that the brain must use
more than clever cells to perform many of the complicated tasks, acknowledging that
there may be something more complex than the average firing rate that carries sensory
information. Hebb’s (1949) theory on organization is based on the idea of independent
assemblies of cells that reorganize with stimulation changes. The assemblies are formed
through a long learning process and the precise spatial and temporal properties of the
assemblies are the reason responses are able to generalize so easily. There is growing
evidence that the brain might take advantage of specialization (for biological efficiency)
and spatiotemporal patterns through subgroups (for computational efficiency) with a
sparse code (Baddeley et al., 1997; Vinje and Gallant, 2000). The sparseness can vary
depending whether the priority is for memory or discrimination (Rolls et al., 1997b).
In this study, we examine what aspects of the spike train are passed between two
neurons. The method we use (Johnson et al., 2001) makes almost no assumptions about
the nature of the neural code and allows us to compare the difference between this
information transfer under different conditions of stimulus. The assumptions we are
forced to make under this method of analysis are the temporal resolution of the
42
interactions and the relevant amount of discharge history. We examine these
relationships across the entire range that is possible under dimensionality restrictions and
find the optimal parameters match well with characteristics of the interspike interval (ISI)
histogram and temporal properties of monosynaptic connectivity.
Cattaneo et al. (1981a,b) demonstrated that over nearly the entire range of
orientation and spatial frequency tuning, only those groups of spikes (bursts) found in the
smaller peak of the ISI (~3 ms) were actually tuned to the stimulus features. Not only are
the number of bursts modulated by orientation, but also the number of spikes within a
burst (DeBusk et al., 1997). Snider at al. (1998) examined the connectivity between pairs
of neurons with cross correlation (Aertsen et al., 1989) to show that on average spikes
found in bursts were twice as efficient in inducing a spike in the post synaptic neuron,
and as the burst length increased, the connectivity became more efficient.
We extend these results to show that feature discrimination is most efficient when
using a temporal resolution that matches the bursting peak found in the ISI histogram and
when considering enough discharge history to include synaptic delay between the
presynaptic and postsynaptic action potentials. The advantages we find in discrimination
are similar to the improvement found by Cattaneo et al. (1981a,b) by considering only
bursts versus all action potentials. However, we do find even greater improvement in
discrimination when considering connectivity for small differences in orientation (less
than 10 degrees). This range is found around the peak of the tuning curve defined by the
average firing rate tuning, where the firing rate is nearly constant.
When we examine the functional connectivity between neurons, we find that the
connectivity tuning continues to modulate significantly over orientation even though the
43
firing rate does not change. We believe this is a result of improvement in the efficiency
of connectivity caused by increasing burst length (Snider et al., 1998). When looking at
the connectivity across time, we find that a delayed inhibition plays a major role in
determining the strength of connectivity. We also find that the connectivity slowly
decreases or adapts over time and that this adaptation is faster for lower contrasts.
Methods
Preparation
Six adult cats (2.5-4.0 kg) were prepared for electrophysiological recordings in
Area 17 (recordings were also made for additional experiments not described in this
paper). Experimental procedures were performed under the guidelines established by the
American Physiological Society and Vanderbilt University’s Animal Care and Use
Committee. Each cat was initially injected intramuscularly with 0.5 ml of acepromazine
maleate and 0.5 ml atropine sulfate. Anesthesia was induced with 5% halothane in O2
and maintained with intravenous injection of 0.3 mg*kg-1*h-1 of Propofol after
cannulating one of the forelimb veins. A second forelimb vein and the trachea were then
cannulated. Once the cat was mounted in a stereotaxic device, a small craniotomy (2x5
mm) was performed over the area centralis represenation (H-C coordinates P4-L2). The
underlying dura was excised and once the electrode was positioned, the hole was covered
with agar mixed with mammalian Ringer’s solution. Melted paraffin was poured over the
agar for stability.
44
During recording, paralysis was induced with 6 mg and maintained intravenously
with 0.3 mg*kg-1*h-1 pancuronium bromide (Pavulon). The cats were artificially
ventilated with a mixture of N2O:O2:CO2 (75:23.5:1.5) and pCO2 was held at 3.9%.
Anesthesia and health were maintained by monitoring the electrocardiogram and
electroencephalograms and making bolus injections of Propofol when necessary. Rectal
temperature was maintained at 37.5oC with a servo-controlled heat pad. The nictitating
membranes were retracted with 10% phenylephrine hydrochloride and pupils were
dilated with 1% atropine sulfate. Contact lenses with 4 mm artificial pupils were fitted,
and auxillary lenses were added to render the retina conjugate at a viewing distance of 57
cm with direct ophthalmoscopy.
Data acquisition and spike classification
Recordings of multiunit activity were done with a single tungsten-in-glass
microelectrode (Levick, 1972). The signal is amplified by 5,000 and band-limited
between 300 and 3,000 Hz, and sampled at 30 kHz by an AT&T DSP32C digital signal
processing board. The threshold for event acceptance was set at 5 standard deviations
above or below the mean noise level (following Chebyshev’s Theorem; see Snider et al.,
1998). The action potential was stored from 1 ms before the trigger point to 3 ms after
the trigger point (a total of 4 ms or 120 sampled points), along with the event time.
The classification procedure of action potentials is described in detail elsewhere
(Snider and Bonds, 1998; Snider et al., 1998). In brief, each waveform is projected as a
120-dimensional vector. Each waveform is represented as a point in space and the
waveform space is partitioned into many small clusters using the method of binary tree
45
bisection. Although waveforms can change shape throughout recording, the method is
able to combine clusters on the assumption that these changes are gradual. A score is
assigned for pairs of clusters based on the individual cluster densities and the density
between each cluster. If the clusters are essentially smeared together (as would be
expected with the gradual non-stationary waveform), the score will be relatively low. A
plot of this score versus the number of clusters can be used to determine a threshold.
This plot typically yields a plateau that represents a threshold for reasonably separated
clusters.
After separating the waveforms, a small number of samples remained unclassified
because they resulted from noise or overlapping waveforms that could not be
unambiguously separated. These waveforms typically represented only 1-3% of the data.
Due to the long recording times (as long as 12 hours for a single group), the data was
broken down into several files for classification. Typically, only a pair of neurons was
present with a steady response throughout all of the files and the entire recording time.
Because neurons beyond the strongest pair usually represented less than 2% of all the
samples and were not consistently recorded, we limited our analysis to pairs.
Stimuli
Initially, bars of light rear-projected onto a large tangent screen were used to
characterize receptive field location and properties. Since multiple cells were recorded,
the receptive field of the aggregate activity was determined and the activity center was
identified. Individual receptive fields could not be distinguished because the spike
sorting is performed offline. Stimuli were then generated using the Cambridge Research
46
Systems VSG2/4 controller board and a 21-inch SONY Trinitron graphics display with a
frame rate of 120 Hz and a mean luminance of 73 cd*m-2. The orientation (angle normal
to direction of drift in degrees), spatial frequency (cycles per degree of visual angle),
temporal frequency (cycles per second), and diameter of sine wave gratings were varied
to determine optimal characteristics for collective stimulation. The optimal
characteristics were determined from the peak of the respective time-averaged spike rate
histogram of the population activity (the highest total spike rate). The grating size varied
from 4-16 degrees (of visual field) with an average of 9 degrees. This measure does not
necessarily represent individual or even multiple or overlapping classical receptive field
sizes. The grating size was determined only by the maximum summed response of all
responding cells to increase the chances of obtaining enough spike samples for type
analysis.
Once the optimal grating parameters were determined for the group, an accurate
peak of the group activity (within 2 degrees or 0.02 c/deg) was measured for the
particular experiment (orientation or spatial frequency discrimination). We then
collected multi-unit recordings from as many 2-second stimulus repetitions as possible
(up to several hundred). We randomly repeated this for variations of orientation from 3,
7, 12, 18, 25, and 33 degrees from both sides of the peak response (maximum combined
firing rate of all neurons) or variations of spatial frequency from 0.03, 0.07, 0.12, 0.18,
0.25, and 0.33 c/deg from both sides of the peak. The randomly interleaved repetitions of
stimulation were done in groups of 20 sweeps to reduce the range of variation in the spike
waveforms, which minimized thresholding and waveform classification problems.
Sometimes the initially determined peak (from 20 sweeps of 2-degree or 0.02-c/deg
47
increments) was different from the peak found after several hundred sweeps, but in all
cases, the protocol yielded fine and gross variations from the peak response while still
covering the majority of the range of stimulus tuning. We measured cells for responses to
spatially optimal stimuli at contrasts (variation from the mean luminance) of 10, 20, 30,
40, 50, 60, 70, 80, 90 and 100%. Some data were also collected under a protocol of 1, 3,
7, 15, and 31-degree orientation variations from the peak; 0.01, 0.03, 0.07, 0.15, 0.31-
c/deg spatial frequency variations from the peak; and at contrasts of 1, 3, 7, 14, 28, 56,
and 100%.
Type analysis
We used the method of type (probability mass function or estimated probability
distribution) analysis described in detail by Johnson et al. (2001), which allows
examination of how neural ensemble responses differ relative to stimulus feature
variations (orientation, spatial frequency, and contrast). Variations in the parameters of
analysis and input data allow testing of redundancy or cooperativity between neurons and
how temporal resolution and discharge history can improve or hinder discrimination.
Type analysis can also be used to examine the temporal dynamics of transneural
correlation and we have extended this approach to examine these dynamics
simultaneously across stimulus feature variations. What makes type analysis so powerful
is that it makes almost no assumptions on the nature of the neural code. The assumptions
that are made are direct results of the amount of data that are available. The procedure
determines how two population responses vary across time in terms of the ability of an
optimal classifier to discriminate them.
48
Each stimulus repetition is first converted into a sequence of “letters”. The letter
is determined by the firing pattern that occurs within a time window (bin width). We use
a binary alphabet where each neuron can have a value of 1 or 0 depending on whether a
spike occurs within the bin. Each neuron represents a place in the binary representation.
For example, if a population of 3 neurons has the first and third neuron fire within a 3-ms
bin, the letter would be 101 (base 2) or:
22*1 + 21*0 + 20*1 = 5 (1)
Once this procedure is complete, each response collected is represented as a sequence of
numbers across 2 seconds ranging from 0 to 2number of neurons-1 or 0 to 7 for our example.
The sequence length is the number of bins that is determined by dividing 2 seconds by
the bin width.
Types or probability mass functions are then formed from the repetitions of each
stimulus. Essentially, a probability distribution is estimated for each bin across time for
each possible letter in the alphabet. Types can then be used from two different stimuli to
calculate a “distance”. The foundations of this method come from modern classification
theory and the distance measure calculated in short provides an estimate of the reduction
in classification error when using an optimal classifier. The classification error is
proportional to 2-distance and therefore an increase in the distance measure results in an
exponential decrease in the classification error.
We use a modified version of the Kullback-Leibler distance described by Johnson
et al. (2001) that provides an estimate of the Chernoff distance. The distance is referred
to as the Resistor Average and is the harmonic average of the Kullback-Leibler distance
from response 1 to response 2 and from response 2 to response 1. The Resistor Average
49
makes the Kullback-Leibler distance symmetrical and one half of the Resistor Average
provides an estimate that will approximate the Chernoff distance. The advantage of the
Resistor Average distance over the Chernoff distance is that it is less complicated to
compute and that it can be added from bin to bin when calculating the accumulated
distance over time. The Kullback-Leibler distance D(p||q) for bins 1 to B and for K
possible letters over M stimulus repetitions is:
Pa(k) = # times the letter k occurs for stimulus a (2) M
∑∑==
=K
x
B
b xPxPxPPPD
0 2
121
121 )(
)(log*)()||( (3)
The Resistor Average:
)||()||()||(*)||(),(
1221
122121 PPDPPD
PPDPPDPPR+
= (4)
The method can be extended to incorporate discharge history into the distance
measure by forming conditional types on the patterns that occur in previous bins. The
number of previous bins examined is the Markov order of analysis (D previous bins) and
is limited by the available data (M stimulus repetitions) and the population size (N
neurons):
)12log()1log(
++
≤ N
MD (5)
The estimation can be in error when using a Markov order too small so the data were
examined to determine how much discharge history was necessary to reach a stable
measure (i.e., when additional bins did not change the measure). If the Markov order was
too large for the available data, the temporal resolution was adjusted to reach a stable
50
measure. We examined the data to minimize the Markov order and still avoid losing any
information with the temporal resolution that was used.
Conditional types are formed from joint types that are the probabilities of
sequences of letters occurring. The conditional type is equal to the joint type of the
sequence of letters from the current bin and all previous bins considered divided by the
joint type of the sequence of letters occurring only in the previous bins. The joint types
are formed by essentially expanding the alphabet to include which bin the pattern occurs
in (alphabet size = 2N*D).
Because a limited amount of data will be available, many bins may end up with
probabilities of zero for certain letters. This would result in possible infinite distances in
the Kullback-Leibler calculations. To avoid this problem, the Krichevsky-Trofimov (K-
T) estimate (Johnson et al., 2001) is used, which initializes each probability to 0.5. The
types are then normalized to compensate for the 0.5 added to each letter’s probability.
Another problem that arises from the limited amount of data that is used to form
the types is the bias that is inherent in a distance estimate. Because the Kullback-Leibler
distance must always have a positive value, there will tend to be an upward bias in the
estimate. The procedure we use to estimate the bias and provide confidence limits on our
measures is the bootstrap method (Efron and Tibshirani, 1993; Johnson et al., 2001). The
bootstrap method creates new datasets from the original by randomly selecting samples
from the M repetitions and allowing for repeats. Then distances are calculated for all
these new datasets (we use 200) and averaged and the bias is obtained by subtracting the
original distance measure from this average. The datasets are sorted and depending on
51
the confidence limits desired, certain datasets are used to produce these limits (i.e., 5th
and 95th percentile for a 90% confidence interval).
To test for the redundancy or cooperativity of neurons in a population, we form
types using the ensemble alphabet (i.e., 8 letters for 3 neurons) and form types for each
individual neuron (having only 2 letters). The procedures for conditional types can then
be repeated for all of the measures to include discharge history. The sum of the
individual neuron’s distances can then be compared to the distance computed from the
ensemble alphabet. When these distances are equal, the neurons can be considered
independent. When the ensemble measure is smaller, the neurons are redundant and
when the ensemble measure is larger, the neurons are cooperative and there is synergy.
The confidence limits can be used to assess the significance of these differences.
The last method we explored from Johnson et al. (2001) was the measurement of
transneural correlation. The dependency of neurons in a population is quantified and
presented as a distance measure over time. A type is formed under the assumption that
the neurons are acting independently (forced-independent type). The first step is to sum
the probabilities of all letters in the original type that indicate a particular neuron
discharged. This is then repeated for all the neurons and the procedure is then repeated
for when the neuron does not discharge. Multiplying the proper sequence of discharges
and non-discharges then forms the independent type for each letter. For example with
two neurons:
Pd1 = Probability of discharge of neuron #1 = P(1) + P(3) (6)
Pnd2 = Probability of no discharge of neuron #2 = P(0) + P(1) (7)
Forced-Independent P(1) = Pd1*Pnd2 (8)
52
This type is compared to the original type to compare how dependent the neurons are and
how this varies over time. The method can be used to determine either dependency that
arises from connectivity or from shared input. We use the method in addition to cross-
correlation measures (Aertsen et al., 1989; Snider et al., 1998) with pairs of neurons to
assess functional connectivity. By extending the measure across another dimension
(variation in a stimulus feature), we are able to produce a surface that reveals properties
of functional connectivity that are not shown with cross-correlation. The absolute values
of the distance represent the strength of connectivity and the surface patterns can
illustrate how the connectivity varies relative to stimulus feature modulation. Cross
sections of the surface can describe how the connectivity changes over time. Because we
use a single electrode, we do not usually record pairs with shared input. In most cases,
the "effective connectivity" measure (Aertsen et al., 1989) shows that the neurons are not
correlated or are correlated with a 2-10-ms lag time indicating connectivity. The
dependency mesh can essentially demonstrate this lag time by varying the temporal
resolution or bin width of the measure and determining when the value of the peak is
greatest.
Results We first describe the general principles and problems of type analysis so that the
results can be understood more easily. Type analysis of neural ensembles produces a
function of “distance” between two responses versus time. The distance might be time-
varying and examination of this distance can reveal what portions of a response (i.e.,
transient or sustained) contribute most to the discrimination between two responses as
53
well as provide insight into how fast this discrimination can take. For example, if the
average firing rate is the primary difference between two responses, the distance will be a
linear function of time with a constant slope. The two responses may differ because of a
variation of a particular stimulus feature (i.e., spatial, temporal, or contrast patterns) and
the distance represents how distinguishable the responses would be using an optimal
classifier. This is expressed in an estimate of the reduction in classification error. The
classification error is proportional to 2-D(t) (where D(t) is the distance at time t), so an
increase in the distance measure results in an exponential decrease in the classification
error.
One difficulty associated with these experiments was the ability to record enough
data to meet the minimum requirements for the dimensionality of the vector space
distances. Since there will always be limitations associated with the amount of data that
can be collected during in vivo electrophysiology recordings, there will always be
limitations on the dimensionality of the measures calculated. The dimensionality can be
broken down into spatial and temporal dimensionality. The spatial dimensionality is how
many neurons (in this case, two) that are included in the calculations and the temporal
dimensionality is the Markov order of analysis or how many previous bins that are
incorporated into conditional probability mass functions. Examination of temporal
resolution and neural dependence on discharge history has provided us an estimated limit
on the temporal dimensionality of our type analysis.
When we examined the distance functions, we had to determine our experimental
approach or what responses to compare that would reveal characteristics of neural
coding. This in turn depended on variation of stimulus features. Since we examined
54
orientation and spatial frequency, the populations were essentially thought of as filters for
these characteristics. Under this assumption, we compared responses to the peak
response (maximum average spike rate of the pair of neurons). The more distinguishable
the response is from this peak response, the more efficient the population acts as a filter.
We examined how the properties of the distance measure changed with fine and gross
differences in stimulus features. Figure 1 shows an example of the orientation response
of a pair of neurons and the differences that were examined.
-40 -30 -20 -10 0 10 20 300
10
20
30
40
50
60
70
80
Firin
g R
ate
(sps
)
Orientation (degrees)
TotalNeuron 1Neuron 2Peak Response5-degree difference14-degree difference
Figure 1: This is an example of the response we recorded from a pair of neurons to variations in the orientation of a sinusoidal grating. The diameter of the grating was adjusted to maximize the total response. We compared fine discrimination (less than 10 degrees) and gross discrimination (greater than 10 degrees) by calculating the Kullback-Leibler Resistor Average distance for a 5-degree and 14-degree difference from the peak response. The response represents the average firing rate calculated from 200 random repetitions of 2-second displays of 13 variations of orientation and the null stimulus (mean luminance, left edge data point).
55
Latency
We first examined the contribution of latency differences in distinguishing
responses. Gawne et al. (1996b) showed that the delay between the start of the stimulus
and the onset of response was strongly modulated by contrast and there was no apparent
modulation with orientation. Peristimulus time histograms (PSTH) in the populations
tested with type analysis showed that latency continuously decreased for increasing
contrast and varied for orientation differences, but did not vary for spatial frequency
changes. The modulation was clearly organized for contrast, but there was no apparent
correlation between latency and orientation. We observed that while the latency varied as
much as 9-10 ms with orientation, there was no relationship between latency and a
particular orientation in the tuning curve.
Figure 2 demonstrates how precise measurement of latency differences can
improve the discrimination between responses to two different orientations. The top two
plots are the PSTHs for the first 100 ms of each neuron in the ensemble pair. The bottom
plot is one half of the Resistor Average distance accumulated over the first 100 ms. The
initial step seen in the distance function calculated between the two responses starting
from stimulus onset is caused by a precise 9 ms difference in latency between the two
responses, which is apparent from observation of the PSTHs. The 0.5 bit of distance can
reduce the error in classification by nearly 30%. To verify that this step was caused by
the latency difference, the responses were tested after shifting one response by 9 ms to
match the onset of the other response. Since the two responses had similar firing rates,
the distance for the first 100 ms between the two responses is nearly zero when
compensating for the latency difference. This particular case demonstrates one of the
56
most precise latency differences observed in the pairs of neurons examined. In some
cases, the latency difference was noticeable in the PSTH, but was not significant enough
in the probability distributions to contribute a significant amount of information. In some
cases, the difference resulted in less than 0.1 bit of additional distance between the
responses.
One of the reasons that we examined latency first is that it is very critical to our
analysis methods. When we compare two responses recorded individually, we must
assume whether the cortex compares responses with reference to the beginning of the
response or the beginning of the stimulus. In the laboratory, absolute latency is
determined by the recording equipment that keeps track of the start of the display relative
to the collection of data, but it is not clear if similar information is accessible by the
brain. Victor (2000) hypothesized that activation of populations of visual cortical cells
during saccades would provide a time-frame reference for the beginning of a stimulus.
This theory is also supported by the results of Park and Lee (2000) that show that a
significant portion of cortical activity is coupled to saccade offset. For each additional
spike train property we examined with type analysis (described below), we examine the
property using both frames of reference when there was a latency difference discovered.
The differences between results were subtle and there was no apparent difference for any
of the qualitative results. No matter which time-frame reference we used, the conclusions
we draw from the study were the same.
57
0 10 20 30 40 50 60 70 80 90 1000
0.05
0.1
Neuron 2
Dis
char
ge P
roba
bility 86 Degrees
90 Degrees
0 10 20 30 40 50 60 70 80 90 1000
0.05
0.1
Dis
char
ge P
roba
bilit
y
Neuron 1
86 Degrees90 Degrees
0 10 20 30 40 50 60 70 80 90 1000
0.2
0.4
0.6
0.8
1
Time (msec)
Dis
tanc
e (b
its)
86 vs. 90 Degrees
Stimulus Time ReferenceResponse Onset Reference
Figure 2: An example of the latency differences that we find in responses to different orientations. The top two plots are the peristimulus time histograms of the pair of neurons with one response superimposed (90 degrees) on top of the other response (86 degrees). Although there is some baseline activity before the onset of the response, the time can be accurately determined by simple observation. There is a 9-ms difference between the onset of the two responses for both neurons. The bottom figure is one half of the zero order Kullback-Leibler Resistor Average distance calculated between the responses as recorded (stimulus time reference) and with the slower response shifted by 9 ms (response onset reference). The solid and dashed lines represent the bootstrap debiased estimate (200 samples) and the dotted lines are the 90% confidence intervals. M = 260 2-second stimulus repetitions. Note: absolute values of latencies are not accurate due to software and hardware restrictions for stimulus and data acquisition components, but relative times have 30- microsecond precision.
58
Temporal Resolution
We next examined the temporal resolution used to bin the responses. Although
recordings were done with 30-microsecond precision, the responses were represented as a
“letter” determined by which neurons had a spike occur within a longer time window.
Temporal resolution was varied from 1 ms to 8 ms for analysis of responses for
discrimination of fine and gross changes of orientation and spatial frequency. Ten pairs
of neurons were examined for orientation differences of less than 10 degrees and greater
than 10 degrees from the peak response, and 10 pairs of neurons for spatial frequency
differences of less than 0.1 c/deg or greater than 0.1 c/deg. The number of stimulus
repetitions collected for each pair of neurons ranged from 30 to 300 with a mean of 195.
In our initial observations of distance rate versus bin width, we found an inversely
proportional relationship with temporal resolution for 8 pairs of neurons. Johnson et al.
(2001) predict that the bin width should be essentially independent of the distance rate or
accumulated distance when discharge probabilities are relatively small. These
predictions are made for a single neuron, zero Markov order scenario where the response
difference is a difference in average firing rates. This of course does not mean that a
distance measure of an actual neural response cannot vary as some function of temporal
resolution. However, a closer look at the 8 pairs of cells with the inversely proportional
relationship revealed that these pairs had either a relatively weak pooled response (<30
sps) or a small number of stimulus repetitions (<100) suggesting that the amount of data
available for the distance measure was insufficient. Smaller bin size results in a lower
probability of nonzero letters occurring in each bin. Across 2000 1-ms bins, even with
100 repetitions there will be instances when a bin is empty. The limited amount of data
59
leads to additional bias resulting in a larger estimate and a wider confidence interval. The
bootstrap method permits an estimation of the bias, but a limited amount of data will still
prevent the method from making accurate estimates. In these cases, the imprecision of
the estimate was indicated by large confidence intervals. As the data approaches larger
firing rates and larger differences in firing rate or larger bin widths and larger numbers of
samples, the distance rate does essentially become independent from the bin width.
There is still another irregularity seen in our results. In the case of stronger responses, we
consistently see a rise in the distance rate versus bin width over the 2-5-ms range. In
some cases, there is still a slight rise from a bin width of 2 ms to 1 ms and wider 90%
confidence intervals at 1ms, but the rise in the 2-5-ms range is still evident.
We examined this relationship for 7 pairs of neurons for orientation
discrimination and 5 pairs for spatial frequency discrimination. These recordings had a
pooled response of 36-109 sps with 120-300 stimulus repetitions (an average of 235).
Our definition of fine discrimination of orientation (<10 degrees) resulted in an average
rate difference of 7.6 sps (range 4-11) or a 10.5% difference (range 6.1-16.5), and the
definition of gross discrimination of orientation (>10 degrees) resulted in an average rate
difference of 30.6 sps (range 21-49) or a 43.0% difference (range 33.3-58.3). For fine
discrimination of spatial frequency (<0.1 c/deg) the average difference was 9.2 sps (range
4-13) or an 11.8% difference (range 5.8-14.6), and for gross discrimination (>0.1 c/deg)
the average difference was 24.6 sps or a 31.7% difference (range 28.7-34.1).
There are two possibilities for the reason there is an advantage in discriminating
neural responses (an increase in distance rate) in this range of temporal resolutions (2-5
ms). First, independent interspike interval (ISI) statistics over the short term (i.e.,
60
bursting) carry information about the stimulus feature being discriminated (e.g., Debusk
et al., 1997) and this information is extracted by filtering the response to emphasize this
time frame. Second, dependent ISI statistics (i.e., connectivity) between the pair of
neurons carries stimulus related information that provides the best discrimination within
this temporal window. To test for both of these possibilities, we first measure the
distance rates versus bin width of the original responses and then measure the function
after shuffling the stimulus repetitions for each neuron to remove spike train
dependencies.
Figure 3 shows an example of the distance rate function for fine (3A) and gross
(3B) discrimination of spatial frequency, along with fine (3C) and gross (3D)
discrimination of orientation. Although in some cases the advantage is subtle, it is clear
there is a peak from 2-4 ms for the original responses for all discriminations. After the
responses were shuffled (dashed lines), the peak remains, demonstrating that independent
ISI properties of the responses provide significant advantages for both orientation and
spatial frequency discrimination. Another subtle characteristic that was seen in all 12
pairs of neurons is that after shuffling the responses, the peak tended to shift to a higher
temporal resolution and the bandwidth of this peak was narrower. The average peak for
the 12 pairs of neurons for fine discrimination was at 3.5 ms with a width of 4.3 ms
(34.4% increase at peak), and after shuffling, the peak was at 2.3 ms with a width of 3.5
ms (67.9% increase at peak). The average peak for gross discrimination was at 3.5 ms
with a width of 6.5 ms (15.8% increase at peak), and after shuffling, the peak was at 2.7
ms with a width of 4.8 ms (29.5% increase at peak). For both small and large stimulus
differences, the results suggest an optimal bin width in the range of 2-5 ms.
61
2 4 6 80
0.5
1
1.5
2D
ista
nce
Rat
e (b
its/s
ec)
A
2 4 6 80
1
2
3
4
5B
OriginalShuffled
2 4 6 80
0.5
1
1.5
2C
2 4 6 80
2
4
6
8
10
Bin Width (msec)
D
OriginalShuffled
Figure 3: These plots demonstrate the distance rate versus temporal resolution for the original response and the response after shuffling (removing dependencies) for A, fine discrimination of spatial frequency; B, gross discrimination of spatial frequency; C, fine discrimination of orientation.; and D, gross discrimination of orientation. For A and B, M = 280 2-second stimulus repetitions and M = 300 2-second stimulus repetitions for C and D.
62
Discharge History
We next tested the contribution that discharge history had on discrimination.
Types are formed using the conditional probabilities that particular spatial patterns occur
depending on the patterns that occur in previous bins. The number of previous bins
examined is the Markov order of analysis or the temporal dimensionality. When the
probabilities of only the patterns occurring in the current bin are considered in the
measure, the order of analysis is zero. In all methods that require temporal binning of
data, there are constraints on the amount of data (the number of stimulus repetitions) that
is necessary to compute a measure with a desired dimensionality. The minimum number
of stimulus repetitions that are needed is equal to or greater than 2ND (where N is the
number of neurons and D is the Markov order). Since we are working with pairs of
neurons and have a limit on the amount of data we can collect, we can set a limit on the
Markov order that can be used. When determining the characteristics of discharge
history that will most improve discrimination between responses, the time is of most
importance, not the Markov order. Of course this becomes more complicated when the
temporal resolution has dramatic effects on the distance calculation as we have shown in
the previous section. As the conditional probabilities are calculated across more history,
the distance measure should reach a point where this is no more to gain. This time, along
with the temporal resolution requirements, determine the Markov order of analysis (D =
maximum discharge history duration divided by the temporal resolution). If the Markov
order falls below the limit set by the number of neurons and the amount of data we
collected, we should be able to determine an accurate estimate. If this value falls above
the limit, the temporal resolution must be sacrificed. Figure 4 shows a representative
63
example of the change in distance as the Markov order is raised from 0 to 2 with a bin
width of 3 ms (discharge history of 0 to 6 ms). There is no improvement by increasing
the Markov order from 1 to 2, but a significant gain from 0 to 1, so we adopted a measure
that includes 3 ms of discharge history into the types.
0 200 400 600 800 1000 1200 1400 1600 1800 20000
0.5
1
1.5
Dis
tanc
e (b
its)
Time (msec)
D = 0D = 1 (3msec)D = 2 (6msec)
Figure 4: This plot demonstrates the steps in determining the relevant discharge history in a response and the Markov order used to form conditional types. The estimate more than doubles when using a Markov order of one versus zero. The measures are bootstrap debiased (200 samples) estimates of one half of the accumulated Kullback-Leibler Resistor Average distance at a resolution of 3 ms. This means that using conditional types depending on the previous bin (3 ms of discharge history) adds nearly a bit of distance over 2 seconds and can halve the classification error. When using a second order conditional type (6 ms of discharge history), there was no increase (actually a slight decline) in the distance estimate so the relevant dependencies appear to fall within 3 ms. M = 200 2-second stimulus repetitions (5-degree difference in orientation).
64
We also wanted to examine how discharge history contributed to discrimination
regardless of the temporal resolution. Because the temporal resolution had such a
significant effect on the distance measure, it distorts the effects of discharge history when
looking at the average distance rate alone. To separate out this distortion, we calculated
the increase in distance from a zero order calculation to a first order calculation at several
bin widths.
Responses from 11 of the 12 pairs of neurons (7 orientation, 4 spatial frequency)
were tested for bin widths from 1 ms to 6 ms to determine the impact of discharge
history. As with temporal resolution, both independent and dependent spike train
characteristics can lead to advantages in discrimination as a result of the discharge history
information. For example, burst length modulation will lead to significant changes in the
probabilities of spikes occurring in the discharge history of the individual neurons. At
the same time, connectivity modulation will be revealed in discharge histories that
include enough time to allow for synaptic delays. To separate the contributions of
independent and dependent properties, we again compared the original results with the
result after shuffling the stimulus repetitions for each neuron.
Figure 5 shows an example of the results for fine (A and C) and gross (B and D)
discrimination of orientation for a pair of moderately connected neurons (we find a
noticeable peak at 2-10 ms in the correlogram or a dependency between the neurons of
greater than 0.1 bits/s—see Functional Connectivity for a detailed explanation). For fine
orientation discrimination (Figure 5A), the optimal amount of discharge history is 2 ms
(providing 110% increase in distance). After shuffling the responses, we see that the
independent properties of discharge history (Figure 5A, black bars) are not as strongly
65
dependent on temporal resolution as the dependent properties (Figure 5C—the shuffled
response subtracted from the original response). The independent properties result in
about 50% increase in distance (including 1-6 ms of history) and the dependent discharge
history increases the distance by another 40% when considering the previous 2 ms of
activity. For gross discrimination of orientation, the discharge history contributes very
little. Figure 5B shows that only about 10-20% is gained from the discharge history and
none of it is a result of dependent properties (Figure 5D).
1 2 3 4 5 60
20
40
60
80
100
120
Incr
ease
with
Dis
char
ge H
isto
ry (%
)
A
1 2 3 4 5 6-10
0
10
20
30
40
50
Discharge History (msec)
C
1 2 3 4 5 6
0
20
40
60
80
100
120B
OriginalShuffled
1 2 3 4 5 6-10
0
10
20
30
40
50D
Figure 5: An example of the contribution of discharge history for orientation discrimination. A, The percent increase from a Markov order of 0 to 1 for the original and shuffled response (removing neuron dependencies) for fine discrimination and B, for gross discrimination. C, The difference between the original and shuffled responses to show the contribution of dependent discharge history for fine discrimination and D, for gross discrimination. M = 260 2-second stimulus repetitions.
66
Figure 6 shows the discharge history results for spatial frequency discrimination for a
pair of neurons with similar response strength and connectivity as the pair reported in
Figure 5. The individual discharge histories of the neurons (see Figure 6A) result in a
40% increase when considering the previous 4 ms of activity. The connectivity (at 2ms)
between the neurons (see Figure 6C) only results in an additional 25% in distance. As
with gross discrimination of orientation, the discharge history hardly improves the gross
discrimination of spatial frequency (see Figure 6B and 6D).
1 2 3 4 5 60
20
40
60
80
100
120A
1 2 3 4 5 6-10
0
10
20
30
40
50
Incr
ease
with
Dis
char
ge H
isto
ry (%
)
Discharge History (msec)
C
1 2 3 4 5 60
20
40
60
80
100
120B
OriginalShuffled
1 2 3 4 5 6-10
0
10
20
30
40
50D
Figure 6: An example of the contribution of discharge history for spatial frequency discrimination. A, The percent increase from a Markov order of 0 to 1 for the original and shuffled response (removing neuron dependencies) for fine discrimination and B, for gross discrimination. C, The difference between the original and shuffled responses to show the contribution of dependent discharge history for fine discrimination and D, for gross discrimination. M = 280 2-second stimulus repetitions.
67
The average overall increase in distance (for 7 pairs) when incorporating 2 ms of
discharge history for fine discrimination of orientation was 110% (range 55-250). After
shuffling the stimulus repetitions for each neuron, the average increase was 60% (range
45-90). Discharge history provided on average only 15% (range 0-35) for gross
discrimination of orientation and the average did not change after removing dependencies
between the neurons. For spatial frequency discrimination, the contribution was less
significant for fine differences, yielding an average increase (for 4 pairs) of 70% (range
40-120) including dependencies and 50% (range 40-75) without dependencies. As with
orientation, there was very little to gain for discrimination of gross changes in spatial
frequency with an average increase in distance of 20% (range 5-25) before and after
shuffling the responses. Overall, the results suggest that the dependent properties of
discharge history between the neurons can provide an average of 50% for fine
discriminations of orientation and 20% for fine discriminations of spatial frequency, and
provide no significant improvement in gross discriminations of both orientation and
spatial frequency. In the next section, we derive similar results using a different approach
to show how dependent ISI properties enhance discriminations of features. We will also
evaluate the confidence of these results with respect to the amount of data we use in the
distance calculations.
Synergy, independence, and redundancy
We next demonstrate how incorporation of discharge history leads to additional
information beyond what would be derived from the individual neurons. Efforts thus far
have demonstrated that visual cortical pairs are independent (Gawne et al., 1996a; Victor,
68
2000; Reich et al., 2001c). We examined our data to see whether the responses of
neurons were independent or if under some conditions, examining them as a population
leads to additional information.
The Kullback-Leibler Resistor Average distances were calculated for the same 11
pairs of neurons mentioned above and for each individual neuron in the pairs. The
individual neurons’ types are formed from a 2-letter alphabet (discharge or no discharge)
whereas the ensemble type is formed from a 4-letter alphabet (both discharged, only
neuron 1 discharged, only neuron 2 discharged, or neither discharged). We also
calculated the first order distances using the optimal parameters for bin width and
discharge history that were determined with the ensemble measures. If the ensemble
measures were greater than the sum of the individual neurons’ measures, then the neurons
were cooperating and producing additional information (synergy). If the ensemble
measure was less than the sum, the neurons contained redundant information about the
stimulus differences. If both the ensemble measure and sum of the individual measures
are equal, the neurons are acting independently. If the first order measures for the
individual neurons show either no gain (from the zero order measure) or a relatively
small gain compared to the gain for the ensemble first order measure, then the gain seen
in the ensemble measure is a result of correlation between the neurons rather than any
auto-correlation found in the individual neurons.
The results we found were similar to the results found when examining discharge
history, implying that a significant portion of the additional information gained from
discharge history is a result of the correlated activity between the neurons. This also
means that a significant portion of the synergy is a result of incorporating the discharge
69
history into the distance measure. We also find that the cooperation is related to the
strength of connectivity between the neurons. The average amount of synergy produced
across all 7 pairs of neurons for fine discrimination of orientation was 50%, with a range
covering 15-120%. Regardless of the connectivity, the neurons for the most part work
independently for gross discrimination of orientation (synergy of only 5% over the range
of 0-15%). The difference between fine and gross discrimination of orientation for
strongly connected neurons was thus nearly 10-fold. As pairs showed less connectivity,
this difference diminished to insignificance for the weakest pair of neurons.
For spatial frequency discrimination, synergy appeared to be lower. The average
amount of distance added as a result of cooperation was 25% for fine discrimination over
the range of 15-40%. The results for gross discrimination of spatial frequency were
nearly the same as gross discrimination of orientation with an average of 8% over the
range of 5-10%. There was at most an 8-fold increase in the percentage gained for fine
discrimination versus gross discrimination for strongly connected neurons and again, the
difference was much smaller for moderately connected neurons.
The example shown in Figure 7A and 7B shows the responses of a pair of neurons
(same pair as in Figure 5) from a 4-degree difference in orientation (4 sps difference with
60 sps peak) and a 16-degree difference (22 sps difference). The ensemble distance is
more than 45% greater than the sum of the individual measures (see Figure 7A). The
larger orientation difference results show that the neurons are acting more independently
(see Figure 7B). The increase is only about 10% and the independent measure falls well
within the lower bound on the 90% confidence interval. This example is for a pair of
70
neurons that shows moderate connectivity and the synergy only becomes significant
when we incorporate 2 ms of discharge history into the distance measure.
The results for all spatial frequency discrimination and for gross discriminations
of orientation agree with previous conclusions that the neurons act essentially
independently. Our results are unique in that we find that when the neurons are strongly
connected, they cooperate for fine discrimination of orientation. If this behavior is found
across larger populations of neurons, relatively small circuits of neurons could perform
finer discriminations than possible by use of differences in the average firing rate. One
reason we believe we find the significant amount of synergy across this small range of
the active region for the pair of neurons is that it is in this region that connectivity is
highly modulated while the average firing rate (and even burst rate) is nearly constant.
We will demonstrate this idea in detail in the next section.
71
0 500 1000 1500 20000
0.5
1
1.5
2
2.5
3A
Ensemble DistanceNeuron 1Neuron 2Neuron 1 + 2
0 500 1000 1500 20000
2
4
6
8
Time (msec)
Dis
tanc
e (b
its)
B
Figure 7: An example of the synergy found in a moderately connected pair of neurons during orientation discrimination (same pair of neurons as in Figure 5). A, One half of the accumulated Kullback-Leibler Resistor Average distances calculated from the 4-letter ensemble type (solid), calculated from the 2-letter types for each neuron (dash-dot), and the sum of the two independent neuron types (dash-dash). The accumulated distance is between the peak response and the response to an orientation 4 degrees from the peak orientation, which had a difference in total firing rate less than 10%. The synergy results in a 45% increase, but it should be noted that the 90% confidence interval (dotted) is relatively large. B, The same four measures as in plot A except for the accumulated distance between the peak response and to an orientation 16 degrees from the peak orientation. The synergy results in a 10% increase and the lower bound on the confidence interval falls well below the independent distance. All calculations were done with bin widths of 2 ms and a Markov order of one. Estimates and confidence intervals were calculated with the bootstrap method (200 samples). M = 260 2-second stimulus repetitions.
72
Confidence in distance estimations
The imprecision in the first order ensemble distance (incorporating 2 ms of
history) is still evident even with our large sample sizes. This is seen in Figure 7A where
the independent distance falls within the lower bound of the 90% confidence interval.
This raises the question as to the accuracy of our distance calculations (both relative and
absolute) for both discharge history and synergy. We performed analysis on all 20 pairs
of neurons to examine how confident we were in our calculations in addition to what the
confidence interval demonstrated. We initially rejected 8 of the 20 possible pairs of
neurons (for orientation and spatial frequency analysis) because the temporal resolution
plots suggested insufficient sampling due to an inversely proportional relationship of
distance rate with bin width. We rejected another pair of neurons for our discharge
history analysis based on significant inconsistencies between the synergy and discharge
history analysis. We also rejected this pair of neurons for synergy analysis after
examining the relationship of our calculations to sample size.
For each pair of neurons, we randomly selected one-half, five-eighths, three-
fourths, and seven-eighths of the available stimulus repetitions and plotted the ensemble
distance rate, the independent distance rate, the two individual neuron distance rates, and
the percent synergy versus the sample size. Figure 8 shows an example of these
calculations for the pair of neurons reported in Figure 7. Figure 8A and Figure 8C show
the distance rates and synergy for fine discrimination of orientation, respectively, and
Figure 8B and Figure 8D for gross discrimination of orientation. Reich et al. (2001a)
examined an information-theoretic-based measure in a similar manner and rejected all
data that resulted in a measure that was in excess of 10% when using a random selection
73
of half the sample versus all of the samples. With our analysis, we could not set such a
criteria without rejecting all of the data, but we do have a sufficient number of samples to
demonstrate that the calculations appear to approach an asymptote versus sample size. At
the same time, the 90% confidence intervals narrow as we select a larger portion of our
sample demonstrating that the precision does continue to increase with the number of
stimulus repetitions.
In general, our sub-sampling analysis showed that with less than 200 stimulus
repetitions, the distance measures and percentage calculations were very unpredictable.
At around 200 repetitions, the distance rates appear to have somewhat of an inversely
proportional relationship with the number of samples and approach asymptotic behavior.
This relationship seems to suggest that fewer samples result in greater discrimination, but
this increase in distance is a result in bias that is not removed with the bootstrap method.
This is clear when observing that the 90% confidence interval grows larger as the sample
size is reduced, and that the lower bound of the interval shows a constant relationship
with sample size. After 250 repetitions, the measures appear to be stable and additional
samples result in less than a 10% difference in the calculations. This behavior is also
dependent on the strength of the response (i.e., the number of spikes) and whether we are
looking at gross versus fine discrimination calculations. In some cases, we found as little
as 120 stimulus repetitions to be sufficient for a pair of neurons with a pooled response of
109 sps. At the same time, we rejected a pair of neurons that only had a response
strength of 31 sps even after 300 stimulus repetitions. A close look at Figure 8A and
Figure 8C versus Figure 8B and Figure 8D reveals that the gross discrimination
74
calculations (8C and 8D) reach asymptotic behavior before fine discrimination
calculations (8A and 8B).
150 200 2500
0.5
1
1.5
2D
ista
nce
Rat
e (b
its/s
ec)
A
150 200 2500
20
40
60
80
100
Number of Sweeps
Syn
ergy
(%)
C
150 200 2500
1
2
3
4
5B
150 200 2500
20
40
60
80
100D
Figure 8: These plots show the relationship between the number of samples and the first order distances and the percent synergy for the response from Figure 7. A, The fine discrimination distance (first-order ensemble—solid, each neuron—dash-dot, and independent—dash-dash) using all 260 stimulus repetitions and a random selection of subsamples of these repetitions. B, The same plot for gross discrimination of orientation. C, The percent synergy calculated versus the number of samples used for fine discrimination and D, for gross discrimination.
75
The behavior of distance rates and synergy versus sample size is very consistent
across all 11 of the pairs we analyzed. Figure 9 shows another example of orientation
discrimination results and the behavior matches well with the behavior seen in Figure 8.
The average increase (from bias in the estimation) from using all samples in the ensemble
calculation to using a random selection of half the samples for fine discriminations was
45% (range 15-100) and for gross discriminations, the average increase was 10% (range
3-40). When looking at Figures 8 and 9 and examining the rest of the pairs’ responses,
our synergy estimations do not accurately convey the precise cooperation between the
neurons (i.e., that more samples would result in lower estimations). However, it is also
apparent that the measurements are reaching an asymptote and that are calculations are
biased upwards and that the error should be less than the difference we find when
comparing half of our data with all of our data.
Functional connectivity
We last examined the dependency between neurons to explore the putative
substrate of the cooperative activity. The probability mass functions collected for an
ensemble were recalculated under the assumption that each neuron was independent from
the others (forced-independent type). If two neurons are independent, then the
probability of both firing within the same bin is equal to the product of the probabilities
of each neuron firing in the bin. If the two neurons are dependent (from a direct
connection, shared input, other correlated circuitry, or stimulus induced), the probability
of both firing within the same bin will be higher (excitatory dependency) or lower
(inhibitory dependency) than the product of the two individual probabilities. The
76
measurement can provide insight into the correlation and connectivity between neurons
and how it varies over the time period of a stimulus and varies across variation of
stimulus features. The method was tested on pairs of neurons to examine their
interactions and the results were compared with the normalized "effective connectivity"
measure (Aertsen et al., 1989; Snider et al., 1998).
150 200 2500
0.5
1
1.5
2
2.5
Dis
tanc
e (b
its)
A
150 200 2500
20
40
60
80
100
Number of Sweeps
Syn
ergy
(%)
C
150 200 2500
1
2
3
4B
150 200 2500
20
40
60
80
100D
Figure 9: These plots show the relationship between the number of samples and the first order distances and the percent synergy for the response of another pair of neurons for orientation discrimination. A, The fine discrimination distance (first-order ensemble—solid, each neuron—dash-dot, and independent—dash-dash) using all 280 stimulus repetitions and a random selection of subsamples of these repetitions. B, The same plot for gross discrimination of orientation. C, The percent synergy calculated versus the number of samples used for fine discrimination and D, for gross discrimination.
77
We have tested this method on 22 pairs of neurons while varying orientation (10
pairs), spatial frequency (10 pairs), and contrast (9 pairs). Two pairs of neurons were
tested for all three stimulus parameters, two pairs for contrast and orientation, and one
pair for contrast and spatial frequency. All neurons were classified as complex (Skottun
et al., 1991). The number of stimuli repetitions ranges from 30 to 300 with a mean of
191.
The results were used to produce a surface mesh that displays the dependency
variation between the neurons across the stimulus feature variations and across the
stimulus display time (2 seconds). Since we used a single electrode for recording multi-
unit activity, our results focus on connectivity between neurons rather than common input
synchrony between neurons. Waveforms correlated around a zero lag time will overlap
on a single electrode and it is usually too difficult to distinguish the individual
waveforms. Six out of the 22 pairs of neurons were shown to be strongly connected with
a peak in the cross-correlogram at 2-4 ms and a width of 1-3 ms (10 out of the 29
experiments: 5 for orientation, 1 for spatial frequency, and 4 for contrast). Seven pairs of
neurons (4 for orientation runs, 4 for spatial frequency runs, and 2 contrast runs) showed
some moderate correlation in the cross-correlogram and the last 9 pairs of neurons (1 for
orientation, 5 for spatial frequency, and 3 for contrast) showed no noticeable correlated
activity.
One half of the Kullback-Leibler Resistor Average distance (between the original
type and forced-independent type) was calculated for all pairs using a temporal resolution
of 1-13 ms until the peak of the connectivity mesh was reached. The resolution that
resulted in the largest distances tended to be slightly larger than the peak that was found
78
in the cross-correlogram. In cases where the actual peak was only 1-3 ms wide, but the
correlated activity still extended to greater than 10 ms, we found optimal bin widths to be
as large as 12 ms (although this was rarely the case). The smallest optimal bin width was
2 ms for a pair of neurons that had a 2-ms width peak at 2 ms in the cross-correlogram.
The average optimal bin width of moderately and strongly connected pairs of neurons
was 3.1 ms. The largest distance of the mesh plot divided by the stimulus time was used
to produce an average distance rate value to classify the pair of neurons as strongly,
moderately, or weakly connected. We found that in a few cases the range for strongly
and moderately connected neurons could overlap and determined that the method would
be best suited to be used in conjunction with normalized cross correlation analysis. The
reason for the overlap is that the distance measure is an entropy measure and the absolute
value of the distance rate will be influenced by the strength (firing rate) of the response.
Strongly connected neurons had distance rates from 0.9-4.0 bits/sec and in one case, the
rate was as low as 0.4 bits/sec. Moderately connected neurons had distance rates from
0.2-0.9 bits/sec with one case as high as 1.1 bits/sec and weakly connected neurons had
average rates in the range of 0-0.1 bits/sec. We determined a cutoff of 0.1 bits/sec to
classify a pair of neurons as at least moderately connected. This was determined by
examining the 90% confidence intervals and observing a lower limit below 0 (no
significant dependencies) for responses of weakly connected pairs of neurons and
shuffled responses of moderately and strongly connected pairs of neurons.
The reason we represent the strength of connectivity in terms of average distance
rate is to present a value that is independent of the stimulus duration. A closer look at the
temporal dynamics of the connectivity reveals that it varies in time. Figure 10A shows an
79
example of a surface mesh between a pair of moderately connected neurons for
orientation tuning. Figure 10B shows the optimal orientation and progressively non-
optimal orientations as a cross section of time to illustrate the modulations of
connectivity. The slope of the response indicates differences from the predicted
independent probabilities. Horizontal regions in the function indicate that the neurons are
firing independently from one another. A comparison of the distance plots for the peak
response (288 degrees) and 10 degrees (or 15 degrees) from the peak (278 or 273
degrees) shows that from 25 ms to 40 ms (see inset in Figure 10), the response of the
connectivity is equal with a very steep slope (>8.0 bits/sec). Immediately following the
initial 40 ms, the slope drops to less than 0.5 bits/sec for 278 degrees and less than 0.2
bits/sec for 273 degrees suggesting the correlated activity is reduced by a slightly delayed
inhibition. The peak response of correlated activity (as well as away from the peak) also
shows a decline in slope (although the reduction is much less significant than the
inhibition) and the slower time scale (on the order of seconds) suggests there is
adaptation occurring in the correlated activity. For the most part, the temporal dynamics
and the role of inhibition were similar for spatial frequency tuning of dependency.
80
0500
10001500
2000
260280
300
0
1
2
Time (msec)
A
Orientation (degrees)
Dis
tanc
e (b
its)
0 500 1000 1500 20000
0.5
1
1.5
2
Time (msec)
Dis
tanc
e (b
its)
B
288 degrees278 degrees273 degrees
30 40 500
0.1
0.2
Figure 10: An example of the temporal dynamics of dependency tuning for orientation. A, The mesh plot of the accumulated distance between the original and forced-independent response across time, while varying orientation. B, A cross section of plot A at the peak orientation (288 degrees) and two orientations away from the peak to show the effect inhibition has on dependency. The inset in B zooms in at 25-50 ms to show the timing of the inhibition. Estimates and confidence intervals (dotted lines) were calculated with the bootstrap method (200 samples). M = 200 2-second stimulus repetitions.
81
In Figure 10, the contrast was set at 50% for the orientation tuning. We also tested how
contrast would modulate the dependency. We would expect contrast tuning of
dependency to match well with average firing rate tuning because Cattaneo et al.
(1981a,b) demonstrated that all spikes regardless of ISI properties (both bursts and
isolated spikes) followed the same modulation. Figure 11B demonstrates this is clearly
not the case. The dependency modulation (solid line representing the accumulated
distance after 2 seconds) drops off much more rapidly than expected when comparing it
to the firing rate contrast response (dotted lines). The time scale at which the adaptation
in dependency occurs appears to be modulated by the contrast (see Figure 11A).
In order to see the role contrast has in this adaptation, we separate the excitatory
and inhibitory effects seen in the initial portion of the dependency from the slower
adaptation effects. We first converted the dependency responses from accumulated
distances to distance rate functions and filtered the response to remove any modulation.
We then normalized the response by the maximum slope (the initial slope) of that
particular response. This yields a clear picture of how the dependency decomposes from
its original strength at the onset of the response. We can see how contrast affects the
adaptation independently from the strength of the dependency. Figure 12 shows an
example of how this decay or adaptation is effected by contrast. The response at 10% has
a much faster time constant for adaptation than at 60% (or even 20%).
82
0500
10001500
2000
0
50
1000
2
4
Time (msec)
A
Contrast (%)
Dis
tanc
e (b
its)
0 20 40 60 80 1000
1
2
3
4
Contrast (%)
Dis
tanc
e (b
its)
B
Figure 11: An example of the dependency modulation while varying contrast. A, The mesh plot of the accumulated distance between the original and forced-independent response across time, while varying contrast. B, A cross-section showing the accumulated distance after 2 seconds (solid line with dashed 90% confidence intervals) with the average firing rate response for both neurons superimposed in the background (dotted lines). The peak firing rates were 90 sps and 25 sps for the two neurons. Estimates and confidence intervals were calculated with the bootstrap method (200 samples). M = 220 2-second stimulus repetitions.
83
0 500 1000 1500 20000
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
Nor
mal
ized
Dis
tanc
e R
ate
Time (msec)
10%20%40%60%
Figure 12: An example of the adapting portion of the dependency rates for the pair of neurons from Figure 11. The accumulated distance plots were converted to distance rates and then were normalized with the maximum rate at each particular contrast.
These results suggest that examination of the initial rise in the dependency plot
can reveal the actual anatomical connectivity between the neurons, since it is independent
of the stimulus features. The slope of the dependency distance after this initial step
reveal how the connectivity is modulated by variation in spatial frequency and
orientation. The time constant of the decline in dependency over this same period can
reveal the contrast of the stimulus. The results also suggest that interpretation of cross-
correlation plots must consider the stimulus duration especially with regard to variation in
contrast.
84
At first glance, the results appear to suggest that the connectivity modulation for
spatial frequency and orientation are a result of the modulation that occurs in the average
firing rate. Figure 13 is an example of the connectivity tuning we find for a pair of
moderately connected neurons for orientation modulation (A and C) and for spatial
frequency modulation (B and D). In the case of orientation, we find, in all 9 cases of
highly or moderately connected neurons, a very sharp peak of correlation. There were
also sometimes multiple peaks and the highest peak was not always found at the peak of
the average firing rate. Figure 13C shows an example of the dependency tuning with the
rate tuning for the two neurons superimposed in the background. The tuning is much
narrower and allows for much finer discrimination between orientations around the peak.
Figure 13D shows that in the case of spatial frequency, the dependency tuning is slightly
narrower than the rate tuning. Across all cases of connected neurons for spatial
frequency, the tuning was definitely narrower than average rate tuning, but never quite as
drastic as the differences found with orientation tuning.
85
0
1000
2000
-40-200200
0.51
1.52
2.5
Time (msec)
A
Dis
tanc
e (b
its)
-40 -20 0 200
0.5
1
1.5
2
2.5C
Orientation (degrees)
0
1000
2000
00.2
0.40
0.5
1
1.5
Time (msec)
0 0.1 0.2 0.3 0.4 0.50
0.5
1
1.5
Spatial Frequency (c/deg)
D
Orientation (degrees) Spatial Frequency (c/deg)
B
Figure 13: A comparison between the connectivity tuning for orientation and spatial frequency. A, One half of the accumulated Kullback-Leibler Resistor Average distance between the observed type and the forced-independent type across the orientation tuning of strongly connected neurons. B, The accumulated dependency distance across spatial frequency for a pair of strongly connected neurons. C, A cross-section of the accumulated dependency distance at 2 seconds to show an example of the connectivity tuning for orientation (solid, with the dashed 90% confidence interval). The very sharp peak is representative of the characteristics seen in the tuning for strongly and moderately connected neurons. The rate tuning for the two neurons is superimposed in the background (dotted with peaks of 74 sps and 19 sps) to show the differences between rate tuning and connectivity tuning. D, A cross-section of the accumulated dependency distance at 2 seconds to show an example of the connectivity tuning for spatial frequency (solid, with the dashed 90% confidence interval). The average rate tuning curves in the background have peaks of 38 sps and 28sps for the two neurons. Estimates and confidence intervals were calculated with the bootstrap method (200 samples). A and C: M = 200 2-second stimulus repetitions, B and D: M = 280 2-second stimulus repetitions.
86
Discussion
Latency
From an information-theoretic point of view, the importance of the first-spike
timing (latency) was demonstrated by Heller et al. (1995) when they found that latency
accounted for 35% of the total information in V1 and 48% of the total information in IT.
We cannot make any direct comparisons with our results because the percent contribution
of latency to the total distance depends on stimulus duration and the slope of the distance
after the initial step caused by the latency difference, but our results are consistent in
expressing that latency could provide a significant amount of information. The
relationship between latency and contrast has also been established in the primary visual
cortex (Gawne et al., 1996b; Reich et al., 2001b). The role that contrast modulation of
latency might play in neural coding has been postulated as an aid in feature binding
(Gawne et al., 1996b) or that it essentially increases the dynamic range of contrast
encoding by providing finer discriminations than with rate coding (Reich et al., 2001b).
A recent study of pairs of visual cortical neurons showed that latency was
correlated across long distances (>2mm) in the brain and that the variability in the
correlation was less than 10 ms (Fries et al., 2001). This demonstrates the possibility of
the relative latencies between populations of neurons contributing information.
Averaging responses might extract this property as an absolute latency. The variability
found in latency from trial-to-trial is further reduced when considering bursts over all
spikes (Guido and Sherman, 1998).
87
We find latency varies relative to contrast and orientation, but not for spatial
frequency. Latency is also modulated for sound intensity (regardless of location) in
auditory cortex (Heil, 1997) and for motion in area MT (Lisberger and Movshon, 1999).
A dynamic network incorporating delays allows functions such as scaling, rotation, and
translation to occur and is very efficient for invariant applications (Hopfield, 1995;
Schalkoff, 1997). Sensory systems incorporate invariant properties in object recognition,
sound location, and even in odor identification. If the fundamental parameter in visual
processing is spatial frequency (or sound frequency in auditory cortex), then the other
parameters such as orientation, contrast, and temporal frequency (or sound intensity)
would need to be rotated or scaled to provide invariant spatial frequency processing. The
results found thus far are consistent with this idea and spatial frequency would likely be
the invariant parameter for edges, surfaces, and textures that are all incorporated into
object recognition. This does not mean that the properties of orientation, contrast, and
temporal frequency cannot also be encoded within the network.
Artificial networks have been used successfully with time delays for auditory
applications and recently to recognize objects regardless of the velocity and direction of
motion (Wohler and Anlauf, 1999a,b). The research in this area has been limited to
theoretical studies beyond the discovery that spatiotemporal patterns are found to be
modulated by stimuli (Grothe and Klump, 2000). One example of a theoretical study
uses “winner-less competitive” (WLC) networks to produce small dynamic ensembles
within a network to encode olfactory information (Rabinovich et al., 2000).
Current research in axon electrotonic architecture suggest axons themselves are
involved with information processing by inducing delays (Segev and Schneidman, 1999)
88
that would be more efficient than synaptic mechanisms, but Hopfield (1995) points out
that delays might be induced by synaptic, axonal, or cellular mechanisms. The
importance of the temporal dynamics has been stressed in recent reviews (Doetsch, 2000;
Milton and Mackey, 2000; Nadasdy, 2000).
Independent ISI characteristics
The most significant finding in our temporal resolution analysis is the importance
of independent ISI properties in feature discrimination. Since Adrian and Zotterman
(1926) first demonstrated that the average spike rate varied as a function of sensory
stimulation, rate coding has become the most straightforward and popular measurement
in neurophysiological studies. At the same time, it is not necessarily the most
straightforward strategy that the brain might use for encoding (Hopfield, 1995).
Segundo et al. (1963) discovered that interval information was passed between
neurons as a result of temporal integration and synaptic properties. The spike train
contains multiple distributions of activity at various temporal resolutions independently
encoding different aspects of the sensory input (Cattaneo et al., 1981a,b; Debusk et al.,
1997; Victor, 2000). Synaptic properties such as facilitation and depression can take
advantage of these ISI characteristics and could multiplex information on a single neuron
level (Victor, 2000). In any case, the synaptic properties make encoding information in
the form of ISI properties much more straightforward for the brain.
We found a maximization in distance with a temporal resolution of 2-5 ms. We
believe the advantage in using this resolution from 2-5 ms is that it acts like a low pass
filter that emphasizes bursts. Because the difference in the number of bursts in responses
89
to non-optimal stimuli will be greater than the difference in total spike count (Cattaneo et
al., 1981a,b), the discrimination between responses will be easier when the temporal
window selects for the burst count. Reinagel et al. (1999) has shown that information
encoding can be much more efficient when bursts are considered as individual events. A
similar result was found in humoral information transfer, where Prank et al. (2000) used
low pass filtering at frequencies for [Ca2+] bursts to improve transmembrane information
transfer.
Bursts and connectivity
Shadlen and Newsome (1994) have argued that neuronal organization on time
scales less than 10 ms is impractical due to synaptic unreliability. Zador (1998) has
shown that an increase in synaptic reliability (due to changes in transmitter release
probabilities or synaptic redundancy) results in a decrease in information transfer for rate
coding, but an increase in information transfer for temporal coding. Bursting (Snider et
al., 1998) suggests one way to enhance synaptic reliability. In fact, the inefficiency of the
synapse makes it the perfect burst detector (Lisman, 1997). Cattaneo et al. (1981a,b)
demonstrated that over a 60-degree range around the peak of orientation tuning and over
a 0.6-cycles/degree range around the peak of spatial frequency tuning, the average rate of
those spikes not contained in bursts remains essentially constant. This implies that the
rate tuning characteristics are actually a result of burst modulation and all other spikes
might be essentially noise. The efficiency of short spike intervals relative to connectivity
has also been demonstrated from retinal inputs to the thalamus (Usrey et al., 1998)
suggesting the importance of bursts as a signal throughout the visual pathway.
90
Information-theoretic approaches have demonstrated the efficiency of bursts in the LGN
(Reinagal et al., 1999) and visual cortex (Reich et al., 2000). Reich et al. (2000) also
found that the bursts were disproportionately influential to the receptive field properties
of neurons. What might make bursts most suggestive as the fundamental signal are their
reliability from trial-to-trial (Victor et al., 1998).
The low probability of neurotransmitter release along with the high threshold in
the postsynaptic neuron makes it highly unlikely that a single spike will result in a post
synaptic spike. How do two neurons then synchronize within 2ms? They manipulate
both of these characteristics using two different properties of cortical networks. The first
is the aforementioned intrinsic bursting (Gray and McCormick, 1996) that essentially
ensures neurotransmitter release (Lisman, 1997). The second is a result of all the isolated
(or non-burst) spikes found in the lower resolutions of the ISI. Although these spikes
have a low probability of resulting in a postsynaptic spike, there are thousands of
connections so that they will still be passed on across cortical layers, but with a large
amount of variability (Shadlen and Newsome, 1998). These isolated spikes have very
broad tuning (Cattaneo et al., 1981a,b) suggesting they are equally represented across a
large population of neurons. Because these spikes activate a large portion of connections
with a high amount of variability, they result in chaos or an activation of a large amount
of excitatory and inhibitory connections (Shadlen and Newsome, 1998). What this chaos
manipulates is the threshold in the postsynaptic neuron by carefully balancing the
excitation and inhibition. The chaos keeps the postsynaptic potential close to threshold,
but below saturation (Bell et al., 1995).
91
When the postsynaptic neuron is closer to threshold, the neuronal time constant is
reduced and increases the precision of the synchronization between the two neurons. The
chaos can be thought of as background activity and Koch et al. (1996) shows that the
time constant reduces as the background level increases. This is not the background level
typically known as the maintained discharge (present during absence of stimulation), but
rather a subthreshold background level that only becomes evident when stimulation
occurs over the broad range of activation of the isolated spikes.
Functional connectivity and synergy
In order to have synergy, the neural response must contain information in the
form of correlation or synchrony that is not already represented in the individual
responses of the neurons. This can occur when connectivity between neurons modulates
while the individual firing rates remain constant. In the auditory cortex, Frostig et al.
(1983) found that in some cases, there was very little change in presynaptic firing rate
after a connectivity change. This was again demonstrated in the frontal cortex (Vaadia et
al., 1995) where the dynamics of correlation varied between two behaviors while the
firing rate remained the same. We showed that the connectivity between two neurons
continued to modulate while the firing remained nearly constant (near the peak). We also
demonstrated that this could provide as much as a 120% increase in distance between
responses to enhance stimulus discrimination.
We have already discussed how the connectivity is modulated with bursting
behavior and chaos, but have not discussed how the connectivity is modulated relative to
the stimulus properties. By analyzing the temporal dynamics of the connectivity, we
92
were able to gain some insight into how this occurs. The first aspect of functional
connectivity that we observed was that inhibition plays a major role. We found that the
very narrow tuning of connectivity relative to orientation and spatial frequency occurred
only after 15 ms. This time scale is in the same range as the temporal dynamics of
average rate tuning (Volgushev et al., 1995; Ringach et al., 1997), suggesting that the
inhibition we see is a result of feedback interactions. The inhibition may also play a role
in reducing the burst length (Debusk et al., 1997) and explaining why the inhibition for
connectivity/synchrony is more dramatic than for average firing rate.
Contrast has been shown to play a role in normalizing responses in the case of
average firing rate studies (Bonds, 1991; Heeger 1992). Contrast might also modulate
the spatial localization of responses (Sceniak et al., 1999). In our study, we show that
contrast modulates the adaptation (or decay) of connectivity during the stimulus period.
Bonds (1991) showed changes in overall firing rate during the stimulus period, but these
were subtle and in the form of both increases and decreases. The adaptation we find is
much more severe and always shows a decline in the connectivity.
How exactly does contrast modulate this adaptation? This relates back to the
discussion on how chaos modulates the temporal precision of synchronization. Contrast
modulates both bursts and isolated spikes (Cattaneo et al., 1981a,b), and has been shown
to have no effect on burst length modulation (Debusk et al., 1997). The burst rate and
burst length modulation control the initial strength of connectivity between the neurons,
but the isolated spike rate causes the differences in the time constant of this adaptation.
At lower contrasts, the background level (the isolated spikes) will be lower and therefore
the neuron's time constant will be longer (Koch et al., 1996) leading to less precision in
93
the connectivity. Because the precision is reduced, the connectivity essentially ‘falls
apart’ much faster at the lower contrast. So overall, the cortex moves from a short,
distributed, and imprecise pattern of code to a long, localized, and precise pattern as the
contrast is increased.
Orientation discrimination
Psychophysical studies have found orientation discriminations as fine as 10-20' of
arc in untrained human observers (Westheimer, 1981). This is substantially greater than
what would be expected when considering physiological evidence of the most highly
tuned neurons in primates (half-width at half-height of 4 degrees), and the performance is
even better than expected considering the resolution of retinal sampling in humans
(hyperacuities) (Wetsheimer, 1981).
Psychologists have proposed population encoding as a solution to the discrepancy
between the findings. The population schemes are not always so clear in terms of
biological mechanism, with vector summation as the most popular solution (Pouget et al.,
2000). Our results demonstrate how ISI characteristics work together both locally
(bursting) and globally (chaos and synchronization) to encode and transmit information
across populations of neurons. There is information as a result of these fast synaptic
modulations of connectivity that can provide significant contributions to orientation
discriminations. In the present study, we provide clues into some of the temporal
characteristics of this framework, and the next logical step is to examine the spatial
characteristics.
94
CHAPTER III
FUTURE EXPLORATIONS
Introduction
Our recordings of pairs of neurons in the visual cortex of the cat and the analysis
of their spike trains with information theory and cross correlation have shown us that
neurons that show strong connectivity can result in synergy to improve fine
discrimination of orientation. The principal results can be summarized as:
• Discrimination is best at the temporal resolution that matches the peak of bursting
activity in the ISI histogram.
• Incorporating the discharge history that matches the timing of connectivity results in
an advantage and significant cooperation for fine discriminations.
• The modulation of connectivity is a result of excitatory and inhibitory interactions,
and the modulation of connectivity with respect to orientation demonstrates the
contributions of bursts and burst length modulation to synaptic coupling.
The analysis of the temporal dynamics of connectivity has shown us fast modulations of
connectivity that occur within the first 50 ms of the response. The assumptions our
analysis make require a temporal resolution of 2-5 ms and knowledge of the interaction
between neurons within 2-5 ms. The next logical step is to extend this analysis to larger
populations of neurons and examine how the cortex might encode multi-dimensional
visual information under these conditions to form perceptions and perform actions within
100’s of milliseconds. We would also like to take a closer look at data on a trial-by-trial
95
basis to discover what properties of the code are repeated across a population of cortical
cells and therefore more reliable.
The difficulties of obtaining larger population recordings found in our earlier
experiments can be alleviated by using a 10 by 10 or 5 by 5 microelectrode array rather
than a single multi-unit electrode. The 100-microelectrode array should allow us to make
simultaneous recordings of as many as 50 neurons in striate cortex (R.A. Normann, pers.
Comm). The larger number of neurons will allow us to examine the spatiotemporal
patterns that vary according to variation in stimulus features. There is a general
consensus that visual processing must take advantage of populations of neurons
regardless of the theory of the actual encoding characteristics (Pouget et al., 2000; Milton
and Mackey, 2000; Doetsch, 2000; Nadasdy, 2000). The only way to test what
repeatable features occur in neural responses and which features are relevant to the
encoding is to examine larger portions of the cortex simultaneously.
Cortical Function Theory
We believe that cortical encoding needs to be in some ways distributed, but that
cortical architecture (Hubel and Weisel, 1962; DeAngelis et al. 1999) and physiology
(Baddeley et al., 1997; Vinje and Gallant, 2000) suggest that some processing might be
divided among smaller groups of neurons to perform specialized functions. Our theory
from the results in our previous experiments suggest that breaking down our
simultaneous recordings into the proper groups by examining their connectivity will
demonstrate how subgroups cooperate to perform finer encoding of stimulus features.
96
Over the last few years, the interest in temporal coding with bursts, latency, and
spatiotemporal patterns has grown significantly in vision (Bair, 1999) and auditory
(Grothe and Klump, 2000) studies. One of the theories against neuronal organization on
time scale less than 10 ms is the unreliability of the synapse (Shadlen and Newsome,
1994). We believe bursting is a mechanism that allows the cortex to be able to exhibit
the cooperative behavior reliably because of temporal integration (Snider et al., 1998) and
synaptic facilitation (Lisman, 1997). The inefficiency of the synapse results in making
bursting the influential signal for inducing an event in the postsynaptic neuron. We
believe that bursts are the fundamental signal in neural encoding and that the
spatiotemporal connectivity represents the neural code. The results of Cattaneo et al.
(1981a,b) demonstrated that over the tuning range of orientation and spatial frequency,
the average rate of those spikes not contained in bursts remains constant, implying the
rate tuning characteristics are actually a result of the burst modulation.
Our previous results (DeBusk et al., 1997) also demonstrated that stimulus
orientation influences the length of bursts, which in turn has later been shown to result in
more efficient functional connectivity (Snider et al., 1998). We believe this is also
demonstrated in the connectivity tuning demonstrated in the current study (see Functional
Connectivity in Chapter II). The efficiency of bursting relative to connectivity has also
been demonstrated from retinal inputs to the thalamus (Usrey et al., 1998) and in the
LGN with information-theoretic methods (Reinagel et al., 1999) suggesting the
importance of bursts as a signal throughout the visual pathway. What might make
bursting most suggestive as the fundamental signal is its reliability for trial to trial (Victor
et al., 1998; Guido and Sherman, 1998).
97
The other property of neural signals revealed by our analysis of pairs of cells was
the contribution of the isolated (or non-burst) spikes to the strength of synaptic coupling.
By modulating contrast and examining the temporal dynamics of the connectivity, we
discovered that the connectivity decays and that this occurs on a faster time scale for
lower contrasts. We believe this is a result of the decrease in the average isolated spike
rate that is only modulated by changes in contrast. The average isolated spike rate only
varies significantly for orientation and spatial frequency when outside their tuning ranges.
Shadlen and Newsome (1998) have postulated that variability (which is most apparent in
the isolated spikes) arises from the balance between excitation and inhibition across the
massive synaptic connections between neurons. This implies a constant level of activity
across a variable population of neurons that provides a background level to reduce the
time constant of the neuron and increase the temporal precision of coupling (Koch et al.,
1996). This has also been demonstrated with theoretical analysis of chaos and its effect
on thresholding and synchronization (Bell et al., 1995; Van Vreeswijk and Sompolinsky,
1996; Hansel, 1996; Karbowski and Kopell, 2000).
We examined latency modulation with respect to stimulus features and found
results consistent with Gawne et al. (1996) and Reich et al. (2001b). We believe that the
modulation of latency suggests that the brain incorporates dynamic spatiotemporal
encoding across populations of neurons. We find latency varies relative to variation of
contrast and orientation, but not for spatial frequency. Latency is also modulated for
sound intensity (regardless of location) in auditory cortex (Heil, 1997) and for motion in
area MT (Lisberger and Movshon, 1999). This all supports a dynamic network
incorporating delays that scale, rotate, and translate to provide invariant representation
98
(Hopfield, 1995; Schallkoff, 1997) of spatial frequency (in striate cortex) and sound
frequency (in auditory cortex).
In order to demonstrate that invariance is achieved with delays, we need to focus
on the timing between subgroups and between the individual neurons to reveal how the
cortex can take advantage of this additional dimension in the neural code. Our analysis
can reveal the precision of these temporal relationships and whether they encode features
alone or are required for faster or invariant discrimination. A lot of the properties
discovered from repeated single-unit recordings have not been shown across populations
and large populations of simultaneous recordings should provide some insight into which
of these properties are relevant to the neural code.
In summary, our theory of cortical function is that neurons use firing rate in a
distributed and variable manner to “turn on” certain subgroups to perform more complex
computations on stimulus features. This serves to raise the membrane potential closer to
threshold (Bell et al., 1995) and increase the temporal precision of synaptic coupling
(Koch et al., 1996). This portion of the response is transmitted in the manner that
Shadlen and Newsome (1998) describe with integration. As the firing rate across regions
becomes large enough, signals are represented in longer (DeBusk et al., 1997) and more
efficient bursts (Snider et al., 1998). The bursts overcome synaptic inefficiency (Lisman,
1997) and allow for fast and accurate progression of a signal through a subgroup. This
increased connectivity is highly tuned among subgroups and allows sharper encoding of a
stimulus feature over the range the subgroup covers. In order to encode the many
dimensions of a visual scene, the cortex can take advantage of space and time. It may be
that some features are more highly distributed while others are encoded along columns
99
and clusters. We also believe that “choosing” the right subgroup is key in encoding the
features. In other words, several features are not apparent in the entire group, but very
apparent in the constitution of their associated and sometimes overlapping subgroups.
The mechanism of transmission of this portion of the code is coincidence detection, but
coincident bursts in addition to the coincidence spikes described by Abeles (1991).
Because our analysis techniques extract connectivity information rather than rate
information, we believe they more strongly resemble the network properties and
dynamics that are repeated across trials.
Abeles (1991) has also proposed that the brain combines the mechanisms of
integration, synchronization, and bursts, but as independent mechanisms. In his theory,
synchronization through one chain leads to bursting behavior in another. He also
proposes that synchronized assemblies occur for local transmission of information, while
an integrated rate code can be used for long-range transmission. The theory we propose
is derived from the idea of assemblies proposed by Hebb (1949) and our representation is
similar to the synaptic modulations described by von der Malsburg (1981), but we also
believe chaotic behavior has an important role in the neural code.
Multidimensional Data
The first step is to present the proper stimuli to obtain recordings that will reveal
population dynamics. We will continue to use sinusoidal gratings and vary the
orientation, spatial frequency, contrast, and temporal frequency. The difficulty in
population recordings will be in choosing the receptive field and the range to cover for
the aforementioned features. In some cases, maximizing the number of active neurons is
100
ideal. Our results have also shown that cooperative properties will only be apparent
when neurons reach their highest levels of activity so it might be beneficial to sacrifice
the number of neurons for the maximum number of highly active neurons (firing rate).
We may discover that the best approach is to run the protocol under several different
parameter options to excite different populations of receptive fields.
We would also like to collect information using a light bar to determine an idea of
the location of specific receptive fields for individual electrodes or regions of the
microelectrode array. Once we have the receptive fields plotted, we would like to either
make rough estimations of the spatiotemporal characterisitics (orientation, spatial
frequency, and temporal frequency) with the light bar or use short runs with a sinewave
grating to determine the tuning for each electrode.
Once the receptive field and feature ranges are determined, we would like to
record signals under all the possible variations of these stimuli and cover relatively small
steps across the range of all the features. Depending on the ranges of each feature, this
might require a substantial number of possible stimuli (i.e., # orientations x # spatial
frequencies x # of temporal frequencies x # of contrasts). To compound the problem, we
want to present these stimuli as many times as possible to allow our analysis to cover the
highest dimensionality possible.
Cortical Clustering
The first step in analyzing the simultaneous recordings will be to determine the
relationships between all of the neurons. In previous experiments, this has been
accomplished using cross correlation analysis (Snider et al., 1998) and with type analysis
101
(see Chapter II). These methods require analysis of the neurons in pairs, which with as
many as 50 neurons would have 1225 different pairs. We can break down the population
into smaller groups several ways. One method is calculating the various average firing
rate tuning curves of the stimulus features for each neuron. Neurons with similar tuning
for stimulus features tend to be clustered or found organized in columns (DeAngelis et
al., 1999). Analysis of the PSTHs for each neuron can also reveal latency properties to
determine possible parallel and serial connected neurons. The gravitational clustering
method developed by Gerstein et al. (1985) is another approach we will use to determine
what neurons are related by synchrony and connectivity. Another option might be
modifying the clustering method based on the complexity measure described by Sporns et
al. (2000) for imaging analysis so that it can be used for multi-unit spike trains. This
would more closely describe the interactions we are searching for since the measure they
use of dependence (Tononi et al., 1994) resembles the type analysis method we have used
for analysis of pairs.
Once we have an idea how the groups of neurons are clustered, we can examine
the smaller groups in pairs to determine the detailed connectivity (i.e., shared input vs.
direct interaction). This will be done with cross-correlation by separating relationships
into common input and directly connected with the “effective connectivity” measure
(Aertsen et al., 1989). With the connected subgroups determined, we can extend our type
analysis research to determine the relationship between synergy and population size.
Analysis of the population relationships (i.e., groupings) will also be key into our
understanding of how the cortex encodes several stimulus features simultaneously. Not
only is determining the subgroups or functional circuits necessary for the type analysis
102
and synergy analysis, but the results in themselves may be even more significant. The
step-by-step procedure of tracing the circuitry can reveal any variations and repeatability
in spatiotemporal patterns. Anatomical analysis of the tissue is another procedure that
can be used to verify some of the spatial relationships.
Spatiotemporal Connectivity
Many of the spike train techniques require several repetitions of a stimulus and
we would like to examine what features of the simultaneous recordings are repeated for
each repetition. One way we can gain some insight into which spike trains to focus on
and under which stimuli, is to produce a “network” diagram of our recordings. The
diagram can display the strength of connectivity between neurons determined by the
slope of the type analysis dependency distance (represented over a color scale). A
diagram can than be produced at different points in time starting with the earliest
response time and continuing over small increments. The diagram can even be
represented as a video to demonstrate how a signal “moves” across the population under
variations in the stimulus.
This diagram of course is produced by probabilities determined from the
repetitions so examination of the interactions of the individual spike trains is necessary to
demonstrate if these spatiotemporal patterns of cooperativity are repeated across these
repetitions. This can be done with either modified spike trains and searching algorithms
similar to Tetko and Villa (2001a) or direct examination of the individual spike trains.
The latter approach might seem tedious with a large population of neurons across a time
of two seconds and for hundreds of repetitions, but the transition points (changes in
103
slope) in the dependency plot will reduce the window of time to examine. One purpose is
to examine if temporal differences occur between the activity of neurons and its precision
from trial to trial, and the other purpose is to see if connectivity between specific neurons
occurs consistently across trials. The results will also provide some insight into the
spatial precision of the connectivity patterns (i.e., a spatial resolution coarser than single
neurons).
The idea of spatiotemporal patterns is not new idea (Hebb, 1949), but finding the
appropriate representation of these patterns has been difficult. The goals of our future
experiments will be to (1) demonstrate that neurons can cooperate in assemblies to
encode with greater efficiency, (2) provide evidence on how cortex can encode at least
two stimulus features simultaneously within the network (using several stimulus
combinations that lead to similar rates in individual neurons), (3) demonstrate that the
network timing has significance especially with regard to invariance, and (4) further
support these theories by demonstrating their reliability across trials.
Our information measure (Johnson et al., 2001) is not based on assumptions of
rate or temporal coding. We choose a simple stimulus because it provides clear feature
properties like orientation and spatial frequency so that we can demonstrate how these
features are encoded and to what precision. It also provides a simple approach to
demonstrate encoding of multiple features simultaneously. Our network diagram (and
dependency in general) is a unique way of displaying the level of influence between
neurons and how it varies over time, and we believe the approach will be more reliable
across trials and will more accurately represent the significant properties of neural
activity.
104
REFERENCES
Abeles M. and Gerstein G. L. Detecting Spatiotemporal Firing Patterns Among Simultaneously Recorded Single Neurons. J. Neurophysiol., 60:909-924, 1988. Abeles M. and Goldstein Jr. M. H. Multispike Train Analysis. Proc. IEEE, 65:762-773, 1977. Abeles M. Corticonics: Neural Circuits of the Cerebral Cortex. Cambridge University Pressl, New York, 1991. Abeles M. Role of cortical neurons: integrator or coincidence detector? Isr. J. Med. Sci., 18:83-92, 1982. Adrian E.D. and Zotterman, Y. The impulses produced by sensory nerve endings, Pt. 3 Impulses set up by touch and pressure. J. Physiol. (Lond.), 61:465-493, 1926. Aertsen A. M. H. J. and Gerstein G. L. Evaluation of Neuronal Connectivity: Sensitivity of Cross-Correlation. Brain Res., 340:341-354, 1985. Aertsen A. M. H. J., Gerstein G. L., Habib M. K., and Palm G. Dynamics of Neuronal Firing Correlation: Modulation of “Effective Connectivity”. J. Neurophysiol., 61:900-917, 1989. Alonso J-M. and Martinez L. M. Functional connectivity between simple cells and complex cells in cat striate cortex. Nature Neurosci., 1:395-403, 1998. Baddeley R., Abbott L. F., Booth M. C. A., Sengpiel F., Freeman T., Wakeman E. A., and Rolls E. T. Responses of neurons in primary and inferior temporal visual cortices to natural scenes. Proc. R. Soc. Lond. B, 264:1775-1783, 1997. Bair W. Spike Timing in the mammalian visual system. Curr. Op. Neurobiol., 9:447-453, 1999. Baker S. N. and Gerstein G. L. Improvements to the sensitivity of gravitational clustering for multiple neuron recordings. Neural Comp., 12:2597-2620, 2000. Barlow H. B. Single units and sensation: a neuron doctrine for perceptual psychology? Perception, 1:371-394, 1972. Bell A., Mainen Z., Tsodyks M., and Sejnowski T. “Balancing” of conductances may explain irregular cortical spiking. Institute for Neural Computation, 1995 (Technical Report INC-9502).
105
Bernasconi C. and Konig P. On the directionality of cortical interactions studied by structural analysis of electrophysiological recordings. Biol. Cybern., 81:199-210, 1999. Bonds A. B. Role of Inhibition in the specification of orientation selectivity of cells in the cat striate cortex. Visual Neuroscience, 2:41-55, 1989. Bonds A.B. Temporal dynamics of contrast gain in single cells of the cat striate cortex. Vis. Neurosci., 6:239-255, 1991. Bowman D. M., Eggermont J. J., and Smith G. M. Effect of Stimulation on Burst Firing in Cat Primary Auditory Cortex. J. Neurophysiol., 74:1841-1855, 1995. Brenner N., Strong S. P., Koberle R., Bialek W., and de Ruyter van Steveninck R. R. Synergy in a Neural Code. Neural Comp. 12:1531-1552, 2000. Brosch M. and Schreiner C. E. Correlations between neural discharges are related to receptive field properties in cat primary auditory cortex. Euro. J. Neurosci., 11:3517-3520, 1999. Cattaneo A., Maffei L., and Morrone C. Patterns in the discharge of simple and complex visual cortical cells. Proc. R. Soc. Lond., 212:279-297, 1981. Cattaneo A., Maffei L., and Morrone C. Two firing patterns in the discharge of complex cells encoding different attributes of the visual stimulus. Exp. Brain Res., 43:115-118, 1981. Christensen T. A., Pawlowski V. M., Lei H. and Hildebrand J. G. Multi-unit recordings reveal context-dependent modulation of synchrony in odor-specific neural ensembles. Nature Neurosci., 3:927-931, 2000. Cover T. M. and Thomas J. A. Elements of Information Theory. John Wiley & Sons, Inc., New York, 1991. Creutzfeldt O. D., Kuhnt U., and Benevento L. A. An Intracellular Analysis of Visual Cortical Neurones to Moving Stimuli: Responses in a Co-Operative Neuronal Network. Exp. Brain Res., 21:251-274, 1974. Creutzfeldt O., Innocenti G.M., and Brooks D. Vertical organization in the visual cortex (area 17) in the cat. Exp. Brain Res., 21:315-36, 1974. Dan Y., Alonso J-M., Usrey W. M., and Reid R. C. Coding of visual information by precisely correlated spikes in the lateral geniculate nucleus. Nature Neurosci., 1:501-507, 1998. Dayhoff J.E. and Gerstein G.L. Favored Patterns in Spike Trains. I. Detection. J. Neurophysiol., 49:1334-1348, 1983a.
106
Dayhoff J.E. and Gerstein G.L. Favored Patterns in Spike Trains. II. Application. J. Neurophysiol., 49:1349-1363, 1983b. de Ruyter van Steveninck R. R., Lewen G. D., Strong S. P., Koberle R., Bialek W. Reproducibility and Variability in Neural Spike Trains. Science, 275:1805-1808, 1997. DeAngelis G. C., Ghose G. M., Ohzawa I., and Freeman R. D. Functional Micro-Organization of Primary Visual Cortex: Receptive Field Analysis of Nearby Neurons. J. Neurosci., 19:4046-4064, 1999. DeBusk B. C., DeBruyn E. J., Snider R. K., Kabara J. F., and Bonds A. B. Stimulus-Dependent Modulation of Spike Burst Length in Cat Striate Cortical Cells. J. Neurophysiol., 78: 199-213, 1997. Diesman M., Gewaltig M., and Aertsen A. Stable propagation of synchronous spiking in cortical neural networks. Nature, 402: 529-531, 1999. Dimitrov A. G. and Miller J. P. Neural coding and decoding: communication channels and quantization. Network: Comput. Neural Syst., submitted 2001. Doetsch G. S. Patterns in the brain: Neuronal population coding in the somatosensory system. Physiol. Behav., 69:187-201, 2000. Efron B. and Tibshirani R. J. An Introduction to the Bootstrap. Chapman & Hall, New York, 1993. Eggermont J. J. and Smith G. M. Burst-firing sharpens frequency-tuning in primary auditory cortex. Neuroreport, 7:753-757, 1996. Eggermont J. J. Azimuth coding in primary auditory cortex of the cat. II. Relative latency and interspike interval representation. J. Neurophysiol., 80:2151-2161, 1998. Eguia M. C., Rabinovich M. I., and Abarbanel H. D. I. Information transmission and recovery in neural communications channels. Phys. Rev. E, 62:7111-7122, 2000. Ernst U., Pawelzik K., and Geisel T. Delay-induced multistable synchronization of biological oscillators. Physical Rev. E, 57:2150-2162, 1998. Ernst U., Pawelzik K., and Geisel T. Synchronization Induced by Temporal Delays in Pulse-Coupled Oscillators. Physical Rev. Letters, 74:1570-1573, 1995. Fagan R. M. Information measures: Statistical confidence limits and inference. J. Th. Biol., 73:61–79, 1978.
107
Farid H. and Adelson E. H. Synchrony does not promote grouping in temporally structured displays. Nature Neurosci., 4:875-876, 2001. Fitzpatrick D. Cortical imaging: Capturing the moment. Curr. Op. Biol, 10:R187-R190, 2000. Fries P., Neuenschwander S., Engel A. K., Goebel R., and Singer W. Rapid feature selective neuronal synchronization through correlated latency shifting. Nat. Neurosci., 4:194-200, 2001. Friston K.J. and Buchel C. Attentional modulation of effective connectivity from V2 to V5/MT in humans. Proc. Nat. Acad. Sci. USA, 97:7591-7596, 2000. Frostig R.D., Gottlieb Y., Vaadia E., and Abeles M. The Effects of Stimuli on the Activity and Functional Connectivity of Local Neuronal Groups in the Cat Auditory Cortex. Brain Res., 272:211-221, 1983. Gabbiani F. and Metzner W. Encoding and Processing of Sensory Information in Neuronal Spike Trains. J. Exp. Biol., 202:1267-1279, 1999. Gawne T.J., Kjaer T.W., Hertz J.A., and Richmond B.J. Adjacent visual cortical complex cells share about 20% of their stimulus-related information. Cereb. Cortex, 6:482-489, 1996a. Gawne T. J., Kjaer T. W., and Richmond B. J. Latency: Another Potential Code for Feature Binding in Striate Cortex. J. Neurophysiol., 76:1356-1360, 1996b. Gershon E. D., Weiner M. C., Latham P. E., and Richmond B. J. Coding Strategies in Monkey V1 and Inferior Temporal Cortices. J. Neurophysiol., 79:1135-1144, 1998. Gerstein G. L. and Perkel D. H. Mutual Temporal Relationships Among Neuronal Spike Trains: Statistical Techniques for Display and Analysis. Biophysical J., 12:453-473, 1972. Gerstein G. L. and Perkel D. H. Representation of Cooperative Firing Activity Among Simultaneously Recorded Neurons. J. Neurophysiol., 54:1513-1528, 1985. Gerstein G. L., Perkel D. H., and Dayhoff J. E. Cooperative Firing Activity in Simultaneously Recorded Populations of Neurons: Detection and Measurement. J. Neurosci., 5:881-889, 1985. Ghose G. M. and Maunsell J. Specialized Representations in Visual Cortex: A Role for Binding? Neuron, 24:79-85, 1999. (111-125 References for binding papers)
108
Gray C. M., Koenig P., Engel A. K., and Singer W. Oscillatory responses in cat visual cortex exhibit inter-columnar synchronization which reflects global stimulus properties. Nature, 338:334-337, 1989. Gray C. M., Maldonado P. E., Wilson M., and McNaughton B. Tetrodes markedly improve the reliability and yield of multiple single-unit isolation from multi-unit recordings in cat striate cortex. J. Neurosci. Methods, 63:43-54, 1995. Gray C. M. and McCormick D. A. Chattering Cells: Superficial Pyramidal Neurons Contributing to the Generation of Synchronous Oscillations in the Visual Cortex. Science, 274:109-113, 1996. Grinvald A. Optical imaging of architecture and function in the living brain sheds new light on cortical mechanisms underlying visual perception. Brain Topogr., 5:71-75, 1992. Grothe B. and Klump G. M. Temporal processing in sensory systems. Curr. Op. Neurobiol., 10:467-473, 2000. Guido W. and Sherman S.M. Response latencies of cells in the cat’s lateral geniculate nucleus are less variable during burst than tonic firing. Vis. Neurosci., 15:231-237, 1998. Hansel D. Synchronized Chaos in Local Cortical Circuits. Int. J. Neural Syst., 7:403-415, 1996. Hebb, D. O. The Organization of Behavior: a Neuropsychological Theory. New York: Wiley, 1949. Heeger D.J. Normalization of cell responses in cat striate cortex. Vis. Neurosci., 9:181-197, 1992. Heil P. Auditory Cortical Onset Responses Revisited. I. First-Spike Timing. J. Neurophysiol., 77:2616-2641, 1997. Heller J., Hertz J. A., Kjaer T. W., and Richmond B. J. Information Flow and Temporal Coding in Primate Pattern Vision. J. Comp. Neurosci., 2:175-193, 1995. Hopfield J. J. Pattern recognition computation using action potential timing for stimulus representation. Nature, 376:33-36, 1995. Hubel D. H. and Wiesel T. N. Receptive Fields, Binocular Interaction and Functional Architecture in the Cat’s Visual Cortex. J. Physiol., 160:106-154, 1962. Johnson D. H., Gruner C. M., Baggerly K., and Seshagiri C. Information-Theoretic Analysis of Neural Coding. J. Comp. Neurosci., 10:47-69, 2001.
109
Karbowski J. and Kopell N. Multispikes and Synchronization in a large Neural Network with Temporal Delays. Neural Comp., 12:1573-1606, 2000. Koch C., Rapp M., and Segev I. A brief history of time (constants). Cereb. Cortex, 6:93-101, 1996. Kruger J. and Bach M. Simultaneous Recording with 30 Microelectrodes in Monkey Visual Cortex. Exp. Brain Res., 41:191-194, 1981. Laubach M., Wessberg J. and Nicolelis M. A. L. Cortical ensemble activity increasingly predicts behaviour outcomes during learning of a motor task. Nature, 405:567-571, 2000. Levick W. Another tungsten microelectrode. Med. Biol. Eng. 10:510-515, 1972. Lindsey B. G., Hernandez Y. M., Morris K. F., Shannon R., and Gerstein G. L. Respiratory related neural assemblies in the brain stem midline. J. Neurophysiol., 67: 905-922, 1992. Lindsey B. G., Hernandez Y. M., Morris K. F., Shannon R., and Gerstein G. L. Dynamic reconfiguration of brain stem neural assemblies: respiratory phase-dependent synchrony versus modulation of firing rates. J. Neurophysiol., 67:923-930, 1992. Lindsey B. G., Morris K. F., Shannon R., and Gerstein G. L. Repeated patterns of distributed synchrony in neuronal assemblies. J. Neurophysiol., 78:1714-1719, 1997. Lindsey B.G., Segers L. S., Morris K. F., Hernandez Y. M., Saporta S., and Shannon R. Distributed actions and dynamic associations in respiratory-related neuronal assemblies of the ventrolateral medulla and brain stem midline: evidence from spike train analysis. J. Neurophysiol., 72:1830-1851, 1994. Lisberger S. G. and Movshov J. A. Visual Motion Analysis for Pursuit Eye Movements in Area MT of Macaque Monkeys. J. Neurosci., 19:2224-2246, 1999. Lisman J. E. Bursts as a unit of neural information: making unreliable synapses reliable. Trends Neurosci., 20:38-43, 1997. Logothetis N. K., Pauls J., Augath M., Trinath T. and Oeltermann A. Neurophysiological investigation of the basis of the fMRI signal. Nature, 412:150-157, 2001. Maldonado P. E. and Gerstein G. L. Neuronal assembly dynamics in the rat auditory cortex during reorganization induced by intracortical microstimulation. Exp. Brain Res., 12:431-441, 1996. Maynard E. M., Hatsopoulos N. G., Ojakangas C. L., Acuna B. D., Sanes J. N., Normann R. A., and Donoghue J. P. Neuronal interactions improve cortical population coding of movement direction. J Neurosci., 19:8083-8093, 1999.
110
Mechler F., Victor J. D., Purpura K. P., and Shapley R. Robust Temporal Coding of Contrast by V1 Neurons for Transient But Not for Steady-State Stimuli. J. Neurosci., 18:6583-6598, 1998. Mehta A. D., Ulbert I., and Schroeder C. E. Intermodal selective attention in monkeys. I: distribution and timing of effects across visual areas. Cereb. Cortex, 10:343-358, 2000. Michalski A., Gerstein G. L., Czarkowska J., and Tarnecki R. Interactions Between Cat Striate Cortex Neurons. Exp. Brain Res., 51:97-107, 1983. Milton J. G. and Mackey M. C. Neural ensemble coding and statistical periodicity: Speculations on the operation of the mind’s eye. J. Physiol., 94:489-503, 2000. Morris K. F., Shannon R., and Lindsey B.G. Changes in cat medullary neurone firing rates and synchrony following induction of respiratory long-term facilitation. J. Physiol., 532:483-497, 2001. Morrone M.C., Burr D.C., and Maffei L. Functional implications of cross-orientation inhibition of cortical visual cells. I. Neurophysiological evidence. Proc R Soc Lond B Biol Sci., 216:335-354, 1982. Nadasdy Z. Spike sequences and their consequences. J. Physiol., 94:505-524, 2000. Nirenberg S., Carcieri S. M., Jacobs A. L., and Latham P. E. Retinal ganglion cells act largely as independent encoders. Nature, 411:698-701, 2001. Nordhausen C.T., Maynard E.M., and Normann R.A. Single unit recording capabilities of a 100 microelectrode array. Brain Res., 726:129-140, 1996. Palm, G., Aertsen, A. M. H. J., and Gerstein, G. L. On the significance of correlations among neuronal spike trains. Biol. Cyber. 59:1-11, 1988. Panzeri S. and Treves A. Analytical estimates of limited sampling biases in different information measures. Network, 7:87-107, 1996. Park J. and Lee C. Neural discharge coupled to saccade offset in the cat visual cortex. Neuroreport, 11:1661-1664, 2000. Pastor J., Lafon M., Trave-Massuyes L., Demonet J-F., Doyon B., and Celsis P. Information processing in large-scale cerebral networks: the causal connectivity approach. Biol. Cybern., 82:49-59, 2000. Perkel, D. H., Gerstein, G. L., and Moore, G. P. Neuronal Spike Trains and Stochastic Point Processes, I. The Single Spike Train. Biophys. J. 7:391-418, 1967.
111
Perkel, D. H., Gerstein, G. L., and Moore, G. P. Neuronal Spike Trains and Stochastic Point Processes, II. Simultaneous Spike Trains. Biophys. J. 7:419-440, 1967. Pouget A., Dayan P., and Zemel R. Information processing with population codes. Nature Rev. Neurosci. 1:125-132, 2000. Prank K., Gabbiani F., and Brabant G. Coding efficiency and information rates in transmembrane signaling. BioSystems, 55:15-22, 2000. Rabinovich M. I., Huerta R., Volkovskii A., Abarbanel H. D. I., Stopfer M., and Laurent G. Dynamical coding of sensory information with competitive networks. J. Physiol., 94:465-471, 2000. Raichle M.E. Behind the scenes of functional brain imaging: A historical and physiological perspective. Proc. Natl. Acad. Sci. USA, 95:765-772, 1998. Reich D. S., Mechler F., and Victor J. D. Formal and Attribute-Specific Information in Primary Visual Cortex. J. Neurophysiol., 85:305-318, 2001a. Reich D. S., Mechler F., and Victor J. D. Temporal Coding of Contrast in Primary Visual Cortex: When, What, and Why. J. Neurophysiol., 85:1039-1050, 2001b. Reich D. S., Mechler F., and Victor J. D. Independent and redundant information in nearby cortical neurons. Science, 294:2566-2568, 2001c. Reich D. S., Mechler F., Purpura K. P., and Victor J. D. Interspike Intervals, Receptive Fields, and Information Encoding in Primary Visual Cortex. J. Neurosci., 20:1964-1974, 2000. Reinagel P., Godwin D., Sherman S. M., and Koch C. Encoding of Visual Information by LGN Bursts. J. Neurophysiol., 81:2558-2569, 1999. Richmond B. J. and Optican L. M. Temporal Encoding of Two-Dimensional Patterns by Single Units in Primate Inferior Temporal Cortex. II. Quantification of Response Waveform. J. Neurophysiol., 57:147-161, 1987. Richmond B. J., Optican L. M., and Spitzer H. Temporal Encoding of Two-Dimensional Patterns by Single Units in Primate Primary Visual Cortex. I. Stimulus-Response Relations. J. Neurophysiol., 64:351-369, 1990. Rieke F., Warland D., de Ruyter van Steveninck R. R., and Bialek W. Spikes: Exploring the Neural Code, MIT Press, Cambridge, MA, 1997. Ringach D.L., Hawken M.J., and Shapley R. Dynamics of orientation tuning in macaque primary visual cortex. Nature, 387:281-284, 1997.
112
Rolls E. T., Treves A., and Tovee M. J. The representational capacity of the distributed encoding of information provided by populations of neurons in primate temporal visual cortex. Exp. Brain Res., 114:149-162, 1997. Rolls E. T., Treves A., Tovee M. J., and Panzeri S. Information in the Neuronal Representation of Individual Stimuli in the Primate Temporal Visual Cortex. J. Comp. Neurosci., 4:309-333, 1997. Sceniak M.P., Ringach D.L., Hawken M.J., and Shapley R. Contrast’s effect on spatial summation by macaque V1 neurons. Nat. Neurosci., 2:733-739, 1999. Schalkoff R. J. Artificial Neural Networks. McGraw-Hill, New York, 1997. Scott S.H., Gribbie P.L., Graham K.M., and Cabel D.W. Dissociation between hand motion and population vectors from neural activity in motor cortex. Nature, 413:161-165, 2001. Segev I. and Schneidman E. Axons as computing devices: Basic insights gained from models. J. Physiol. (Paris), 93:263-270, 1999. Segundo J.P., Bullock T.H., Moore G.P., and Stensaas L.J. Sensitivity of Neurones in Applysia to Temporal Pattern of Arriving Impulses. J. Exp Biol., 40:643, 1963. Shadlen M. N. and Movshon, J. A. Synchrony unbound: a critical evaluation of the temporal binding hypothesis. Neuron, 24:67-77, 1999. Shadlen M.N. and Newsome W.T. Noise, neural codes and cortical organization. Curr. Op. Neurobiol., 4:569-579, 1994. Shadlen M. N. and Newsome W. T. The Variable Discharge of Cortical Neurons: Implications for Connectivity, Computation, and Information Coding. J. Neurosci., 18:3870-3896, 1998. Shannon C. E. A Mathematical Theory of Communication. Bell Systems Tech. J., 27:379-423, 623-656, 1948. Sherrington, C. S. Man on His Nature. London: Cambridge University Press, 1941. Sillito A.M. The contribution of inhibitory mechanisms to the receptive field properties of neurones in the striate cortex of the cat. J. Physiol. (London), 250:305-29, 1975. Sillito A.M. and Jones H.E. Context-dependent interactions and visual processing in V1. J. Physiol. (Paris), 90:205-209, 1996. Singer W. and Gray C. M. Visual feature integration and the temporal correlation hypothesis. Annu. Rev. Neurosci., 18:555-586, 1995.
113
Skottun B. C., De Valois R. L., Grosof D. H., Movshon J. A., Albrecht D. G., and Bonds A. B. Classifying Simple and Complex Cells on the Basis of Response Modulation. Vision Res., 31:1079-1086, 1991. Snider R. K. and Bonds A. B. Classification of non-stationary neural signals. J. Neurosci. Methods, 84:155-166, 1998. Snider R. K., Kabara J. F., Roig B. R., and Bonds A. B. Burst Firing and Modulation of Functional Connectivity in Cat Striate Cortex. J. Neurophysiol., 80:730-744, 1998. Sporns O., Tononi G., and Edelman G. M. Connectivity and complexity: the relationship between neuroanatomy and brain dynamics. Neural Networks, 13:909-922, 2000. Strangman G. Detecting synchronous cell assemblies with limited data and overlapping assemblies. Neural Comp., 9:51-76, 1997. Strehler B.L. and Lestienne R. Evidence on precise time-coded symbols and memory of patterns in monkey cortical neuronal spike trains. Proc. Nat. Acad. Sci. USA, 83:9812-9816, 1986. Tetko I. V. and Villa A. E. P. A pattern grouping algorithm for analysis of spatiotemporal patterns in neuronal spike trains. 1. Detection of repeated patterns. J. Neurosci. Methods, 105:1-14, 2001. Tetko I. V. and Villa A. E. P. A pattern grouping algorithm for analysis of spatiotemporal patterns in neuronal spike trains. 2. Application to simultaneous single unit recordings. J. Neurosci. Methods, 105:15-24, 2001. Tononi G., McIntosh A.R., Russell D.P., and Edelman G.M. Functional Clustering: Identifying Strongly Interactive Brain Regions in Neuroimaging Data. Neuroimage, 7:133-149, 1998. Tononi G., Sporns O., and Edelman G. M. A measure for brain complexity: Relating functional segregation and integration in the nervous system. Proc. Natl. Acad. Sci USA, 91:5033-5037, 1994. Toyama, K., Kimura, M., and Tanaka, K. Cross-Correlation Analysis of Interneuronal Connectivity in Cat Visual Cortex. J. Neurophysiol. 46:191-201, 1981. Toyama, K., Kimura, M., and Tanaka, K. Organization of Cat Visual Cortex as Investigated by Cross-Correlation Technique. J. Neurophysiol. 46:202-214, 1981. Toyama, K., Matsunami, K., Ohno, T., and Tokashiki, S. An Intracellular Study of Neuronal Organization in the Visual Cortex. Exp. Brain Res. 21:45-66, 1974.
114
Traub R.D. and Miles R. Neuronal Networks of the Hippocampus. New York: Cambridge University Press, 1991. Ts’o D.Y., Gilbert C.D., and Wiesel T.N. Relationships between horizontal interactions and functional architecture in cat striate cortex as revealed by cross-correlation analysis. J. Neurosci., 6:1160-1170, 1986. Usrey W.M., Reppas J.B., and Reid R.C. Paired-spike interactions and synaptic efficacy of retinal inputs to the thalamus. Nature, 395:384-387, 1998. Vaadia E., Haalman I., Abeles M., Bergman H., Prut Y., Slovin H., and Aertsen A. Dynamics of neuronal interactions in monkey cortex in relation to behavioural events. Nature, 373:515-518, 1995. van der Togt C., Lamme V.A.F., and Spekreijse H. Functional connectivity within the visual cortex of the rat shows state changes. Eur. J. Neurosci., 10:1490-1507, 1998. van Vreeswijk C. and Sompolinsky H. Chaos in Neuronal Networks with Balanced Excitatory and Inhibitory Activity. Science, 274:1724-1726, 1996. van Vreeswijk C. Partial synchronization in populations of pulse-coupled oscillators. Physical Rev. E, 54:5522-5537, 1996. Victor J. D. and Purpura K. P. Nature and Precision of Temporal Coding in Visual Cortex: A Metric-Space Analysis. J. Neurophysiol., 76:1310-1326, 1996. Victor J. D. How the brain uses time to represent and process visual information. Brain Research, 886:33-46, 2000. Victor J. D., Mechler F., Reich D. S., and Purpura K. P. Spatiotemporal origin of bursts and “reliable” spikes generated by neurons in V1. Soc. Neurosci. Abstr., 24:1258, 1998. Vinje W. E. and Gallant J. L. Sparse Coding and Decorrelation in Primary Visual Cortex During Natural Vision. Science, 287:1273-1276, 2000. Volgushev M., Vidyasagar T.R., and Pei X. Dynamics of the orientation tuning of postsynaptic potentials in the cat visual cortex. Vis. Neurosci., 12:621-628, 1995. von der Malsburg C. The correlation theory of brain function. Internal Report, Max-Planck Institute for Biophysical Chemistry. Goettingen, Germany, 1981. Warland D. K., Reinagel P., and Meister M. Decoding Visual Information From a Population of Retinal Ganglion Cells. J. Neurophysiol., 78:2336-2350, 1997.
115
Wessberg J., Stambaugh C. R., Krallk J. D., Beck P D., Laubach M., Chapin J. K., Kim J., Biggs S. J., Srinivasan M. A. and Nicolelis M. A. L. Real-time prediction of hand trajectory by ensembles of cortical neurons in primates. Nature, 408:361-365, 2000. Westheimer G. Visual hyperacuity. In: Progress in Sensory Physiology Vol. 1, edited by Autrum H., Ottoson D., Perl E. R., Schmidt R. F. Berlin: Springer Verlag, 1981. Wilson H. R. and Wilkinson F. Evolving concepts of spatial channels in vision: from independence to nonlinear interaction. Perception, 26:939-960, 1997. Wohler C. and Anlauf J. K. A time delay neural network algorithm for estimating image-pattern shape and motion. Image Vision Comput. J., 17:281-294, 1999. Wohler C. and Anlauf J. K. An Adaptable Time Delay Neural-Network Algorithm for Image Sequence Analysis. IEEE Trans. Neural Networks, 10:1531-1536, 1999. Wu J., Cohen L. B. and Falk C. X. Neuronal Activity During Different Behaviors in Aplysia: A Distributed Organization? Science, 263: 820-823, 1994. Zador A. Impact of Synaptic Unreliability on the Information Transmitted by Spiking Neurons. J. Neurophysiol., 79:1219-1229, 1998.
116