19th INTERNATIONAL CONGRESS ON ACOUSTICS

MADRID, 2-7 SEPTEMBER 2007

EVALUATION OF INTERAURAL TIME AND INTENSITY DIFFERENCES: A NEURAL MODEL

PACS: 43.64.Bt Bures, Zbynek1

1Dept. of Radioelectronics, FEE, Czech Technical University in Prague; Technicka 2, Praha 6, 16627, Czech Republic; buresz@fel.cvut.cz ABSTRACT According to the duplex theory of sound localization, two distinct neural mechanisms exist that evaluate the interaural disparities in time and intensity. In mammals, these basic binaural localization clues are detected and encoded by neurons found in the superior olivary complex. The paper presents a neural computational model of binaural interactions taking place in the medial and lateral superior olive. The evaluation of interaural time differences (ITD) employs detection of coincidence of spikes in the two auditory channels, taking advantage of the random distribution of spikes over the sound period. The interaural intensity difference (ILD) evaluation is based on subtraction of firing rates, making use of the excitatory and inhibitory inputs. The interaural disparities at the input of the model are encoded by variable spike rate at model output. The paper attempts, by means of modeling, to investigate the preconditions, capabilities and limitations of the assumed biological mechanism. The response of the model to basic sound stimuli is presented and discussed with respect to the latest physiological data. INTRODUCTION The ability of vertebrates to localize the position of a sound source relative to their head relies on two basic clues available to their ears: different times of arrival of the sound wave and different spectra at the ears. In mammals, this ability is supported by two pairs of neural nuclei that receive bilateral inputs, the medial (MSO) and lateral (LSO) superior olive. It is assumed that the former nucleus is sensitive predominantly to the interaural time differences (ITDs) while the latter one detects the intensity differences (ILDs) in the corresponding frequency bands. It follows from the physical properties of sound that ITD can be evaluated unambiguosly in the low frequency band only, whereas the ILD plays major role in the high frequency band. However, it is yet not known with confidence what neural mechanisms are involved in evaluation of these interaural disparities. The classic model of ITD detection, Jeffress’ delay line, predicted a structure that was found in owls but it was never reliably identified in mammals. Nevertheless, an agreement exists that the ITD evaluation is based on detection of coincidence of excitatory action potentials (AP) and that the ILD is assessed utilizing subtraction of firing rates. PHYSIOLOGICAL BACKGROUND The ILD and ITD pathways are shown schematically in Fig. 1. Cells of the MSO receive bilateral excitatory inputs from the spherical bushy cells in the anteroventral cochlear nucleus (AVCN), which in turn receive inputs directly from the auditory nerve (AN). According to the latest findings [7], the MSO cells are also the target of bilateral inhibitory projections which probably provide the necessary systematical time shift of the excitatory APs from the two auditory channels. For frequencies below ca. 2kHz, both the excitatory and the inhibitory inputs are precisely phase-locked to the sound wave; this is the precondition of correct functioning of the ITD evaluation mechanism. Cells of the LSO receive excitatory input from the ipsilateral AVCN cells, however, the input from the contralateral AVCN is inhibitory due to the additional synapse in the medial nucleus of the trapezoid body (MNTB). In the high frequency band, the APs are not phase-locked to the sound wave, therefore the ILD-evaluating mechanism has to be relatively insensitive to minor time shifts of the coinciding spikes. The neurons located in the superior olivary complex translate the interaural disparities to changes in spike rate at their output [11], [2]. These dependences are shown (in a simplified

19th INTERNATIONAL CONGRESS ON ACOUSTICS – ICA2007MADRID

2

form) in the inset of Fig. 1 – it is clear that the response of MSO is symmetric, exhibiting an intermediate activity for zero ITDs, while the LSO response is asymmetric, reaching zero near zero ILD and increasing rapidly for favorable ILDs only.

cochlea cochlea

AN AN

AVCNAVCN

MNTB

LSO

cochlea cochlea

AN AN

AVCNAVCN

MSO

excitatory

inhibitory

ILD pathway

ITD pathway

+- ITDILD

spike rate

Figure 1.- Simplified diagram of neural connections for ITD and ILD evaluation (left) and idealized response of LSO and MSO cells to binaural disparities (right).

MODEL INPUTS The presented model of binaural interactions is the final link of the chain of succesive model stages simulating the preceding auditory pathway. If the neural processing is to be modeled at the level of individual spikes, the properties of the preceding model stages have to faithfully reflect the physiological reality. Therefore, the models of the outer, middle and inner ear including the mechanoelectric transduction of the inner hair cells will be shortly described. The complete auditory model is shown schematically in Fig. 2, with the outputs of the individual model stages depicted in the lower panes. The effect of outer and middle ear plays minor role in the binaural interactions of our interest provided that both channels are identical; nevertheless it is included in the complete model. The implementation utilizes a cascade of linear filters. The transfer function of outer ear (red), middle ear (green) and the sum of the two (blue) is shown in the leftmost lower pane of Fig. 2. The model of cochlear mechanics employs the electromechanical analogy, consisting of a cascade of resonators tuned to the resonant frequencies of the adjacent cochlear partitions [3]. The active non-linear feedback is included to reflect the outer hair cells’ amplification and the compressive non-linearity of the cochlear response. The cochlear model output represents the basilar membrane (BM) velocity. The response of BM to three tonal stimuli of various intensities is shown in the bottom row of Fig. 2, the second pane from the left. The model of inner hair cells (IHC) is an important part of the whole system – it translates the vibrations at the cochlear model output to the sequences of action potentials advancing through the auditory nerve. It comprises a model of fluid-cilia coupling relating the cilia displacement to the basilar membrane velocity, a model of the cell membrane depolarisation (receptor potential, RP) resulting from the cilia movement, and finally a model of the potential-driven transmitter release. The fluid-cilia coupling is modeled as a low-pass filter with 2ms time constant [15]. It is followed by a half-way rectifier (the IHC fire mostly during the positive half-wave of the signal, [1]) and a low-pass filter accounting for the several stages of integration in the process of RP generation. The compressive dependence of RP amplitude on the cilia displacement is modeled using an envelope transformation. The output of the RP model matches well the behavior observed during in-vivo measurements [12]. The probability of transmitter release directly relates to the RP magnitude. An altered version of the model published in [17] is used; the modifications include increased amount of available transmitter vessicles and less stochastic transmitter recycling. As a result, the RP magnitude (which in fact corresponds to the stimulus intensity) is encoded not only by the average rate of released transmitter vessicles but also by the number of vessicles emitted at a time.

19th INTERNATIONAL CONGRESS ON ACOUSTICS – ICA2007MADRID

3

Considering that a single IHC is innervated by several fibers and assuming that one vessicle can produce a post-synaptic AP in one fiber at maximum, the RP magnitude translates to the total rate of spikes summed over the bunch of neighbouring fibers – that means that by a set of correlated fibers it is possible to encode much wider dynamic range of RP magnitudes than by a single fiber. This feature is very important especially for precise intensity discrimination in the LSO. Furthermore, the output spikes are synchronized with the sound wave for frequencies below ca. 2kHz; the distribution of events over the sound period well matches the physiologically observed results [1]. To be complete, the fiber refractoriness is modeled by a combination of absolute and exponentially decaying relative refractory period. A period histogram showing the relative distribution of spikes over the sound period is depicted in the bottom row of Fig. 2, the third pane from the left. In agreement with the above described physiology, the binaural model is divided in two independent parts which correspond to the MSO and LSO, respectively. In order to retain reasonable complexity of the model, the neural pathways are somewhat simplified: the AVCN and MNTB synapses are omitted, the MSO and LSO models receive inputs directly from the model of auditory nerve (AN). The excitatory and inhibitory character of spikes is modeled using impulses of inverted polarity.

outer and

middle ear

cochlear

mechanicsRP and IHC

MSO

LSO

outer

middle

total

0 1

S [

dB

]

sp

ike

ra

te

tone period

S [

dB

]

frequency [Hz]frequency [Hz] ITD / ILD- +

sp

ike

ra

te

input

(wave)

output

(spike train)

Figure 2.- The complete auditory model; outputs of the model stages shown in the bottom row. MSO MODEL The proposed MSO model is based on detection of coincidence of spikes within a short time window [4], [8]. It assumes that a single MSO cell can operate as a coincidence detector, emitting a spike whenever it receives spikes from both inputs. The neuron in fact performs binary AND operation. For correct functioning, the model requires that the incoming AP be synchronized with the sound wave which guarantees that the ITD can be evaluated according to mutual shift of the two sequences. This condition holds only for a limited range of frequencies, the synchrony rapidly declines for frequencies above ca. 2kHz due to refractoriness of neurons and also as a consequence of several stages of integration present in the auditory pathway. However, this synchrony must never be ideal – in case of spikes occuring always at the same position within the sound period, the coincidence model would fail as the spikes would meet solely under zero ITD condition, while the non-zero ITDs would remain unresolved. From the recordings of activity in the auditory nerve it is known that the distribution of spikes is rather Gaussian when plotted against the sound period [1]. The coincidence model thus relies upon the random jitter, present in the auditory nerve [4]. MSO model preconditions The information about the oncoming sound is encoded into series of impulses that advance up the auditory nerve. With respect to our coincidence model, certain conditions must hold so that the system might work: 1) the transformation has to preserve the time shift; 2) the impulses have to be dense enough so that the decision time is ecologically relevant; 3) both auditory channels must have approximately the same properties. In the low-frequency band, the spikes are phase-locked to the sound wave, yet randomly displaced from their ideal position. Our model takes advantage of this jitter, making it an integral part of the evaluation mechanism [4], [8]. Let’s assume, for the sake of clarity, that the displacement is a random variable with rectangular PDF. Let’s further assume that the width of the rectangle is equal to the half-period of the sound wave. We assign such a random process

19th INTERNATIONAL CONGRESS ON ACOUSTICS – ICA2007MADRID

4

to both auditory channels and evaluate the probability of coincidence of spikes in relation to the mutual delay of the channels, ie. the ITD. It is clear that the probability of coincidence is maximum for zero ITD – the rectangles overlap entirely – and decreases with increasing mutual delay, reaching zero for delays equal to the half-period of the sound wave – the rectangles share one edge but have no common area. The resulting function has a shape of triangular non-monotonic function, peaking at zero ITD. Such a peaking response is, however, in contrast to the published physiological data [2], see also Fig. 1. A possible solution is to introduce a systematical delay of the channels (approximately of ¼-period of the sound wave) that would shift the peak probability aside from zero ITD. We will show that this modification can be well justified with respect to physiological data: in the measurements of McAlpine et. al. [9], neurons with different characteristic frequency (CF) exhibited peak activity at different ITDs, ensuring that the medium activity occurs always at zero ITD. Vice versa, it follows from the measurements of Brand et. al. [2] that for a single neuron, the peak activity occurs at constant ITD regardless of stimulus frequency, indicating a constant delay proportional to the neuron's CF. Measurements in the MSO published in [11] and [16] reveal that the distribution of spikes over the sound period is shifted in time with ipsilateral and contralateral monaural stimulation. In our opinion, these facts prove that the model requirements are realistic and plausible. The distribution of spikes over the sound period has direct consequences for localization. Obviously, the wider the jitter, the wider range of ITDs can be detected. On the other hand, the jitter width must not exceed half-period of the signal for the sake of unambiguity. Under these conditions, the model predicts that the range of detectable ITDs should decrease with frequency. Indeed, this prediction agrees with human psychophysical data [14]. Moreover, the available recordings of discharge patterns of single nerve fibers (eg. [1]) indicate that the spread of the spikes over the sound period complies with the above stated requirements. As shown in Fig. 2, the distribution is not uniform, it can be approximated eg. by a Beta distribution. MSO model implementation and output The implemented MSO model is based on evaluation of coincidence of spikes within a short time window. The spikes are represented as idealized binary events occuring at discrete times with 100kHz sample rate. A spike is generated at the MSO output if two spikes from the two auditory channels meet within the coincidence window. This output sequence is passed through a model of refractory neuron, allowing to adjust the absolute and relative refractory period. The width of the coincidence interval is a crucial parameter of the model, it has a strong impact on the shape of the output function. Furthermore it is possible, by summing up the outputs from several nerve fibers, to simulate the convergence of fibers at a single MSO cell. This feature enables to investigate the possible neural connections with respect to the output data. Generally, the less wide the coincidence window, the more fibers have to converge to provide input dense enough for relevant ITD coding. The dependence of spike rate at the ouput on the ITD is shown in the left pane of Fig. 3. The stimulus was a 500Hz tone burst at 60dB SPL. The shape of the curve resembles closely the results from neurophysiology (see Fig. 1). The range of detectable ITDs is optimal, maximum positive and negative ITD corresponding to ¼-period of the sound wave. LSO MODEL It is believed that the neural mechanism underlying the ILD sensitivity is located in the LSO, and that it is based on subtraction of spike rates in the two auditory channels. By its very principle, spike rate is an integral quantity that may be enumerated solely by integration over a period of time. A principal LSO cell has no access to this type of information, it must operate directly on individual spikes. We attemtped, by means of modeling, to investigate the possibility of ILD assessment using detection of spike coincidence. In case of the LSO, as opposed to the MSO, the oncoming spikes cancel each other – in a simplified form, the output can be described by a discrete-time equation

))()(,0max()( iRiLiy −= ( Eq. 1) where y(i) is the LSO output, L(i) and R(i) represents the binary information whether there’s a spike in the i-th sample in the left and right channel, respectively.

19th INTERNATIONAL CONGRESS ON ACOUSTICS – ICA2007MADRID

5

LSO model preconditions In the frequency range where ILD is the dominant clue for sound localization, that is above ca. 2kHz, the APs are no longer phase-locked to the sound wave. The activity in a high frequency auditory fiber can rather be described as a random dead-time Poisson process (eg. [5]). Considering that the spikes in the two channels may appear at almost any time, they either have to be dense enough to increase the probability of coincidence or the inhibitory effect of a contralateral AP has to last quite long. Let’s now address the temporal density of spikes at the LSO input. First, it is known [18] that the LSO output spike rate can change by approx. 20spikes/sec per decibel of ILD; such a rapid change of spike rate per decibel can hardly rely upon a single nerve fiber [13], [19]. Second, the just noticeable ILD in the median plane can be as small as 0.5dB [10]; such a precision can not be achieved by a single fiber [5]. Third, the ILD has to be evaluated as fast as possible to provide the subject with the life-saving information. It is thus concluded that an LSO cell must receive inputs from several converging fibers which supply the cell with numerous spikes. Even in the case of several converging fibers at one cell, it is required that the inhibitory effect of a contralateral spike be relatively long, due to the necessity to cancel as many excitatory spikes as possible. According to [6], the inhibition is maximum at least for 2ms after the inhibitory stimulus offset and can still be observed some 9ms after the stimulus offset.

-400 -200 0 200 400 0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

ITD [µs]

Nor

mal

ized

spi

ke ra

te

-15 -10 -5 0 5 10 150

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

ILD [dB]

Nor

mal

ized

spi

ke ra

te

Figure 3.- MSO model output (left), LSO model output (right). LSO model implementation and output The spike representation was described earlier in this paper. The algorithm of ILD detection is based on cancellation of spikes from the two auditory channels. A contralateral spike causes 100% inhibition persisting for 2.5ms. After this absolute inhibitory period, a 7ms relative inhibitory period follows, with linear decrease of the probability of inhibition. For each excitatory spike, the probability of inhibition is calculated, the spike is then either cancelled or passed to the output. Furthermore, one contralateral spike can cancel one excitatory spike at maximum. The resulting sequence is then passed through a simple model of refractory neuron. As in the case of the MSO model, the “coincidence window” and the number and type of converging fibers are the main parameters. Fig. 3 (right pane) shows the dependence of spike rate at the ouput on the ILD. The excitatory stimulus was a 4kHz tone burst at 55dB SPL. The temporal characteristics of the inhibitory effect were set to the above mentioned values. Ten inhibitory and five excitatory high-spontaneous-rate fibers converged at one cell. The shape of the curve resembles closely the results from neurophysiology (see Fig. 1). It is worth mentioning that the spontaneous rate and sensitivity thresholds of the converging fibers are also important parameters affecting the shape of the output. If the density of inhibitory spikes is too low, the

19th INTERNATIONAL CONGRESS ON ACOUSTICS – ICA2007MADRID

6

cell output in fact reproduces the excitatory input. If, on the contrary, the inhibitory spike density is too high, the output function is flat and almost zero. This dependence is in rough agreement with our preliminary (unpublished) results from the measurements of guinea-pig inferior colliculus cells. These data suggest that an ILD-sensitive cell is capable of ILD evaluation in a limited dynamic range only. CONCLUSION An implementation of a probabilistic model for evaluation of interaural time and level differences was presented and related to known physiological and psychophysical data. The model is based on direct interaction of spikes from the two auditory channels; the interaction is excitatory-excitatory in the case of MSO and excitatory-inhibitory in the case of LSO. The coincidence detection model takes advantage of the random position of spikes relative to the sound period. The interaural disparity at the input is encoded by means of variable spike rate at the output. The presented implementation suggests that the model might be a good approximation of MSO and LSO functioning; with suitable parameter settings, the model is capable of reproducing the observed physiological data. ACKNOWLEDGMENTS This work has been supported by the grant of GA CR no.102/05/2054 "Qualitative Aspects of Audiovisual Information Processing in Multimedia Systems" and research project MSM 6840770014 "Research in the Area of Prospective Information and Communication Technologies". References: [1] D. J. Anderson, et.al.: Temporal Position of Discharges in Single Auditory Nerve Fibers within the Cycle of a Sine-Wave Stimulus: Frequency and Intensity Effects. J. Acoust. Soc. Am. 49 (1971) 1131–1139 [2] A. Brand: Precise inhibition is essential for microsecond interaural time difference coding. Nature 417 (2002) 543-547 [3] Z. Bures, F. Rund: Binaural Psychoacoustic Model Applicable To Sound Quality Estimation. In: Proceedings of ICSV13 [CD-ROM]. Wien: Technische Universität (2006) 1-8. ISBN 3-9501554-5-7 [4] Z. Bures: Neural Mechanism of Localization of Low Frequency Sound Stimuli. In: POSTER 2006 [CD-ROM]. Prague: CTU (2006) 1-4 [5] H. S. Colburn, et. al.: Quantifying the Information in Auditory-Nerve Responses for Level Discrimination. Journal of the Association for Research in Otolaryngology 04 (2003) 294-311 [6] P. G. Finlayson, D. M. Caspary: Synaptic potentials of chincilla lateral superior olivary neurons. Hearing Research 38 (1989) 221-228 [7] B. Grothe: New roles for synaptic inhibition in sound localization. Nat. Rev. Neurosci 04 (2003) 540 – 550 [8] P. Marsalek, J. Kofranek: Sound localization at high frequencies and across the frequency range. Neurocomputing 58-60 (2004) 999 – 1006 [9] D. McAlpine, et. al.: A Neural Code for Low-Frequency Sound Localization in Mammals. Nat. Neuroscience 4 (2001) 396 – 401 [10] A. W. Mills: Lateralization of High-Frequency Tones. J. Acoust. Soc. Am. 32 (1960) 132-134 [11] G. Moushegian, et. al.: Functional Characteristics of Superior Olivary Neurons to Binaural Stimuli. Journal of Neurophysiology 5 (1975) 1037 – 1048 [12] I. J. Russell, P. M. Sellick: Low-Frequency Characteristics of Intracellularly Recorded Receptor Potentials in Guinea-Pig Cochlear Hair Cells. J. Physiol. 338 (1983) 179-206 [13] M. B. Sachs, P. J. Abbas: Rate versus level functions for auditory-nerve fibers in cats: tone-burst stimuli. J. Acoust. Soc. Am. 56 (1974) 1835-1847 [14] B. McA. Sayers: Acoustic-Image Lateralization Judgments with Binaural Tones. J. Acoust. Soc. Am. 36 (1964) 923-926 [15] S. A. Shamma, et.al.: A biophysical model of cochlear processing: Intensity dependence of pure tone responses. J. Acoust. Soc. Am. 60 (1986) 133-145 [16] M. W. Spitzer, M. N. Semple: Neurons Sensitive to Interaural Phase Disparity in Gerbil Superior Olive: Diverse Monaural and Temporal Response Properties. Journal of Neurophysiology 73 (1995) 1668 – 1690 [17] C. J. Sumner, et. al.: A revised model of the inner-hair cell and auditory-nerve complex. J. Acoust. Soc. Am. 111 (2002) 2178-2188 [18] D. J. Tollin: The Lateral Superior Olive: A Functional Role in Sound Source Localization. The Neuroscientist 9 (2003) 127–143 [19] I. M. Winter, A. R. Palmer: Intensity coding in low-frequency auditory-nerve fibers of the guinea-pig. J. Acoust. Soc. Am. 90 (1991) 1958-1967