Noise in the nervous systems: Stochastic Resonance

Jaeseung Jeong, Ph.D

Department of Bio and Brain Engineering,

KAIST

Several sources of Noise in the Brain

• Thermal noise

• Cellular noise: Stochastic opening and closing of ion channels• Membrane voltage fluctuations in the axons and dendrites

• Synaptic noise: Spontaneous release of vesicles in the synapses

• Sensory and motor noises: Random generation of voltage fluctuations in the fibers

• Environmental (Stimulus) noises

Cortical variability

Cortical variability: cellular noise

• a. The shift of the overall spike pattern across rows reflects the average propagation speed of the APs. The raster plot of the somatic measurement reflects spike-time variability from AP initiation. Owing to channel noise, the spike-time variability rapidly increases the further the AP propagates, and it eventually reaches millisecond orders.

• b. Trial-to-trial variability of synaptic transmission measured in vitro by paired patch-clamp recordings in rat somatosensory cortex slices. Six consecutive postsynaptic responses (black traces) to an identical presynaptic-stimulation pattern (top trace) are shown, along with the ensemble mean response (grey trace) from over 50 trials.

What is Stochastic Resonance?

• A stochastic resonance is a phenomenon in which a nonlinear system is subjected to a periodic modulated signal so weak as to be normally undetectable, but it becomes detectable due to resonance between the weak deterministic signal and stochastic noise.

• The earliest definition of stochastic resonance was the maximum of the output signal strength as a function of noise (Bulsara and Gammaitoni 1996).

Ice-age cycle of Earth

Noise-aided hopping events

• Surmounting the barrier requires a certain amount of force. Suppose the ball is subjected to a force which varies in time sinusoidally, but is too weak to push the ball over the barrier.

• If we add a random “noise” component to the forcing, then the ball will occasionally be able to hop over the barrier.

• The presence of the sinusoid can then be seen as a peak in the power spectrum of the time series of noise-aided hopping events.

Signal and noise

• We can visualize the sinusoidal forcing as a tilting of the container. In the time series below, the gray background represents the time-varying depth of the wells with respect to the barrier. The red trace represents the position of the ball.

Physical picture of Stochastic Resonance

• If the particle is excited by a small sinusoidal force, it will oscillate within one of the two wells. But if the particle is also excited by a random force (i.e. noise plus sine) it will hop from one well to the other, more or less according to the frequency of the sine: the periodical force tends to be amplified.

• It can intuitively be sensed that if the particle is excited by the sine plus a very small noise it will hop a few times. In return, if the noise is too powerful, the system will become completely randomized. Between these two extreme situations, there exists an optimal power of input noise for which the cooperative effect between the sine and the noise is optimal.

Bistability in Stochastic Resonance

Power spectra of hopping events

• In this power spectra of hopping events, the gray bars mark integer multiples of the sinusoidal forcing frequency.

Kramers rate for Stochastic Resonance

• Physically, the sine posses a characteristic time that is its period. The dynamical system has also a characteristic time system that is the mean residence time in the absence of the sine, i.e. the mean (in statistical sense) time spent by the particle inside one well.

• This time is the inverse of the transition rate, known as Kramers rate, and is function of the noise level. (i.e. the inverse of the average switch rate induced by the sole noise: the stochastic time scale).

• For the optimal noise level, there is a synchronization between the Kramers rate and the frequency of the sine, justifying the term of resonance.

• Since this resonance is tuned by the noise level, it was called stochastic resonance (SR).

Peak SNRs correspond to maximum spatiotemporal synchronization.

• SNR of the middle oscillator of an array of 65 as a function of noise for two different coupling strengths, 0.1 and 10.

Stochastic Resonance in multi Array oscillators

• Time evolution (up) of an array of 65 oscillators, subject to different noise power and coupling strength. The temporal scale of the patterns decreases with increasing noise while the spatial scale of the patterns increases with increasing coupling strength.

• For this range of noise and coupling, spatiotemporal synchronization (and peak SNR) correspond to a coupling of about 10 and a noise of about 35 dB, as indicated by the striped pattern in the third column of the second row from the top.

Stochastic Resonance in the nervous system

• Since its first discovery in cat visual neurons, stochastic-resonance-type effects have been demonstrated in a range of sensory systems.

• These include crayfish mechanoreceptors, shark multimodal sensory cells, cricket cercal sensory neurons and human muscle spindles.

• The behavioural impact of stochastic resonance has been di

rectly demonstrated and manipulated in passive electrosensing paddlefish and in human balance control.

N oise produces nonlinearity• in spike-generating neurons, sub-threshold signals have no effect on the output of

the system. Noise can transform such threshold nonlinearities by making sub-threshold inputs more likely to cross the threshold, and this becomes more likely the closer the inputs are to the threshold.

• Thus, when outputs are averaged over time, this noise produces an effectively smoothed nonlinearity.

• This facilitates spike initiation and can improve neural-network behaviour, as was shown in studies of contrast invariance of orientation tuning in the primary visual cortex.

• Neuronal networks in the presence of noise will be more robust and explore more states, facilitating learning/adaptation to the changing demands of a dynamic environment.

SR-based techniques

• SR-based techniques has been used to create a novel class of medical devices (such as vibrating insoles) for enhancing sensory and motor function in the elderly, patients with diabetic neuropathy, and patients with stroke.

Balancing act using vibrating insoles

• Using a phenomenon called stochastic resonance, the human body can make use of random vibrations to help maintain its balance.

• In experiments on people in their 20s and people in their 70s, actuators embedded in gel insoles generated noisy vibrations with such a small amount of force that a person standing on the insoles could not feel them. A reflective marker was fixed to the research subject's shoulder, and a video camera recorded its position.

• People always sway a small amount even when they are trying to stand still. The amount of sway increases with age. But under the influence of a small amount of vibration, which improves the mechanical senses in the feet, both old and young sway much less.

• Remarkably, noise made people in their 70s sway about as much as people in their 20s swayed without noise.

‘Noise reduction’ mechanisms in the Brain

• Thresholding systems in the neurons

• Low Reliability (Bursts) between neurons

• Rate coding hypothesis

• Averaging (Neuronal Population coding)

• Using prior knowledge about the noise characteristics

Accuracy in the Information processing

of the Brain

vs.

Noise

How can neural networks maintain stable activity in the presence of noise?

Part a shows convergence of signals onto a single neuron. If the incoming signals have independent noise, then noise levels in the postsynaptic neuron will scale in proportion to the square root of the number of signals (N), whereas the signal scales in proportion to N.

cf. If the noise in the signals is perfectly correlated, then the noise in the neuron will also scale in proportion to N.

H omeostatic plasticity mechanisms

• Experimental evidence suggests that average neuronal activity levels are maintained by homeostatic plasticity mechanisms that dynamically set synaptic strengths, ion-channel expression or the release of neuromodulators.

• This in turn suggests that networks of neurons can dynamically adjust to attenuate noise effects. Moreover, these networks might be wired so that large variations in the response properties of individual neurons have little effect on network behaviour.

Principles of how the CNS manages noise

• The principle of averaging can be applied whenever redundant information is present across the sensory inputs to the CNS or is generated by the CNS.

• Averaging can counter noise if several units (such as receptor molecules, neurons or muscles) carry the same signal and each unit is affected by independent sources of noise.

• Averaging is seen at the very first stage of sensory processing.

D ivergence

• Counterintuitively, divergence (one neuron synapsing onto many) can also support averaging.

• When signals are sent over long distances through noisy axons, rather than using a single axon it can be beneficial to send the same signal redundantly over multiple axons and then combine these signals at the destination.

• Crucially, for such a mechanism to reduce noise the initial divergence of one signal into many must be highly reliable.

• Such divergence is seen in auditory inner hair cells, which provide a divergent input to 10–30 ganglion cells through a specialized 'ribbon synapse'.

• Averaging is used in many neural systems in which information is encoded as patterns of activity across a population of neurons that all subserve a similar function: these are termed neural population codes.

Prior knowledge about noises

• Prior knowledge can also be used to counter noise. If the structure of the signal and/or noise is known it can be used to distinguish the signal from the noise. This principle is especially helpful in dealing with sensory signals that, in the natural world, are highly structured and redundant.

• Signal-detection theory shows that the optimal signal detector, subject to additive noise, is obtained by matching all parameters of the detector to those of the signal to be detected: in neuroscience this is termed the matched-filter principle.

• Thus, the structures of receptive fields embody prior knowledge about the expected inputs and thereby allow neurons to attenuate the impact of noise.

Bayesian inference: combining averaging and prior knowledge

• The principles of averaging and prior knowledge can be placed into a larger mathematical framework of optimal statistical estimation and decision theory, known as Bayesian inference.

• Bayesian inference assigns probabilities to propositions about the world (beliefs). These beliefs are calculated by combining prior knowledge (for example, that an animal is a predator) and noisy observations (for example, the heading of animal) to infer the probability of propositions (for example, animal attacks).

• Psychophysical experiments have confirmed that humans use these Bayesian inferences to allow them to cope with noise (and, more generally, with uncertainty) in both perception and action.