+ All Categories

Outline

Date post: 24-Feb-2016
Category:
Upload: giona
View: 49 times
Download: 0 times
Share this document with a friend
Description:
Week 11: The Role of Noradrenaline in Learning and Plasticity – Part 1. Gabor Stefanics (TNU) Neuropharmacology (Computational Psychiatry Seminar: Spring 2014). Outline. -Noradrenaline (norepinephrine) -NE activity and effects -Involvement in cognition - PowerPoint PPT Presentation
Popular Tags:
32
D-ITET/IBT Week 11: The Role of Noradrenaline in Learning and Plasticity – Part 1. Gabor Stefanics (TNU) Neuropharmacology (Computational Psychiatry Seminar: Spring 2014) 16.05.2014 D-ITET / IBT / TNU 1
Transcript
Page 1: Outline

1D-ITET / IBT / TNU

D-ITET/IBT

Week 11: The Role of Noradrenaline in Learning and Plasticity – Part 1.

Gabor Stefanics (TNU)

Neuropharmacology (Computational Psychiatry Seminar: Spring 2014)

16.05.2014

Page 2: Outline

16.05.2014 2D-ITET / IBT / TNU

Outline

-Noradrenaline (norepinephrine)

-NE activity and effects

-Involvement in cognition

-Theory of locus coeruleus noradrenaline function

-Computational model of decision making

-Twenty-Five Lessons from Computational Neuromodulation

Page 3: Outline

16.05.2014 3D-ITET / IBT / TNU

The noradrenergic system

NEmonoamine, an organic compound that has a catechol (benzene with two hydroxyl side groups) and a side-chain amine (basic nitrogen atom with a lone pair)

dopamine norepinephrine

Page 4: Outline

.

16.05.2014 4D-ITET / IBT / TNU

Neurochemical Modulation of Response Inhibition and Probabilistic Learning in Humans

Chamberlain et al. (2006) Science

Chamberlain et al. investigated the differential involvement of NA and 5-HT transmitter systems in these processes in humans, using the

1) selective NA reuptake inhibitor (SNRI) atomoxetine and the

2) selective 5-HT reuptake inhibitor (SSRI) citalopram.

These agents are among the most selective inhibitors for brain NA and 5-HT reuptake transporters available for human use.

Page 5: Outline

.

16.05.2014 5D-ITET / IBT / TNU

Neurochemical Modulation of Response Inhibition and Probabilistic Learning in Humans

Chamberlain et al. (2006) Science

~NE

Stop-signal taskSubjects respond rapidly to left- or right-facing arrows on screen with corresponding motor responses, and they attempt to inhibit responses when an auditory stop-signal sounds. Over the course of the task, the time between stimulus onset and occurrence of the stop-signal is varied by means of a tracking algorithm. This permits calculation of the SSRT, which reflects an estimate of the time taken to internally suppress prepotent motor responses.

Page 6: Outline

.

16.05.2014 6D-ITET / IBT / TNU

Stop-signal inhibition disrupted by damage to right inferior frontal gyrus in humans

Aron et al. (2003) Nat Neurosci

Page 7: Outline

16.05.2014 7D-ITET / IBT / TNU

Neurochemical Modulation of Response Inhibition and Probabilistic Learning in Humans

Chamberlain et al. (2006) Science

Probabilistic learning taskVolunteers make a two-alternative forced choice between two stimuli (one red, one green) on each trial. The ‘‘correct’’ stimulus (always the first stimulus touched) receives an 4:1 ratio of positive:negative feedback, and the opposite ratio is given for the ‘‘incorrect’’ stimulus. Feedback is provided in the form of ‘‘CORRECT’’ or ‘‘INCORRECT’’ appearing on screen after each choice. Ability to acquire the stimulus-reward association on the basis of this degraded feedback is assessed by the number of errors made before reaching criterion, defined as eight consecutive correct responses to the maximally rewarded stimulus. After 40 trials (stage 1), the contingencies reverse for the subsequent 40 trials (stage 2). The detrimental effect of misleading negative feedback on learning is assessed by means of an overall ‘‘feedback sensitivity’’ score. This is defined as the overall likelihood that the volunteer inappropriately switched to choose the incorrect stimulus after misleadingly being informed that his or her correct response on the previous trial was not correct.

~5-HTstage 1

stage 2

Page 8: Outline

09.05.2014 8D-ITET / IBT / TNU

Phasic norepinephrine: A neural interrupt signal for unexpected events.

Dayan & Yu (2006) Network: Computation in Neural Systems

NE activity and effectsA) Rats solved a sequential decision problem with two sets of cues (spatial and visual). When the relevant cues were switched after a few days of learning (from spatial to visual), rats with pharmacologically boosted NE (idazoxan) learned to use the alternative set of cues faster than the controls. Adapted from Devauges & Sara (1990).

B) Monkeys solved a vigilance task in which they had to react to rare targets and ignore common distractors. The trace shows the activity of a single norepinephrine neuron around the time of a reversal between target and distractor cues (vertical dashed line). The tonic activity is elevated for a considerable period. Adapted from Aston-Jones et al. (1997).

C) In the same vigilance task, single NE cells are activated on a phasictime-scale stimulus locked (vertical line) to the target (upper plot) and not the distractor (lower plot). Adapted from Rajkowski et al. (2004).

D) Again in the same task, the average responses of a large number of norepinephrine cells (over a total of 41,454 trials) stimulus locked (vertical line) to targets or distractors, sorted by the nature and rectitude of the response. Adapted from Rajkowski et al. (2004).

E) In a GO/NO-GO olfactory discrimination task for rats, single units are activated by the target odor (and not by the distractor odor), but are temporally much more tightly locked to the response (right) than the stimulus (left). Trials are ordered according to the time between stimulus and response, with the red and blue marks showing the time of the response and stimulus respectively. From Bouret & Sara (2004).

F) Correlation between the gross fluctuations in the tonic activity of a single NE neuron (upper) and performance in the simple version of the vigilance task (lower, measured by false alarm rate).

Page 9: Outline

.

16.05.2014 9D-ITET / IBT / TNU

Norepinephrine. Involvement in cognition

The noradrenergic system might be involved in learning and memory, there have been increasingly complex theories concerning the functional role of this system, beginning with vigilance, attention and memory processes, and culminating in complex models concerning prediction errors, decision making and unexpected uncertainty.

Bouret & Sara (2005) TINS

Page 10: Outline

09.05.2014 10D-ITET / IBT / TNU

NE and psychopathologyRelationship between depression symptomps and some noradrenergic projections from the locus coeruleus

Multi-stage model of attention-deficit hyperactivity disorder (ADHD)

Himelstein et al. (2000) Front Bioscience

The posterior attention system receives dense NE innervation from the LC . NE inhibits the spontaneous discharge of neurons, which enhances the signal-to-noise ratio of target cells and primes the posterior system to orient to and engage novel stimuli . Attentional function then shifts to the anterior executive system, which consists of the prefrontal cortex (PFC) and the anterior cingulate gyrus. The responsiveness of the PFC and anterior cingulate to the incoming signals is modulated primarily by dopaminergic (DA) input from the ventral tegmental area in the midbrain. Ascending DA fibers stimulate postsynaptic D1 receptors on pyramidal neurons in the PFC and anterior cingulate, which in turn, facilitate excitatory NMDA receptor inputs from the posterior attention system. Thus, DA selectively gates excitatory inputs to the PFC and cingulate, thereby reducing irrelevant neuronal activity during the performance of executive functions. Inability of NE to prime the posterior attention system could account for the attentional problems seen in children with ADHD, while the loss of DA's ability to gate inputs to the anterior executive system may be linked to the deficit in executive functions characteristic of ADHD.

Page 11: Outline

09.05.2014 11D-ITET / IBT / TNU

Network reset: a simplified overarching theory of locus coeruleus noradrenaline function

Bouret & Sara (2005) TINS

A new, simplified and overarching theory of noradrenaline function is inspired by an invertebrate model: neuromodulators in crustacea abruptly interrupt activity in neural networks and reorganize the elements into new functional networks determining the behavioral output.

Analogously in mammals, phasic activation of noradrenergic neurons of the locus coeruleus in time with cognitive shifts could provoke or facilitate dynamic reorganization of target neural networks, permitting rapid behavioral adaptation to changing environmental imperatives.

A behavioral state is characterized by a given functional network that could be defined by a specific spatiotemporal pattern of neuronal activity, here represented by a pattern of activated neurons (green circles). Gray circles represent cells that do not participate in the network. When a stimulus induces a cognitive shift, activation of LC appears immediately before the behavioral shift and, through a simultaneous action on its multiple target structures, can promote the underlying modification of network interactions. These modifications are schematized by engagement (red arrows) or disengagement (red crosses) of several cells. Such an action, analogous to that described in invertebrates, could underlie the implication of the noradrenergic system in cognitive and behavioral flexibility.

Current state of art

Page 12: Outline

.

16.05.2014 12D-ITET / IBT / TNU

Modulators of decision making

1. Computational model of decision making

Doya (2008) Nat Neurosci

The process of decision making can be decomposed into four steps.

1) one recognizes the present situation (or state).

2) one evaluates action candidates (or options, e.g., going out tonight) in terms of how much reward or punishment each potential choice would bring.

3) one selects an action in reference to one’s needs.

4) one may reevaluate the action based on the outcome.

Page 13: Outline

.

16.05.2014 13D-ITET / IBT / TNU

Modulators of decision making

1. Computational model of decision making

Doya (2008) Nat Neurosci

Computational model of decision making

1) Perceptual decision making, depends on veridical perception

2) Evaluation of action candidatesValue of a reward given by an action at a state is a function of reward amount, delay and probability

V = f(amount) × g(delay) × h(probability)

when there are multiple possible outcomesV = ∑ f(amounti) × g(delayi) × h(probabilityi)

In standard ‘expected utility’ theory, h is assumed to be identity, resulting in a simpler form

V = E[f(amount) × g(delay)]

where E denotes expectation. However, subjects often undervalue probabilistic outcomes, which is better modeled by a function h that is smaller than unity for probability < 1 (except at very small probabilities, which can be overvalued). Such deviations from expected utility theory are summarized in ‘prospect theory’.

i

Page 14: Outline

16.05.2014 14D-ITET / IBT / TNU

Modulators of decision making

Doya (2008) Nat Neurosci

Standard models of evaluation of amount, delay and probability of reward

utility function ‘temporal discounting’ of delayed rewards

over- or undervaluationof stochastic outcomes

Page 15: Outline

.

16.05.2014 15D-ITET / IBT / TNU

Modulators of decision making

1. Computational model of decision making

Doya (2008) Nat Neurosci

3) Action selection. After evaluating the value of each action candidate, the next issue is how to select an appropriate one. Given the values for action candidates V(a1),…, V(an), the most straightforward way is to choose the one with the highest value. This is called greedy action selection.

Another common solution is ‘Boltzmann selection’ in which selection probabilities p are proportional to the exponentials of the estimated values

By an analogy with thermodynamics, the scaling parameter β is called the ‘inverse temperature’; β = 0 means all actions are taken with an equal probability of 1/n, and the larger the β, the greedier the selection.

In animal behavior studies, a well known principle is the matching law, in which an action is selected in proportion to its value

This is a nearly optimal strategy in ‘baited’ tasks, in which a reward becomes available at a given location with a certain probability and will stay there until it is taken. In such an environment, a less rewarded action becomes more profitable after a long interval.

Page 16: Outline

16.05.2014 16D-ITET / IBT / TNU

Modulators of decision making

1. Computational model of decision making

Doya (2008) Nat Neurosci

4) LearningIn learning the values of actions in dynamic environments, a critical issue is to identify which action in time caused a given outcome (the problem of ‘temporal credit assignment’).

Three basic ways for learning values in dynamic environments:

-First, keep in memory which action was taken at which state in the form of ‘eligibility traces’, and when a reward is given, reinforce the state-action associations in proportion to the eligibility traces. -Second, use so-called temporal difference learning (sampling the environment). In the case of exponential temporal discounting, this involves following a model using a recursive relationship of the values of subsequent states and actions to update the previous state-action pair.

V(state, action) = E[reward + γV(new state, new action)]

-Third, learn a model of action-dependent state-transition probability and, given the present state, predict the future rewards for hypothetical actions in order to select the best evaluated.Temporal difference learning is more efficient but depends on the appropriate choice of the state variable. Model-based planning requires more contrived operations but can provide flexible adaptation to changes in behavioral goals.

Page 17: Outline

16.05.2014 17D-ITET / IBT / TNU

Modulators of decision making

Factors that affect decisions and learning

Doya (2008) Nat Neurosci

Needs and desires. The utility curve f should reflect the decision maker’s physiological or economic needs. The utility of any amount exceeding the maximal consumption should also saturate. Thus utility functions often have sigmoid shape with threshold and saturation. In people, different desires leads to different thresholds of nonlinear valuation.

Risk and uncertainty. Buying insurance and a lotto ticket. Deviations from simple linear evaluation can be regarded as ‘risk-averse’ or ‘risk-seeking’ decisions and be modeled by nonlinearity in either the utility function f or the probability evaluation function h.

Knowledge and uncertainty about the environment are also important in decision making. E.g., expected uncertainty vs. unexpected uncertainty. Stochastic environmental dynamics limits predictability, thus in reinforcement learning, the temporal horizon needs to be set long enough, but not too long.

Time spent and time remaining. A general issue in learning is how fast one should learn from new experiences and how stably old knowledge should be retained. The appropriate choice of the learning rate depends on both the character of the environment and the experience of the subject. In a constant environment, the theoretically optimal way to learn is to start with rapid memory updating and then to decay the learning rate as an inverse of the number of experiences. When the dynamics of the environment change over time, the setting of the learning rate should depend on the estimate of the time for which the past experiences remain valid. (+ exclusiveness of commitment)

Page 18: Outline

.

16.05.2014 18D-ITET / IBT / TNU

Modulators of decision making

Neural substrates modulating decision making

Doya (2008) Nat Neurosci

Decisions are made in the circuit linking the cerebral cortex and the basal ganglia. Reward-predictive neural activities are found in a variety of areas in the cortex, the striatum, the globus pallidus and the thalamus. Functional brain imaging in humans show activity related to reward prediction error in the striatum, which receives strong dopaminergic projections. Dopamine-dependent plasticity in the striatum seems to be important in learning of reward-predictive neural activities. The dynamic interaction of these areas composing the cortex–basal ganglia loops, as well as other subcortical structures, especially the amygdala, is believed to result in reward-dependent selection of particular actions.

The network is affected by the sensory and contextual information represented in the cortex, as well as in diffuse neurochemical systems, such as serotonin, norepinephrine and acetylcholine.

Page 19: Outline

16.05.2014 19D-ITET / IBT / TNU

Modulators of decision making

Neural substrates modulating decision making

Doya (2008) Nat Neurosci

-Gains and losses. The amygdala is involved in processing of aversive stimuli and avoidance learning. Human brain imaging shows response of the amygdala to expectation of losses as opposed to gains. FMRI reveals that different parts in the striatum respond to gains and losses. Neurons in the lateral habenula respond to no-reward predictive cues as well as reward omission, in exactly the opposite way as dopamine neurons, and that stimulation of the lateral habenula causes inhibition of dopamine neurons. The result highlights the lateral habenula as a possible center of aversive learning.

-Cost and effort. In a T-maze with a small reward behind a low wall on one side and a large reward behind a high wall on the other, lesions of the anterior cingulate cortex (ACC) cause choices of small rewards obtained by smaller effort. Choice of a larger reward with a larger effort is impaired by a dopamine D2 receptor antagonist.

-Risk and variance. Brain imaging shows that especially the ventral striatum is involved in expectation of rewards. Imaging studies show activity in the anterior insula and the lateral orbitofrontal cortex (OFC) in response to variance in the predicted reward. Risk-seeking choice also activates the ventral striatum.

5-HT, serotonin; ACC, anterior cingulate cortex; DA, dopamine; DLPFC, dorsolateral prefrontal cortex; DS, dorsal striatum; NE, norepinephrine; OFC, orbitofrontal cortex; VS, ventral striatum

Page 20: Outline

16.05.2014 20D-ITET / IBT / TNU

Modulators of decision making

Neural substrates modulating decision making

Doya (2008) Nat Neurosci

-Delay discounting. Deficits in the serotonergic system are implicated in impulsivity, both in suppression of maladaptive motor behaviors and in choices of larger but delayed rewards. FMRI studies using a game in a dynamic environment show the dorsolateral prefrontal cortex, dorsal premotor cortex, parietal cortex and insula are more activated in conditions requiring long-term prediction of rewards rather than in conditions requiring short-term predictions. Activation of the ventral striatum, medial OFC, ACC and posterior cingulate cortex encodes expectation of immediate rewards. Serotonin efflux increases in medial prefrontal cortex while rats perform a delay discounting task.

-Learning and exploration. The optimal setting of the learning rate depends on how quickly the world is changing. Subjects’ learning rates vary depending on the volatility of the task environment, which is also correlated with the activity of ACC. After an abrupt change of the environment, it is more appropriate to totally reset what has been learned (or switch to another learning module) and start over. Norepinephrine is implicated in such ‘resets’ of ongoing activities. NE is also suggested to be important in regulating the decision to explore alternatives versus exploiting a known resource. Deficits in serotonin, especially in the medial prefrontal cortex, disturb adaptation to changes in the required action for a given cue (reversal learning) by making the subjects more likely to stick to pre-learned behaviors.

5-HT, serotonin; ACC, anterior cingulate cortex; DA, dopamine; DLPFC, dorsolateral prefrontal cortex; DS, dorsal striatum; NE, norepinephrine; OFC, orbitofrontal cortex; VS, ventral striatum

Page 21: Outline

16.05.2014 21D-ITET / IBT / TNU

Modulators of decision making

Neural substrates modulating decision making

Doya (2008) Nat Neurosci

Summary

-Expectation of a high reward motivates subjects to choose an action despite a large cost, for which dopamine in the anterior cingulate cortex is important.

-Uncertainty of action outcomes can promote risk taking and exploratory choices, in which norepinephrine and the orbitofrontal cortex seem to be involved.

-Predictable environments promote consideration of longer-delayed rewards, for which serotonin and the dorsal part of the striatum as well as the dorsal prefrontal cortex are key.

-Much work will be required to build quantitative models of how decision parameters should be regulated depending on the environment and experience, and then to elucidate how they could be realized by network, cellular and neurochemical mechanisms.

Page 22: Outline

16.05.2014 22D-ITET / IBT / TNU

Twenty-Five Lessons from Computational Neuromodulation

Neural processing faces three rather different, and perniciously* tied, communication problems:

1) Computation is radically distributed, yet point-to-point interconnections are limited (small-world network).

2) The bulk of these connections are semantically uniform, lacking differentiation at their targets that could tag particular sorts of information (but also there are lots of cell types with different connection patterns).

3) The brain’s structure is relatively fixed (but there are also many forms of plasticity), and yet different sorts of input, forms of processing, and rules for determining the output are appropriate under different, and possibly rapidly changing, conditions (context dependence).

Neuromodulators address these problems by their multifarious and broad distribution, by enjoying specialized receptor types in partially specific anatomical arrangements, and by their ability to mold the activity and sensitivity of neurons and the strength and plasticity of their synapses.* per.ni.cious.ly adv, mean exceedingly harmful, implies causing irreparable or deadly injury through evil or insidious corrupting or undermining

Dayan (2012) Neuron

Page 23: Outline

16.05.2014 23D-ITET / IBT / TNU

Diversity of Cortical Interneurons

Kepecs & Fishell (2014) Nature 505, 318–326

Multiple dimensions of interneuron diversity

Page 24: Outline

16.05.2014 24D-ITET / IBT / TNU

Diversity of Cortical Interneurons

Adding more interneurons of the same type linearly increases the network’s combinatorial properties. However, adding novel interneuron types to the old network, even in small numbers, offers a nonlinear expansion of qualitatively different possibilities.

Simplified summary of the salient features of the basic cortical circuit, consisting of only one type of pyramidal cell and a set of local circuit GABAergic cells

P. Somogyi et al., Brain Research Reviews 26 (1998) 113–135

Page 25: Outline

.

09.05.2014 25D-ITET / IBT / TNU

Twenty-Five Lessons from Computational Neuromodulation

Dayan (2012) Neuron

Page 26: Outline

.

09.05.2014 26D-ITET / IBT / TNU

Twenty-Five Lessons from Computational Neuromodulation (in decision making)

Dayan (2012) Neuron

A first group of the 25 general lessons about neuromodulation emerges from what we know about dopamine and reward (utility):

(A) Neuromodulatory neurons can report very selective information (i.e., reward predictionerrors for dopamine) on a (B) very quick timescale. To put it another way, there is no reason why anatomical breadth should automatically be coupled with either semantic or temporal breadth. Nevertheless (C), neuromodulators can also signal over more than one timescale, with at least partially separable tonic and phasic activity, and different receptor types may be sensitive to the different timescales; additionally (D) by having different affinities (as do D1 and D2 receptors), different types can respond selectively to separate characteristics of the signal. Along with their different properties (E), different receptor types can be localized on different pathways, and these pathways are also potentially subject to modulation from a variety of other systems, such as the local, tonically active interneurons in the striatum that release Ach.(F) observe the multiplexing inherent in having a neuromodulator report a signal (e.g., a reward prediction error) that has a variety of important, but distinct, functions.(G) a key role is played by autoreceptors that are typically inhibitory to the release of the neuromodulator concerned, e.g., dopamine receptors on dopamine neurons and their terminals. An obvious role for these is feedback control. Autoinhibition is a way for tonic signaling to set a baseline for phasic signaling.

Page 27: Outline

09.05.2014 27D-ITET / IBT / TNU

Twenty-Five Lessons from Computational Neuromodulation (in decision making)

Dayan (2012) Neuron

(H) interneuromodulatory interactions, such as the influence of one set of neuromodulators on others are very widespread. Structures that drive dopamine activity might themselves be directly sensitive to motivational state—for instance, it has been suggested that the amygdala’s sensitivity to the neuromodulator oxytocin will change its responding in the face of social threats or opportunities.(I) forms of opponency between different neuromodulators are a common motif, both in the central nervous system and indeed in the periphery. However (J) this opponency is rarely simple or symmetric: for instance, although it appears as if the dominantinfluence of 5-HT on behaviors associated with dopamine in practice is inhibitory, there are many typesof serotonin receptor that have an excitatory net effect on dopamine.(K) a complex tapestry of (receptor) heterogeneity is revealed, particularly within the serotonin system.

Page 28: Outline

09.05.2014 28D-ITET / IBT / TNU

Twenty-Five Lessons from Computational Neuromodulation

Dayan (2012) Neuron

A second group of the 25 general lessons about neuromodulation emerges from what we know about uncertainty and learning:

(M) neuromodulators control the course of activity by regulating which of a number of gross pathways determines the activity of neurons is a common scheme. There are also other potential neuromodulatory routes for this influence: for instance, (N) ACh helps regulate oscillations, which is a critical dynamical effect of neuromodulators in many circumstances, that simultaneously affect multiple sub-regions of the hippocampal formation.(O) effects of neuromodulators on various timescales of plasticity are among their mostinfluential.(P) although there is structure in the loops connecting cholinergic nuclei to sensory processing and prefrontal cortices, as indeed with other loops between prefrontal regions and neuromodulatory nuclei, there is only rather little work as to how the relatively general forms of uncertainty that could be represented even by a wired neuromodulatory system might interact with the much more specific uncertainty that could be captured in, say, a cortical population code.(R) There is evidence for local, presumably glutamatergic, control of the release of neuromodulators in the cortex, independent of the spiking activity of the neuromodulatory neurons themselves, which could allow for more specificity in their local effects, but the computational implications of this are not clear.(S) NE helps organize a massive response to stress, notably in conjunction with cortisol, a steroid hormone that acts as another neuromodulator. This involves everything from changing energy storage and usage, via glucocorticoids (involvement with energy regulation is itself a more general principle of neuromodulation.

Page 29: Outline

.

09.05.2014 29D-ITET / IBT / TNU

Twenty-Five Lessons from Computational Neuromodulation

Dayan (2012) Neuron

(T) a commonly reported finding for neuromodulators, namely a inverted U-shaped curve of efficacy. An example finding is that drugs that boost a neuromodulator such as dopamine have a beneficial effect for subjects whose baseline levels are low, but a harmful effect for subjects for whom these levels are high.(U) a general issue for (relatively slow) phasic activity, namely that the time it takes for the neuromodulator to be delivered to its site of action (norepinephrine fibers are not myelinated) appears to be at the margins of the period in which there is a chance to have a suitable effect on the on-going computation.(V) problems or manipulations of neuromodulatory systems are tied to debilitating neurological and psychiatric diseases, such as addiction and Parkinson’s disease, and they are also major therapeutic targets, as in schizophrenia, depression, Alzheimer’s disease, etc.(W), individual (e.g., genetic) differences in factors such as the properties of particular receptor types, or the efficacy of transporters controlling the longevity of neuromodulators following release, have been associated with differences in decision making behavior, such as the propensity to explore or to learn from positive or negative feedback.(X) There is a number of forms of control, including self-regulation by autoreceptors, complexforms of interneuromodulator interaction, and even the possibility of local glutamatergic control over release (co-release of glutamate).(Y) not only do we know very little about the coupling between activity and the blood oxygenationlevel-dependent (BOLD) signal that is measured in fMRI in areas such as the striatum that are the main targets of key neuromodulators, but also these neuromodulators might be able to affect local blood flow directly themselves.

Page 30: Outline

.

09.05.2014 30D-ITET / IBT / TNU

Twenty-Five Lessons from Computational Neuromodulation

Dayan (2012) Neuron

Summary

From a computational perspective, there is much work to do to understand the overall network and systems effects of the changes that we know different neuromodulators lead to in individual elements in those circuits.

The most compelling computational issue is the relationship between specificity and generality and cortical versus neuromodulatory contributions to representation and processing.

For utility, this issue centers on the interactions between model-free and model-based systems, with the former being substantially based on neuromodulators such as dopamine and serotonin, whereas the latter depends on cortical processing (albeit itself subject to modulation associated with specific stimulus values).

For uncertainty, the question is how representations of uncertainty associated with cortical populationcodes, with their exquisite stimulus discrimination, interact with those associated with neuromodulators, with their apparent coarseness.

Page 31: Outline

.

16.05.2014 31D-ITET / IBT / TNU

Reading List

• Dayan, Peter. 2012. “Twenty-Five Lessons from Computational Neuromodulation.” Neuron 76 (1): 240–56.

• Chamberlain SR, Müller U, Blackwell AD, Clark L, Robbins TW, Sahakian BJ. 2006. “Neurochemical modulation of response inhibition and probabilistic learning in humans.” Science 311(5762): 861-3.

• Doya K. 2008. “Modulators of decision making.” Nature Neuroscience 11(4): 410-6. • Preuschoff, Kerstin, Bernard Marius ’t Hart and Wolfgang Einhäuser. 2011. “Pupil Dilation

Signals Surprise: Evidence for Noradrenaline’s Role in Decision Making.” Frontiers in Neuroscience 5: 115.

• Yu, Angela J., and Peter Dayan. 2005. “Uncertainty, Neuromodulation, and Attention.” Neuron 46 (4): 681–92.

• Eldar, Eran, Jonathan D. Cohen, and Yael Niv. 2013. “The Effects of Neural Gain on Attention and Learning.” Nature Neuroscience 16 (8): 1146–53.

• Reading List+• Aron AR, Fletcher PC, Bullmore ET, Sahakian BJ, Robbins TW. 2003. Stop-signal inhibition

disrupted by damage to right inferior frontal gyrus in humans. Nat Neurosci 6(2):115-6.• Bouret S, Sara SJ. 2005. Network reset: a simplified overarching theory of locus coeruleus

noradrenaline function. Trends Neurosci 28(11):574-82.

Page 32: Outline

16.05.2014 32D-ITET / IBT / TNU

Thank you!


Recommended