doi: 10.1152/jn.00806.2011108:513-527, 2012. First published 28 March 2012;J Neurophysiol
Mark A. Bourjaily and Paul Millerassociations and discriminationsimprove performance in tasks requiring stimulus Dynamic afferent synapses to decision-making networks
You might find this additional info useful...
for this article can be found at: Supplementary materialhttp://jn.physiology.org/http://jn.physiology.org/content/suppl/2012/06/07/jn.00806.2011.DC1.html
81 articles, 26 of which you can access for free at: This article citeshttp://jn.physiology.org/content/108/2/513.full#ref-list-1
including high resolution figures, can be found at: Updated information and serviceshttp://jn.physiology.org/content/108/2/513.full
can be found at: Journal of Neurophysiology about Additional material and informationhttp://www.the-aps.org/publications/jn
Dynamic afferent synapses to decision-making networks improve performancein tasks requiring stimulus associations and discriminations
Mark A. Bourjaily2,3 and Paul Miller1,2,3
1Department of Biology, Brandeis University, Waltham, Massachusetts; 2Neuroscience Program, Brandeis University,Waltham, Massachusetts; and 3Volen Center for Complex Systems, Brandeis University, Waltham, Massachusetts
Submitted 1 September 2011; accepted in final form 26 March 2012
Bourjaily MA, Miller P. Dynamic afferent synapses to decision-making networks improve performance in tasks requiring stimulusassociations and discriminations. J Neurophysiol 108: 513–527, 2012.First published March 28, 2012; doi:10.1152/jn.00806.2011.—Ani-mals must often make opposing responses to similar complex stimuli.Multiple sensory inputs from such stimuli combine to produce stim-ulus-specific patterns of neural activity. It is the differences betweenthese activity patterns, even when small, that provide the basis for anydifferences in behavioral response. In the present study, we investi-gate three tasks with differing degrees of overlap in the inputs, eachwith just two response possibilities. We simulate behavioral output viawinner-takes-all activity in one of two pools of neurons forming abiologically based decision-making layer. The decision-making layerreceives inputs either in a direct stimulus-dependent manner or via anintervening recurrent network of neurons that form the associativelayer, whose activity helps distinguish the stimuli of each task. Weshow that synaptic facilitation of synapses to the decision-makinglayer improves performance in these tasks, robustly increasing accu-racy and speed of responses across multiple configurations of networkinputs. Conversely, we find that synaptic depression worsens perfor-mance. In a linearly nonseparable task with exclusive-or logic, thebenefit of synaptic facilitation lies in its superlinear transmission:effective synaptic strength increases with presynaptic firing rate,which enhances the already present superlinearity of presynapticfiring rate as a function of stimulus-dependent input. In linearlyseparable single-stimulus discrimination tasks, we find that facilitatingsynapses are always beneficial because synaptic facilitation alwaysenhances any differences between inputs. Thus we predict that foroptimal decision-making accuracy and speed, synapses from sensoryor associative areas to decision-making or premotor areas should befacilitating.
A FUNDAMENTAL ROLE of an animal’s brain is to allow the animalto adapt to an ever-changing environment such that the ani-mal’s response, or behavior, leads to most reward or least harmgiven a particular confluence of environmental stimuli. Subtledifferences in stimuli can require opposite behaviors. Forexample, those characteristics differentiating a friend from afoe or a stronger opponent from a weaker one could appearminor amidst the general overlap of features pertaining to afellow member of the same species. However, the animal mustlearn to discriminate differences in such salient features toproduce opposite responses, even if much of the sensoryinformation (and neural activity within the brain) is similar.Thus how appropriate responses, or decisions that lead to those
responses, are learned and produced from patterns of overlap-ping and similar neural activity is a matter of intense investi-gation, both experimentally (Curtis and Lee 2010; Glimcherand Rustichini 2004; Gold and Shadlen 2007, Harris et al.2008; Histed et al. 2009; Jay 2003; Kennerley and Walton2011; Takeda et al. 2005) and theoretically (Eckhoff et al. 2008; Fenget al. 2009; Rigotti et al. 2010a, 2010b; Sakai et al. 2006; Sakaiand Miyashita 1991; Shen et al. 2008; Soltani and Wang 2008,2010; Wang 2008).
In our studies, we focus on the inputs to a binary winner-takes-all (WTA) decision-making circuit (Wang 2002). TheWTA network serves as a model for perceptual decision-making and the animal’s behavioral choice in a two-alternativeforced-choice task (Wang 2002; Wong and Wang 2006). Re-sponse of the WTA network depends on bias in the input to itstwo decision-making pools. In our model, input synapses to thedecision-making neurons are modified by a dopamine (DA)-modulated Hebbian reward-based plasticity rule (Reynolds andWickens 2002; Soltani and Wang 2006) that, in principle, canlead the decision-making pools to produce responses thatmaximize reward, given sufficient information about the stim-uli within the neural activity of its input cells. For the networksof spiking neurons in our study, the cells providing input to thedecision-making circuit reside in a randomly recurrently con-nected associative layer. Within such a layer, stimulus-selec-tive cells can arise, even when the inputs are broadly overlap-ping (Bourjaily and Miller 2011a; Rigotti et al. 2010b). Ageneral linear readout, as used in support vector machines,could produce appropriate responses to information within thehigh-dimensional space of associative layer neural firing ratesvia a trained or fitted set of synaptic weights (Burges 1998).However, the requirement that such weights are produced bybiologically realistic learning rules generally constrains thereadout process such that information can be contained withinneurons in the associative layer, but firing of those neuronscannot produce a reliable response. In this report, we suggestthat the use of dynamic synapses between the associative layerand decision-making layer can enhance the readout processbeyond that achievable by a general linear classifier.
Multiple decision-making studies have focused on the for-mation and function of the internal dynamics of the decision-making networks (Albantakis and Deco 2011; Cisek et al.2009; Daw et al. 2006; Lee and Seo 2007; Rolls et al. 2010;Theodoni et al. 2011; Wang 2002; Wong and Wang 2006; Xieet al. 2002). In computational studies of decision-making, theinputs have been primarily via static synapses (i.e., synapseswithout short-term synaptic plasticity), whose postsynapticoutput increases approximately linearly with presynaptic firing
Address for reprint requests and other correspondence: P. Miller, Dept. ofBiology and Volen Center for Complex Systems, Brandeis Univ., 415 SouthSt., Waltham, MA 02454-9110 (e-mail: [email protected]).
J Neurophysiol 108: 513–527, 2012.First published March 28, 2012; doi:10.1152/jn.00806.2011.
rate (until saturation at a rate proportional to the inverse of thepostsynaptic receptor’s time constant) (Wang 2002; Wong andWang 2006). However, numerous experimental studies acrossdifferent cortical regions (Abbott et al. 1997; Chance et al.1998; Markram et al. 1998b; Nelson and Turrigiano 1998;Wang et al. 2006) have shown that synaptic strength is dy-namic on the timescale of interspike intervals and that synaptictransmission increases nonlinearly with presynaptic firing rate(Chapman et al. 2000; Grillner et al. 2005; Tsodyks et al.1998). Computational studies have shown a number of func-tions for both facilitating and depressing synapses, such aspersistent neural activity (Barak and Tsodyks 2007; Conboy etal. 2010; Markram 1997; Martinez-Garcia et al. 2011), filteringneural signals (Farajidavar et al. 2008; Lindner et al. 2009),and gain control (Abbott et al. 1997), among others.
Some of the most difficult behavioral tasks are linearlynonseparable (Grand and Honey 2008; Harris 2006; Harris etal. 2008; Melchers et al. 2008; Sanderson et al. 2006); that is,the choice of response cannot be determined by an appropri-ately weighted combination of individual stimuli (Minsky andPapert 1987). Behavioral tasks that are linearly nonseparableinclude those that employ exclusive-or (XOR) logic, such asbiconditional discrimination (Melchers et al. 2008; Sandersonet al. 2006) and negative patterning (Harris et al. 2008),inference tasks such as associative transitive inference (Bunseyand Eichenbaum 1993), certain paired association tasks(Takeda et al. 2005), and transverse patterning (Sanderson etal. 2006). Although these tasks vary in design and sensorymodality, they all require animals to produce stimulus pair-selective responses to solve the task. In biconditional discrim-ination, as in other linearly nonseparable tasks, successfuldecision-making requires responses selective to stimulus pairs(e.g., A � B vs. C � B vs. C � D) (Bourjaily and Miller2011a), with different responses to the most similar pairs.Thus, if the animal only learns a preferred response to onemember of the pair (e.g., stimulus B from stimulus pair A �B), then if it responds correctly to stimulus pair A � B, itwould respond incorrectly to stimulus pair C � B. Our resultsfor the biconditional discrimination task can be applied to theother linearly nonseparable associative learning tasks men-tioned above.
In artificial and/or feedforward neural networks with non-linear behavior, XOR has been solved using methods such asselective suppression of feedback (Hasselmo and Cekic 1996)and associative learning (Alkon et al. 1990) of input correla-tions in conjunction with stochastic gradient ascent reinforce-ment learning (Iannella and Back 2001; Park et al. 2000). Atthe synaptic level, XOR has been solved using stochasticgradient ascent learning, where stochastic synapses and/orirregular spiking provided requisite noise for stochastic gradi-ent ascent learning to a readout cell (Seung 2003; Xie andSeung 2004).
A critical property of dynamic synapses is that they lead tononlinear synaptic transmission (Dayan and Abbott 2001; Fieteet al. 2004; Markram et al. 1998a, 1998b), which couldenhance the circuit’s ability to produce solutions to linearlynonseparable tasks. Here, we address how short-term plasticityin afferent synapses to a decision-making network (Wang2002; Wong and Wang 2006) with a biophysically basedreinforcement learning rule (Jay 2003; Reynolds and Wickens2002; Shen et al. 2008) can affect performance and reaction
times in both a linearly nonseparable task and simpler tasks.Although individual synapses are known to have both facili-tating and depressing dynamics (Grillner et al. 2005; Maassand Zador 1999; Markram et al. 1998b), to investigate thespecific role of each, we look at either purely facilitating orpurely depressing synapses (Chapman et al. 2000; Dayan andAbbott 2001; Grillner et al. 2005; Markram et al. 1998a)compared with static synapses in each network and task.
For the overwhelming majority of network types trained onlinearly nonseparable tasks and for all tests of linearly separa-ble tasks, we find that facilitation’s superlinear synaptic trans-mission improves performance. Specifically, facilitation im-proves decision-making accuracy and speed of response,whereas depression’s sublinear synaptic transmission worsensboth relative to static synapses. The beneficial role of facilita-tion suggests that synapses from sensory and associative areasof the brain to decision-making regions such as prefrontalcortex and premotor areas should be facilitating.
MATERIALS AND METHODS
Simplified Firing Rate Model
We produced a simplified firing rate model network to demonstrateclearly the interaction between the following three factors: nonlinear-ity of associative layer responses, dynamics due to short-term plas-ticity at the synapse, and the details of the reward-based learning rule.The simplified model was written in Matlab (The MathWorks, Natick,MA), and the code is available online (http://people.brandeis.edu/�pmiller/code.html).
As in the full network (see below), four stimulus pairings were used(A � B, A � D, C � B, or C � D). We assume four types of neuralresponse in the associative layer (AB, AD, CB, or CD), correspondingto groups of cells whose preferred stimulus is one of the four stimuluspairings (see Table 1). In the associative layer, each cell group i isrepresented by a single firing rate, which evolves during each stimuluspair presentation s according to
�dri
dt� �ri � f(Ii,s
App),
where � � 5 ms and the applied current Ii,sApp � 40 for the group’s
preferred stimulus pair, Ii,sApp � 20 for the opposite (nonoverlapping)
stimulus pair, and Ii,sApp � 0 for each of the two intermediate stimulus
pairs, as given in Table 1.The steady-state firing rate, f(IApp), in response to an applied
current, IApp, can take on three forms. The response function may belinear, f lin(IApp) � IApp; sublinear, f sub(IApp) � IApp/(1 � �IApp); orsuperlinear, f supra(IApp) � IApp·(1 � �IApp). We set the degree ofnonlinearity as � � 0.1 in the results (presented in Fig. 3).
The four associative layer cell groups i provide synaptic input toeach of two decision layer cell groups, j � 1, 2, whose firing ratesevolve according to
Table 1. Firing rate response of linear neurons to paired inputs
where IjIn is input from the associative layer, Ij
Rec is recurrent self-excitation and recurrent cross-inhibition from within the decisionlayer, and Ij
Noise is uncorrelated noise input in the decision layer toproduce stochastic responses. The total input Ij
In to a decision layercell group j depends on the synaptic strength, Wij
AD, which is modi-fied through reward-based plasticity and the rate-dependent synaptictransmission, si(t), from presynaptic associative layer cell group i:IjIn�t� � �i Wij
ADsi�t�.For static (i.e., linear) synapses, si
stat�t� � ri�t�; for depressing (i.e.,sublinear) synapses, si
dep�t� � ri�t�⁄�1 � �ri�t��; and for facilitating(i.e., superlinear) synapses, si
fac�t� � ri�t�·�1 � �ri�t��. Note that theabove formulas for the simplified rate model include only the steady-state change in amplitude wrought by the short-term plasticity, todemonstrate its effect on decision-making; they do not account for thetransient dynamics that occur when firing rates change (Abbott et al.1997). We modeled the complete dynamics in the full spiking-neuronnetwork model [see Supplemental Table S1, where simulation detailsare presented in a form suggested by Nordlie et al. (2009)]. (Supple-mental data for this article is available online at the Journal ofNeurophysiology website.)
The synaptic strengths from the four associative layer groups to thetwo decision layer groups were initially equal, set to Wij
�0� � 10⁄�si����, wheresi(�) is the steady-state synaptic input of cell group i and the average(in angle brackets) was over the four associative layer cell groups.Thus the initial mean input to the decision layer before training washeld constant when we varied the nonlinearity in the associative layeror the short-term dynamics of the afferent synapses. The synapticstrengths were updated by �Wij (defined below) on a trial-by-trialbasis in a reward-dependent manner to the “winning” decision layerpool with the higher firing rate after a 500-ms stimulus presentation.If the winning decision layer pool was the designated one for obtain-ing reward (group 1 for stimulus pair A � B or C � D and group 2for stimulus pair A � D or C � B), then �Wij � �W��Ai � �Ai��;otherwise, �Wij � ��W��Ai � �Ai��. Ai depended on the firing rateof the associative layer cell and could be linear, Ai � ri(t
max);sublinear, Ai � ri(t)/[1 � �ri(t
max)]; or superlinear, Ai � ri(t)·[1 ��ri(t
max)]. Thus we present our results in a 3 � 3 � 3 matrix form,because they depend on the linearity, sublinearity, or superlinearity ofneural response, synaptic dynamics, and the presynaptic rate depen-dence of the reinforcement rule. All other terms in the decision layerwere kept constant across these 27 combinations of network.
The recurrent current to the decision layer cell groups (j � 1, 2) isgiven by Ij
Rec�t� � Wsymrj�t� � WXr2�j�t� with Wsym � 0.8 and WX �0.6. The noise current to the decision layer cell groups is given byIjNoise�t� � ��t�, where � � 0.2 and (t) is a white-noise process with
unit variance. The firing rate curve of the decision layer neurons issigmoidal, f dec(I) � rmax/{1 � exp [(ITh � I)/I�]} with rmax � 50 Hz,ITh � 50, and I� � 8.
We simulated 10 independent sessions of 1,000 consecutive trialswith each linearity-type configuration and calculated the fraction ofcorrect responses across these trials. We also calculated the netasymmetry, SW, of the final weight matrix produced by the learningrule as a number that can vary from �1 to �1:
SW �W11
AD � W12AD � W21
AD � W22AD � W31
AD � W32AD � W41
AD � W42AD
W11AD � W12
AD � W21AD � W22
AD � W31AD � W32
AD � W41AD � W42
AD .
A weight asymmetry of SW � �1 means that associative layer cellgroups selectively potentiate to decision layer cell groups that producethe incongruous response, i.e., opposite to the one desired for theirpreferred cue.
Two-stimulus discrimination task. To assess the effect of dynamicsynapses on the inputs to a decision-making network in a two-
stimulus discrimination task, we used the rate model, as describedabove. In this task, we used two distinct associative layer pools, eachconnected to a separate decision-making pool. Inputs to the twoassociative layer pools differed (by 5% of the mean in the resultspresented) to simulate concurrent visual input of different strengths inopposing directions (Wang 2002), as used in visual motion coherencediscrimination tasks (Shadlen and Newsome 1996). To compensatefor the change of effective synaptic strength inherently produced bydynamic synapses, we evaluated discrimination accuracy, namely, theprobability that the “winning” decision-making pool was the one withgreater stipulated input, as a function of baseline synaptic strength ofafferent synapses to the decision-making circuit.
Spiking Network Simulations
Task logic and inputs. In the biconditional discrimination task(Sanderson et al. 2006), four stimulus pairings were activated, one pertrial (A � B, C � D, A � D, or C � B), with each stimulus producinginputs as Poisson spike trains. If either both A and B were activatedor neither was activated (i.e., C � D), the decision was to release thelever. If either A or B but not both were present (i.e., A � D or C �B), the opposite decision was made, namely, to hold the lever until theend of the trial. In the stimulus-response association task, singlestimuli were presented and the decision was to release for inputs A orD and to hold for inputs B or C.
We produced networks with different degrees of input correlations(as shown in Fig. 4A) by altering the number of independent inputs perstimulus as 2, 4, 6, 10, or 20 to excitatory and inhibitory cells in theassociative layer (see Fig. 4B). Each input comprised a train ofindependent Poisson spikes with a mean firing rate defined by r� � 480Hz/(no. of inputs per stimulus). For example, in a network with 20inputs, any stimulus produced 20 independent Poisson spike trains at24 Hz, with each input projecting to independent sets of cells withinthe associative layer, whereas with 2 inputs per stimulus, the 2 trainsof 240-Hz Poisson spikes (a firing rate much higher than that pro-duced by an individual cell) could be considered as 20 independentPoisson spike trains of 24 Hz grouped into 2 sets of 10: while thereceiving cells between sets were uncorrelated, the receiving cellswithin such a set of 10 would be identical. Thus, as the number ofinputs per stimulus decreased, the correlation in connectivity fromafferent cells increased (see Fig. 4A).
Input sparseness was defined via the probability of any input groupprojecting to any given cell. As input connection probability in-creased, sparseness decreased. We used the following five values forinput connection probability: 1/2, 1/3, 1/5, 1/10, and 1/20 (1/5 in thedefault network). Five levels of input sparseness, combined with fivedifferent degrees of input correlations, led to 25 variant networks ineach regime.
Associative layer connectivity. Both input and recurrent connectiv-ity in the initial network were sparse and random. We simulated 400neurons with an excitatory-to-inhibitory ratio of 4:1 (Abeles 1991).Excitatory-to-excitatory connections were sparse and random. Inhibi-tion was feedforward only. Inhibitory-to-inhibitory connections wereall-to-all. Finally, inhibitory-to-excitatory synapses connected ran-domly. Connections to the decision layer were initially all-to-all fromexcitatory neurons with a uniform strength. We used a stronger initialconstant input weight for the sparse networks (1/10, 1/20), becausetheir initial low average activity was too small to drive the decision-making layer otherwise (Bourjaily and Miller 2011a).
Decision layer connectivity. The decision-making network, basedon Wang (2002), comprised two 200-cell excitatory and 50-cellinhibitory pools. Connections within each pool were all-to-all. Strongrecurrent excitation within each pool and cross-inhibition from eachinhibitory pool to the opposing excitatory pool together generatedWTA activity so that only one pool was stably active, modeling thedecision. The decision-making network received an “urgency-gating”
input, which ramped up linearly from cue onset to offset, ensuring thata decision was made each trial (Cisek et al. 2009).
Neuron and synapse properties. We used leaky integrate-and-fireneurons (LIF) (Tuckwell 1988), defined according to the equations inSupplemental Table S1D, with parameters defined in SupplementalTable S2, D1 and D2. The spiking threshold potential was dynamic,because it depolarized with increased firing and decreased exponen-tially to the base value as the firing rate decreased. After a spike, therewas a dynamic refractory conductance, gref(t).
We included voltage noise by a Gaussian distribution of zero meanwith unit variance and amplitude �V in the associative layer. We alsomodeled noise as independent excitatory AMPA and inhibitoryGABAA conductance variables, drawn from a uniform distribution [0�S] (see Supplemental Table S2 for parameters).
Synaptic conductances were modeled as instantaneous steps after aspike followed by an exponential decay (described in SupplementalTable S1). Recurrent excitatory currents were modeled by AMPA andvoltage-dependent NMDA receptors. Inhibitory currents were mod-eled by GABAA receptors. NMDA receptors were also defined by thevoltage term (Compte et al. 2000; Jahr and Stevens 1990). Totalconductance was the weighted sum of all inputs (see SupplementalTable S1). Initial values of weights depended on synapse type as givenin Supplemental Table S2 and were modified according to the plas-ticity rules described below and in Supplemental Table S1.
Short-term synaptic plasticity. Short-term facilitation and depres-sion were modeled as instantaneous steps after a presynaptic spikefollowed by an exponential decay (Dayan and Abbott 2001), de-scribed by the equations below, which mimic in vitro measurements(Markram and Tsodyks 1996; Markram et al. 1998a). The totalconductance of the presynaptic synapse to the decision-making networkwas a multiplication of the facilitation or depression variable (defined below)by the normal conductance [e.g., s�t�AMPA
fac � sAMPA�t� F�t�, whereF(t) is the facilitation variable].
Dynamics of the facilitation or depression variables are given inSupplemental Table S1. The facilitation variable represents empiri-cally the gradual decay of calcium concentration and calcium-depen-dent receptor activation that follows calcium influx at the axonterminal following each action potential. The depression variablerepresents empirically the fraction of vesicles available for release,which decreases following each presynaptic spike but recovers to itsbase level with a time constant dependent on vesicle recycling.
Long-term plasticity. For all plastic connections, changes in syn-aptic strength per trial were limited to a maximum of 50% per trial toconstrain the model to biologically observed change from in vitro andslice procedures (Bi and Poo 1998; Maffei et al. 2006), while acrossall trials, synaptic strength was hard bounded between 0 and 20�WEE (associative excitatory-to-excitatory base synaptic weight), 20�WEI (associative inhibitory-to-excitatory base synaptic weight), and10� Input synE (excitatory input synaptic weight), the initial meansynaptic weights given in Supplemental Table S2.
We modeled long-term potentiation of inhibition (LTPi) fromrecent experimental work by Maffei et al. (2006) and as in our priorarticle (Bourjaily and Miller 2011a). LTPi occurred when an inhibi-tory cell spiked and its postsynaptic excitatory cell was depolarized(explicitly modeled with a voltage threshold) but silent, as describedin Supplemental Table S1. If the excitatory cell was co-active (i.e.,spiking), synapse strength underwent no change. We referred to thisas a “veto” effect in our model of LTPi. Any excitatory spike withina uniform temporal window about the inhibitory spike, the LTPiwindow, resulted in a veto. For each inhibitory spike (“non-vetoed”),the synapse was potentiated. LTPi has been reported experimentallyas a mechanism for increasing (but not decreasing) the strength ofinhibitory synapses in cortex (Maffei et al. 2006). We used homeo-stasis by multiplicative postsynaptic scaling (Turrigiano et al. 1998) atthe inhibitory-to-excitatory synapses to compensate for LTPi’s inabil-ity to depress synapses. Hard upper bounds on inhibitory synaptic
strength limited further potentiation of the inhibitory synapses of themost inhibited cells.
Triplet spike-timing-dependent plasticity (STDP) was modeledaccording to the rule published by Pfister and Gerstner (2006) usingthe parameters cited from the full model “all-to-all” cortical parametersets (see Supplemental Table S2). The weight changes are detailed inSupplemental Table S1.
Synaptic stability was maintained by multiplicative postsynapticscaling (Turrigiano et al. 1998) that was updated each trial. Thechange in synaptic strength was proportional to the difference be-tween the mean rate, r�, and a goal rate, rgoal, with a rate constant, �H,shared by all synapses. The goal rates, rgE and rgIE, were heteroge-neous about their means with an added 5-Hz random spread from auniform distribution.
DA-modulated Hebbian reward rule and reaction times. Rewardplasticity updated the associative layer excitatory to decision layerexcitatory synapses according to Supplemental Table S1. If thedecision matched the instructed cue, then reward was delivered (e.g.,for stimulus pair A � B, a “release” response resulted in reward � 1,producing LTP when pre- and postsynaptic cells were active, whereasa “hold” response to A � B resulted in reward � 0, and the sign ofthe change to the synapse was negative, i.e., LTD between coactivecells). We used this rule because it has been suggested by the DAdependence of correlational synaptic plasticity in vitro (Jay 2003;Reynolds and Wickens 2002; Shen et al. 2008) and has been appliedsuccessfully by others in computational studies (Soltani and Wang2006, 2010) for biologically based reinforcement learning. If theassociative layer contains sufficient numbers of stimulus pair-selec-tive cells, then this rule will maximize reward and generate selectivepotentiation from excitatory associative layer to excitatory decisionlayer cells necessary for reliable decision-making.
Decisions and reaction times were calculated as follows. Thereaction time for a decision was made when the mean activity of oneexcitatory pool (e.g., release) in a 20-ms time bin was at least 20 Hzgreater than that of the opposite excitatory pool (e.g., hold). Weaveraged reaction times across the final 80 trials and then across the4 random instantiations for each of the 25 networks. For a reward of1, we required the “correct” excitatory pool to maintain at least 20 Hzgreater activity than that of the opposite pool in the final time bin ofthe stimulus presentation. Repeating these calculations with alternatebin sizes of 10, 50, and 100 ms did not produce significant differencesin reaction times.
Single-Stimulus and Stimulus Pair Selectivity Metric
Single-stimulus selectivity defined each neuron’s selectivity forone stimulus (e.g., A) over the other three stimuli (e.g., B, C, and D).Likewise, stimulus pair selectivity defined each neuron’s selectivityfor one stimulus pair (e.g., A � B) over the other three stimulus pairs(e.g., A � D, C � B, and C � D). We defined selectivity, Si, of eachexcitatory neuron i as its maximum firing rate minus its meanresponse across all four stimuli, normalized by the mean response:Si � �ri
max � �ri��⁄�ri�, where rimax was its maximal response and �ri�
was its mean response.
Task Performance Metrics, Statistics, and Curve Fitting
We addressed how short-term plasticity affected a network’s ac-curacy at biconditional discrimination, a nonlinear XOR operation.Linear operations can produce up to 75% correct responses; thus wedesignated networks with response accuracy in the range of 76–84%as borderline networks, whereas those with �85% correct weredesignated as reliable. Final decision-making accuracy (% correct)was calculated from the mean of the final 80 trials.
We computed statistical significance between networks with dif-ferent synaptic dynamics using a two-sample t-test. Specifically, wecompared 4 instantiations of each of the 25 network input configura-
tions. We also combined the network input configurations and instan-tiations to strengthen statistical power (4 instantiations, 25 networkinput configurations, 100 networks total) before statistical comparisonby paired t-test for an overall global change in accuracy.
To assess whether cells preferentially strengthened connections tothe congruous decision-making pool, for each cell we calculated aresponse bias as the mean firing rate to stimuli requiring a response of“release” minus that to stimuli requiring a response of “hold” (i.e.,rrelease � rhold) and a connection bias as the mean synaptic weight tocells in the release pool minus that to cells in the hold pool (i.e.,Wrelease � Whold).
Numerical Procedures
Simulations were run for at least 400 trials using the Euler-Maruyama method of numerical integration with a time step dt � 0.02ms. To ensure network stability, we ran key networks up to 2,000trials with sustained results. For all configurations, we repeated withfour random instantiations of initial network structure, cell/synapseheterogeneity, and noise. Robustness was further ensured by simulat-ing key networks using 10 distinct random instantiations. Simulationswere written in C�� and run on Intel Xeon machines. Matlab r2011awas used for rate models, as well as all data analysis and visualization.
RESULTS
Biconditional Discrimination: Firing Rate Model
Necessity for nonlinearity of inputs to the decision-makingnetwork. In a task such as biconditional discrimination, oneexpects to find in a successful final network that those asso-ciative layer cells most selective to a particular stimulus pairconnect preferentially to congruous decision-making neurons,namely, those whose activity generates the correct response tothat stimulus pair (Fig. 1B). That is, associative layer neuronsmost responsive to stimulus pairs A � B or C � D shouldconnect to the release pool of neurons, whereas those associa-tive layer neurons most responsive to stimulus pairs A � D orC � B should connect more strongly to decision layer neuronsin the hold pool. However, even with this appropriate connec-tivity, the output from these cells to the decision layer neuronsmust be a nonlinear function of the associative layer inputs forthe network to produce a correct response in a biconditionaldiscrimination task.
If the associative layer neurons operate in a linear regime,the response of a cell to its preferred stimulus pair (e.g., its
Fig. 1. Synaptic nonlinearity can combine with asymmetric connectivity to produce above-chance performance in biconditional discrimination. Simplifiednetworks are shown with linear input-output associative layer neurons. A: the initial pretrained linear network has all possible connections, so excitatoryassociative layer cells do not preferentially project to either decision-making pool. B: after training with synaptic plasticity, linear excitatory associative layerneurons project selectively to either the release or hold decision-making pool. For the example A � B trial, each input (A, B) fires at 20 Hz, generating a 40-HzAB cell that projects to release but also is matched by the combined output of 2 cells, AD and CB, that each fire at 20 Hz and project to hold. Thus the totalsynaptic transmission from information-containing cells in the associative layer is equal to the 2 decision-making pools, leading to chance performance.C: synaptic facilitation produces superlinear synaptic transmission, which generates nonlinear input to the 2 decision-making pools, leading to the ability to formreliable decisions. (Arrow thickness represents strength of synaptic transmission.) D: synaptic depression with the same connectivity as static or facilitatingsynapses causes the opposite pool to win, because the congruous pool is driven by associative layer neurons firing at a higher rate but with lower synaptictransmission, which is sublinear as a function of firing rate or depressing synapses. E: however, with synaptic depression, many neurons can potentiate to theincongruous decision-making pool (e.g., A � B ¡ hold). Given sufficient numbers of such oppositely connected neurons from the associative layer to thedecision-making circuit, networks with depressing synapses can drive reliable decisions, because lower synaptic transmission from the cells with the highest firingrates to the incongruous pool allows the congruous pool to win and generate selective potentiation over trials. DA, dopamine; WTA, winner take all.
firing rate rAB to stimulus pair A � B, see Table 1) is equal tothe sum of two responses to nonpreferred stimulus pairs (e.g.,its firing rate rAD to A � D plus firing rate rCB to C � B, seeTable 1) (Fig. 1B). The corollary is that on a trial, for example,A � B, AB-selective cells that drive activity in the release poolof the decision-making layer are opposed and matched byequal drive from CB-selective and AD-selective cells, eachfiring at one-half the rate, promoting activity in the hold pool(Fig. 2A).
However, nonlinearity in the synaptic output as a function ofpresynaptic firing rate as produced by dynamic synapses (Fig.2A) can produce preferential input to one of the decision-making pools and improve performance. Short-term facilita-tion produces superlinear synaptic transmission rates, whereasdepression produces sublinear synaptic transmission rates. Be-cause the facilitation function of the rates is superlinear, thesynaptic transmission rates to the decision-making network forAB neurons during trials A � B is more than double the sumof the synaptic transmission rates from AD and CB neurons
firing at one-half the rate [e.g., F(rAB) � F(rAD) � F(rCB)](Fig. 2A). Thus, on each trial in the presence of facilitation, themaximally driven cells will reliably drive the correct decision-making pool (Fig. 1C), producing improved task performance(Fig. 3A).
Short-term depression produces the opposite result ofsynaptic facilitation such that a maximally driven cell pro-duces less synaptic transmission than the sum of that pro-duced by the two secondarily driven cells, which fire atone-half the rate [e.g., D(rAB) D(rAD) � D(rCB)] (Fig.2A). This leads to poor decisions if the associative layerneurons become connected to the decision layer neurons viadopaminergic learning (i.e., reward-based plasticity) asshown in Fig. 1, A and D. Intriguingly, if the dopaminergiclearning could lead to the connectivity shown in Fig. 1E,with associative layer cells more strongly connected to theincongruous decision-making pool, then the nonlinearity ofsynaptic depression would be sufficient to solve the taskgiven linear associative layer responses. Indeed, Fig. 3, A
Fig. 2. Impact of nonlinear synaptic transmission as a function of presynaptic activity in a biconditional discrimination task. A: synaptic transmission rates asa function of mean firing rates are shown as solid curves. Static synapses produce linear synaptic transmission (blue line). This is problematic for biconditionaldiscrimination, because on an A � B trial the activity of the most responsive cell (e.g., AB; black marker on blue line) is always equal to the combined activityfrom 2 cells receiving input from either A or B, but not both (e.g., AD and CB; blue circle at double the value of the blue cross). Thus these synapses produceequal input to the 2 decision-making pools, as shown in B. Facilitating synapses produce superlinear synaptic transmission: r�Prel
F � � r�P0F � fFr�F�⁄�1 �
fFr�F� (Dayan and Abbott 2001), which results in the synaptic transmission rates produced by the maximal AB cell (black marker on green line) being greaterthan the sum of the synaptic transmission rates from the AD and CB cells (green circle at double the value of the green cross). Depressing synapses producesublinear synaptic transmission rates: r�Prel
D � � rP0D⁄�1 � �1 � Dfrac�r�D�, which results in the synaptic transmission rates produced by the maximal AB cell (black
marker on red line) being less than the sum of the synaptic transmission rates from the AD and CB cells (red circle at double the value of the red cross).B–D: linearly nonseparable tasks [e.g., exclusive-or (XOR)] such as biconditional discrimination can be aided by the appropriate type of efferent synapse type(blue, static; red, depressing; or green, facilitating) depending on the nonlinearity of the input-output function in the operating regime of neurons in the associativelayer. Colored lines represent constant synaptic output as a function of the firing rate of 2 cells, 1 receiving A and B inputs (y-axis) and the other receiving Cand D inputs (x-axis). The mean rate of the cells (across all 4 input combinations) is r0. B: neurons with linear responses and static synapses cannot solve linearlynonseparable tasks, because no straight line can separate AB/CD from AD/CB, but the nonlinear synaptic transmission of facilitation or depression can separatethe responses to solve the task. C: if neurons in the associative layer have a superlinear response, as produced by the spiking threshold, then responses to theoptimal stimulus pair are more than double the response to a single component of the pair. In this case, static synapses are sufficient, but the superlinear synaptictransmission of facilitation enhances separability. Depression, however, with its sublinear synaptic transmission, compromises the threshold nonlinearity,reducing or eliminating any separability achieved. D: if neurons are in a saturating regime, for example, if firing near their maximal rates, the separability achievedby static synapses with linear output is enhanced by depressing synapses but is compromised by facilitating synapses.
and C (left columns, middle rows) demonstrates such aresult in the linear associative layer network.
In Fig. 2, B–D, we motivate an expected interaction betweenany nonlinearity of the firing rate responses of associative layerneurons with nonlinearity in their synaptic transmission to thedecision layer neurons. We assume four types of associativelayer neurons, identified by their preferred stimulus pair. Thusthe network’s response to any input combination can be sum-marized by the firing rates of these four types of cells andexpressed as a single point in a 4-dimensional (4-D) space offiring rates. For visualization, we project the 4-D space into2-D, plotting the rate of the AB-responsive cells against therate of the CD-responsive cells to each of the four possiblestimulus pairs. Figure 2B is an example of such a plot andsummarizes the results of Fig. 2A. In particular, the fourresponses fall on a straight line (the response to A � D is equalto the response to C � B in this projection, so only 3 circles arevisible), indicating the linear dependence for all linearly re-sponsive neurons:
riA�B � ri
C�D � riA�D � ri
C�B � 0
for all cells i whose firing rate increases linearly with inputs A,B, C, and D. Equivalently, the combination of firing rates of thefour cell types is linearly dependent:
rABX�Y � rCD
X�Y � rADX�Y � rCB
X�Y � 0
for each stimulus pair X � A, C; Y � B, D. Thus, if synaptictransmission is simply proportional to presynaptic firing rate
(dashed blue line denotes constant synaptic transmission), thenthe stimulus pairs cannot generate opposite responses. How-ever, since the sign of
F(rABX�Y) � F(rCD
X�Y) � F(rADX�Y) � F(rCB
X�Y) � 0
is positive for the pairs X � Y �A � B and X � Y � C � Dand opposite for the pairs X � Y � A � D and X � Y � C �B, synaptic transmission through facilitating synapses doesseparate these stimulus pairs (green line). Similarly, for de-pressing synapses,
D(rABX�Y) � D(rCD
X�Y) � D(rADX�Y) � D(rCB
X�Y) � 0,
although the sign of the result is opposite, so the points fall onthe opposite side of the line of constant synaptic transmission(red curve) compared with facilitation (green curve).
In Fig. 3, the left columns demonstrate how either facilitat-ing or depressing synaptic transmission can indeed producehigh mean reward accumulation from neurons with linearresponses. The differences in signs of the nonlinearity leaddepressing synapses to connect preferentially to the incongru-ous decision-making pool (Fig. 3C, left column, middle row),as expected.
We should note that the schematic diagrams of Fig. 2, B–D,have two shortcomings. First, they are a 2-D projection of whatis a 4-D space. However, the symmetry in our simplified modelmeans the arguments are still valid; in particular, when the 4points are linearly dependent, they fall on a 3-D hyperplane of4-D space but are seen to fall on a line in this 2-D projection.
Fig. 3. Interaction of firing rate and synaptic nonlinearity in a firing rate model. A and B: fraction of reward accumulated over all trials (blue � 0; green � 0.5;red � 1). C and D: asymmetry in the synaptic strengths from associative layer excitatory cells to decision layer pools (blue � asymmetry of �1, i.e., input tothe incongruous pool; green � asymmetry of 0, i.e., equal input to both pools; red � asymmetry of 1, i.e., input to the congruous pool). In A and C, the presynapticrate dependence of the reinforcement learning rule matches the synaptic input to the decision layer cells. In B and D, the reinforcement learning rule producingthe maximum reward for each combination of firing rate and synaptic nonlinearity is used. Stat, static; Dep, depression; Fac, facilitation; Sublin, sublinear;Supralin, supralinear; f-l, firing rate-linearity relationship.
Similarly, when a line can correctly separate the points in the2-D projection, then any 3-D hyperplane that intersects the linewill separate them in the full 4-D space.
The second shortcoming lies in the fact that just because thepoints can be correctly separated, meaning a set of synapticweights can be chosen to correctly bias the decision-makingpools, it does not mean that all reinforcement learning ruleswill lead to such a solution. Indeed, Fig. 3 demonstrates adependence on the reinforcement learning rule. In biologicalterms, if the DA-modulated reward-based learning rule ispurely postsynaptic, then any dependence on presynaptic firingrate must be through synaptic transmission. Thus a depressingsynapse would lead to the presynaptic rate dependence of thereinforcement rule being sublinear, whereas for a facilitatingsynapse it would be superlinear, and for a static synapse itwould be linear. Figure 3, A and C, show the results with sucha reinforcement rule that matches the synaptic transmission.Alternatively, if we select the reinforcement rule (either linear,sublinear, or superlinear in presynaptic rate) that maximizesreward, we obtain the results from Fig. 3, B and D. The keyfeatures of the results do in fact match our insight fromnonlinear synaptic transmission rates from Fig. 2, B–D. Inparticular, for a sublinear (saturating) neural response, thesublinear, depressing synapses produce greatest reward,whereas for superlinear (threshold) neural responses, the su-perlinear, facilitating synapses produce greatest reward. If theneural response is sublinear, or in some cases, if synapses aredepressing, then a negative asymmetry arises in the synapticconnections, meaning that neurons connect preferentially to theincongruous decision-making pool, as in Fig. 3C (middle row).
Static synapses. Static synapses from excitatory cells in theassociative layer (Fig. 4B) to the decision-making layer (Fig.4C) served as a control to assess the effect of facilitating ordepressing synapses. In the biconditional discrimination task,one untrained associative layer with sparse input (1/20) couldproduce reliably correct decisions (at least 85% correct) bytraining only the excitatory synapses from the associative layerto the decision-making layer’s excitatory cells via a DA-modulated reward-based Hebbian learning rule (5 other un-trained networks produced borderline reliability of 76–84%correct). However, the addition of correlation-based synapticplasticity (LTPi and triplet STDP), at inputs to and at recurrentsynapses within the associative layer, increased stimulus pairselectivity and led to greater reward compared with the un-trained associative layer (both with reward-based plasticity)(Bourjaily and Miller 2011a). Multiple trained associativelayers (24%, 6 of 25) generated reliably correct decision-making with additional borderline networks (20%, 5 of 25)(Fig. 4F).
Facilitating synapses. Because facilitating synapses producesuperlinear synaptic transmission rates (Fig. 2A), we expectedthat facilitating synapses would improve task performancerelative to static synapses by enhancing the superlinearityalready present in the responses of associative layer cells.Indeed, the addition of facilitation increased the number ofrandomly connected initial networks that produced reliablycorrect decisions (8%, 2 of 25) as well as the number ofborderline networks (20%, 5 of 25). Furthermore, nearly one-half of the trained associative networks (44%, 12 of 25)
generated reliable decisions, whereas 20% were borderline (5of 25) (Fig. 4G). Of the 12 reliable networks, 6 were signifi-cantly improved (P 0.01) over networks using static syn-apses. Overall, 11 of 25 trained facilitating networks weresignificantly improved over those using static synapses (P 0.05), and across all trained networks and input configurations,there was a significant improvement of �7% in decision-making accuracy (P 0.001) in trained networks with facili-tating synapses compared with those with static synapses to thedecision-making layer.
To test whether facilitation simply increased accuracy onlybecause it increased the strength of input from the associativelayer to the decision layer, we varied initial synaptic strength.We observed the same number of reliable networks across arange of �25% to �25% of the default strength and a decreaseof 2–3 reliable networks at 50% of the default value. The lossof reliability arose because either no decision-making poolresponded (too little drive) or both pools responded (too muchdrive), whereas networks with static synapses always produceda single active decision-making pool; indeed, their synapticinput strengths had been optimized to avoid such extremes.Thus the cause of increased performance was not simply amatter of linearly scaling the synaptic input strength.
The addition of facilitating synapses reduced the minimalamount of stimulus pair selectivity necessary to generate reli-able decisions to a value of 0.88, compared with the value of1.23 with static synapses. Furthermore, facilitating synapsesreduced the mean time to reach a decision across all networks,on average 52 ms, compared with a value of 118 ms for staticsynapses, with the difference being significant for all 25 inputconfigurations (P 0.01 by 2-sample t-test in each case).Thus, for the biconditional discrimination task, facilitationenhanced both speed and accuracy of decisions.
Depressing synapses. Depressing synapses produce sublin-ear synaptic transmission rates, counter to the observed super-linear activity responses in the associative network, so weexpected they would worsen performance. Indeed, no un-trained random associative networks with depressing synapsesproduced reliable decisions. Furthermore, only two of thetrained associative networks (8%, 2 of 25) generated reliabledecisions, of which one was significantly worse than theequivalent network with static synapses (P 0.05), as well asa small number of networks with borderline accuracy (25%, 7of 25) (Fig. 4E). Eight of 25 trained networks with depressingsynapses were significantly worse than those with static syn-apses (P 0.05), and there was a corresponding significantdecrement in average decision-making accuracy of �13%(P 0.001). The reduction in accuracy corresponded to a needfor greater stimulus pair selectivity, a minimal value of 1.35,for reliable responses. Reaction times were also significantlylonger for networks with depressing synapses (mean 158 ms)compared with static synapses, (mean 118 ms), with 17 of 25networks producing significantly slower responses (P 0.01).Thus depressing synapses worsened performance by both in-creasing reaction time and decreasing accuracy.
Selective potentiation of afferent synapses to the decision-making network. We quantified selective potentiation acrossnetworks by the strength of afferent synapses to the congruousdecision-making pool, that is, the decision-making pool, pro-ducing reward for the preferred stimuli of cells (e.g., thestrength of synapse from an AB-tuned cell to “release”), versus
Fig. 4. Inputs and architecture for the full spiking network with decision-making accuracy. In this study, we have examined the robustness of our findings acrossnetworks with varying input correlation and input probability, thus creating multiple network conditions. In A, we provide a general description of our inputsand show an example of input configurations from a single stimulus-responsive input population for 2 different input correlations (see MATERIALS AND METHODS,Spiking Network Simulations). Each input population is represented by 20 cells, which output independent Poisson spike trains but correlated connections. Thetotal number of independent sets of connections ranges from 2 to 20. Top: with 2 independent inputs per stimulus population and low input connection probability(1/5 is shown), associative layer cells receive initially sparse selective inputs. Bottom: with 10 independent inputs per stimulus population and low inputconnection probability (1/5 is shown), associative layer cells receive more uniform, less correlated inputs. Poisson input groups randomly project to asparse-random recurrent network of excitatory (red) and inhibitory (blue) cells. Input projection probability ranges from 1/20 (sparse) to 1/2 (dense), with inputconnections selected independently between each set of independent input cells per input population. B: the network consists of an input population of Poissonindependent inputs per stimulus that project randomly (represented by dashed lines) to a random recurrent network of excitatory (red) and inhibitory (blue) cellscalled the associative layer. Excitatory-to-excitatory connections (arrows) and inhibitory-to-excitatory connections (colored circles) are probabilistic and plastic.All-to-all inhibitory-to-inhibitory synapses are also present but not plastic. Triplet spike-timing-dependent plasticity (STDP) occurs at excitatory-to-excitatoryand input-to-excitatory synapses, whereas long-term potentiation of inhibition (LTPi) occurs at inhibitory-to-excitatory synapses. Homeostasis by multiplicativesynaptic scaling is present on all plastic synapses within the associative layer. Inhibition is feedforward only (i.e., the network does not include recurrentexcitatory-to-inhibitory synapses) (Bourjaily and Miller 2011a). C: excitatory cells from the associative layer project all-to-all with dynamic synapses (redarrowheads), initially with equal synaptic strength to excitatory cells in both the hold and release pools of the decision-making network, which also receive anurgency-gating signal (Cisek et al. 2009), modeled as a linear ramping conductance to ensure decisions are made each trial. The decision-making network consistsof 2 excitatory pools with strong recurrent connections, which compete via cross-inhibition. Strong self-recurrent excitation ensures bistability for each pool,while the cross-inhibition generates winner-take-all (WTA) dynamics such that only 1 population can be active following the stimulus, resulting in 1 decision(Wang 2002). Whether the motor output (based on the decision of hold vs. release) is correct or incorrect for the corresponding cue (e.g., A � B), accordingto the rules of the behavioral tasks, determines the presence or absence of dopamine (DA) at the input synapses (dashed lines, double arrowhead).D–G: biconditional discrimination decision-making network accuracy depends on afferent short-term plasticity. Each matrix contains the results for 25 networkstrained with LTPi and triplet STDP, with 5 levels of input correlation (x-axis) and 5 levels of sparseness (y-axis) in 1 of 3 conditions. The color bar in D representsstimulus pair selectivity at the population level with a range of 0–2. For decision-making networks in E–G, the color bar represents percent correct with a rangeof 50–100% correct. D: trained network stimulus pair selectivity. E: depressing synapses produce 2 reliable networks via sublinear synaptic transmission. F: staticsynapses generate 6 reliable networks due to associative learning generating selectivity within the associative layer (Bourjaily and Miller 2011a). G: networkswith facilitating synapses generate 12 reliable networks via superlinear synaptic transmission. These results demonstrate the beneficial role of facilitation forreliable decision-making. [Panels A–C reprinted from Bourjaily and Miller (2011b).]
that to the incongruous decision-making pool (e.g., the strengthof synapse from an AB-tuned cell to “hold”). Compared withcontrol networks, those with facilitating synapses produced anincrease in the number of neurons projecting to the congruousdecision-making pool and a corresponding decrease in thepercentage of synapses to the incongruous decision-makingpool. For networks with depressing synapses, we found theopposite to be the case and indeed found many cells to beconnected preferentially to the incongruous decision-makingpool (cf. Figs. 1E and 3C).
We simulated a more common task, stimulus-response as-sociation, to assess the value of nonlinear synaptic transmis-sion when the task is linearly separable. Only a single stimuluswas shown per trial and required to drive a decision. Becausesingle-stimulus selectivity is inherently greater than stimuluspair selectivity, many untrained random associative layers withstatic synapses could produce reliable decision-making re-sponses (84%, 21 of 25), a number that did not change upontraining the associative layer with LTPi and triplet STDP.However, facilitating synapses caused all but one of the trainednetworks (96%, 24 of 25) to produce reliable decision-making,without affecting the number of reliable untrained networks. Inthe same task, depressing synapses worsened accuracy, with76% (19 of 25 networks) producing reliable decision-making
from both trained and untrained associative layers. Thus again,we found facilitation to be beneficial and depression to bedetrimental to performance.
Two-Stimulus Discrimination Task: Firing Rate Model
Models of decision-making have mostly been based on atask requiring a comparison of the magnitude of two distinctstimuli (Ratcliff 1978; Usher and McClelland 2001; Wang2002), such as whether coherence of rightward or leftwardmotion is greater in a pattern of randomly flickering dots(Shadlen and Newsome 1996). Thus we assessed how accuracyin such a task was affected by dynamic synapses at the inputsto the decision-making circuit. Figure 5A indicates that facili-tating synapses boosted decision-making accuracy, whereasdepressing synapses reduced accuracy. We found this to betrue both when all of the noise in the system was in the inputs(using a deterministic decision-making circuit) as well as whenthe decision-making circuit itself was noisy. The deterministiceffect was a straightforward consequence of the nonlinearity ofsynaptic transmission of dynamic synapses as a function offiring rate: facilitation enhanced whereas depression decreasedthe difference in postsynaptic conductance resulting from adifference in presynaptic firing rates (Dayan and Abbott 2001).
Dynamic synapses increase noise in the postsynaptic con-ductance, because they add variability to the amplitude ofconductance pulses in addition to the variability in timing of
Fig. 5. Facilitating synapses enhance accuracy in a 2-stimulusdiscrimination task. A–C: inputs differing by 5% of their meanare transmitted to separate decision-making pools throughfiring rate model cell groups, which respond either linearly (A),superlinearly (B), or sublinearly (C). Baseline synaptic strengthof the inputs to the decision-making circuit is varied anddecision-making accuracy is measured for input synapses thatare static (blue), facilitating (green), or depressing (red). Per-formance peaks for a given total input that produces activationof just 1 of the 2 decision-making pools (no pools are active ifinput is too weak, whereas both are active if input is toostrong). D: signal-to-noise ratio (SNR) of the postsynapticconductance resulting from a Poisson input train is enhancedthrough facilitating synapses (green) and reduced through de-pressing synapses (red) compared with static synapses (blue).Solid lines indicate simulated data; dashed lines represent theanalytic formula, assuming the synaptic time constant is muchshorter than interspike intervals, an approximation that fails athigh firing rates (see Appendix). Only the steady-state re-sponses are measured and compared. Signal is produced by adifference in the presynaptic firing rate equal to 5% of the meanrate.
spikes (see Lindner et al. 2009 and Appendix). However, forthe typical parameters used, and even if all noise was at thelevel of inputs, via a combination of Poisson spike timing andfluctuating input firing rates, facilitating synapses led to higherdecision-making accuracy (Fig. 5A) by increasing the signal-to-noise ratio (Fig. 5B). The benefit of facilitating synapses waseven more striking if noise was greater within the decision-making circuitry than at the inputs, because in that case thedeterministic enhancement of the signal by synaptic facilitationdominated its effect (data not shown). For all conditions withwhich we have studied the two-stimulus discrimination task,depressing input synapses reduced accuracy, as expected, be-cause they reduced the downstream effect of any differencebetween input signals (Dayan and Abbott 2001) as well asenhancing noise (Lindner et al. 2009).
DISCUSSION
Short-term plasticity is a well-established feature of syn-apses in cortical circuits (Hempel et al. 2000; Markram et al.1998b; Tsodyks and Markram 1997; Wang et al., 2006).Although presynaptic frequency-dependent dynamic changesin effective synaptic strength may be an inevitable conse-quence of the biological mechanisms underlying synaptictransmission, theoretical studies have shown that these dynam-ics, on a short timescale, can be extremely beneficial formultiple cortical functions (Maass and Zador 1999): filteringinputs in the frequency domain (Farajidavar et al. 2008; Mat-veev and Wang 2000; Natschlager et al. 2001; Tsodyks et al.1998), dynamically controlling gain to enhance sensitivity tochanges in inputs (Abbott et al. 1997), producing a temporallag allowing for motion direction selectivity (Chance et al.1998), providing a necessary slow time constant for memorymaintenance (Barak and Tsodyks 2007; Martinez-Garcia et al.2011; Mongillo et al. 2008), and producing an optimal estima-tor of presynaptic membrane potential dynamics (Pfister et al.2010), among others. Here, we show how facilitating synapsescan produce improved classifiers based on a nonlinear combi-nation of presynaptic firing rates. Specifically, facilitation im-proves task performance by selectively enhancing synaptictransmission to a decision-making circuit from those input cellsmost responsive to a given stimulus. On the contrary, depress-ing synapses worsen performance by relatively enhancingsynaptic transmission from less responsive cells. The effects offacilitation and depression depended monotonically on the timeconstant and amplitude of short-term plasticity. Enhancing theamplitude of facilitation and depression boosted their respec-tive positive and negative changes in task performance more sothan increasing the time constants.
Biconditional discrimination employs XOR logic, makingit linearly nonseparable; synaptic facilitation provides non-linearity necessary to solve such a task, even when associa-tive layer neural responses are a linear combination of theirinputs. In principle, any nonlinearity should allow a networkto produce suitable performance in an XOR-like/linearlynonseparable task; thus we assessed whether depressingsynapses could also be beneficial given the proper condi-tions. Indeed, for linearly responsive neurons, depressingsynapses lead to good performance if associative layer neu-rons connect most strongly to the decision-making pool thatproduces the incongruous response to that of their preferred
stimulus (e.g., AB associative neurons to “hold” decisionpool neurons) (Figs. 1E and 5). Consistent with this expec-tation based on linear neurons, in simulated networks ofspiking neurons trained with depressing synapses, we foundan excess of these “incongruous” connected neurons. How-ever, performance in these simulated networks proved to beworse with depressing synapses, because the saturating ef-fect of synaptic depression on neural output opposed thesuperlinearity of the firing rate curve of spiking neurons inthe associative layer. Thus, although synaptic facilitationenhanced the existing nonlinearity in the response of abiophysically based neural circuit, enhancing performance,synaptic depression reduced the nonlinearity, producingworse performance.
In addition to testing the afferent synapses to the deci-sion-making network from the associative layer, we alsoincorporated short-term plasticity at the associative layersynapses plastic for triplet STDP and LTPi, independently/in turn.However, we did not find that the addition of short-termplasticity at these locations to and within the associativelayer led to any appreciable difference in decision-makingperformance compared with static synapses as when appliedto afferent synapses to the decision-making network, aspresented here.
Many decision-making studies have looked at single-stimulus discrimination tasks based on motion coherence(Mensh et al. 2004; Wang 2002), natural images (Seung2003; Theodoni et al. 2,011), and orientation (Suh et al.2003), among others. Thus we included a stimulus-responseassociation task and a discrimination task, to generalize ourfindings. Although responding selectively to distinct singlestimuli is much easier than to stimulus pairs, facilitation anddepression of inputs to the decision-making circuit stillimproved and worsened performance, respectively, in suchtasks. Similarly, in a model of a discrimination task basedon motion coherence (Shadlen and Newsome 1996), facili-tation of synaptic input to the decision-making circuit en-hanced discriminability, even when all of the noise in thesystem was in the stimulus and so transmitted through thesame facilitating synapses, whereas depression of input syn-apses decreased discriminability (Fig. 5).
In summary, we find for three distinct types of tasks thatnetworks with facilitating afferent synapses to a biologicallybased decision-making circuit produce responses that are bothsignificantly more accurate (i.e., more often correct) and sig-nificantly quicker than those with either static or depressingsynapses. In linearly nonseparable tasks, the superlinear syn-aptic transmission of facilitating synapses (Dayan and Abbott2001) is advantageous because it boosts the inherent superlin-earity in neural firing rates (Anderson et al. 2000; Carandiniand Ferster 2000; Miller and Troyer 2002). In linearly separa-ble tasks, facilitation can boost the signal-to-noise ratio byincreasing the signal without producing a concomitant increasein the input noise (Lindner et al. 2009). Thus we expectsynapses from associative areas to decision-making areas to befacilitating rather than depressing, as has been shown in hip-pocampal projections to prefrontal cortex (Mulder et al. 1997)and internal prefrontal connections (Berger et al. 2009; Hempelet al. 2000; Wang et al. 2006). We suggest that the optimaldynamics of a synapse may reflect not only the statistics of thepresynaptic membrane potential (Pfister et al. 2010) but also
the type of information it needs to transmit, given the compu-tational function of the circuit.
APPENDIX: SIGNAL-TO-NOISE RATIO FOR THE DIFFERENCEIN SYNAPTIC INPUT BETWEEN TWO POISSON SPIKE TRAINS
A decision-making circuit produces a decision based on the differ-ence between two sets of synaptic inputs. We define a signal-to-noiseratio (SNR) for the difference between two synaptic conductanceinputs, g1 and g2, from two presynaptic cells with respective rates r1
and r2, as
SNR ��g1 � g2�
�g1�g2
��g1� � �g2�
��12 � �2
2, (A1)
where �g1� and �g2� are the expected means of the two inputs withstandard deviations �1 and �2, respectively. Note that any staticscaling of the conductance of synapses will scale the means andstandard deviations equally, not affecting the SNR.
However, the nonlinear rate-based filtering of a facilitating synapseincreases the mean conductance of one set of inputs more than it doesthe other. In particular, if r1 � r2, then synaptic conductances arescaled, respectively, by the facilitation variables F1 and F2, with F1 �F2. Yet, because spike trains are not regular, the dynamics of facili-tating synapses produces spike-to-spike variability in the postsynapticconductance, which adds to the variability in postsynaptic input. Thuswe must ask whether the boost to the signal produced by facilitatingsynapses is greater than the boost to the postsynaptic noise. We followthe methods of Lindner et al. (2009) below, to produce the formulasfor mean and variance of synaptic transmission at steady state, via thesynapses of our model.
Postsynaptic responses are defined by the conductance g(t), whichincreases to g0F(t�)D(t�) immediately following a presynaptic spikeat time t and then decays exponentially according to
dg(t)
dt� �
g(t)
�s. (A2)
We assume the time constant �s is significantly shorter than theinterspike interval. For AMPA receptor-mediated synapses, �s � 2ms, so this condition is met for most realistic presynaptic spike trains.Note that F(t�) � 1 for static and depressing synapses, whereasD(t�) � 1 for static and facilitating synapses. In the following, wereproduce the analysis for facilitating synapses and later state thecorresponding results from the equivalent derivation for depressingsynapses.
For a spike train of average firing rate r, the mean postsynapticconductance is
�g� � g0r�s�F�� (A3)
where �F�� � �F�t��� is the expected mean facilitation variableimmediately prior to a spike. The difference in mean conductance oftwo inputs to a decision-making circuit (the numerator in Eq. A1) is
��g� � �g1� � �g2� g0�r�sF� � g0r��s��F� , (A4)
where we assume �r r� and r1 � r� � �r ⁄ 2, r2 � r� � �r ⁄ 2,
�F1� � F� � ��F� ⁄ 2, and �F2� � F� � ��F� ⁄ 2.The variance in postsynaptic conductance is calculated as
�g2� �g0
2r�s�F2�2
�g0
2r�s�F�2
2�
g02r�s
2�F
2 , (A5)
where �F2 is the variance in F. Thus the variance in conductance can
be calculated as
�g2 � �g2� � �g�2 � g0
2r�s�F�21
2� r�s� �
g02r�s
2�F
2 , (A6)
where we have assumed r�s 1. It can be seen for synapticfacilitation that F � 1, so the conductance variance, and hence noisein the postsynaptic cell, is higher because of both a multiplica-tive increase in the conductance (first term) and, through �F
2, thevariance in F. For the input to a decision-making circuit, the variance(the noise term) in the denominator of Eq. A1 is increased as
�12 � �2
2 2g02 r��sF�
21
2� r��s� � g0
2 r��s�F0� . (A7)
Combining Eqs. A4 and A7 with Eq. A1 leads to the SNR for dynamicsynapses,
SNR �g0�r�sF� � g0r��s��F�
�2g02 r��s�F��21
2� r��s� � g0
2 r��s�F2�
, (A8)
which requires us to calculate the mean and variance of the facilitationvariable, F.
We note that F decays to 1 between spikes with a time constant�F as
dF
dt�
1 � F
�F, (A9)
whereas immediately following a spike, it increases from its priorvalue, F�, to F� according to
F� � F� � fF 1
p0� F�� � F� � fF(Fmax � F�), (A10)
where fF is the facilitation factor indicating the amount of facilitationand p0 is the initial, baseline vesicle release probability in the absenceof spikes. The term in brackets prevents the release probabilityincreasing above unity so that the maximum value of facilitation,Fmax, is 1/p0. The postsynaptic conductance produced by each spike isproportional to F�, the value of F immediately before the spike,whose mean and variance we now calculate.
For a Poisson train, the expected value of F� is given by (seeDayan and Abbott 2001)
�F�� � 1 �
r�FfF 1
p0� 1�
1 � r�FfF. (A11)
Thus, for the two input trains to a decision-making circuit, thedifference in facilitation factor becomes
��F� � �F1� � �F2� �r
�FfF 1
p0� 1�
(1 � r��FfF)2 ��r(�F�� � 1)
r�(1 � r��FfF).
(A12)
From calculations similar to those leading to Eq. A9 (see Dayan andAbbott 2001 and also Lindner et al. 2009), the variance in F� is given by
Var(F�) �
r�FfF2 1
p0� 1�2
(1 � r�FfF)2[2 � r�FfF(2 � fF)]; (A13)
and thus the standard deviation in F� can be written as
which becomes much smaller than the increase in synaptic response(proportional to �F��) if r�F is large compared with 1. Because thetime constant for facilitation is on the order of hundreds of millisec-onds, for typical firing rates needed to drive a decision-making circuit,the added variability in synaptic response due to variation in excit-atory postsynaptic potential (EPSP) amplitudes is small comparedwith the boost in EPSP amplitude.
Substituting Eqs. A11, A12, and A14 into Eq. A8 and assuming�r r� leads to the SNR for the difference in Poisson spike trainswith facilitating synapses as
SNR
� �r�s
1 �
r��FfF 1
p0� 1�
(1 � r��FfF)(1 � r��FfF ⁄ p0)�
�2r��s1
2� r��s� �
r�2�s�FfF2 1
p0� 1�2
[2 � r��FfF(2 � fF)](1 � r��FfF ⁄ p0)2
.
(A15)
Dashed curves in Fig. 5D are plotted using Eq. A15 with fF � 0.4 andfF � 0 for facilitating and static synapses, respectively, demonstratinga boost to SNR by synaptic facilitation.
Depressing synapses, with a fraction Dfrac of vesicles released perspike and recovery with time constant �D, lead to similar results, butthe equivalent of ��F� is negative,
��D� � �D1� � �D2� ��r
�DDfrac
�1 � r��DDfrac)2 , (A16)
so depressing synapses reduce the signal as well as increasing thenoise. Thus, for depressing synapses, the SNR is always reduced by anamount that can be calculated as
Var(D�) �r�DDfrac
2
(1 � r�DfD)2[2 � r�DDfrac(2 � Dfrac)], (A17)
to give
SNR � �r�s
�1 �r��DDfrac
(1 � r��DDfrac)�
�2r��s1
2� r��s� �
r�2�s�DDfrac2
[2 � r��DDfrac(2 � Dfrac)],
(A18)
which is shown as the dashed red curve in Fig. 5D, with Dfrac � 0.5and �D � 200 ms.
GRANTS
We are grateful for financial support for this work from National Instituteof Deafness and Other Communicative Diseases Award DC009945, the De-partment of Biology, Brandeis University, and from a National ScienceFoundation Integrative Graduate Education and Research Traineeship Awardto the Neuroscience Graduate Program of Brandeis University.
DISCLOSURES
No conflicts of interest, financial or otherwise, are declared by the authors.
ENDNOTE
At the request of the authors, readers are herein alerted to the fact thatadditional materials related to this manuscript may be found at the institutional
website of one of the authors, which at the time of publication they indicate is:http://people.brandeis.edu/�pmiller/code.html. These materials are not a partof this manuscript, and have not undergone peer review by the AmericanPhysiological Society (APS). APS and the journal editors take no responsibil-ity for these materials, for the website address, or for any links to or from it.
AUTHOR CONTRIBUTIONS
Author contributions: M.A.B. and P.M. conception and design of research;M.A.B. and P.M. performed experiments; M.A.B. and P.M. analyzed data;M.A.B. and P.M. interpreted results of experiments; M.A.B. and P.M. preparedfigures; M.A.B. and P.M. drafted manuscript; M.A.B. and P.M. edited andrevised manuscript; M.A.B. and P.M. approved final version of manuscript.
REFERENCES
Abbott LF, Varela JA, Sen K, Nelson SB. Synaptic depression and corticalgain control. Science 275: 220–224, 1997.
Abeles M. Corticonics: Neural Circuits of the Cerebral Cortex. Cambridge,UK: Cambridge University Press, 1991.
Albantakis L, Deco G. Changes of mind in an attractor network of decision-making. PLoS Comput Biol 7: e1002086, 2011.
Alkon DL, Blackwell KT, Barbour GS, Rigler AK, Vogl TP. Pattern-recognition by an artificial network derived from biologic neuronal systems.Biol Cybern 62: 363–376, 1990.
Anderson JS, Lampl I, Gillespie DC, Ferster D. The contribution of noise tocontrast invariance of orientation tuning in cat visual cortex. Science 290:1968–1972, 2000.
Barak O, Tsodyks M. Persistent activity in neural networks with dynamicsynapses. PLoS Comput Biol 3: e35, 2007.
Berger TK, Perin R, Silberberg G, Markram H. Frequency-dependentdisynaptic inhibition in the pyramidal network: a ubiquitous pathway in thedeveloping rat neocortex. J Physiol 587: 5411–5425, 2009.
Bi GQ, Poo MM. Synaptic modifications in cultured hippocampal neurons:dependence on spike timing, synaptic strength, and postsynaptic cell type. JNeurosci 18: 10464–10472, 1998.
Bourjaily MA, Miller P. Synaptic plasticity and connectivity requirements toproduce stimulus-pair specific responses in recurrent networks of spikingneurons. PLoS Comput Biol 7: e1001091, 2011a.
Bourjaily MA, Miller P. Excitatory, inhibitory, and structural plasticityproduce correlated connectivity in random networks trained to solve paired-stimulus tasks. Front Comput Biol 5: 37, 2011b.
Bunsey M, Eichenbaum H. Critical role of the parahippocampal region forpaired-associate learning in rats. Behav Neurosci 107: 740–747, 1993.
Burges CJ. A tutorial on support vector machines for pattern recognition.Data Min Knowl Discov 2: 121–167, 1998.
Carandini M, Ferster D. Membrane potential and firing rate in cat primaryvisual cortex. J Neurosci 20: 470–484, 2000.
Chance FS, Nelson SB, Abbott LF. Synaptic depression and the temporalresponse characteristics of V1 cells. J Neurosci 18: 4785–4799, 1998.
Chapman RS, Seung HK, Schwartz SE, Bird EK. Predicting languageproduction in children and adolescents with Down syndrome: the role ofcomprehension. J Speech Lang Hear Res 43: 340–350, 2000.
Cisek P, Puskas GA, El-Murr S. Decisions in changing conditions: theurgency-gating model. J Neurosci 29: 11560–11571, 2009.
Compte A, Brunel N, Goldman-Rakic PS, Wang XJ. Synaptic mechanismsand network dynamics underlying spatial working memory in a corticalnetwork model. Cereb Cortex 10: 910–923, 2000.
Conboy L, Bisaz R, Markram K, Sandi C. Role of NCAM in emotion andlearning. Adv Exp Med Biol 663: 271–296, 2010.
Curtis CE, Lee D. Beyond working memory: the role of persistent activity indecision making. Trends Cogn Sci 14: 216–222, 2010.
Daw ND, O’Doherty JP, Dayan P, Seymour B, Dolan RJ. Cortical sub-strates for exploratory decisions in humans. Nature 441: 876–879, 2006.
Dayan P, Abbott LF. Theoretical Neuroscience. Cambridge, MA: MIT Press, 2001.Eckhoff P, Holmes P, Law C, Connolly PM, Gold JI. On diffusion processes
with variable drift rates as models for decision making during learning. NewJ Phys 10: nihpa49499, 2008.
Farajidavar A, Saeb S, Behbehani K. Incorporating synaptic time-dependentplasticity and dynamic synapse into a computational model of wind-up.Neural Netw 21: 241–249, 2008.
Feng S, Holmes P, Rorie A, Newsome WT. Can monkeys choose optimallywhen faced with noisy stimuli, and unequal rewards? PLoS Comput Biol 5:e1000284, 2009.
Fiete IR, Hahnloser RH, Fee MS, Seung HS. Temporal sparseness of thepremotor drive is important for rapid learning in a neural network model ofbirdsong. J Neurophysiol 92: 2274–2282, 2004.
Glimcher PW, Rustichini A. Neuroeconomics: the consilience of brain, anddecision. Science 306: 447–452, 2004.
Gold JI, Shadlen MN. The neural basis of decision making. Annu RevNeurosci 30: 535–574, 2007.
Grand C, Honey RC. Solving XOR. J Exp Psychol Anim Behav Process 34:486–493, 2008.
Grillner S, Markram H, De Schutter E, Silberberg G, LeBeau FE.Microcircuits in action—from CPGs to neocortex. Trends Neurosci 28:525–533, 2005.
Harris JA. Elemental representations of stimuli in associative learning.Psychol Rev 113: 584–605, 2006.
Hasselmo ME, Cekic M. Suppression of synaptic transmission may allowcombination of associative feedback and self-organizing feedforward con-nections in the neocortex. Behav Brain Res 79: 153–161, 1996.
Hempel CM, Hartman KH, Wang XJ, Turrigiano GG, Nelson SB. Mul-tiple forms of short-term plasticity at excitatory synapses in rat medialprefrontal cortex. J Neurophysiol 83: 3031–3041, 2000.
Histed MH, Pasupathy A, Miller EK. Learning substrates in the primateprefrontal cortex, and striatum: sustained activity related to successfulactions. Neuron 63: 244–253, 2009.
Iannella N, Back AD. A spiking neural network architecture for nonlinearfunction approximation. Neural Netw 14: 933–939, 2001.
Jahr CE, Stevens CF. Voltage dependence of NMDA-activated macroscopicconductances predicted by single-channel kinetics. J Neurosci 10: 3178–3182, 1990.
Jay TM. Dopamine: a potential substrate for synaptic plasticity and memorymechanisms. Prog Neurobiol 69: 375–390, 2003.
Kennerley SW, Walton ME. Decision making, and reward in frontal cortex:complementary evidence from neurophysiological and neuropsychologicalstudies. Behav Neurosci 125: 297–317, 2011.
Lee D, Seo H. Mechanisms of reinforcement learning and decision making in theprimate dorsolateral prefrontal cortex. Ann NY Acad Sci 1104: 108–122, 2007.
Lindner B, Gangloff D, Longtin A, Lewis JE. Broadband coding withdynamic synapses. J Neurosci 29: 2076–2088, 2009.
Maffei A, Nataraj K, Nelson SB, Turrigiano GG. Potentiation of corticalinhibition by visual deprivation. Nature 443: 81–84, 2006.
Markram H. A network of tufted layer 5 pyramidal neurons. Cereb Cortex 7:523–533, 1997.
Markram H, Gupta A, Uziel A, Wang Y, Tsodyks M. Information processingwith frequency-dependent synaptic connections. Neurobiol Learn Mem 70:101–112, 1998a.
Markram H, Tsodyks M. Redistribution of synaptic efficacy between neo-cortical pyramidal neurons. Nature 382: 807–810, 1996.
Markram H, Wang Y, Tsodyks M. Differential signaling via the same axonof neocortical pyramidal neurons. Proc Natl Acad Sci USA 95: 5323–5328,1998b.
Martinez-Garcia M, Rolls ET, Deco G, Romo R. Neural and computationalmechanisms of postponed decisions. Proc Natl Acad Sci USA 108: 11626–11631, 2011.
Matveev V, Wang XJ. Differential short-term synaptic plasticity and trans-mission of complex spike trains: to depress or to facilitate? Cereb Cortex 10:1143–1153, 2000.
Melchers KG, Shanks DR, Lachnit H. Stimulus coding in human associativelearning: flexible representations of parts and wholes. Behav Processes 77:413–427, 2008.
Mensh BD, Aksay E, Lee DD, Seung HS, Tank DW. Spontaneous eyemovements in goldfish: oculomotor integrator performance, plasticity, anddependence on visual feedback. Vision Res 44: 711–726, 2004.
Miller KD, Troyer TW. Neural noise can explain expansive, power-law nonlin-earities in neural response functions. J Neurophysiol 87: 653–659, 2002.
Minsky M, Papert S. Perceptrons: An Introduction to Computational Geom-etry. Cambridge, MA: MIT Press, 1987.
Mongillo G, Barak O, Tsodyks M. Synaptic theory of working memory.Science 319: 1543–1546, 2008.
Mulder AB, Arts MP, Lopes da Silva FH. Short- and long-term plasticity ofthe hippocampus to nucleus accumbens and prefrontal cortex pathways inthe rat, in vivo. Eur J Neurosci 9: 1603–1611, 1997.
Nelson SB, Turrigiano GG. Synaptic depression: a key player in the corticalbalancing act. Nat Neurosci 1: 539–541, 1998.
Nordlie E, Gewaltig MO, Plesser HE. Towards reproducible descriptions ofneuronal network models. PLoS Comput Biol 5: e1000456, 2009.
Park H, Amari SI, Fukumizu K. Adaptive natural gradient learning algo-rithms for various stochastic models. Neural Netw 13: 755–764, 2000.
Pfister JP, Dayan P, Lengyel M. Synapses with short-term plasticity areoptimal estimators of presynaptic membrane potentials. Nat Neurosci 13:1271–1275, 2010.
Pfister JP, Gerstner W. Triplets of spikes in a model of spike timing-dependent plasticity. J Neurosci 26: 9673–9682, 2006.
Ratcliff R. A theory of memory retrieval. Psychol Rev 85: 59–108, 1978.Reynolds JN, Wickens JR. Dopamine-dependent plasticity of corticostriatal
synapses. Neural Netw 15: 507–521, 2002.Rigotti M, Ben Dayan Rubin D, Morrison SE, Salzman CD, Fusi S.
Attractor concretion as a mechanism for the formation of context represen-tations. Neuroimage 52: 833–847, 2010a.
Rigotti M, Rubin DB, Wang XJ, Fusi S. Internal representation of task rulesby recurrent dynamics: the importance of the diversity of neural responses.Front Comput Neurosci 4: 24, 2010b.
Rolls ET, Grabenhorst F, Deco G. Decision-making, errors, and confidencein the brain. J Neurophysiol 104: 2359–2374, 2010.
Sakai K, Miyashita Y. Neural organization for the long-term memory ofpaired associates. Nature 354: 152–155, 1991.
Sakai Y, Okamoto H, Fukai T. Computational algorithms and neuronal networkmodels underlying decision processes. Neural Netw 19: 1091–1105, 2006.
Sanderson DJ, Pearce JM, Kyd RJ, Aggleton JP. The importance of the rathippocampus for learning the structure of visual arrays. Eur J Neurosci 24:1781–1788, 2006.
Seung HS. Learning in spiking neural networks by reinforcement of stochasticsynaptic transmission. Neuron 40: 1063–1073, 2003.
Shadlen MN, Newsome WT. Motion perception: seeing and deciding. ProcNatl Acad Sci USA 23: 628–633, 1996.
Shen W, Flajolet M, Greengard P, Surmeier DJ. Dichotomous dopaminer-gic control of striatal synaptic plasticity. Science 321: 848–851, 2008.
Soltani A, Wang XJ. A biophysically-based neural model of matching law be-havior: melioration by stochastic synapses. J Neurosci 26: 3731–3744, 2006.
Soltani A, Wang XJ. From biophysics to cognition: reward-dependent adap-tive choice behavior. Curr Opin Neurobiol 18: 209–216, 2008.
Soltani A, Wang XJ. Synaptic computation underlying probabilistic infer-ence. Nat Neurosci 13: 112–119, 2010.
Suh YG, Lee YS, Min KH, Park OH, Seung HS, Kim HD, Park HG, ChoiJ, Lee J, Kang SW, Oh UT, Koo JY, Joo YH, Kim SY, Kim JK, ParkYH. Novel non-vanilloid VR1 antagonist of high analgesic effects, and itsstructural requirement for VR1 antagonistic effects. Bioorg Med Chem Lett13: 4389–4393, 2003.
Takeda M, Naya Y, Fujimichi R, Takeuchi D, Miyashita Y. Activemaintenance of associative mnemonic signal in monkey inferior temporalcortex. Neuron 48: 839–848, 2005.
Wang Y, Markram H, Goodman PH, Berger TK, Ma J, Goldman-RakicPS. Heterogeneity in the pyramidal network of the medial prefrontal cortex.Nat Neurosci 9: 534–542, 2006.
Wong KF, Wang XJ. A recurrent network mechanism of time integration inperceptual decisions. J Neurosci 26: 1314–1328, 2006.
Xie X, Hahnloser RH, Seung HS. Double-ring network model of thehead-direction system. Phys Rev E Stat Nonlin Soft Matter Phys 66: 041902,2002.
Xie X, Seung HS. Learning in neural networks by reinforcement of irregularspiking. Phys Rev E Stat Nonlin Soft Matter Phys 69: 041909, 2004.
