Effects of asymmetric coupling and self-coupling on metastabledynamical transient rotating waves in a ring of sigmoidal neuronsYo Horikawa ∗
Faculty of Engineering, Kagawa University, Takamatsu, 761-0396, Japan
h i g h l i g h t s
• Metastable dynamical transient rotating waves in a ring neural network are studied.• A kinematical equation for a change in bump length of rotating waves is derived.• Conditions for the stabilization and pinning of rotating waves are obtained.• An exponential growth rate of the duration of transient rotating waves is obtained.• The growth rate depends on the strength of asymmetric coupling and self-coupling.
a r t i c l e i n f o
Article history:Received 6 May 2013Received in revised form 8 January 2014Accepted 24 January 2014
Transient rotating waves in a ring of sigmoidal neurons with asymmetric bidirectional coupling and self-coupling were studied. When a pair of stable steady states and an unstable traveling wave coexisted,rotating waves propagating in a ring were generated in transients. The pinning (propagation failure) ofthe traveling wave occurred in the presence of asymmetric coupling and self-coupling, and its conditionswere obtained. A kinematical equation for the propagation of wave fronts of the traveling and rotatingwaves was then derived for a large output gain of neurons. The kinematical equation showed that theduration of transient rotating waves increases exponentially with the number of neurons as that in aring of unidirectionally coupled neurons (metastable dynamical transients). However, the exponentialgrowth rate depended on the asymmetry of bidirectional coupling and the strength of self-coupling. Therate was equal to the propagation time of the traveling wave (a reciprocal of the propagation speed),and it increased near pinned regions. Then transient rotating waves could show metastable dynamics(extremely long duration) even in a ring of a small number of neurons.
Rings of coupled neurons (ring neural networks) have been ofwide interest since they can generate various spatiotemporal pat-terns and oscillations despite their simple structure. Central pat-tern generators, which control rhythmic motion such as walking,flying and swimming, have beenmodeled by neural networkswithclosed loops, e.g., for early work (Friesen & Stent, 1977; Kling &Székely, 1968) and reviews (Friesen & Stent, 1978; Ijspeert, 2008;Pearson, 1993). A lot of work has been carried out on the syn-chronization and bifurcations of oscillations in rings of variouskinds of neuron models, e.g., (Bazhenov, Huerta, Rabinovich, & Se-jnowski, 1998; Bonnin, 2009; Enjieu Kadji, Chabi Orou, & Woafo,2007; Ermentrout, 1985; Friesen & Stent, 1977; Grasman & Jansen,
1979; Kitajima, Yoshinaga, Aihara, & Kawakami, 2001; Kypriani-dis & Stouboulos, 2003; Linkens, Taylor, & Duthie, 1976; Oprisan,2010; Perlikowski, Yanchuk, Popovych, & Tass, 2010; Somers &Kopell, 1993; Still & Le Masson, 1999; Wang, Lu, & Chen, 2007;Wang, Lu, Chen, & Guo, 2006; Yanchuk & Wolfrum, 2008). Ringnetworks with identical neurons and coupling have been dealtwith from the viewpoint of group theory due to their symmetry(Bressloff, Coombes, & de Souza, 1997; Buono, Golubitsky, & Pala-cios, 2000; Collins & Stewart, 1994; Golubitsky, Stewart, & Schaef-fer, 1988). In this paper, a ring of coupled simple neuron modelswith sigmoidal input–output relations is considered. It has beenshown that a ring of unidirectionally coupled sigmoidal neuronscan show oscillations if it has an odd number of inhibitory cou-pling (Amari, 1978). Such a ring is qualitatively the same as a ringoscillator, which is a ring of inverters and buffers in wide use as avariable-frequency oscillator in digital circuits (Gutierrez, 1999).Although a ring of sigmoidal neurons is quite simple, its proper-ties have been studied as a basic model of recurrent neural net-works (Atiya & Baldi, 1989; Hirsh, 1989) and as a cyclic feedback
system with monotone dynamics (Gedeon, 1998). Pattern forma-tion in a ring of piecewise linear neurons and its application tosignal processing have been studied as a cellular neural network(Chua&Yang, 1988; Crounse, Chua, Thiran, & Setti, 1996; Setti, Thi-ran, & Serpico, 1998; Thiran, 1997; Thiran, Crounse, Chua, & Hasler,1995; Thiran, Setti, & Hasler, 1998). It has been shown that itsdiscrete-time version has multiple periodic orbits (Blum & Wang,1992; Pasemann, 1995). Effects of delays have also been exten-sively studied on a ring of sigmoidal neurons with delays. Variousoscillations, spatiotemporal patterns and their bifurcations havebeen shown in the cases of unidirectional coupling (Guo & Huang,2007b; Li &Wei, 2005b; Wei & Li, 2004; Wei & Velarde, 2004; Wei& Zhang, 2008), both unidirectional and self-coupling (Campbell,1999; Campbell, Ruan, & Wei, 1999; Li & Wei, 2005a; Yuan & Li,2010), bidirectional coupling (Bélair, Campbell, & van den Driess-che, 1996; Marcus & Westervelt, 1989; Wu, 1998; Zou, Huang, &Chen, 2006; Zou, Huang, & Wang, 2010), both bidirectional andself-coupling (Campbell, Yuan, & Bungay, 2005; Guo, 2005; Guo &Huang, 2003, 2006, 2007a; Huang &Wu, 2003; Hu & Huang, 2009;Yuan & Campbell, 2004) and asymmetric coupling (Lin, Lemmert,& Volkmann, 2001; Xu, 2008; Yan, 2006; Zhang & Guo, 2003). Ofparticular interest are long-lasting transient oscillations in a ring ofunidirectionally coupled sigmoidal neurons with delays (Babcock&Westervelt, 1987; Baldi & Atiya, 1994; Pakdaman, Malta, Grotta-Ragazzo, Arino, & Vibert, 1997).
Several kinds of information processing in nervous systems areconsidered to be carried out during transient states, not asymp-totically stable states such as equilibrium and limit cycles, whichrange from sensory response to decision making (Fingelkurts &Fingelkurts, 2004; Friston, 1997, 2001; Kelso, 1995; Rabinovich,Huerta, & Laurent, 2008; Rabinovich, Varona, Selverston, & Abar-banel, 2006; Werner, 2007). It is because asymptotic states maynot be realized within a short response time to stimuli. Thustransient dynamics can play a more important role in neural in-formation processing than asymptotic states. For instance, chaoticitinerancy (Tsuda, 1991, 2001, 2009) and winnerless competitionnetworks (Komarov, Osipov, & Suykens, 2010; Nekorkin, Kasatkin,& Dmitrichev, 2010; Rabinovich et al., 2001) for olfactory systems,which are related to heteroclinic cycles, have been widely stud-ied. Further, long-lasting chaotic transients have been found inseveral neural networks: diluted random networks of integrate-and-fire neurons (Jahnke, Memmesheimer, & Timme, 2008;Zillmer, Brunel, & Hansel, 2009; Zillmer, Livi, Politi, & Torcini,2006, 2007; Zumdieck, Timme, Geisel, &Wolf, 2004), discrete timerecurrent networks of spiking neurons (Cessac, 2008; Cessac &Viéville, 2008; Destexhe, 2009; El Boustani & Destexhe, 2009; Ku-mar, Schrader, Aertsen, & Rotter, 2008) and a ring of Bonhoef-fer–van der Pol models (Horikawa & Kitajima, 2012d).
Recently, metastable dynamical transient rotating waves havebeen found in a ring of unidirectionally coupled sigmoidal neuronswithout delays (Horikawa & Kitajima, 2009a). The system hasa pair of stable spatially uniform steady states and is globallybistable. In addition, unstable traveling waves rotating in a ringcan be generated owing to unidirectional coupling, and theirinstability decreases exponentially with the number of neurons.In the presence of these weakly unstable traveling wave solutions,rotatingwaves generated in transients showmetastable dynamics:their duration (life time) increases exponentially with the numberof neurons. Then the states of neurons can continue to oscillatewithin a practical time when the number of neurons is large,although the oscillations are unstable. As characteristics of thesemetastable dynamical transients, it has been shown that theduration of randomly generated rotating waves is distributed in apower law form and that spatiotemporal noise with intermediateintensity could increase the duration of rotating waves (noise-sustained propagation) (Horikawa & Kitajima, 2009c). These
properties have been derived from a kinematical equation for thepropagation of wave fronts in the rotating waves.
Their kinematics is qualitatively the same as that of a kink(front) and pairs of kink and antikink (pulses) in a symmet-ric bistable reaction–diffusion equation (the time-dependentGinzburg–Landau equation, also called as the Allen–Cahn equationand the Schlöglmodel) (Bronsard & Kohn, 1990; Carr & Pego, 1989;Ei &Ohta, 1994; Fusco &Hale, 1989; Kawasaki &Ohta, 1982). Thereis interaction between a kink and antikink, the strength of whichdecreases exponentially with the distance between them. Themo-tion of these transient spatial patterns is very slow when distancebetween a kink and antikink is large. Such extremely slow motionof interfaces is referred to as dynamicalmetastability ormetastabledynamics. Metastable dynamical patterns have been found in sev-eral reaction–diffusion (-convection) systems as well as in two andthree-dimensional spaces (Ward, 1996, 1998, 2001). Exponentiallyweak interaction also exists between pulses in neural field mod-els described by nonlocal integro-differential equations (Bressloff,2005), which are concerned with the propagation of electricalactivity in brain slices. This interaction is due to an exponentialdecrease in a synaptic weight function with distance. Dispersionrelation between speeds and interspike intervals in spikes propa-gating in a nerve fiber, and generally in excitable media describedby reaction–diffusion equations, also has the form of an exponen-tial function (Meron, 1992; Rinzel & Keller, 1973). This exponentialdependence of a propagation speed on an interspike interval comesfrom the linear relaxation of membrane activity in a recovery pro-cess. In both neural cases, a single pulse (spike) is stable in itselfand pulse locations and interspike intervals change. The changesin transients might thus not be of much interest, but they can haveconsiderable effects on signal transmission in the refractory periodin a nerve fiber (Horikawa, 1992).
The metastable dynamical rotating wave in a ring of unidirec-tionally coupled sigmoidal neurons is a spatially discretized ver-sion of metastable dynamical patterns in the spatially continuoussystems. It has been shown that such metastable dynamical tran-sient rotating waves exist commonly in bistable rings of unidi-rectionally coupled systems: bistable elements (Horikawa, 2012),cubic maps (Horikawa & Kitajima, 2012a), Lorenz systems(Horikawa&Kitajima, 2012b) andparametric oscillators (Horikawa& Kitajima, 2012c). Further, metastable dynamical rotating wavesin the form of propagating oscillations have been found in a ringof synaptically coupled Bonhoeffer–van der Pol models, in whichneurons alternate between firing and resting states in transients(Horikawa, 2011). Long-lasting transient rotating waves shownin a ring of ferromagnetic cores (Lindner & Bulsara, 2006) anda generalized repressilator model (one of genetic regulatory net-works) (Strelkowa & Barahona, 2010, 2011) are also considered tobe caused by the same metastable dynamics.
In this study, we consider the effects of asymmetric cou-pling and self-coupling onmetastable dynamical transient rotatingwaves in a ring of sigmoidal neurons. Bidirectional coupling existswidely in nervous systems and has been employed in models ofcentral pattern generators (Friesen & Stent, 1977, 1978; Ijspeert,2008; Kling & Székely, 1968; Pearson, 1993), for instance. It playsan important role in applications of neural networks, e.g., asso-ciative memory and recurrent neural networks. Self-coupling orself-feedback can also have a large effect on the dynamics ofneural networks. It has been shown that self-coupling can causeoscillations in a pair of excitatory and inhibitory sigmoidal neu-rons (Amari, 1972; Wilson & Cowan, 1972). Although spatiotem-poral patterns (steady states and limit cycles) generated in a ringof piecewise linear neurons with asymmetric coupling and self-coupling have been studied (Setti et al., 1998; Thiran, 1997; Thi-ran et al., 1998), its transient dynamics hardly seems to havebeen dealt with. In a ring of sigmoidal neurons with symmetricbidirectional coupling and without self-coupling, pairs of steady
28 Y. Horikawa / Neural Networks 53 (2014) 26–39
patterns are generated from the origin through pitchfork bifurca-tions successively. It has been shown that metastable dynamicaltransient patterns exist (Horikawa, 2013), which are similar to akink and antikink in the symmetric reaction–diffusion equation.However, these patterns are easily stabilized as a gain of a sig-moidal function increases so that metastable dynamical transientsdisappear. When there is asymmetry in bidirectional coupling,traveling waves rotating in a ring are generated instead of steadypatterns. In the presence of an unstable travelingwave, metastabledynamical transient rotating waves emerge as in a ring of uni-directionally coupled neurons. When asymmetry in bidirectionalcoupling is small, however, rotating waves are pinned, i.e., theirpropagation speeds become zero and they change into stablesteady patterns.
First, bifurcations of solutions to a ring of sigmoidal neuronswith asymmetric bidirectional coupling and self-coupling will beshown. A pair of spatially uniform steady states and an unstabletraveling wave will be generated from the origin as an outputgain of sigmoidal neurons will increase. The unstable travelingwave will be stabilized when the strength of self-coupling willbe large, and it will be pinned when the strength of self-couplingwill be large or asymmetry in bidirectional coupling will be small.There can be unsaturated neurons at wave fronts in the travelingwaves and its number depends on the strength of self-coupling andasymmetric coupling. Conditions for the pinning of these travelingwaves will be obtained by piecewise linear approximation to asigmoidal output function. These pinned steady states will agreewith those obtained in the case of symmetric bidirectional coupling(Thiran et al., 1995).
Then, a kinematical equation for the motion of wave frontsin the traveling wave will be derived for a large output gain.Conditions for the stabilization of the traveling wave will beobtained by the kinematical equation. Rotating waves will begenerated in transients and they will change into stable spatiallyuniform steady states eventually when the traveling wave willbe unstable. Changes in the length of bumps in transient rotatingwaves will be described by the kinematical equation, and it will beshown that metastable dynamics is retained. That is, the durationof transient rotating waves will increase exponentially with thenumber of neurons. It will be shown that the growth rate of theexponential increase in the duration of transient rotating waves isequal to the reciprocal of the propagation speed of the travelingwave. The growth rate will then tend to infinity as the travelingwave will become pinned. As a result, long-lasting transientrotating waves can exist in rings of small numbers of neurons.
The rest of the paper is organized as follows. In Section 2, amodel equation of a ring of sigmoidal neurons with asymmetricbidirectional coupling and self-coupling is introduced. Then thebifurcations of its steady states and traveling wave solution areshown. In Section 3, conditions for the pinning of the travelingwave are obtained by piecewise linear approximation. A kinemati-cal equation for the propagation of wave fronts in rotatingwaves isthen derived with a sign function. In Section 4, it is shown that theduration of transient rotating waves increases exponentially withinitial bump length (the number of neurons in a smaller bump) byusing the kinematical equation. This exponential increase in theirduration agrees with the results of computer simulation for largeoutput gains. It is also shown that the duration of randomly gen-erated rotating waves is distributed in a power law form over awide range when rotating waves are nearly pinned even thoughthe number of neurons is small. Finally, a conclusion is given inSection 5. In the Appendix, conditions for the pinning of rotatingwaves are derived from a system of linear differential equationsfor unsaturated neurons.
2. A ring of coupled sigmoidal neurons and its bifurcation
We consider the following ring of bidirectionally coupledsigmoidal neurons.
dxn/dt = −xn +c + d2
f (gxn−1) +c − d2
f (gxn+1) + sf (gxn)
f (x) = tanh(x) (1 ≤ n ≤ N, xn±N = xn, g ≥ 0, d ≥ 0)(1)
where xn is the state of the nth neuron, f is a sigmoidal outputfunction of a neuron, g (≥0) is an output gain, c and d arethe strength of symmetric coupling and asymmetric couplingbetween adjacent neurons, respectively, and s is the strength ofself-coupling. A periodic boundary condition is imposed so that atotal of N neuronsmake a closed loop.We let d ≥ 0without loss ofgenerality. The origin (xn = 0 (1 ≤ n ≤ N)) is always a steady stateof Eq. (1), and the eigenvalues λk of the Jacobian matrix evaluatedat the origin are given by
λk = −1 + g[s + c cos(2kπ/N) + id sin(2kπ/N)]
(0 ≤ k < N). (2)
They are located on an ellipsewith center at (−1+gs, 0), horizontalaxis of 2g|c| and vertical axis of 2gd in the complex plane. Theorigin is stable when g = 0 since λk = −1 (0 ≤ k < N).
A change in the stability of the origin with an increase in gdepends on the sign of the strength c of symmetric coupling aswell as the parity of the numberN of neurons.When c ≥ 0 orwhenc < 0 andN is even, the largest real eigenvalue:λ0 = −1+g(s+c)for c ≥ 0 or λN/2 = −1 + g(s − c) for c < 0 becomes positive atg = 1/(s+|c|)when s+|c| > 0. The origin then becomes unstableand a pair of stable spatially uniform steady states: xn = ±xs(1 ≤ n ≤ N) for c ≥ 0 or xn = ±(−1)nxs (1 ≤ n ≤ N) forc < 0 with xs = (s + |c|)f (gxs) is generated through a pitchforkbifurcation. The origin is always stable when s + |c| < 0.
When c < 0 and N is odd, however, the eigenvalue withthe largest real part is λk = −1 + g[s + c cos(π ± π/N) +
id cos(π ± π/N)] for k = (N ± 1)/2. The origin causes a pitchforkbifurcation when d = 0 or the Hopf bifurcation when d > 0 atg = 1/[s + c cos(π ± π/N)] when s + c cos(π ± π/N) > 0. Apair of generated stable steady states for d = 0 has a shape withx1 = 0 and xn = −xN−n+2 (2 ≤ n ≤ (N + 1)/2) with xnxn+1 < 0(2 ≤ n ≤ N − 1). A generated stable limit cycle for d > 0 is atraveling wave rotating in the ring, which is qualitatively the sameas a ring oscillator. The inconsistency in the signs of the states ofneurons, i.e., the location of the same signs in the states of adjacentneurons, rotates in the ring as (x1, x2, x3, x4, x5): (+, −, −, +, −) →(+, −, +, +, −) → (+, −, +, −, −) → · · ·.
In this study, we let c ≥ 0 and consider the case where Eq. (1)becomes bistable when g > 1/(s + c) > 0 through a pitchforkbifurcation at the origin with a pair of stable spatially uniformsteady states (xn = ±xs (1 ≤ n ≤ N)). The results can be appliedwhen c < 0 and N is even by reversing the signs of the states ofneurons of even numbers: x2m → −x2m (1 ≤ m ≤ N/2).
When d = 0, a further increase in g causes a degeneratedpitchfork bifurcation at the origin as a pair of the second largestreal eigenvalue λ1 and λN−1 becomes positive at g = 1/[s +
c cos(2π/N)] when s + c cos(2π/N) > 0. Two pairs of unstablespatially periodic steady states are then generated. When s = 0,the following results have been obtained (Horikawa, in prepara-tion). When N is even, one pair (type-2) has two zero-state neu-rons (x1 = xN/2+1 = 0) and a pair of positive and negative bumps(xn > 0 for 2 ≤ n ≤ N/2 and xn < 0 for N/2 + 2 ≤ n ≤ N), whilethe other (type-0) has a pair of positive and negative bumps (xn> 0 for 1 ≤ n ≤ N/2 and xn < 0 for N/2 + 1 ≤ n ≤ N) withno zero-state neurons. When N is odd, one pair (type-1) has onezero-state neuron (x(N+1)/2 = 0) and a pair of positive and neg-ative bumps (xn > 0 for 1 ≤ n ≤ (N − 1)/2 and xn < 0 for
Y. Horikawa / Neural Networks 53 (2014) 26–39 29
(N+1)/2+1 ≤ n ≤ N), while the other (type-0a) has a pair of pos-itive andnegative bumps (xn > 0 for 1 ≤ n ≤ (N−1)/2 and xn < 0for (N + 1)/2 ≤ n ≤ N). On one hand, the type-0 steady statesare stabilized through a pitchfork bifurcation as g increases so thatEq. (1) becomes multistable. On the other hand, the instability ofthe unstable steady states for small g decreases exponentially withthe number of neurons. Consequently, transient spatially nonuni-form patterns with positive and negative bumps consisting of un-equal numbers of neurons show dynamical metastability. Theirduration increases exponentially with the number of neurons ina smaller bump.
When d > 0, an increase in g causes Hopf bifurcations at theorigin as the real parts of λk become positive so that unstable limitcycles are generated. The limit cycle generated first at g = 1/[s +
c cos(2π/N)] is a traveling wave solution rotating in a ring withthe wave number k = 1, which has a pair of positive and negativebumps of equal length (consisting of a half of neurons (N/2) whenN is even). When s = 0, the following results have also beenobtained (Horikawa, in preparation). On one hand, the unstabletraveling wave is pinned as g increases further when d is small(d ≈ 0.01c). It breaks up into a saddle–node loop (a heterocliniccycle) and N pairs of unstable steady states are generated. Thegenerated steady states are quickly stabilizedwith a slight increasein g through pitchfork bifurcations. On the other hand, spatiallyasymmetric rotating waves with bumps of unequal length aregenerated in transients and they show dynamical metastability(Horikawa & Kitajima, 2009a). It is because the instability of thespatially symmetric traveling wave decreases exponentially withthe number of neurons.
First, Fig. 1 shows an example of spatiotemporal patterns oftraveling waves in Eq. (1) with c = 1.0, d = 0.5, s = −0.3, g =
10.0 and N = 10. Although the traveling waves are unstable, theyare obtained with computer simulation by imposing a symmetricinitial conditionwhenN is even. Eq. (1)was numerically integratedwith the Runge–Kuttamethod and a time step 0.001 under xn(0) =
c + s for 1 ≤ n ≤ N/2 and xn(0) = −(c + s) for N/2+ 1 ≤ n ≤ N .A top panel shows a time course of the state x1(t) (150 < t <200) of the first neuron, a middle panel shows a spatiotemporalpatterns of the states xn(t) (1 ≤ n ≤ N , 150 < t < 200), inwhich black andwhite regions correspond to positive and negativesigns, respectively, and a bottom panel shows a snapshot of xn(t)(1 ≤ n ≤ N) at t = 200. The spatial shape of the travelingwave is symmetric with respect to N/2, i.e., xn(t) = −xn+N/2(t)(1 ≤ n ≤ N/2). As d decreases to zero, the propagation speedof the traveling wave decreases and it is pinned at d ≈ 0.016.The traveling wave is one-dimensionally unstable, and the largesteigenvalue ν of its Poincaré map decreases double exponentiallywith the number N of neurons. Fig. 2 shows a semi-log plot oflog(ν)/τp of the traveling wave solution against the number N ofneurons in Eq. (1)with c = 1.0, d = 0.5, s = −0.3 and g = 2 (solidcircles), 10 (open circles) and 100 (solid squares), where τp is theperiod of the traveling wave. These values were obtained with thesoftware package AUTO (Doedel, 1996). This exponential decreasein the instability of the traveling wave indicates the existence ofexponentially long transients with the number of neurons.
Next, Fig. 3 shows bifurcation diagrams in the s–d plane withc = 1.0, N = 10 and g = 2 (a), g = 10 (b) or g = 100 (c) (d).The diagrams were obtained with AUTO and numerical calculationof Eq. (1). When s < 1/g − c , the origin is globally stable. A pairof spatially uniform nonzero states is generated at s = 1/g − c(λ0 = −1 + g(s + c) = 0), and Eq. (1) is bistable when 1/g − c <s < 0. The unstable traveling wave with k = 1 is generated ats = 1/g − c cos(2π/N) (Re[λ1] = −1 + g[s + c cos(2π/N)] = 0)when d = 0. It is stabilized through a pitchfork bifurcation at alocus PF (a dashed line) and it is pinned at a locus PIN (a dotted line).Note that it is difficult to evaluate the stability of the travelingwave
Fig. 1. Example of spatiotemporal patterns of traveling waves in Eq. (1) withc = 1.0, d = 0.5, s = −0.3, g = 10.0 and N = 10. Time course of the statex1(t) (150 < t < 200) (top panel); spatiotemporal patterns of xn(t) (1 ≤ n ≤ N ,150 < t < 200), in which black and white regions correspond to positive andnegative signs, respectively (middle panel); snapshot of xn(t) (1 ≤ n ≤ N) att = 200 (bottom panel).
Fig. 2. Semi-log plot of log(ν)/τp of the travelingwave vs the numberN of neuronsin Eq. (1) with c = 1.0, d = 0.5, s = −0.3 and g = 2 (solid circles), 10 (opencircles) and 100 (solid squares). ν: the largest eigenvalue of the Poincaré map of thetraveling wave; τp: the period of the traveling wave.
near the pinned region since its period tends to infinity, and thenthe point of the intersection of the loci PF and PIN was not traced.The loci PF and PINmove toward left as g increases so that the locusPF becomes s = c (d > c) and the locus PIN becomes s = d in thelimit of g → ∞, which will be shown in Section 3 and Fig. 5(a). Inaddition, there are other three pinned regions of the travelingwavefor small d in the left-hand side of the locus PIN. These regions alsomove toward left as g increases and they are shown for g = 100 inFig. 3(d) (Pj, 0 ≤ j ≤ 3). Actually, the traveling wave is first pinnedthrough a saddle–node loop and then the pinned steady state isstabilized through a pitchfork bifurcation, but these two loci arealmost the same for large g .
In these pinned regions, steady states have different shapes,which are shown in Fig. 4 for d = 0, g = 100 and N = 10. Theoutput of all neurons is saturated (f (gxn) ≈ −1 (1 ≤ n ≤ N/2),f (gxn) ≈ 1 (N/2+ 1 ≤ n ≤ N)), and the stable steady state has nounsaturated neurons in the region P0 (s = 0.2) (a), i.e., it is of type-0. A pair of neurons is unsaturated (f (gxn) ≈ gxn (n = N/2,N)),and the others are saturated in the region P1 (s = −0.1) (b),which is referred to as type-2. Further, two pairs of neurons areunsaturated (f (gxn) ≈ gxn (n = 1,N/2,N/2+1,N)) in the regionP2 (s = −0.5) (c), which is referred to as type-4. Traveling waves
30 Y. Horikawa / Neural Networks 53 (2014) 26–39
(a) g = 2. (b) g = 10.
(c) g = 100. (a) g = 100 (magnified).
Fig. 3. Bifurcation diagrams in the s–d plane of solutions to Eq. (1) with c = 1.0, N = 10 and g = 2 (a), g = 10 (b) or g = 100 (c) (d).
Fig. 4. Patterns of pinned steady states in Eq. (1) with c = 1.0, d = 0, g = 100 andN = 10. Pinned region P0 of type-0 (s = 0.2) (a), P1 of type-2 (s = −0.1) (b) and P2of type-4 (s = −0.5) (c).
also have different shapes corresponding to the pinned states. Inthe next section, boundaries between these regions are derivedwith piecewise linear approximation to f (x). Then, the propagationof wave fronts in traveling waves is described with a kinematicalequation for large g (→∞).
3. Kinematics of rotating waves
In this section, we derive the propagation speeds of rotatingwaves for large g (≫1). The shapes of spatially nonuniform steadystates and traveling waves vary with the strength of coupling (c , dand s) as in Fig. 4. The s–d plane is divided into N/2 − 1 regionsin which the states of 2j neurons remain in an unsaturated regionof f (gx) (0 ≤ j ≤ N/2 − 2). Fig. 5 shows a bifurcation diagram inthe s–d plane with c = 1.0 and N = 20 in the limit of g → ∞ (a)and its magnification (b). The regions of pinned steady states andtraveling waves with 2j unsaturated neurons are denoted with Pjand Tj (0 ≤ j ≤ 8), respectively. We refer to the solutions of Eq.(1) in the regions Pj and Tj as type-2j. Type-0, type-2 and type-2j(j ≥ 2) solutions are dealt with in Sections 3.1–3.3, respectively.We first derive conditions for the existence and pinning of type-2jsolutions by using piecewise linear approximation to f (x).We thenderive a kinematical equation for the propagation ofwave fronts byusing the following sign function for f (x) in the limit of g → ∞.
f (gx) = −1 (x < 0), 1 (x > 0). (3)
3.1. Type-0 solution
We let the state xn of a neuron be negatively saturated andf (gxn) = −1 when xn < −1/g , be unsaturated and f (gxn) = gxn
Y. Horikawa / Neural Networks 53 (2014) 26–39 31
Fig. 5. Bifurcation diagram in the s–d plane of solutions to Eq. (1) with c = 1.0 andN = 20 in the limit of g → ∞ (a) and its magnification (b). Regions Pj of pinnedsteady states and Tj of traveling waves with 2j unsaturated neurons (0 ≤ j ≤ 8).
when |xn| < 1/g , and be positively saturated and f (gxn) = 1whenxn > 1/g . We here denote s′ = s − 1/g (note that s′ = s forg → ∞).
The origin is unstable when s′ > −c , and a pair of stablespatially uniform steady states is xn = ±xs0 = ±(c + s) (1 ≤ n ≤
N). A travelingwavewith k = 1 is generated at s′ = −c cos(2π/N)when d > 0. Let xn−2 < −1/g , xn−1 < −1/g , xn > 1/g andxn+1 > 1/g (all saturated) in the traveling wave for N ≥ 4. Then
dxn−1/dt = −xn−1 −c + d2
+c − d2
− s
= −xn−1 − d − s = −xn−1 + xT0 (xT0 = −d − s) (4)
dxn/dt = −xn − d + s = −xn + xP0 (xP0 = −d + s). (5)
The state xn−1 remains negatively saturated if xT0 < −1/g (s′ >−d) and xn remains positively saturated if xP0 > 1/g (s′ > d)so that a wave front cannot propagate. Thus, a condition for thepinning of a type-0 solution is given by s′ > d (P0), and the locusPIN (a dotted line) in Figs. 3 and 5(a) corresponds to s′ = d. Thispinning condition has been obtained in a cellular neural networkwith a piecewise linear output function (Thiran et al., 1998). When|s′| < d, xn−1 remains negatively saturated while xn decreasesto be unsaturated so that a wave front propagates. This gives acondition for the propagation of a type-0 solution (T0). Note thatxn cannot remain unsaturated and becomes negatively saturatedsince xT0 < −1/g (s′ > −d) in Eq. (4). Strictly, we must considera change in an unsaturated xn, which is shown in the Appendix.When s′ < −d, both xn−1 and xn move to be unsaturated and oneof them can remain unsaturated so that a type-2j solution (j ≥ 1)is generated, which is dealt with in Sections 3.2 and 3.3.
Next, we consider the propagation speed of the traveling wavein the region T0 (|s′| < d). We here consider large g (→∞) and
let s′ = s. We then use a sign function (Eq. (3)) for f (x) to derivethe kinematics of a wave front. Fig. 6(a) shows time courses ofthe states of neurons at a wave front in the traveling wave in T0(c = 1.0, d = 0.5, s = 0.2, g = 1000 and N = 10). Let xn−1become from positive to negative at t = 0 (xn−1(0) = 0). Wedefine propagation time tp by xn(tp) = 0, i.e., it is the time requiredfor the propagation over one unit distance (one neuron). It is givenby solving Eq. (5) as
xn(tp) = (xn(0) − xP0) exp(−tp) + xP0= (xn(0) + d − s) exp(−tp) − d + s = 0
The propagation speed is 1/tp and it becomes zero as s → d, whichcorresponds to the above pinning condition (s = d). When thenumber of neurons is large (N ≫ 1), xn(0) is close to xs0 (=c + s)because dxn/dt = −xn+c+s and xn → xs0 for t < 0 since xn+i > 0(−1 ≤ i ≤ 1). (It takes a half period τp/2 of the traveling wavesince the preceding other wave front passed (xn−1(−τp/2) = 0),which is long when N ≫ 1.) The propagation time is then given by
The propagation time tp0 becomes zero and the speed tends toinfinity as s → −c or d → ∞.
We then consider the dependence of the propagation speed onthe length lh of bumps (a half (N/2) of the number of neurons foreven values of N) in the traveling wave. The state xn changes fort > tp as
where lh = N/2, lhtp is a half period, and we approximate tp bytp0. The state xn changes toward −d − s in tp0 < t < 2tp0 andtoward −c − s in 2tp0 < t < lhtp0. The state xn(0) is equal to−xn(lhtp0) because of the symmetry of the travelingwave, i.e., xn =
−xn+N/2. The propagation time tp(lh) of the travelingwavewith thebump length lh is then given by substituting −xn(lhtp0) for xn(0) inEq. (6).
The propagation time depends exponentially on the bump length.When the traveling wave is unstable, rotating waves are
generated in transient states until they eventually converge to thestable spatially uniform steady states. Fig. 7 shows an exampleof spatial patterns of the states of neurons in transient rotatingwaves of Eq. (1) with c = 1.0, d = 0.5, s = −0.1, g = 100
32 Y. Horikawa / Neural Networks 53 (2014) 26–39
(a) T0 . (a) T1 .
(c) T2 .
Fig. 6. Time courses of the states of neurons at a wave front in the traveling wave of Eq. (1) with c = 1.0 and g = 1000 in T0 (d = 0.5, s = 0.2 and N = 10) (a), T1 (d = 0.1,s = −0.4 and N = 12) (b) and T2 (d = 0.04, s = −0.7 and N = 12).
and N = 20, in which l1 and l2 be the locations of wave fronts(l2 > l1). The rotating wave has two bumps of unequal lengthl = l2 − l1 and N − l. The length l of a smaller bump decreasesand becomes zero. Consequently, two wave fronts collide and therotating wave disappears. The kinematical equation (10) can beapplied to the speeds of wave fronts in transient rotating waveswith bumps of unequal length. In its derivation, the propagationtime tp(lh) depends on how close the state xn of the neuronapproaches xs0 at t = lhtp after xn crosses zero at t = tp. That is, itdepends on the elapsed time after the preceding zero-crossing ofxn. Consequently, the bump length lh in tp(lh) corresponds to thelength of the spatially forward bump of each front in the directionof propagation, not the length of the backward bump. The locations(l1 and l2) of wave fronts and the bump length (l = l2 − l1) thuschange as
Eq. (12) has a steady solution l = lh (=N/2), which correspondsto the traveling wave with bumps of equal length in Eq. (1). Thelinear stability of the traveling wave is evaluated by letting l =
A perturbation l′ in the bump length increases (decreases) and thetravelingwave is unstable (stable) when b > 0 (b < 0), i.e., c2 > s2(s2 > c2). Since c + s > 0 when the traveling wave exists, itsinstability (stability) condition is s < c (s > c). The locus PF (adashed line) in Figs. 3 and 5(a) corresponds to its boundary s = c(=1.0). That is, the traveling wave is stabilized in the presenceof excitatory self-coupling when its strength s is larger than thestrength c of symmetric bidirectional coupling. It is because theabsolute value of xn(2tp) becomes larger than xs0 so that xn(0) =
−xn(lhtp) > xs0, i.e., the state xn overshoots its asymptotic valueat t = 2tp. This is similar to the stabilization of traveling wavesdue to the overdamping of the states of neurons in the presence ofan inertial term (Horikawa & Kitajima, 2009b). Finally, it should benoted that the stable traveling wave (not pinned) exists when c <s < d. Then the strength (c + d)/2 and (c − d)/2 of coupling withadjacent neurons are positive and negative, respectively. That is,coupling in one direction must be excitatory and that in the other
Fig. 7. Spatial pattern of the states of neurons in a transient rotating wave of Eq.(1) with c = 1.0, d = 0.5, s = −0.1, g = 100 and N = 20.
direction must be inhibitory for the occurrence of the stabilizationof the traveling wave, which has been shown in a cellular neuralnetwork (Thiran, 1997).
3.2. Type-2 solution
Next, we consider kinematics of the traveling wave in s′ < −din the region T1 and a pinning condition for the region P1 in Fig. 3(d)and 5. Let xn−2 < −1/g , xn−1 < −1/g , xn > 1/g and xn+1 > 1/g(saturated) as above, and then Eqs. (4) and (5) hold.When s′ < −d,xn−1 increases to be unsaturated: xn−1 → −(d+s)(>−1/g)whilexn decreases to be unsaturated: xn → −d + s (<1/g). Both statesxn−1 and xn then move to be unsaturated.
We here fix xn to be positive and saturated (xn > 1/g , f (gxn) =
1) and let xn−j < −1/g (j ≥ 1). When s′ < −d, the valueof the state xn−1 increases to be unsaturated (|xn−1| < 1/g ,f (gxn−1) = gxn−1). However, xn−1 never increases to be positivelysaturated since thewave front cannotmove in the direction inverseto asymmetric coupling (d > 0). In fact, when xn−1 = 0,dxn−1/dt = −xn−1 −d and xn−1 turns toward−d (<0). When xn−1is unsaturated (|xn−1| < 1/g) and xn−2 < −1/g , a change in xn−1is described as
Note that a steady solution (xn−1 = d/(s′g)) to Eq. (14) is stablesince s′ < −d (<0). Then xn−2 changes as
dxn−2/dt = −xn−2 −c + d2
+c − d2
· gxn−1 − s
= −xn−2 −c + d2
+c − d2
·ds′
− s
xn−2 → xT1 = (cd − cs′ − ds′ − d2)/2s′ − s.
(15)
Y. Horikawa / Neural Networks 53 (2014) 26–39 33
If xT1 < −1/g and xn−2 remains saturated, only one neuron isunsaturated (|xn−1| < 1/g). Thus cd – cs′ – ds′ – d2 – 2s′2 = 0gives a boundary between the regions T1 and T2. It is shown with asolid line in Fig. 5, which connects (s, d) = (−c/2, 0) to (−c, c).
We then let xn be free and xn+1 > 0. A change in xn is describedas
dxn/dt = −xn +c + d2
gxn−1 +c − d2
+ s
= −xn +c + d2
·ds′
+c − d2
+ s
xn → xP1 = (cd + cs′ − ds′ + d2)/2s′ + s.
(16)
If xP1 < 1/g and xn decreases to be unsaturated, then xn−1 beginsto decrease to be negatively saturated since only one neuron canremain unsaturated as mentioned above. The wave front thenpropagates. If xP1 > 1/g and xn remains saturated, the wave frontcannot propagate. Thus cd + cs′ − ds′ + d2 + 2s′2 < 0 is a pinningcondition for the region P1 in Figs. 3(d) and 5(a). It is –c/2 < s′ < 0for d = 0. Themaximum value of d in the pinned region P1 is givenby d = (4
√2−5)c/7 ≈ 0.094c at s′ = (−3+
√2)c/7 ≈ −0.227c.
We again let g → ∞ and s′ = s, and use a sign function (Eq.(3)) for f (x) to derive the kinematics of a wave front. In the regionT1 (s < −d, xT1 < −1/g (=0) and xP1 < 1/g (=0)), a change inthe state of a neuron at a wave front in the traveling wave consistsof four steps as shown in Fig. 6(b) (c = 1.0, d = 0.1, s = −0.4,g = 1000 and N = 12). Let xn−2 < 0, xn > 0 and let the signof xn−1 change from positive to negative at t = 0 (xn−1(0) = 0)in the traveling wave. Actually we consider that xn−1 becomesunsaturated (xn−1 → d/(sg)) and Eq. (16) becomes applicable forxn at t = 0. However, unless xP1 ≈ 0 we can let xn−1(0) = 0 forlarge g since d/(sg) ≈ 0. First, we obtain the propagation time tp0for large N (≫1) by letting xn(0) = xs0 (=c + s) as
xn(tp0) = (xs0 − xP1) exp(−tp0) + xP1 = 0
tp0 = log[(xP1 − xs0)/xP1]
= log[(c + d)(d − s)/(cd + cs − ds + d2 + 2s2)].
(17)
In tp0 < t < 2tp0, the state xn remains d/(sg)(xn(t) = d/(sg) (tp0< t < 2tp0)) since xn−1 < 0 and xn+1 > 0. Then xn+1 becomeszero (unsaturated, xn+1 → d/(sg)) at t = 2tp0, and xn changes in2tp0 < t < 3tp0 as
where b > 0 since −c < s < −d < 0. The traveling wave is thusalways unstable in the region T1. The propagation of wave frontsand a change in bump width in transient rotating waves are alsodescribed by the kinematical equations (11) and (12) with tp0 andb in Eq. (20).
3.3. Type-2j solutions (j > 1)
We then consider the case in which the states of j (>1) neuronsremain in an unsaturated region of f (gx) (|xn−i| < 1/g (1 ≤ i ≤ j),xn−i < −1/g (i > j)) in the traveling wave. Let xn be fixed to bepositively saturated (xn > 1/g , f (gxn) = 1), let xn−i (1 ≤ i ≤ j)be unsaturated (f (gxn−i) = gxn−i) and let xn−i (i > j) be negativelysaturated (f (gxn−i) = −1). Changes in the states of neurons aredescribed as
dxn−1/dt = −xn−1 +(c + d)
2gxn−2 +
(c − d)2
+ sgxn−1
dxn−i/dt = −xn−i +(c + d)
2gxn−i−1
+(c − d)
2gxn−i+1 + sgxn−i (2 ≤ i ≤ j − 1)
dxn−j/dt = −xn−j −(c + d)
2+
(c − d)2
gxn−j+1 + sgxn−j
dxn−j−1/dt = −xn−j−1 −(c + d)
2+
(c − d)2
gxn−j − s.
(21)
We then assume that xn−i (1 ≤ i ≤ j) remain unsaturated in thesteady states, i.e., they are the following stable solutions to Eq. (21)
satisfying |gxn−i| < 1 (1 ≤ i ≤ j). We denote these stable steadysolutions in Eq. (22) by xn−i (1 ≤ i ≤ j). A condition for theexistence of the wave front with j unsaturated neurons is
so that xn−j−1 remains negatively saturated. That is, xTj = −1/ggives a boundary between the regions Tj and Tj+1 (Fig. 5(b)), wherexTj < −1/g in Tj and xTj > −1/g in Tj+1.
We then let xn be free and xn+1 > 1/g . A change in xn isdescribed as
dxn/dt = −xn +c + d2
gxn−1 +c − d2
+ s. (24)
A condition for the pinning of thiswave frontwith j unsaturatedneurons is
so that xn remains positively saturated. Thus xPj = 1/g gives aboundary between the regions Tj and Pj (Fig. 5(b)), where xPj < 1/gin Tj and xPj > 1/g in Pj. Boundaries between the regions Tj−1 andTj and those between the regions Tj and Pj (j ≥ 1) are shown inFig. 5(b) with solid and dashed lines, respectively. In a ring of Nneurons (for even N ≥ 4), pinned regions up to (N/2 − 2)th exist(Pj, 0 ≤ j ≤ N/2 − 2) since at least two saturated neurons mustexist in one bumpwhen s′ < c . It has been shown that the value ofs′ at the boundary between the pinned regions Pj−1 and Pj for d = 0is given by s′ = −c cos[π/(j + 1)] (j ≥ 1) (Thiran et al., 1995).
It should be noted that the boundary condition (xTj−1 = −1/g)between Tj−1 and Tj is equivalent to gxn−j = −1 in Eq. (22), wheregxn−j < −1 in Tj−1 and gxn−j > −1 in Tj. It is because there are j−1(or less) unsaturated neurons at a wave front if gxn−j < −1 while
34 Y. Horikawa / Neural Networks 53 (2014) 26–39
there are j (ormore) unsaturated neurons at awave front if gxn−j >−1. Also, the boundary condition (xPj−1 = 1/g) between Tj−1 andPj−1 is equivalent to gxn−1 = 1 in Eq. (22), where gxn−1 > 1 in Tj−1and gxn−1 < 1 in Pj−1. It is because there are unstable unsaturatedsolutions xn−i (1 ≤ i ≤ j) to Eq. (21) in Pj−1. Then, either xn−1becomespositively saturated or xn−j becomesnegatively saturated,which results in the generation of a pinned wave front with j − 1unsaturated neurons. An intuitive explanation for these equivalentconditions is also given in the Appendix.
We then consider a change in the state of a neuron at a wavefront in the traveling wave in the region Tj. We again let g → ∞
and s′ = s, and we use a sign function (Eq. (3)) for f (x). Fig. 6(c)shows time courses of the states of neurons at a wave front in T2(c = 1.0, d = 0.04, s = −0.7, g = 1000 and N = 12). Let the signof xn−1 change from positive to negative at t = 0 (xn−1(0) = 0)in the traveling wave. Actually we consider that xn−1 becomesunsaturated (xn−1 → xn−1) and Eq. (24) becomes applicable forxn at t = 0, as mentioned in Section 3.2. The propagation time tp0for large N is given by
where xn−1 is the solution in Eq. (22). Note that tp0 > 0 since|gxn−1| < 1 (f (gxn−1) < 1) and xs0 = (c+ s) > 0. Then xn remainsin an unsaturated region (|xn| < 1/g) in tp0 < t < (j + 1)tp0. In(j + 1)tp0 < t < (j + 2)tp0, xn changes as
The values of tp0 and b in Eq. (29) are used in Eq. (12) for a change inbumpwidth in transient rotatingwaves. Eq. (29) can be applied forj = 1 by letting gxn−1 = d/s and for j = 0 by letting gxn−1 = −1and gxn−j = 1. The coefficient b is always positive for j ≥ 1 since|gxn−i| < 1 (i ≥ 1) and (c − d)gxn−j − (c + d)gxn−1 − 2s >(c − d)(1+ gxn−j) > 0 because xPj < 1/g (≈ 0). Consequently, thetraveling wave is always unstable in the regions Tj (j ≥ 1).
4. Duration of rotating waves
In this section, we derive the duration of transient rotatingwaves using the kinematical equation (12) for a change in thelength of bumps. We assume that Eq. (1) is bistable and theunstable traveling wave exists (1/g − c cos(2π/N) < s < c andb > 0 in Eq. (12)). By imposing the initial condition l(0) = l0, asolution to Eq. (12) is obtained by (Horikawa & Kitajima, 2009a)
where l is the length of a smaller bump. The duration T of transientrotating waves thus increases exponentially with the initial lengthl0 of a smaller bump, i.e., the number of neurons in a smaller bump.A growth rate of the duration with the initial bump length is equalto the propagation time tp0 of the traveling wave. When couplingis unidirectional (d = c and s = 0), the propagation time and thegrowth rate are tp0 = log[(c + d)/(d − s)] = log 2 so that theduration is doubled by increasing one neuron (T ∼ 2l0). They varyin the presence of asymmetric coupling and self-coupling as Eqs.(7), (17) and (26). On one hand, the propagation time increases sothat the growth rate increases as the strength of coupling becomesclose to the pinned regions. This happens when the strength ofself-coupling is close to that of asymmetric coupling (s′ ≈ d)in the region T0 or when the strength of asymmetric coupling issmall (d ≈ 0) for negative self-coupling (s′ < 0) in the regionsTj (j ≥ 1). On the other hand, the propagation time and thegrowth rate become zero when the strength of self-coupling isnegative (s′ ≈ −c) or when the strength of asymmetric couplingis large (d ≫ c , s′). The latter limit corresponds to antisymmetricbidirectional coupling (c = 0).
The bifurcation (generation and pinning) of traveling wavesdepends on s′ = s − 1/g for small g rather than the strengths of self-coupling itself as shown in Section 3. We then comparethe duration of rotating waves for different values of an outputgain g with the same value of s′. Fig. 8 shows semi-log plots of theduration T of rotating waves against initial smaller bump length l0with c = 1.0, d = 0.5, s′ = 0.48 and N = 11 in T0 (a), c = 1.0,d = 2.0, s′ = −0.4 and N = 101 in T0 (b), c = 1.0, d = 0.1,s′ = −0.3 and N = 15 in T1 (c), and c = 1.0, d = 0.02, s′ = −0.7and N = 21 in T2 (d). Plotted are results of computer simulationof Eq. (1) with g = 2 (solid circles), 10 (open circles), 100 (solidsquares) and 1000 (open squares). Note that the strength of self-coupling is given by s = s′ + 1/g , e.g., s = s′ + 0.5 for g = 2. Incomputer simulation, the initial condition was given by
xn(0) = −xs0 (1 ≤ n ≤ l0) = xs0 (l0 + 1 ≤ n ≤ N). (35)
The duration was obtained as the time at which the signs of thestates of all neurons become the same. Please note that data forg = 2 and 100 in (a) overlap and that data for g = 10, 100and 1000 in (c) overlap. Plotted also are Eq. (34) (solid lines) with
Y. Horikawa / Neural Networks 53 (2014) 26–39 35
Fig. 8. Semi-log plots of the duration T of rotating waves vs initial smaller bumplength l0 with c = 1.0, d = 0.5, s′ = 0.48 and N = 11 in T0 (a), c = 1.0, d = 2.0,s′ = −0.4 and N = 101 in T0 (b), c = 1.0, d = 0.1, s′ = −0.3 and N = 15 in T1(c) and c = 1.0, d = 0.02, s′ = −0.7 and N = 21 in T2 (d) (s′ = s − 1/g). Resultsof computer simulation of Eq. (1) with g = 2 (solid circles), 10 (open circles), 100(solid squares), 1000 (open squares). Eq. (34) with Eq. (7) (tp0 = 4.32, k = 1924)in (a) (tp0 = 0.223, k = 0.146) in (b) for the region T0 , with Eq. (17) (tp0 = 3.09,k = 3388) in (c) T1 , or with Eq. (26) (j = 2, tp0 = 2.29, k = 7218) in (d) for T2 (solidlines).
Eq. (10) (tp0 = 4.32, b = 1924) in (a) (tp0 = 0.223, b = 0.146)in (b) for the region T0, with Eq. (20) (tp0 = 3.09, b = 3388) in(c) for the region T1, or with Eq. (29) (j = 2, tp0 = 2.29, b =
7218) in (d) for the region T2. Eq. (34) agrees with the simulationresults for large g (=100 and 1000), while the growth rates differfor small values of g (=2, 10) in (b) and (d) since the kinematicalequation was derived for large g . When d is large (b), the growthrate (tp0 = log[(c+d)/(d−s)]) for small g becomes large since the
Fig. 9. Log–log plot of a normalized histogram of the duration T of rotating waves.Results of 1000 runs of computer simulation of Eq. (1) with c = 1.0, d = 0.5, s′ =
0.48 and N = 9 under the random Gaussian initial condition: xn(0) ∼ N(0, 0.12)
(1 ≤ n ≤ N); g = 2 (solid circles), 10 (open circles), 100 (open triangles) and 1000(crosses), and Eq. (36) with tp0 = 4.32 and k = 1920 (a solid line).
propagation time tp0 never becomes zero and remains finite in thelimit of d → ∞. When s′ is close to −1 (d), each region (Tj (j ≥ 1))is narrow so that the growth rate may vary sensitively with s′and d.
It is worth noting that the values of tp0 and b can be estimatedwith the graph of log(ν)/τp against N (Fig. 2). A solution toEq. (13) for the perturbation l′(=l − N/2) of bump length leadsto ν ≈ l′(τp)/l′(0) = exp[2b/t2p0 exp(−tp0lh)τp]. Thus it holds thatlog(ν)/τp = 2b/t2p0 exp(−tp0N/2). The estimates can be obtainedby optimal fitting of this equation to the graphs. It can be shownthat T (l0) in Eq. (34) with the estimates gives good agreementwiththe simulation results.
Further, it has been shown that the duration T of rotatingwavesgenerated under random initial conditions is distributed in a powerlaw form up to a cut-off Tc (Horikawa & Kitajima, 2009a).
h(T ) =1
tp0T + 1/b·2N
(0 < T < Tc = tp0/b · exp(tp0N/2)). (36)
The cut-off Tc also increases exponentially with the number Nof neurons. Fig. 9 shows a log–log plot of normalized histogramsof the duration T of rotating waves obtained with 1000 runs ofcomputer simulation of Eq. (1) with c = 1.0, d = 0.5, s′ = 0.48and N = 9 under the random Gaussian initial condition: xn(0) ∼
N(0, 0.12) (1 ≤ n ≤ N). The values of g are 2 (solid circles), 10(open circles), 100 (open triangles) and 1000 (crosses). Eq. (36)with tp0 = 4.32 and b = 1920 is also plotted with a solid line.It agrees with power law distributions of the simulation results upto the cut-off: Tc ≈ 6.2×105. The duration is distributed in a powerlaw formover five decades. Although the kinematical equation (12)is derived for large N , it is applicable for a ring of such a smallnumber of neurons when the growth rate (the propagation time,tp0) is large.
5. Conclusion and future work
Effects of asymmetric bidirectional coupling and self-couplingon traveling waves and transient rotating waves in a ring ofcoupled sigmoidal neurons were studied. Conditions for the
36 Y. Horikawa / Neural Networks 53 (2014) 26–39
pinning of the traveling waves were obtained by piecewise linearapproximation to a sigmoidal output function of neurons. Therewere N/2 − 1 kinds of traveling waves or pinned states (type-2j,0 ≤ j ≤ N/2−2, whereN was the number of neurons). In a type-2jsolution, the output of 2j neurons at wave fronts was unsaturatedwhile the output of the otherN−2j neuronswas saturated. The s–dplane of the strength of self-coupling (s) and asymmetric coupling(d) was divided into Tj and Pj regions (0 ≤ j ≤ N/2 − 2), in whichthe type-2j traveling wave and the type-2j pinned steady stateexisted, respectively. In addition to pinning in P0 for excitatory self-coupling (s − 1/g > d > 0, where g was an output gain), therewere pinned regions Pj (j ≥ 1) for s < 1/g (inhibitory self-couplingfor g ≫ 1) when the strength of asymmetric coupling was small(d ≈ 0). Inhibitory self-coupling tended to change the signs of thestates of neurons and thus tended to make wave fronts propagate.However, the states of neurons at wave fronts became unsaturatedso that the pinning of wave fronts occurred.
A kinematical equation for the propagation of wave fronts ineach region Tj was then derived by using a sign function for alarge output gain of neurons (g ≫ 1). The traveling wave wasstabilized when the strength of self-coupling was larger than thatof symmetric bidirectional coupling (c) and smaller than that ofasymmetric coupling (c < s < d). When the traveling wave wasunstable, transient states showedmetastable dynamics in the samemanner as a ring of unidirectionally coupled sigmoidal neurons.The duration of rotating waves generated in transients increasedexponentially with the number l0 of neurons in the initial smallerbump. However, the growth rate of the exponential increase intheir duration depended on the strength of asymmetric couplingand self-coupling. The rate was equal to the propagation time tp0of the wave front, and the duration became exp(tp0) times byincreasing one neuron. When coupling was unidirectional withoutself-coupling, the growth rate was log 2 so that the durationdoubled per neuron. The growth rate tended to infinity whenthe traveling wave was nearly pinned since the propagation timeincreased. This occurred not onlywhen the strength of asymmetriccoupling was small (d ≈ 0) but also when the strength of self-coupling was close to that of asymmetric coupling (s ≈ d). Thena power law distribution of the duration of randomly generatedrotating waves, which was characteristic of metastable dynamicaltransient rotating waves, was shown in a ring of a small number(N = 9) of neurons.
In this study, a ring network consisting of simple sigmoidalneurons, which were firing rate models of neurons or theirpopulations, was dealt with. This simple coupled system wassuitable to study the mechanism and properties of metastabledynamical transients, and the pinning conditions and kinematicsof metastable dynamical rotating waves were derived with linearanalysis. In general,metastable dynamical transients emerge in thepresence of a one-dimensionally unstable solution the instabilityof which decreases exponentially with system size (the numberof elements or the size of domains), which is due to competitionbetween symmetric bistable states. In a ring of sigmoidal neuronswith asymmetric bidirectional coupling, the unstable solutionhad a form of the traveling wave, and rotating waves eventuallyconverged to one of bistable spatially uniform states only throughchanges in the distance between wave fronts (the length ofbumps). This transient motion reflected the one-dimensionalinstability of the traveling wave. The kinematics of wave frontsrevealed that the linear relaxation of the states of neurons towardone of the stable states causes extremely small difference inthe speeds of wave fronts which decreases exponentially withthe bump length (the number of neurons between wave fronts).Thus, metastable dynamical transients in the form of rotatingwaveswere attributed phenomenologically to the linear relaxationand exponential convergence. Asymmetric bidirectional coupling
and self-coupling had an effect on the kinematics and relaxationprocess of the states of neurons. Unsaturated neurons at wavefronts played an important role in the presence of inhibitory self-coupling.
The results obtained in this study will be applicable to ringsof biologically more plausible spiking neuron models. As notedin Section 1, metastable dynamical transient rotating waves havebeen found in a ring of Bonhoeffer–van der Pol models coupledwith slow inhibitory synapses unidirectionally (Horikawa, 2011),which have the same kinematics as those in a ring of sigmoidalneurons. The rotating wave has a form of propagating oscillations,in which neurons in firing states and resting states are locatedalternately while there are two successive neurons in the samestates. The locations of these inconsistencies propagate in thedirection of coupling, and each neuron alternates between firingand resting as the inconsistencies pass. Asymmetric bidirectionalcoupling and self-coupling might have similar effects on suchrotating waves in rings of spiking neurons to those obtained inthis study. The growth rate of the exponential increase in theirduration with the number of neurons might vary according tothe propagation speed, while the growth rate is about log 2 in aring with unidirectional coupling. Then, extremely long transientrotating waves can emerge even in rings of small numbers ofspiking neurons when rotating waves are nearly pinned. However,traveling waves in a ring of spiking neurons might be easilystabilized like those in rings of inertial neurons (Horikawa &Kitajima, 2009b) and parametric oscillators (Horikawa & Kitajima,2012c) because of oscillatory recovery. Further, the resting andfiring states of a spiking neuron differwith each other qualitatively.The effects of this asymmetry in its output might have to be takeninto account. Also, the unsaturated state of a sigmoidal neuronmight correspond to the low-frequency firing of a spiking neuronormight be an artifact accompanying simplification. It is of interestto what extent the properties of metastable dynamical rotatingwaves in a ring of sigmoidal neurons hold for rings of spikingneurons and what their difference comes from.
Acknowledgment
The author would like to acknowledge valuable discussionswith Dr. H. Kitajima.
Appendix. Stability of solutions to a system of linear differen-tial equations
We consider the stability of steady solutions to the followingsystem of linear differential equations for j unsaturated neurons(xn−i (1 ≤ i ≤ j)) in Eq. (21).
ddt
xn−1
xn−2
xn−j+1
xn−j
= g
s′ (c + d)/2 0 0
(c − d)/2 s′ (c + d)/20 (c − d)/2 0
s′ (c + d)/20 0 (c − d)/2 s′
×
xn−1
xn−2
xn−j+1
xn−j
+
(c − d)/2
00
−(c + d)/2
(A.1)
where s′ = s − 1/g (s′ = s for g → ∞). Steady solutions toEq. (A.1) are given by xn−i (1 ≤ i ≤ j) in Eq. (22). The eigenvalues
Y. Horikawa / Neural Networks 53 (2014) 26–39 37
Fig. A.1. Re[µ1(j)] = 0 (1 ≤ j ≤ 5) in Eq. (A.2) (solid lines) and boundaries forpinned regions Pj (1 ≤ j ≤ 4) (dashed lines) in the s′–d plane.
µk(j) of the tridiagonal coefficient matrix in Eq. (A.1) are given by(Yamamoto, 1976)
Steady solutions to Eq. (A.1) are stable if Re[µ1(j)] < 0, i.e., ifs′ < −
√c2 − d2 cos[π/(j + 1)] for d < c or if s′ < 0 for d > c
(j ≥ 1).Numerical calculation shows that there are stable and unsta-
ble unsaturated steady solutions (|gxn−i| < 1 (1 ≤ i ≤ j)) toEq. (A.1) outside and inside of the ellipse (s′2/ cos[π/(j + 1)]2 +
d2 = c2), respectively, which connect with each other at (s′, d) =
(−c cos[π/(j + 1)], 0) in the s′–d plane. Fig. A.1 shows the linesRe[µ1(j)] = 0 (1 ≤ j ≤ 5) (solid lines) and the boundaries for thepinned regions of type-2j solutions Pj (1 ≤ j ≤ 4) (dashed lines) inthe s′–d plane. On one hand, the stable unsaturated steady solutioncan exist for s′ < −
√c2 − d2 cos[π/(j + 1)] (d < c), and its exis-
tence condition is given by gxn−j > −1. This means that a (stable)type-2j solution to Eq. (1) can exist, and thus gxn−j = −1 corre-sponds to the boundary (xTj−1 = −1/g) between Tj−1 and Tj. A sta-ble j + 1 unsaturated steady solution (|gxn−i| < 1 (1 ≤ i ≤ j + 1))to Eq. (A.1) with j → j+1 exists for s′ < −
√c2 − d2 cos[π/(j+2)]
(gxn−j−1 > −1) in the samemanner. Then a condition for the exis-tence of a type-2j solution for d = 0 is given by−c cos[π/(j+2)] <s′ < −c cos[π/(j + 1)] (j ≥ 1), which has been derived by Thiranet al. (1995).
On the other hand, the unstable unsaturated steady solution canexist in −
√c2 − d2 cos[π/(j + 1)] < s′ < −
√c2 − d2 cos(π/j)
(j ≥ 2), which is bounded by two ellipses (s′2/ cos2[π/(j + 1)] +
d2 < c2 and s′2/ cos2(π/j)+d2 < c2), or s′ > 0 (j = 1). A conditionfor its existence is given by gxn−1 < 1. Intuitively, the existence ofthis unstable unsaturated solutionmeans that an unsaturated xn−1can move to be positively saturated and an unsaturated xn−j canmove to be negatively saturated. Then, one of them becomes satu-rated, which results in the generation of a pinned wave front withj − 1 unsaturated neurons, i.e., a pinned type 2(j − 1) solution. Inother words, a positively saturated xn−1 (>1/g) cannot move to beunsaturated so that awave frontwith junsaturatedneurons cannotbe generated when gxn−1 < 1. When gxn−1 > 1, a saturated xn−1(1 < gxn−1 < gxn−1) decreases to be unsaturated so that the wavefront with j unsaturated neurons propagates. Thus, gxn−1 = 1 cor-responds to the boundary (xPj−1 = 1/g) between Tj−1 and Pj−1.
When j = 1, a steady solution is gxn−1 = d/s′, the eigenvalue isµ1(1) = gs′, and the steady solution is stable (unstable) for s′ < 0(s′ > 0).
There are a stable unsaturated solution in s′ < −d (gxn−1 >−1) and an unstable unsaturated solution in s′ > d (gxn−1 > 1),
Fig. A.2. Steady solutions gxn−1 (a solid line) and gxn−2 (a dashed line) to Eq. (A.1)with j = 2, c = 1.0 and d = 0.05 vs s′ .
which correspond to xT0 > −1/g for T1 and xP0 > 1/g for P0,respectively. In −d < s′ < 0, there is a stable negatively saturatedsolution (gxn−1 < −1) so that xn−1 becomes negatively saturated.In 0 < s′ < d, there is an unstable positively saturated solution(gxn−1 > 1) so that xn−1 also becomes negatively saturated. Thena wave front propagates for s′ < |d|, which gives the region T0 forthe propagation of a type-0 solution.
When j = 2, the largest eigenvalue µ1(2) is negative and thesteady solution is stable when s′ < −
√c2 − d2/2 (4s′2 + d2 > c2,
s′ < 0). Fig. A.2 shows gxn−1 (a solid line) and gxn−2 (a dashedline) as a function of s′ for c = 1.0 and d = 0.05. A steady so-lution to Eq. (A.1) diverges at s′ = −
√c2 − d2/2 (≈−0.5) since
the determinant of the coefficient matrix is zero. A stable unsatu-rated solution exists for s′ < −0.57 (gxn−2 > −1) so that a type-2 solution can exist (T2). An unstable unsaturated solution existsin −0.41 < s′ < −0.064 (gxn−1 < 1) so that a pinned type-1 solution exists (P1). There is a stable solution with a negativelysaturated xn−2 in −0.57 < s′ < −0.5 so that xn−2 becomes neg-atively saturated. There is an unstable solution with a positivelysaturated xn−1 in −0.5 < s′ < −0.41 so that a positively sat-urated xn−1 becomes unsaturated while xn−2 becomes negativelysaturated. Then a wave front with one unsaturated neuron propa-gates in −0.57 < s′ < −0.41, which corresponds to the region T1,in which a type-1 solution propagates.
References
Amari, S. (1972). Characteristics of random nets of analog neuron-like elements.IEEE Transactions on Systems, Man, and Cybernetics, SMC-2, 643–657.
Amari, S. (1978). Mathematics of neural networks. Tokyo: Sangyo-Tosho (inJapanese).
Atiya, A. F., & Baldi, P. (1989). Oscillations and synchronizations in neural networks:an exploration of the labeling hypothesis. International Journal of NeuralSystems, 1, 103–124.
Babcock, K. L., & Westervelt, R. M. (1987). Dynamics of simple electronic neuralnetworks. Physica D, 28, 305–316.
Baldi, P., & Atiya, A. F. (1994). How delays affect neural dynamics and learning. IEEETransactions on Neural Networks, 5, 612–621.
Bazhenov, M., Huerta, R., Rabinovich, M. I., & Sejnowski, T. (1998). Cooperativebehavior of a chain of synaptically coupled chaotic neurons. Physica D, 116,392–400.
Bélair, J., Campbell, S. A., & van den Driessche, P. (1996). Frustration, stability anddelay-induced oscillations in a neural network model. SIAM Journal on AppliedMathematics, 56, 245–255.
Blum, E. K., & Wang, X. (1992). Stability of fixed points and periodic orbits andbifurcations in analog neural networks. Neural Networks, 5, 577–587.
Bonnin, M. (2009). Waves and patterns in ring lattices with delays. Physica D, 238,77–87.
Bressloff, P. C. (2005). Weakly interacting pulses in synaptically coupled neuralmedia. SIAM Journal on Applied Mathematics, 66, 57–81.
Bressloff, P. C., Coombes, S., & de Souza, B. (1997). Dynamics of a ring of pulse-coupled oscillators: group theoretic approach. Physical Review Letters, 79,2791–2794.
Bronsard, L., & Kohn, R. V. (1990). On the slowness of phase boundary motion inone space dimension. Communications on Pure and Applied Mathematics, 43,983–998.
Buono, P.-L., Golubitsky, M., & Palacios, A. (2000). Heteroclinic cycles in rings ofcoupled cells. Physica D, 143, 74–108.
Campbell, S. A. (1999). Stability and bifurcation of a simple neural network withmultiple time delays. Fields Institute Communications, 21, 65–79.
Campbell, S. A., Ruan, S., & Wei, J. (1999). Qualitative analysis of a neural networkmodel with multiple time delays. International Journal of Bifurcation and Chaos,9, 1585–1595.
Campbell, S. A., Yuan, Y., & Bungay, S. D. (2005). Equivariant Hopf bifurcation in aring of identical cells with delayed coupling. Nonlinearity, 18, 2827–2846.
Carr, J., & Pego, R. L. (1989). Metastable patterns in solutions of ut = ε2uxx − f (u).Communications on Pure and Applied Mathematics, 42, 523–576.
Cessac, B. (2008). A discrete time neural network model with spiking neurons.Rigorous results on the spontaneous dynamics. Journal of Mathematical Biology,56, 311–345.
Cessac, B., & Viéville, T. (2008). On dynamics of integrate and-fire neural networkswith conductance based synapses. Frontiers in Computational Neuroscience, 2,21–220.
Chua, L. O., & Yang, L. (1988). Cellular neural networks: theory. IEEE Transactions onCircuits and Systems, 35, 1257–1272.
Collins, J. J., & Stewart, I. (1994). A group-theoretic approach to rings of coupledbiological oscillators. Biological Cybernetics, 71, 95–103.
Crounse, K. R., Chua, L. O., Thiran, P., & Setti, G. (1996). Characterization anddynamics of pattern formation in cellular neural networks. International Journalof Bifurcation and Chaos, 6, 1703–1724.
Destexhe, A. (2009). Self-sustained asynchronous irregular states and up-downstates in thalamic, cortical and thalamocortical networks of nonlinearintegrate-and-fire neurons. Journal of Computational Neuroscience, 27, 493–506.
Doedel, E.J. 1996 AUTO: software for continuation and bifurcation problemsin ordinary differential equations. Concordia University, Montreal, Canada,http://indy.cs.concordia.ca/auto/.
Ei, S., & Ohta, T. (1994). Equation of motion for interacting pulses. Physical ReviewE, 50, 4672–4678.
El Boustani, S., & Destexhe, A. (2009). Amaster equation formalism formacroscopicmodeling of asynchronous irregular activity states. Neural Computation, 21,46–100.
Enjieu Kadji, H. G., Chabi Orou, J. B., & Woafo, P. (2007). Spatiotemporaldynamics in a ring of N mutually coupled self-sustained systems. Chaos, 17,0331091–03310911.
Ermentrout, G. B. (1985). The behavior of rings of coupled oscillators. Journal ofMathematical Biology, 23, 55–74.
Fingelkurts, A. A., & Fingelkurts, A. A. (2004). Making complexity simpler:multivariability and metastability in the brain. International Journal ofNeuroscience, 114, 843–862.
Friesen, W. O., & Stent, G. S. (1977). Generation of a locomotory rhythm by a neuralnetwork with recurrent cyclic inhibition. Biological Cybernetics, 28, 27–40.
Friesen, W. O., & Stent, G. S. (1978). Neural circuits for generating rhythmicmovements. Annual Review of Biophysics and Bioengineering , 7, 37–61.
Friston, K. J. (1997). Transients, metastability, and neuronal dynamics. Neuroimage,5, 164–171.
Friston, K. J. (2001). Brain function, nonlinear coupling, and neuronal transients.Neuroscientist , 7, 406–418.
Fusco, G., & Hale, J. K. (1989). Slow motion manifolds, dormant instability andsingular perturbations. Journal of Dynamics and Differential Equations, 1, 75–94.
Gedeon, T. (1998). Cyclic feedback systems. InMemoirs of the americanmathematicalsociety. Providence: AMS.
Golubitsky, M., Stewart, I., & Schaeffer, D. G. (1988). Singularities and groups inbifurcation theory II. New York: Springer.
Grasman, J., & Jansen, M. J. W. (1979). Mutually synchronized relaxation oscillatorsas prototypes of oscillating systems in Biology. Journal of Mathematical Biology,7, 171–197.
Guo, S. (2005). Spatio-temporal patterns of nonlinear oscillations in an excitatoryring network with delay. Nonlinearity, 18, 2391–2407.
Guo, S., & Huang, L. (2003). Hopf bifurcating periodic orbits in a ring of neuronswithdelays. Physica D, 183, 19–44.
Guo, S., & Huang, L. (2006). Non-linear waves in a ring of neurons. IMA Journal ofApplied Mathematics, 71, 496–518.
Guo, S., & Huang, L. (2007a). Stability of nonlinear waves in a ring of neurons withdelays. Journal of Differential Equations, 236, 343–374.
Guo, S., & Huang, L. (2007b). Pattern formation and continuation in a trineuron ringwith delays. Acta Mathematica Sinica, 23, 799–818.
Gutierrez, G. (1999). Variable-frequency oscillators. In J. G. Webster (Ed.), Wileyencyclopedia of electrical and electronics engineering: Vol. 23 (pp. 75–84). JohnWiley.
Hirsh, M. H. (1989). Convergent activation dynamics in continuous time networks.Neural Networks, 2, 331–340.
Horikawa, Y. (1992). Spike propagation during the refractory period in thestochastic Hodgkin–Huxley model. Biological Cybernetics, 67, 253–258.
Horikawa, Y. (2011). Exponential transient propagating oscillations in a ring ofspiking neurons with unidirectional slow inhibitory synaptic coupling. Journalof Theoretical Biology, 289, 151–159.
Horikawa, Y. (2012). Exponential dispersion relation and its effects on unstablepropagating pulses in unidirectionally coupled symmetric bistable elements.Communications in Nonlinear Science and Numerical Simulation, 17, 2791–2803.
Horikawa, Y. (2013).Metastable dynamical patterns and their stabilization in arraysof bidirectionally coupled sigmoidal neurons. Physical Review E, 88, 062902/1-18.
Horikawa, Y. (in preparation). Effects of asymmetry in an output function onthe pinning of rotating waves in a ring neural oscillator with asymmetricbidirectional coupling and self-coupling. Submitted to Neurocomputing.
Horikawa, Y., & Kitajima, H. (2009a). Duration of transient oscillations in ringnetworks of unidirectionally coupled neurons. Physica D, 238, 216–225.
Horikawa, Y., & Kitajima, H. (2009b). Bifurcation and stabilization of oscillations inring neural networks with inertia. Physica D, 238, 2409–2418.
Horikawa, Y., & Kitajima, H. (2009c). Effects of noise and variations on the durationof transient oscillations in unidirectionally coupled bistable ring networks.Phyical Review E, 80, 0219341–02193415.
Horikawa, Y., & Kitajima, H. (2012a). Exponential transient rotating waves in abistable ring of unidirectionally coupled maps. Physica D, 241, 106–114.
Horikawa, Y., & Kitajima, H. (2012b). Exponential transient rotating waves andtheir bifurcations in a ring of unidirectionally coupled bistable Lorenz systems.Procedia IUTAM , 5, 283–287.
Horikawa, Y., & Kitajima, H. (2012c). Quasiperiodic and exponential transient phasewaves and their bifurcations in a ring of unidirectionally coupled parametricoscillators. Nonlinear Dynamics, 70, 1079–1094.
Horikawa, Y., & Kitajima, H. (2012d). Transient chaotic rotating waves in a ring ofunidirectionally coupled symmetric Bonhoeffer-van der Pol oscillators near acodimension-two bifurcation point. Chaos, 22, 0331151–03311512.
Huang, L., & Wu, J. (2003). Nonlinear waves in networks of neurons with delayedfeedback: pattern formation and continuation. SIAM Journal on MathematicalAnalysis, 34, 836–860.
Hu, H., & Huang, L. (2009). Stability and Hopf bifurcation analysis on a ring of fourneurons with delays. Applied Mathematics and Computation, 213, 587–599.
Ijspeert, A. J. (2008). Central pattern generators for locomotion control in animalsand robots: A review. Neural Networks, 21, 642–653.
Jahnke, S., Memmesheimer, R. M., & Timme, M. (2008). Stable irregular dynamicsin complex neural networks. Physical Review Letters, 100, 0481021–0481024.
Kawasaki, K., & Ohta, T. (1982). Kink dynamics in one-dimensional nonlinearsystems. Physica, 116A, 573–593.
Kelso, J. A. S. (1995). Dynamic patterns: the self-organization of brain and behavior.Massachusetts: MIT Press.
Kitajima, H., Yoshinaga, T., Aihara, K., & Kawakami, H. (2001). Chaotic bursts andbifurcation in chaotic neural networks with ring structure. International Journalof Bifurcation and Chaos, 11, 1631–1643.
Kling, U., & Székely, G. (1968). Simulation of rhythmic nervous activities I. functionof networks with cyclic inhibitions. Kybernetic , 5, 89–103.
Komarov, M. A., Osipov, G. V., & Suykens, J. A. K. (2010). Metastable statesand transient activity in ensembles of excitatory and inhibitory elements.Europhysics Letters, 91, 200061–200066.
Kumar, A., Schrader, S., Aertsen, A., & Rotter, S. (2008). The high-conductance stateof cortical networks. Neural Computation, 20, 1–43.
Kyprianidis, I. M., & Stouboulos, I. N. (2003). Chaotic synchronization of threecoupled oscillators with ring connection. Chaos, Solitons and Fractals, 17,327–336.
Li, X., &Wei, J. (2005a). On the zeros of a fourth degree exponential polynomialwithapplications to a neural networkmodelwith delays. Chaos, Solitons and Fractals,26, 519–526.
Li, X., & Wei, J. (2005b). Stability and bifurcation analysis on a delayed neuralnetwork model. International Journal of Bifurcation and Chaos, 15, 2883–2893.
Lin, Y., Lemmert, R., & Volkmann, P. (2001). Bifurcation of periodic solution in athree-unit neural network with delay. Acta Mathematicae Applicatae Sinica, 17,375–382.
Lindner, J. F., & Bulsara, A. R. (2006). One-way coupling enables noise-mediatedspatiotemporal patterns inmedia of otherwise quiescentmultistable elements.Physical Review E, 74, 0201051–0201054(R).
Linkens, D. A., Taylor, I., & Duthie, H. L. (1976). Mathematical modeling of thecolorectal myoelectrical activity in humans. IEEE Transactions on BiomedicalEngineering , BME-23, 101–110.
Marcus, C. M., & Westervelt, R. M. (1989). Stability of analog neural network withdelay. Physical Review A, 39, 347–359.
Meron, E. (1992). Pattern formation in excitable media. Physics Reports, 218, 1–66.Nekorkin, V. I., Kasatkin, D. V., & Dmitrichev, A. S. (2010). Transient dynamics in a
small ensemble of synaptically coupledMorris-Lecar neurons. Radiophysics andQuantum Electronics, 53, 45–53.
Oprisan, S. A. (2010). Existence and stability criteria for phase-lockedmodes in ringneural networks based on the spike time resetting curve method. Journal ofTheoretical Biology, 262, 232–244.
Pakdaman, K., Malta, C. P., Grotta-Ragazzo, C., Arino, O., & Vibert, J.-F. (1997).Transient oscillations in continuous-time excitatory ring neural networks withdelay. Physical Review E, 55, 3234–3248.
Pasemann, F. (1995). Characterization of periodic attractors in neural ringnetworks. Neural Networks, 8, 421–429.
Pearson, K. (1993). Common principles of motor control in vertebrates andinvertebrates. Annual Review of Neuroscience, 16, 265–297.
Perlikowski, P., Yanchuk, S., Popovych, O. V., & Tass, P. A. (2010). Periodic patterns ina ring of delay-coupled oscillators. Physical Review E, 82, 0362081–03620812.
Rabinovich, M. I., Huerta, R., & Laurent, G. (2008). Transient dynamics for neuralprocessing. Science, 321, 48–50.
Rabinovich, M. I., Varona, P., Selverston, A. I., & Abarbanel, H. D. I. (2006). Dynamicalprinciples in neuroscience. Reviews of Modern Physics, 78, 1213–1265.
Rabinovich,M. I., Volkovskii, A., Lecanda, P., Huerta, R., Abarbanel, H. D. I., & Laurent,G. (2001). Dynamical encoding by networks of competing neuron groups:winnerless competition. Physical Review Letters, 87, 0681021–0681024.
Rinzel, J., & Keller, J. B. (1973). Traveling wave solutions of a nerve conductionequation. Biophysical Journal, 13, 1313–1337.
Setti, G., Thiran, P., & Serpico, C. (1998). An approach to information propagation in1-D cellular neural networks-part II: Global propagation. IEEE Transactions onCircuits and Systems I , 45, 790–811.
Somers, D., & Kopell, N. (1993). Rapid synchronization through fast thresholdmodulation. Biological Cybernetics, 68, 393–407.
Still, S., & LeMasson, G. (1999). Travelingwaves in a ring of three inhibitory coupledmodel neurons. Neurocomputing , 26–27, 533–539.
Strelkowa, N., & Barahona, M. (2010). Switchable genetic oscillator operating inquasi-stable mode. Journal of the Royal Society Interface, 7, 1071–1082.
Strelkowa, N., & Barahona, M. (2011). Transient dynamics around unstable periodicorbits in the generalized repressilator model. Chaos, 21, 0231041–02310410.
Thiran, P. (1997). Dynamics and self-organization of locally coupled neural networks.Lausanne: Presses Polytechniques et Universitaires Romandes.
Thiran, P., Crounse, K. R., Chua, L. O., & Hasler, M. (1995). Pattern formationproperties of autonomous cellular neural networks. IEEE Transactions on Circuitsand Systems I , 42, 757–776.
Thiran, P., Setti, G., & Hasler, M. (1998). An approach to information propagation in1-D cellular neural networks-part I: local diffusion. IEEE Transactions on Circuitsand Systems I , 45, 777–789.
Tsuda, I. (1991). Chaotic itinerancy as a dynamical basis of hermeneutics of brainand mind.World Futures, 32, 167–185.
Tsuda, I. (2001). Toward an interpretation of dynamic neural activity in terms ofchaotic dynamical systems. Behavioral and Brain Sciences, 24, 793–847.
Tsuda, I. (2009). Hypotheses on the functional roles of chaotic transitory dynamics.Chaos, 19, 0151131–01511310.
Wang, Q. Y., Lu, Q. S., & Chen, G. R. (2007). Ordered bursting synchronizationand complex wave propagation in a ring neuronal network. Physica A, 374,869–878.
Wang, Q. Y., Lu, Q. S., Chen, G. R., & Guo, D. H. (2006). Chaos synchronization ofcoupled neurons with gap junctions. Physics Letters A, 356, 17–25.
Ward, M. J. (1996). Metastable bubble solutions for the Allen-Cahn equation withmass conservation. SIAM Journal on Applied Mathematics, 56, 1247–1279.
Ward, M. J. (1998). Dynamic metastability and singular perturbations. In M. C.Delfour (Ed.), CRM proceedings and lecture notes: Vol. 13. Boundaries, interfaces,and transitions (pp. 237–263). Providence: AMS.
Ward, M. J. (2001). Metastable dynamics and exponential asymptotics in multi-dimensional domains. In C. K. R. T. Jones, & A. I. Khibnik (Eds.), The IMA volumesin mathematics and its applications: Vol. 122. Multiple time-scale dynamicalsystems (pp. 233–259). New York: Springer.
Wei, J., & Li, M. Y. (2004). Global existence of periodic solutions in a tri-neuronnetwork model with delays. Physica D, 198, 109–119.
Wei, J., & Velarde,M. (2004). Bifurcation analysis and existence of periodic solutionsin a simple neural network with delays. Chaos, 14, 940–953.
Wei, J., & Zhang, C. (2008). Bifurcation analysis of a class of neural networks withdelays. Nonlinear Analysis: Real World Applications, 9, 2234–2252.
Werner, G. (2007). Metastability, criticality and phase transitions in brain and itsmodels. BioSystems, 90, 496–508.
Wilson, H. R., & Cowan, J. D. (1972). Excitatory and inhibitory interactions inlocalized populations of model neurons. Biophysical Journal, 12, 1–24.
Wu, J. (1998). Symmetric functional differential equations and neural networkswith memory. Transactions of the American Mathematical Society, 350,4799–4838.
Xu, X. (2008). Complicated dynamics of a ring neural network with time delays.Journal of Physics A: Mathematical and Theoretical, 41, 0351021–03510223.
Yamamoto, T. (1976). Introduction to numerical analysis. Tokyo: Science Co. Ltd(in Japanese).
Yan, X. P. (2006). Hopf bifurcation and stability for a delayed tri-neuron networkmodel. Journal of Computational and Applied Mathematics, 196, 579–595.
Yanchuk, S., & Wolfrum, M. (2008). Destabilization patterns in chains of coupledoscillators. Physical Review E, 77, 0262121–0262127.
Yuan, Y., & Campbell, S. A. (2004). Stability and synchronization of a ring of identicalcells with delayed coupling. Journal of Dynamics and Differential Equations, 16,709–744.
Yuan, S., & Li, X. (2010). Stability and bifurcation analysis of an annular delayedneural network with self-connection. Neurocomputing , 73, 2905–2912.
Zhang, Z., & Guo, S. (2003). Periodic oscillation for a three-neuron network withdelays. Applied Mathematics Letters, 16, 1251–1255.
Zillmer, R., Brunel, N., & Hansel, D. (2009). Very long transients, irregular firing, andchaotic dynamics in networks of randomly connected inhibitory integrate-and-fire neurons. Physical Review E, 79, 0319091–03190913.
Zillmer, R., Livi, R., Politi, A., & Torcini, A. (2006). Desynchronization in dilutedneural networks. Physical Review E, 74, 0362031–03620310.
Zillmer, R., Livi, R., Politi, A., & Torcini, A. (2007). Desynchronized stable states indiluted neural networks. Neurocomputing , 70, 1960–1965.
Zou, S., Huang, L., & Chen, Y. (2006). Linear stability and Hopf bifurcation in a three-unit neural network with two delays. Neurocomputing , 70, 219–228.
Zou, S., Huang, L., & Wang, Y. (2010). Bifurcation of a three-unit neural network.Applied Mathematics and Computation, 217, 904–917.
Zumdieck, A., Timme, M., Geisel, T., & Wolf, F. (2004). Long chaotic transients incomplex networks. Physical Review Letters, 93, 2441031–2441034.
