journa l homepage: www.e lsev ier .com/ locate /b iosystems
Asynchronous spatial evolutionary games
David Newtha,∗, David Cornforthb
a CSIRO Centre for Complex Systems Science, CSIRO Marine and Atmospheric Research, GPO Box 284, Canberra, ACT 2601, Australiab CSIRO Energy Technology, 10 Murray Dwyer Cct, Steel River Estate, Mayfield West, NSW 2304, Australia
a r t i c l e i n f o
Article history:Received 20 January 2008Received in revised form 23 July 2008Accepted 10 September 2008
Keywords:Prisoner’s DilemmaAsynchrony
a b s t r a c t
Over the past 50 years, much attention has been given to the Prisoner’s Dilemma as a metaphor forproblems surrounding the evolution and maintenance of cooperative and altruistic behavior. The bulkof this work has dealt with the successfulness and robustness of various strategies. Nowak and May(1992) considered an alternative approach to studying evolutionary games. They assumed that playerswere distributed across a two-dimensional (2D) lattice, interactions between players occurred locally,rather than at long range as in the well mixed situation. The resulting spatial evolutionary games displaydynamics not seen in their well-mixed counterparts. An assumption underlying much of the work onspatial evolutionary games is that the state of all players is updated in unison or in synchrony. Using the
Cellular automata
Evolutionary stable strategy framework outlined in Nowak and May (1992), we examine the effect of various asynchronous updatingschemes on the dynamics of spatial evolutionary games. There are potential implications for the dynamics
lly exCrow
1
gbufgeMdpt1
ife(bttS
(
Fsttacrtct(e1
wTaoa
0d
of a wide variety of spatia
. Introduction
Since its conception by Maynard Smith and Price, evolutionaryame theory (Maynard Smith and Price, 1973) has proven itself toe an interesting tool in the study of phenotypic evolution, in sit-ations where the fitness of an individual is dependant upon therequency of a particular trait within a population. Evolutionaryame theory has been applied to a wide range of biological phenom-na including: animal contests (Maynard Smith and Price, 1973;aynard Smith, 1974, 1982), sex allocation (Charnov et al., 1978),
ispersal in a uniform environment (Hamilton and May, 1977),lant growth and reproduction (Mirmirani and Oster, 1979), andhe evolution of cooperation (Axelrod and Hamilton, 1981; Axelrod,984).
Typically, the evolution of cooperation and altruistic behaviors studied through the game Prisoner’s Dilemma. In its canonicalorm, the Prisoner’s Dilemma is a game consisting of two players,ach of whom can elect to adopt one of two strategies: to cooperatei.e. play C); or to defect (i.e. play D), in any given encounter. Should
oth players choose to play C, then they both receive the payoff R; ifhey choose different strategies, then the player choosing D receiveshe highest payoff T, while the player choosing C receives the payoff; should both players play D, then they both receive the payoff P.
or a game to be considered a Prisoner’s Dilemma, the payoffs mustatisfy the following conditions: (1) defection is always worth morehan cooperation T > R and P > S; (2) mutual cooperation is bet-er than mutual defection R > P; and (3) alternating does not pays 2R > (T + S). From this, it is immediately clear that in a gameonsisting of a single round, an individual should always play D,egardless of their opponent’s choice. But in a sequence of encoun-ers, repeatedly playing D leaves both players worse off, as mutualooperation pays more than mutual defection (R > P). Followinghe pioneering work of Axelrod and Hamilton (1981) and Axelrod1984), many studies have sought to understand what strategiesvolve under various conditions and constraints (Maynard Smith,982; Frean, 1994; Nowak and Sigmund, 1994).
In this paper, we shall consider evolutionary games from a some-hat different perspective to that which is commonly adopted.
his approach was first proposed by Nowak and May (1992),nd involves studying evolutionary games, where the playersccupy regions distributed across some spatial domain. Inter-ctions between players are constrained to be between nearesteighbors, rather than long distance interactions as per the clas-ical mean-field approach. A player occupying a particular regionhanges his strategy if his neighbors are more successful. This repre-ents the evolutionary scenario where more successful phenotypes
eplace less successful ones. This derivation of the game is knowns the Spatial Prisoner’s Dilemma. An assumption underlying muchf the work on Spatial Prisoner’s Dilemma is that all players updateheir strategy in unison, or in synchrony with some global clock.n many biological contexts, this is an unrealistic assumption, as
D. Newth, D. Cornforth / BioSystems 95 (2009) 120–129 121
Fig. 1. Synchronous and asynchronous updating of players in the spatial Prisoner’s Dilemma. Both simulations were seeded with a single defector surrounded by a sea ofcooperators and b = 1.9 (see Section 3 for more details). The color coding is as follows: blue represents a player who on the current and previous time step adopted thestrategy C; green is a player who adopted C following a D on the previous time step; yellow is a player who adopted strategy D but played C on the previous time step; andred is a player who played D on the current and previous time step. The snapshot of the system was taken at time step 217. In the synchronous case (left) (Nowak and May,1992) the system is made up of a cooperator–defector polymorphic population that persists indefinitely. This configuration also generates complex spatial patterns. In thea strater
accootIietS
astiesemB2tTict
raanptfo
aesdtc
2
eueihusuaWa
2
tI(2ic
synchronous case (right), the population collapses and defection is the only stableeferred to the web version of the article.)
clock that causes all system elements to update their state syn-hronously seldom exists. While seasonal effects and other externallocks can synchronize metabolic and reproductive cycles in somerganisms, in many social settings individuals act at different andften uncorrelated time scales, making decisions on informationhat may be imperfect or delayed (Huberman and Glance, 1993).n a social context for example, two players may be just conclud-ng an encounter, while two other players may be just initiating anncounter. However, the result of the first encounter may influencehe strategies adopted by the individuals in the second encounter.uch situations are captured by asynchronous updating schemes.
Fig. 1 illustrates the contrast between the synchronous andsynchronous version of the Spatial Prisoner’s Dilemma. In bothimulations a 99 × 99 square lattice with fixed boundary condi-ions was used and both simulations were seeded with the samenitial condition of a single defector surrounded by a sea of coop-rators. Fig. 1(left) shows the state of the system after 217 timeteps, where each player is updated in synchrony. As the systemvolves, symmetry is maintained, and the system consists of a poly-orphic population containing both cooperators and defectors.
y contrast, Fig. 1(right) illustrates, the state of the system after17 time steps when players are updated asynchronously. Underhis updating scheme, the population is dominated by defectors.his serves to illustrate how a change in state update, while keep-ng every other parameter of the model constant, can completelyhange the dynamics of the system. This implies that more atten-ion should be paid to such details (Huberman and Glance, 1993).
Synchronous and asynchronous updating are only two of a wideange of possible updating schemes. In previous work (Cornforth etl., 2005), we have shown that random asynchronous and orderedsynchronous updating schemes can generate complex behaviors
ot seen in their synchronous counterparts. The main focus of thisaper is to explore the effect of alternative updating schemes onhe evolutionary dynamics of spatial Prisoner’s Dilemma. In theollowing section, we briefly review a number of updating schemesbserved in biological systems. Following on from this, we provide
ovpve
gy. (For interpretation of the references to color in this figure legend, the reader is
formal definition of spatial evolutionary games and detail howach of the updating schemes is implemented, along with the mea-ures used to analyze the behavior of the system. In Section 4, weescribe the simulation experiments and results, and finally in Sec-ion 5 we provide a discussion of the findings and some concludingomments.
. Asynchronous Processes in Biological Systems
Biological systems provide abundant evidence that agents (play-rs) update their state in accordance with one of a wide array ofpdating schemes. In this section we will provide a number ofxamples, drawn from biology and ecology, where the dynam-cs of the system are driven by different updating schemes. Weave already seen the effect of synchronous and asynchronouspdating on artificial systems; however, many natural systems fallomewhere between these two schemes. Often these alternativepdating schemes contain some degree of structure or orderingnd can even demonstrate the ability to synchronize over time.e collectively refer to these semi-structured updating schemes
s ordered asynchronous processes.
.1. Social Networks
Social networks are groups of people interacting via social con-acts. The “states” of people include their opinions and beliefs.nteractions and changes in state may take place synchronouslye.g. the influence of mass media) or asynchronously (Stocker et al.,001). We can identify two types of random asynchronous updat-
ng in social networks. The first type occurs when people meet byhance and subsequent interactions cause them to re-evaluate their
pinions. This is independent random sampling, that is, an indi-idual is chosen and updated at random with replacement. So therobability of a state update is independent of the number of pre-ious state updates. The second type occurs, for example, during anlection. In this case, once a person has voted, they are not allowed
122 D. Newth, D. Cornforth / BioSystems 95 (2009) 120–129
F value( s upda
tuoi
2
gdaasc
2
elf
o(ttRt1tb
2
ceo
ig. 2. Typical asymptotic patterns for each of the updating schemes, for a range of bc) random asynchronous updating without replacement; (d) random asynchronou
o vote again until the next election. In this scenario, the order ofpdate is random, but each individual is updated once and oncenly during each round. At each stage, the individual to be updateds chosen at random without replacement.
.2. Neural Activity
The behavior of interconnected neurons in the brain leads tolobal patterns of behavior across the whole brain. This activityoes not exhibit stationary patterns, but periodic, quasi-periodicnd chaotic patterns (Freeman, 1992). There is no known mech-nism such as a global clock in the brain, yet neurons exhibitynchronized behavior for a time, suggesting a mechanism of asyn-hronous updating leading to periods of synchronous updating.
.3. Forest Succession
The competition between different species within a forestcosystem, coupled with catastrophic events such as forest fires,eads to a complex system of interactions. Succession between dif-erent community classes (such as a transition from rainforest to
ah(ta
s. (a) Synchronous updating; (b) random asynchronous updating with replacement;ting with a fixed order; (e) clocked updating; (f) self-synchronizing updating.
pen savanna) requires vastly different time scales to completeNoble and Slatyer, 1980). A fire-induced transition from woodlando grassland may be virtually instantaneous, whereas a transitiono mature rainforest might take thousands of years to complete.ecognition of the asynchronous nature of forest succession led tohe adoption of the semi-Markov model in this context (Howard,971). Although an ordered-asynchronous updating is implicit inhese models, forest succession has not been widely recognized aselonging to this category of processes.
.4. Coordination of Resource Sharing
Within a large population, a sub-population may act in syn-hrony. Separation of flowering times in eucalypts is a goodxample of this behavior. This separation of time scales (wherene sub-species will flower followed by another and so on) is
mechanism for maintaining genetic “identity”, and limitingybridization (Keatley et al., 2004). In the late 1950s Hutchinson1959) noted that the reproductive cycles of sub-species of bee-les were ordered in lock-step, so as to avoid direct competition forvailable resources.
D. Newth, D. Cornforth / BioSystems 95 (2009) 120–129 123
F , averu ent;s
2
nIdttmecae
3
fspaE
atlNof
tta
(
ig. 3. Fraction of cooperators (h) occupying the lattice for each updating schemepdating with replacement; (c) random asynchronous updating without replacemelf-synchronizing updating.
.5. Synchronization of Ovulation
The synchronization of reproductive cycles via pheromone sig-alling is one of the best examples of a self-synchronizing process.
nitially individuals within a population have a well defined repro-uctive cycle. Pheromone signals passed between members ofhe population “nudge” the phase of the individuals reproduc-ive cycle until they fall into phase. This is believed to be the
ajor driver in ovarian synchrony, and has been reported in sev-ral mammalian species including: humans (McClintock, 1971);himpanzees (Wallis, 1985); golden lion tamarins, Leontideus ros-lia (French and Stribley, 1987); and golden hamsters (Handelmannt al., 1985).
. Spatial Games
We now give a general outline of the spatial evolutionary games
ramework studied here. We consider a game consisting of a finiteet of pure strategies S = {C, D} and a finite set of players. Eachlayer I adopts a strategy in S. Let E(i, j) be the payoff to an individualdopting strategy i ∈ S against an opponent adopting strategy j ∈ S.ach player I occupies a site on a regular lattice A —which represents
(
aged over 100 simulations. (a) Synchronous updating; (b) random asynchronous(d) random asynchronous updating with a fixed order; (e) clocked updating; (f)
landscape, and each site on the lattice represents a territory withinhe landscape. In this account A is a regular two-dimensional squareattice. Every site on the lattice has a set of neighbors denoted by(I). For this study, N(I) is chosen to be I’s eight nearest neighborsr the Moore neighborhood. A fixed boundary condition was usedor all experiments presented here.
A spatial evolutionary game is defined by an association at timeof a strategy �t(I) ∈ S to each cell I ∈ A, with a rule that determineshe strategy occupying I at t + 1. The dynamical process is defineds follows:
1) the total st(I) for player I time t is defined as the sum of thepayoffs resulting from playing all the neighboring cells. That is:
st(I) =∑
J ∈ N(I)
E(�t(I), �t(J)). (1)
2) Using these scores, we can associate a strategy with each playerI at time t + 1. For any player I ∈ A, let î ∈ S be the strategyassociated at generation t with the cell J ∈ N(I) which has themaximum score smax
t (J).
124 D. Newth, D. Cornforth / BioSystems 95 (2009) 120–129
F ged ow om asu
(
omcatc
3
aSstua
(
(
(
(
ig. 4. Fraction of strategy changes (h′) in the lattice for each updating scheme, averaith replacement; (c) random asynchronous updating without replacement; (d) randpdating.
3) We then set �t+1(I) = î. More specifically, the strategy occupy-ing site I at t + 1 is the strategy within N(I) that receives thehighest payoff.
We follow Nowak and May (1992) in the selection of the pay-ff values. Specifically, R = 1, T = b (with b > 1), S = P = 0. That is,utual cooperation scores 1, mutual defection scores 0, and double
rossing your opponent pays b. The parameter b characterizes thedvantage of defectors over cooperators, and as such forms a con-rol parameter through which dynamical behavior of the systeman be studied.
.1. Updating Schemes
Within the framework defined above, it is possible to define
lternative updating schemes that mimic the behaviors outlined inection 2. In our alternative updating schemes we consider a timetep to be completed when n updates have occurred. We define no equal the number of players within the model. We refer to thepdate of a single player as a micro-time-step, and the update ofll n players as a macro-time-step.
(
ver 100 simulations. (a) Synchronous updating; (b) random asynchronous updatingynchronous updating with a fixed order; (e) clocked updating; (f) self-synchronizing
1) Synchronous updating. This is the behavior described above. Inshort, at each time step, every player’s score is evaluated andstate updated in parallel.
2) Random asynchronous updating with replacement. Under thisupdating scheme, each macro-time-step is divided into nmicro-time-steps. At each micro-time-step a player is selectedat random from the population, and updated. As a consequence,as each player is updated, they “awake” to see a slightly differentworld from that of the players updated before and after them.
3) Random asynchronous updating without replacement. Under thisupdating scheme, for each micro-time-step, a player is cho-sen at random and updated. Unlike the random asynchronousupdating with replacement scheme, once a player is updated itcannot be updated again.
4) Random asynchronous updating with a fixed order. This updat-ing scheme is identical to the random asynchronous updatingwithout replacement scheme; however, players are updated in
a fixed random order throughout the entire simulation (Kanada,1994).
5) Clocked updating. The Clock updating scheme (Low and Lapsley,1999) assigns a clock or an oscillator to each player. Initially eachclock is set to a random starting position along its period. During
D. Newth, D. Cornforth / BioSystems 95 (2009) 120–129 125
F lation( s upda
(
3
sf(
ig. 5. Neighborhood entropy (H) for each updating scheme, averaged over 100 simuc) random asynchronous updating without replacement; (d) random asynchronou
each micro-time-step, the oscillator moves along its period bysome fixed amount. When the oscillator reaches the top of itsperiod, the player awakens and updates its state. The process isrepeated until n updates are completed.
6) Self-synchronizing updating. This scheme is similar to theclocked updating scheme, except the clock of each player isinfluenced by the clocks of other players. Here, clocks are mod-eled as a Kuramoto style self-synchronizing oscillator (Strogatz,2000):
�̇i = ωi +∑
j
�ij(�i − �j), (2)
where �i is the phase, ωi is the natural frequency of oscillatori. Updating is identical to the clocked scheme, however eachplayer can influence the clocks of all other players through thephase difference function � (·). Here � (�i − �j) is a function ofthe phase difference between the two clocks. This model is used
when there is total coupling between all oscillators. For ourmodel we modified this and use:
�i(t+1) = �i(t) + ωi + ˇ(�̄ − �i), (3)
3
st
s. (a) Synchronous updating; (b) random asynchronous updating with replacement;ting with a fixed order; (e) clocked updating; (f) self-synchronizing updating.
where ˇ is a multiplying factor representing the couplingstrength of the oscillators, and �̄ is the mean phase in the systemgiven by
�̄ = arctan
⎛⎜⎜⎝
∑i
sin(�i)
∑i
cos(�i)
⎞⎟⎟⎠ . (4)
The value of ˇ was set to 0.25 for all experiments reported here.
.2. Measures
To compare the dynamical properties of the various updatingchemes, we employ the use of four measures. These are: (1) theraction of cooperators (h); (2) the fraction of strategy changes (h′);3) neighborhood entropy (H); and (4) Lyapunov exponent (�).
.2.1. Fraction of CooperatorsThis statistic characterizes the number of cooperators (and
ubsequently the number of defectors) making up the popula-ion (Nowak and May, 1992). The fraction of cooperators h is
126 D. Newth, D. Cornforth / BioSystems 95 (2009) 120–129
F tions.( s upda
c
h
wa
3
o
h
wa
3
cc
H
wustf
3
cdwioe
�
ig. 6. Lyupanov exponent (�) for each updating scheme, averaged over 100 simulac) random asynchronous updating without replacement; (d) random asynchronou
alculated by:
= Cn
L2, (5)
here Cn is the number of lattice sites occupied by a cooperator,nd L is the side length of the lattice.
.2.2. Fraction of Strategy ChangesThis measure characterizes the number of strategy changes
ccurring in the entire model. This fraction is calculated as
′ = C ′n
L2, (6)
here C ′n is the number of lattice sites where the strategy at time t
nd t − 1 are different. Again L is the side length of the lattice.
.2.3. Neighborhood EntropyThe third measure used here is the entropy of the neighborhood
onfigurations, also known as the Shannon–Weaver informationontent (Shannon and Weaver, 1963):
= −∑
i
pi log2pi, (7)
wiip
(a) Synchronous updating; (b) random asynchronous updating with replacement;ting with a fixed order; (e) clocked updating; (f) self-synchronizing updating.
here pi is the relative frequency of the ith neighborhood config-ration. As there are eight neighbors, and each one can be in twotates (C or D), as a result there are 28 configurations. The statis-ic shows the diversity and thus the complexity of the structuresormed upon the lattice.
.2.4. Lyapunov ExponentTo measure the sensitivity of the model to initial conditions, we
alculate the Lyapunov exponent � for each updating scheme. Too this we maintain a second model that is identical to the first,ith the state at a single site changed. Both models are updated
n the same way as they evolve. At each time step, the numberf differences in the two models Nd is calculated. The Lyupanovxponent is then calculated as:
= Nd
L2t, (8)
here L is the side length of the lattice, and t is the number ofterations since initialization. The Lyapunov exponent character-zes the divergence or instability within the system due to a smallerturbation.
ioSyst
4
Fiwacccsdaotdvbstbtttt
ubfuwTo
cospaicaItfisltn
srcstvsw1
blsab
lettasttsTvsc1osr
c(lCtIoeCrIdIttob
tuaggouGame of Life. Under the synchronous updating scheme, these glid-ers are blinkers. More work is needed to examine if these glidersare capable of creating new gliders and ultimately more complexbehavior.
Table 1Classification of the complexity created by each updating schemes for variousranges of b values. (a) Synchronous updating; (b) random asynchronous updatingwith replacement; (c) random asynchronous updating without replacement; (d)random asynchronous updating with a fixed order; (e) clocked updating; and (f)self-synchronizing.
(a) (b) (c) (d) (e) (f)
1 ≤ b < 1.25 II a II a II a II a II a II a
1.25 ≤ b < 1.28 II b II a II a II a II a II a
1.28 ≤ b < 1.3 II b III III III III III1.3 ≤ b < 1.5 II b II b II b II b II b III
D. Newth, D. Cornforth / B
. Experiments and Results
As an initial analysis of the dynamics of the updating schemes,ig. 2 illustrates typical asymptotic patterns for each of the updat-ng schemes and various b values, on a 99 × 99 square lattice
ith fixed boundary conditions. The models were seeded withrandom starting configuration containing approximately 50%
ooperators and 50% defectors. For b = 1.1, all updating schemesonverged to a static configuration, with defectors surrounded byooperators. When b = 1.3 and b = 1.5, the synchronous and self-ynchronizing schemes created spatial patterns with stable lines ofefectors, and small sub-populations that “flip” between cooper-tion and defection (blinkers). For these parameter settings, thether schemes produced a network of defector lines, that con-inually pulled themselves apart and reassembled in a slightlyifferent configuration. When b = 1.7 all updating schemes con-erged to almost static configurations of defector lines, with a fewlinkers. When b = 1.9, both synchronous and self-synchronizingchemes generated rich dynamical behavior creating complex spa-ial patterns, while the clocked and random asynchronous schemesoth lead to a point attractor where defection was the only longerm viable strategy. Finally, we show that when the game condi-ions no longer hold (e.g. b = 2.1), all the updating schemes leado a point attractor where the only surviving strategy is defec-ion.
The asymptotic pictures of Fig. 2 show that the alternativepdating schemes are capable of creating an array of interestingehaviors. To examine these behaviors further, 100 runs were madeor each b value and for each updating schemes on a 200 × 200 reg-lar lattice with fixed boundary conditions. The control parameter bas varied systematically between 1 ≤ b ≤ 2 in increments of 0.01.
he asymptotic values for h, h′, H, and � were recorded and averagedver the 100 runs. Figs. 3–6 show the results of these experiments.
First, we examine the fraction of the population made up ofooperators (h). Fig. 3 shows the fraction of cooperators h for eachf the updating schemes. The most notable difference between theynchronous and asynchronous schemes is that the former sup-orts more cooperators than the others. The greatest difference ispparent when b > 1.8, the same region indentified as the mostnteresting in previous studies. In this parameter range, the syn-hronous scheme displays rich dynamical behavior with h ≈ 0.3,s described by Nowak and May (1992) and shown in Fig. 1(left).n contrast, all the asynchronous schemes converge to a popula-ion composed entirely of defectors (h = 0), confirming previousndings (Huberman and Glance, 1993). The exception to this is theelf-synchronizing scheme, where h ≈ 0.2. The latter shows simi-arity with both synchronous and asynchronous schemes, becausehe value of coupling selected allows pockets of players to synchro-ize their behavior.
The fraction of strategy changes (h′) is shown in Fig. 4. Allchemes show a very low count of changes apart from severalegions of the parameter b. At b ≈ 1.3 and b ≈ 1.55, all asyn-hronous schemes show large peaks up to a value of 0.2. Theynchronous scheme shows lower but more spread out peaks inhese regions, with an additional peak at b ≈ 1.15, plus the ele-ated section between 1.8 ≤ b ≤ 2. The self-synchronous schemehows features of both synchronous and asynchronous schemes,ith a reduced peak at b ≈ 1.3 and an elevated section between
.8 ≤ b ≤ 2.The neighborhood entropy H, describes the diversity of neigh-
orhood configurations of cooperators and defectors upon theattice. Fig. 5 shows the neighborhood entropy for each updatingcheme. This value is generally higher for asynchronous schemes,s there is less coordination of the states in neighboring cells. When> 1.8, the comments made in relation to Fig. 3 apply.
111
ems 95 (2009) 120–129 127
The previous measures consider features of the evolving popu-ation (their composition and diversity of structure). The Lyapunovxponent (�) describes how sensitive the model is to initial condi-ions, by measuring how fast a small disturbance grows throughhe entire population. Fig. 6 shows the Lyapunov exponent forll six update schemes. The Lyapunov exponent is substantiallyimilar for all the schemes considered, except for elevated sec-ions apparent in the asynchronous schemes. In particular there arewo regions where the asynchronous updating schemes are mostensitive to initial conditions: 1.28 < b < 1.34 and 1.5 < b < 1.6.hese correlate with regions of Figs. 3–5 that display elevatedalues. As the graphs in Fig. 6 have a logarithmic y-axis, theseections represent substantially large relative values for �. The syn-hronous scheme shows some sensitivity to initial conditions when.8 < b < 2. Again the self-synchronizing scheme shows featuresf the synchronous scheme with elevated values for b ≥ 1.8, andhows features of the asynchronous schemes with the elevatedegions.
It is interesting to note the relationship between b values andomplexity classes of cellular automata as proposed by Wolfram1984). Under this classification scheme, the behavior of a cellu-ar automata can be classified as belonging to one of four classes.lass I behavior is very simple behavior, where almost all ini-ial conditions lead to exactly the same uniform final state. ClassI behavior displays many different possible final states, but allf them consist just of a certain set of simple structures, thatither remain the same forever or repeat every few time steps.lass III behavior is more complex, and seems in many respectsandom, although coherent structures appear repeatedly. ClassV behavior involves a mixture of order and randomness to pro-uce complex patterns. Cellular automata that can produce Class
V behavior, generally create a variety of simple local structures,hat interact with each other to produce more complicated struc-ures (Wolfram, 2002). Table 1 provides a course grain classificationf the behavior created by each updating scheme for variousvalues.
On further examination of each updating scheme, we foundhat all the random asynchronous updating schemes and clockedpdating scheme, are able to create “gliders” that randomly walkcross the lattice for 1.13 ≤ b < 1.14. Fig. 7 (left) illustrates theseliders. However, due to the fixed boundary conditions, as theliders are lost the system converges to a fixed state containingnly cooperators (see Fig. 7, right). For this narrow range of b val-es, the Spatial Prisoner’s Dilemma can be compared to Conway’s
.5 ≤ b < 1.6 II b III III III III III
.6 ≤ b < 1.8 II b II b II b II b II b II b and III
.8 ≤ b < 2 IV I I I I and III I and IV
a Fix points only.b Fixed points and short cycles.
128 D. Newth, D. Cornforth / BioSystems 95 (2009) 120–129
F the cl( e worr
5
lpaepaalaParstacd
atstcesstwfdyes
une
auuph
aisAidtWtDtpSatsbdge
R
AA
B
C
ig. 7. Random walking gliders. The random asynchronous updating schemes andleft). However the fixed boundary conditions lead to a fixed point (right) where thange of b values (1.13–1.14), these updating schemes display Class II complexity.
. Discussion
The Prisoner’s Dilemma in an interesting metaphor for the bio-ogical problem of how cooperative behavior may emerge andersist. Many of the previous studies are confined to individu-ls or groups of individuals who have memories of past events;xpectations of future encounters and can formulate more com-lex strategies such as Tit-for-tat (Axelrod, 1984). By contrast, thepproach taken here was to study the emergence of cooperationmongst simple organisms with limited memories, who formu-ate very simple strategies. Specifically we examined the effect ofsynchrony in the spatial version of Prisoner’s Dilemma. Spatialrisoner’s Dilemma is analogous to a population distributed acrosslandscape with interactions between players occurring locally
ather than long distance. The inclusion of alternative updatingchemes means that players make decisions about future interac-ions based on delayed and imperfect information. When playersre updated in these alternative ways, it does not always follow thatooperation will emerge and persist, or even that the system willisplay rich dynamical behavior.
In our experiments, we found that the inclusion of varioussynchronous interactions between players lead to simulationshat produced complicated dynamical features, not seen in theynchronous counterparts. The Spatial Prisoner’s Dilemma is essen-ially a 2D cellular automata with a rule space of 225. This rule spacereates a rich “universe” of dynamical objects (such as rotators, glid-rs, blinkers, etc.) (Nowak and May, 1992). The alternative updatingchemes, presented here, altered the rule space by limiting the tran-itions that can/cannot occur. As a result new structures appearedhat were not observed in the synchronous version (e.g. the randomalking glider, which in the synchronous case is a blinker). Also, we
ound that the degree of cooperation achieved was strongly depen-ant upon the updating scheme and the reward of double-crossingour opponent. Populations that were able to support both coop-rators and defectors in the synchronous world, were unable to do
o in any of the alternative schemes examined here.
In the real world, purely synchronous or random asynchronouspdating appears to be rare property of complex systems. Manyatural processes seem to lie somewhere between these twoxtremes. We refer to the processes that fill this gap as ordered
C
F
F
ocked scheme are capable of creating gliders that randomly walk across the worldld is dominated by cooperators. As a result, for these boundary conditions and this
synchronous processes. One such example is the self-synchronouspdating scheme, where the degree of coupling between individ-als controls the amount of synchrony or asynchrony within theopulation. This seems to offer an interesting mechanism to explainow living systems could manipulate the degree of synchrony.
As our results show, the exact manner of updating can haveprofound effect on overall system behavior and structure. One
mplication of this is the realization that a majority of naturalystems are able to function without external synchronization.nother is that when building models of natural systems, it is
mportant to consider the updating scheme used. Simulations con-ucted without such consideration may lead to results that failo fully represent the rich behavior of the systems under study.
hile the motivation for this work was to examine how alterna-ive updating schemes change the dynamics of Spatial Prisoner’silemma, the results here are also relevant to the dynamics of spa-
ially extended systems such as Ising systems, spatially distributedredatory systems (Frean and Abrahams, 2001; Johnson andeinen, 2002), and even models of pre-biotic evolution (Boerlijstnd Hogeweg, 1991); as the inclusion of asynchrony alters the struc-ure and nature of the interactions taking place. Existing modelsuch as those outlined in Section 2 may be made more realisticy taking asynchrony into account. The inclusion of asynchronousynamics may also provide a deeper understanding of the emer-ence of system level order from the interactions of its low levellements.
eferences
xelrod, R., 1984. The Evolution of Cooperation. Basic Books, New York.xelrod, R., Hamilton, W.D., 1981. The evolution of cooperation. Science 211,
hypercycles stable against parasites. Physica D 48, 17–28.harnov, E.L., Gotshall, D.W., Robinson, J.G., 1978. Sex ratio: adaptive response to
population fluctuations in pandalid shrimp. Science 200, 204–206.
ornforth, D., Green, D.G., Newth, D., 2005. Ordered asynchronous processes in
multi-agent systems. Physica D 204, 70–82.rean, M.R., 1994. The prisoner’s dilemma without synchrony. Proceedings of the
Royal Society, Series B 257, 75–79.rean, M., Abrahams, E.R., 2001. Rock–paper–scissors and the survival of the weakest.
Proceedings of the Royal Society of London, Series B 268, 1323–1327.
ioSyst
F
F
HH
H
H
H
J
K
K
L
M
MM
M
M
N
N
N
S
S
D. Newth, D. Cornforth / B
reeman, W., 1992. Tutorial on neurobiology: from single neurons to brain chaos.International Journal of Bifurcation and Chaos 2, 451–482.
rench, J.A., Stribley, J.A., 1987. Synchronization of ovarian cycles within and betweensocial groups in golden lion tamarins (Leontopithecus rosalia). American Journalof Primatololgy 12, 469–478.
amilton, W.D., May, R.M., 1977. Dispersal in stable habitats. Nature 269, 578–581.andelmann, G., Ravizza, R., Ray, W.J., 1985. Social dominance determines estrus
entrainment among female hamsters. Hormones and Behavior 14, 107–115.oward, R.A., 1971. Dynamic Probabilistic Systems: Semi-Markov and Decision Pro-
cesses. New York.uberman, B.A., Glance, N.S., 1993. Evolutionary games and computer simulations.
Proceedings of the National Accademy of Science 90, 7716–7718.utchinson, G.E., 1959. Homage to santa rosalia or why are there so many kinds of
animals? The American Naturalist XCIII, 145–158.ohnson, C.R., Seinen, I., 2002. Selection for restraint in competitive ability in spatial
competition systems. Proceedings of the Royal Society of London, Series B 269,655–663.
anada, Y., 1994. The effects of randomness in asynchronous 1D cellular automata.
In: Proceedings of the Artificial Life, vol. IV.
eatley, M.R., Hudson, I.L., Fletcher, T.D., 2004. Long-term flower synchrony of box-ironbark eucalypts. Australian Journal of Botany 52, 47–54.
ow, S.H., Lapsley, D.E., 1999. Optimization flow control. Part I. Basic algorithm andconvergence. IEEE/ACM Transactions on Networking.
cClintock, M.K., 1971. Menstrual synchrony and suppression. Nature 229, 244–245.
S
W
WW
ems 95 (2009) 120–129 129
aynard Smith, J., Price, G.R., 1973. The logic of animal confilict. Nature 246, 15–18.aynard Smith, J., 1974. The theory of games and the evolution of animal conflicts.
Journal of Theortical Biology 47, 209–221.aynard Smith, J., 1982. Evolution and the Theory of Games. Cambridge University
Press.irmirani, M., Oster, G., 1979. Competition, kin selection and evolutionary stable
strategies. Thoretical Population Biology 13, 36–40.oble, I.R., Slatyer, R.O., 1980. The use of vital attributes to predict successional
changes in plant communities subject to recurrent disturbances. Vegetation 43,5–21.
owak, M.A., May, R.M., 1992. Evolutionary games and spatial chaos. Nature 359,826–829.
owak, M.A., Sigmund, K., 1994. The alternating prisoners dilemma. Journal of The-ortical Biology 168, 219–226.
hannon, C.E., Weaver, W., 1963. The Mathematical Theory of Communication. Uni-versity of Illinois Press, Illinois.
tocker, R., Green, D.G., Newth, D., 2001. Consensus and Cohesion in Simulated SocialNetworks. The Journal of Artificial Societies and Social Simulation (JASSS) 4.
trogatz, S.H., 2000. From Kuramoto to Crawford: exploring the onset of synchro-nization in populations of coupled oscillators. Physica D 143, 1–20.
allis, J., 1985. Synchrony of estrous swelling in captive group-living chimpanzees(Pan troglodytes). International Journal of Primatology 6, 335–350.
olfram, S., 1984. Cellular automata as models of complexity. Nature 311, 419–424.olfram, S., 2002. A New Kind of Science. Wolfram Media, Illinois.
