12.3.2003AS-74.330 Complex Systems1 Towards new order Lasse Eriksson 12.3.2003.

12.3.2003 AS-74.330 Complex Systems 1

Towards new order

Lasse Eriksson

12.3.2003


Towards new order - outline

• Introduction• Self-Organized Criticality (SOC)

– Sandpile model

• Edge of Chaos (EOC)– Two approaches

• Measuring Complexity– Correlation distance– Phase transition

• Highly Optimized Tolerance (HOT)• Summary and conclusions


Introduction

• From Catastrophe to Chaos? [12]– Catastrophe theory studies and classifies phenomena

characterized by sudden shifts in behavior arising from small changes in circumstances

– Originated by the French mathematician Rene Thom in the 1960s, catastrophe theory is a special branch of dynamical systems theory

• ”... the big fashion was topological defects. Everybody was ... finding exotic systems to write papers about. It was, in the end, a reasonable thing to do. The next fashion, catastrophe theory, never became important for anything.” - James P. Sethna (Cornell University, Ithaca) [11]


• We are moving from Chaos towards new Order– From Chaotic to Complex systems– What is the difference, just a new name? Is it really

something new?

• Langton's famous egg diagram (EOC, [7])

Introduction


Introduction [13]

• Chaos vs. Complex– Advances in the scientific study of chaos have been

important motivators/roots of the modern study of complex systems

– Chaos deals with deterministic systems whose trajectories diverge exponentially over time

– Models of chaos generally describe the dynamics of one (or a few) variables which are real. Using these models some characteristic behaviors of their dynamics can be found

– Complex systems do not necessarily have these behaviors. Complex systems have many degrees of freedom: many elements that are partially but not completely independent

– Complex behavior = "high dimensional chaos”


Introduction

• Chaos is concerned with a few parameters and the dynamics of their values, while the study of complex systems is concerned with both the structure and the dynamics of systems and their interaction with their environment

• Same kind of phenomena in catastrophe, chaos and complexity theory– Stable states become unstable– Sudden changes in system’s behavior– Critical points, edges...


Introduction [13]

• Complexity is …(the abstract notion of complexity has been captured in many different ways. Most, if not all of these, are related to each other and they fall into two classes of definitions):– 1) ...the (minimal) length of a description of the system.– 2) ...the (minimal) amount of time it takes to create the

system

• The length of a description is measured in units of information. The former definition is closely related to Shannon information theory and algorithmic complexity, and the latter is related to computational complexity


Introduction

• Emergence is...– 1) ...what parts of a system do together that they would not

do by themselves: collective behavior• How behavior at a larger scale of the system arises from

the detailed structure, behavior and relationships on a finer scale

– 2) ...what a system does by virtue of its relationship to its environment that it would not do by itself: e.g. its function

• Emergence refers to all the properties that we assign to a system that are really properties of the relationship between a system and its environment

– 3) ...the act or process of becoming an emergent system


About fractals [1]

• Many objects in nature are best described geometrically as fractals, with self-similar features on all length scales– Mountain landscapes: peaks of all sizes, from kilometres

down to millimetres– River networks: streams of all sizes– Earthquakes: occur on structures of faults ranging from

thousands of kilometres to centimetres

• Fractals are scale-free (you can’t determine the size of a picture of a part of a fractal without a yardstick)

• How does nature produce fractals?


About fractals

• The origin of the fractals is a dynamical, not a geometrical problem

• Geometrical characterization of fractals has been widely examined but it would be more interesting to gain understanding of their dynamical origin

• Consider earthquakes:– Earthquakes last for a few seconds– The fault formations in the crust of the earth are built up

during some millions of years and the crust seems to be static if the observation period is a human lifetime


About fractals

• The laws of physics are local, but fractals are organized over large distances

• Large equilibrium systems operating near their ground state tend to be only locally correlated. Only at a critical point where continuous phase transition takes place are those systems fractal


Example: damped spring

• Consider a damped spring as shown in the figure

• Let’s model (traditionally!) and simulate the system– mass m

– spring constant k

– damping coefficient B

– position x(t)x

mk

B



x

To Workspaces

1

Integrator1s

1

Integrator

k

Gain2

B

Gain1

-1/m

Gain

s

mxkgm

s

kgB

m

Nk

tkxtxBm

tx

tkxtxBtxm

1)0(,1,08.0,1.0

)()(1

)(

0)()()(

.

...

...


145 150 155 160 165 170 175 180 185 190 195 200

-0.01

-0.005

0

0.005

0.01

Time

Pos

ition

x(t

)

0 20 40 60 80 100 120 140 160 180 200-2

-1.5

-1

-0.5

0

0.5

1

1.5

2

2.5

3

Time

Pos

ition

x(t

)


Zoom:


Damped spring [1]

• In theory, ideal periodic motion is well approximated by sine wave

• Oscillatory behavior with decreasing amplitude theoretically continues forever

• In real world, the motion would stop because of the imperfections such as dust

• Once the amplitude gets small enough, the emotion suddenly stops– This generally occurs at the end of an oscillation where the

velocity is smallest– This is not the state of smallest energy!– In a sense, the system is most likely to settle near a

”minimally stable” state


Multiple Pendulums

• The same kind of behavior can be detected when analysing pendulums

• Consider coupled pendulums which all are in a minimally stable state

• The system is particularly sensitive to small perturbations which can ”avalanche” through the system

• Small disturbances could grow and propagate through the system with little resistance despite the damping and other impediments


Multiple Pendulums

• Since energy is dissipated through the process, the energy must be replenished for avalanches to continue

• If ”Self-Organized Criticality” i.e. ”SOC” is considered, the interest is on the systems where energy is constantly supplied and eventually dissipated in the form of avalanches


Self-Organized Criticality [1],[2]

• Concept was introduced by Per Bak, Chao Tang, and Kurt Wiesenfeld in 1987

• SOC refers to tendency of large dissipative systems to drive themselves to a critical state with a wide range of length and time scales

• The dynamics in this state is intermittent with periods of inactivity separated by well defined bursts of activity or avalanches

• The critical state is an attractor for the dynamics• The idea provides a unifying concept for large-scale

behavior in systems with many degrees of freedom


Self-Organized Criticality

• SOC complements the concept ”chaos” wherein simple systems with a small number of degrees of freedom can display quite complex behavior

• Large avalanches occur rather often (there is no exponential decay of avalanche sizes, which would result in a characteristic avalanche size), and there is a variety of power laws without cutoffs in various properties of the system

• The paradigm model for this type of behavior is the celebrated sandpile cellular automaton also known as the Bak-Tang-Wiesenfeld (BTW) model


Sandpile model [1],[2]

• Adding sand slowly to a flat pile will result only in some local rearrangement of particles

• The individual grains, or degrees of freedom, do not interact over large distances

• Continuing the process will result in the slope increasing to a critical value where an additional grain of sand gives rise to avalanches of any size, from a single grain falling up to the full size of the sand pile

• The pile can no longer be described in terms of local degrees of freedom, but only a holistic description in terms of one sandpile will do

• The distribution of avalanches follows a power law


Sandpile model

• If the slope were too steep one would obtain a large avalanche and a collapse to a flatter and more stable configuration

• If the slope were too shallow the new sand would just accumulate to make the pile steeper

• If the process is modified, for instance by using wet sand instead of dry sand, the pile will modify its slope during a transient period and return to a new critical state– Consider snow screens: if you build them to prevent

avalanches, the snow pile will again respond by locally building up to steeper states, and large avalanches will resume


[14]


Simulation of the sandpile model [1]

• 2D cellular automaton with N sites• Integer variables zi on each site i

represent the local sandpile height• When height exceeds critical

height zcr (here 3), then 1 grain is transferred from unstable site to each 4 neighboring site

• A toppling may initiate a chain reaction, where the total number of topplings is a measure of the size of an avalanche

• Figure: after 49152 grains dropped on a single site (fractals?)


Simulation of the sandpile model

• To explore the SOC of the sandpile model, one can randomly add sand and have the system relax

• The result is unpredictable and one can only simulate the resulting avalanche to see the outcome

• State of a sandpile after adding pseudo-randomly a large amount of sand on a 286*184 size lattice

• Figure: open boundaries• Heights: 0 = black, 1 = red,

2 = blue, 3 = green



• Configuration seems random, but some subtle correlations exist (e.g. never do two black cells lie adjacent to each other, nor does any site have four black neighbors)

• Avalanche is triggered if a small amount of sand is added to a site near the center

• To follow the avalanche, a cyan color has been given to sites that have collapsed in the following figures



time


Sandpile model

• Figure shows a log-log plot of the distribution of the avalanche sizes s (number of topplings in an avalanche), P is the probability distribution

1.2 ,)( 1 ssP


Sandpile model

• Because of the power law, the initial state was actually remarkably correlated although it seemed at first featureless

• For random distribution of z’s (pile heights), one would expect the chain reaction of an avalanche to be either– Subcritical (small avalanche)– Supercritical (exploding avalanche with collapse of the entire

system)

• Power law indicates that the reaction is precisely critical, i.e. the probability that the activity at some site branches into more than one active site, is balanced by the probability that the activity dies


Simulation of the sandpile model [3]

Sandpile Java applet

http://cmth.phy.bnl.gov/~maslov/Sandpile.htm


The Edge of Chaos [4]

• Christopher Langton’s 1-D CA– States ”alive”, ”dead”: if a cell and its neighbors are dead,

they will remain dead in the next generation

• Some CA’s are boring since all cells either die after few generations or they quickly settle into simple repeating patterns– These are ”highly ordered” CA’s– The behavior is predictable

• Other CA’s are boring because their behavior seems to be random– These are ”chaotic” CA’s– The behavior is unpredictable


The Edge of Chaos

• Some CA’s show interesting (complex, lifelike) behavior– These are near the border of between chaos and order– If they were more ordered, they would be predictable– If they were less ordered, they would be chaotic

• This boundary is called the ”Edge of Chaos”• Langton defined a simple number that can be used to

help predict whether a given CA will fall in the ordered realm, in the chaotic realm, or near the boundary

• The number (01) can be computed from the rules of the CA. It is simply the fraction of rules in which the new state of the cell is living


The Edge of Chaos

• Remember that the number of rules (R) of a CA is determined by: ,

where K is the number of states and N is the number of neighbors

• If = 0, the cells will die immediately; if = 1, any cell with a living neighbor will live forever

• Values of close to zero give CA's in the ordered realm and values near one give CA's in the chaotic realm. The edge of chaos is somewhere in between

• Value of does not simply represent the edge of chaos. It is more complicated

NKR


The Edge of Chaos

• You can start with = 0 (death) and add randomly rules that lead to life instead of death => > 0

• You get a sequence of CA’s with values of increasing from zero to one

• In the beginning, the CA’s are highly ordered and in the end they are chaotic. Somewhere in between, at some critical value of , there will be a transition from order to chaos

• It is near this transition that the most interesting CA's tend to be found, the ones that have the most complex behavior

• Critical value of is not a universal constant


The Edge of Chaos

Edge of Chaos CA-appletEdge of Chaos CA-applet

http://math.hws.edu/xJava/CA/EdgeOfChaosCA.html


EOC – another approach [5]

• Consider domino blocks in a row (stable state, minimally stable?)– Once the first block is nudged, an ”avalanche” is started– The system will become stable once all blocks are lying

down– The nudge is called perturbation and the duration of the

avalanche is called transient

• The strength of the perturbation can be measured in terms of the effect it had i.e. the length of time the disturbance lasted (or the transient length) plus the permanent change that resulted (none in the domino case)

• Strength of perturbation is a measure of stability!


EOC – another approach

• Examples of perturbation strength:– Buildings in earthquakes: we require short transient length

and return to the initial state (buildings are almost static)– Air molecules: they collide with each other continually,

never settling down and never returning to exactly the same state (molecules are chaotic)

• For air molecules the transient length is infinite, whereas for our best building method it would be zero. How about in the middle?



• A room full of people:– A sentence spoken may be ignored (zero transient), may

start a chain of responses which die out and are forgotten by everyone (a short transient) or may be so interesting that the participants will repeat it later to friends who will pass it on to other people until it changes the world completely (an almost infinite transient - e.g. the Communist Manifesto by Karl Marx is still reverberating around the world after over 120 years)

• This kind of instability with order is called the Edge of Chaos, a system midway between stable and chaotic domains



• EOC is characterised by a potential to develop structure over many different scales and is an often found feature of those complex systems whose parts have some freedom to behave independently

• The three responses in the room example could occur simultaneously, by affecting various group members differently



• The idea of transients is not restricted in any way and it applies to different type of systems– Social, inorganic, politic, psychological…– Possibility to measure totally different type of systems with

the same measure

• It seems that we have a quantifiable concept that can apply to any kind of system. This is the essence of the complex systems approach, ideas that are universally applicable


Correlation Distance [5]

• Correlation is a measure of how closely a certain state matches a neighbouring state, it can vary from 1 (identical) to -1 (opposite)

• For a solid we expect to have a high correlation between adjacent areas, but the correlation is also constant with distance

• For gases correlation should be zero, since there is no order within the gas because each molecule behaves independently. Again the distance isn't significant, zero should be found at all scales


Correlation Distance

• Each patch of gas or solid is statistically the same as the next. For this reason an alternative definition of transient length is often used for chaotic situations i.e. the number of cycles before statistical convergence has returned– When we can no longer tell anything unusual has happened,

the system has returned to the steady state or equilibrium

• Instant chaos would then be said to have a transient length of zero, the same as a static state - since no change is ever detectable. This form of the definition will be used from now on


Measuring Complex Systems [5]

• For complex systems we should expect to find neither maximum correlation (nothing is happening) nor zero (too much happening), but correlations that vary with time and average around midway

• We would also expect to find strong short range correlations (local order) and weak long range ones– E.g. the behavior of people is locally correlated but not

globally

• Thus we have two measures of complexity– Correlations varying with distance– Long non-statistical transients


Phase Transitions [5],[6]

• Phase transition studies came about from the work begun by John von Neumann and carried on by Steven Wolfram in their research of cellular automata

• Consider what happens if we heat and cool systems– At high temperatures systems are in gaseous state (chaotic)– At low temperatures systems are in solid state (static)

• At some point between high and low temperatures the system changes its state between the two i.e. it makes a phase transition

• There are two kinds of phase transitions: first order and second order


Phase Transitions

• First order we are familiar with when ice melts to water– Molecules are forced by a rise in temperature to choose

between order and chaos right at 0° C, this is a deterministic choice

• Second order phase transitions combine chaos and order– There is a balance of ordered structures that fill up the

phase space

• The liquid state is where complex behaviour can arise


Phase Transitions [7]

• Schematic drawing of CA rule space indicating relative location of periodic, chaotic, and ``complex'' transition regimes


Phase Transitions

• Crossing over the lines (in the egg) produces a discrete jump between behaviors (first order phase transitions)

• It is also possible that the transition regime acts as a smooth transition between periodic and chaotic activity (like EOC experiments with ). This smooth change in dynamical behavior (smooth transition) is primarily second-order, also called a critical transition


Phase Transitions

• Schematic drawing of CA rule space showing the relationship between the Wolfram classes and the underlying phase-transition structure


Wolfram’s four classes [8]

• Different cellular automata seem to settle down to– A constant field (Class I)– Isolated periodic structures (Class II)– Uniformly chaotic fields (Class III)– Isolated structures showing complicated internal behavior

(Class IV)


Phase Transitions

• Phase transition feature allows us to control complexity by external forces– Heating or perturbing system => chaotic behavior– Cooling or isolating system => static behavior

• This is seen clearly in relation to brain temperature– Low = static, hypothermia– Medium = normal, organised behaviour– High = chaotic, fever


Highly Optimized Tolerance [9]

• HOT is a mechanism that relates evolving structure to power laws in interconnected systems

• HOT systems arise, e.g. in biology and engineering where design and evolution create complex systems sharing common features– High efficiency– Performance– Robustness to designed-for uncertainties– Hypersensitivity to design flaws and unanticipated

perturbations– Nongeneric, specialized, structured configurations– Power laws


Highly Optimized Tolerance

• Through design and evolution, HOT systems achieve rare structured states which are robust to perturbations they were designed to handle, yet fragile to unexpected perturbations and design flaws

• E.g. communication and transportation systems– Systems are regularly modified to maintain high density,

reliable throughput for increasing levels of user demand– As the sophistication of the systems is increased, engineers

encounter a series of tradeoffs between greater productivity and the possibility of the catastrophic failure

– Such robustness tradeoffs are central properties of the complex systems which arise in biology and engineering


Highly Optimized Tolerance [9],[10]

• Robustness tradeoffs also distinguish HOT states from the generic ensembles typically studied in statistical physics under the scenarios of the edge of chaos and self-organized criticality

• Complex systems are driven by design or evolution to high-performance states which are also tolerant to uncertainty in the environment and components

• This leads to specialized, modular, hierarchical structures, often with enormous “hidden” complexity with new sensitivities to unknown or neglected perturbations and design flaws


Highly Optimized Tolerance [10]

• Robust, yet fragile!Robustness of HOT

systems

Robust

Fragile

Robust(to known anddesigned-foruncertainties)

Fragile(to unknown

or rareperturbations)

Uncertainties


A simple spatial model of HOT

• Square site percolation or simplified “forest fire” model

• Carlson and Doyle,

PRE, Aug. 1999


A “spark” that hits an empty site does

nothing

Assume one “spark” hits the lattice at a

single site




A “spark” that hits a cluster causes loss of

that cluster



Yield = the density after one spark

yield density loss


0 0.2 0.4 0.6 0.8 10

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

Y =(avg.)yield

= density

“critical point”

N=100(size of the lattice)

no sparks

sparks


0 0.2 0.4 0.6 0.8 10

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

limit N

Y =(avg.)yield

c = 0.5927

= density

“critical point”


Cold

Fires don’t matter.

Y


Y

Burned

Everything burns.


Critical point

Y


010

110

210

310

410

-1

100

101

102

Power laws

Criticality

cluster size

cumulativefrequency


Edge-of-chaos, criticality, self-organized criticality

(EOC/SOC)

yield

density

Essential claims:

• Nature is adequately described by generic configurations (with generic sensitivity)• Interesting phenomena are at criticality (or near a bifurcation)


critical

Cold

Highly Optimized Tolerance (HOT)

Burned


Almost any distribution of sparks

OptimizeYield

Power law distributionof events

both analytic and numerical results.

Why power laws?Why power laws?

12.3.2003 AS-74.330 Complex Systems 675 10 15 20 25 30

5

10

15

20

25

30

0.1902

2.8655e-011 4.4486e-026

High probability region

Probability distribution of sparks

2.9529e-016


1

DDOF=1

• The goal is to optimize yield (push it towards the upper bound)• This is done by increasing the design degrees of freedom (DDOF)• Design parameter ρ for a percolation forest fire model

Increasing Design Degrees of Freedom


1 2

3 4

4 tunable densities

Each region is characterized by the ensemble ofrandom configurations atdensity ρi

DDOF= 4



1 2 3 4

5 6 7 8

9 10 11 12

13 14 15 16

16 tunabledensities…

DDOF= 16



SOC = 1 DDOF

Design Degrees of Freedom = Tunable Parameters


HOT= many DDOF

Blue: ρ = ρc Red: ρ = 1

•The HOT states specifically optimize yield in the presence of a constraint•A HOT state corresponds to forest which is densely planted to maximize the timber yield, with firebreaks arranged to minimize the spread damage

Design Degrees of Freedom = Tunable Parameters


grid evolved DDOF

All produce:

HOT: Many mechanisms

• High densities• Modular structures reflecting external disturbance patterns• Efficient barriers, limiting losses in cascading failure• Power laws


0 0.2 0.4 0.6 0.8 10

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

random

grid

High yields

Optimized grid

density = 0.8496yield = 0.7752

large events are unlikely

Small events likely


Increasing DDOF: increases densities,increases yields, decreases losses butincreases sensitivity

Robust, yet fragile

Y


HOT – summary

• It is possible to drive a system over the critical point, where SOC systems collapse. These overcritical states are called ”highly optimized tolerance” –states

• SOC is an interesting but extreme special case…• HOT may be a unifying perspective for many systems• HOT states are both robust and fragile. They are

ultimately sensitive for design flaws• Complex systems in engineering and biology are

dominated by robustness tradeoffs, which result in both high performance and new sensitivities to perturbations the system was not designed to handle


HOT – summary

• The real work with HOT is in– New Internet protocol design (optimizing the throughput of a

network by operating in HOT state)– Forest fire suppression, ecosystem management– Analysis of biological regulatory networks– Convergent networking protocols


Summary and Conclusions

• Catastrophe – Chaos – Complex– Common features, different approaches– Complexity adds dimensions to Chaos– Lack of useful applications

• Self-Organized Criticality– Refers to tendency of large dissipative systems to drive

themselves to a critical state– Coupled systems may collapse during an ”avalanche”

• Edge of Chaos– Balancing on the egde between periodic and chaotic

behavior


Summary and Conclusions

• Parts of a system together with the environment make it all function

• Complex systems– Structure is important in complex systems– Between periodic (and static) and chaotic systems– Order and structure to chaos– Increasing the degrees of freedom

• HOT– Optimizing the profit/yield/throughput of a complex system– By design one can reduce the risk of catastrophes– Yet fragile!


References

• [1]:Bunde, Havlin (Eds.): Fractals in Science 1994• [2]:http://cmth.phy.bnl.gov/~maslov/soc.htm• [3]:http://cmth.phy.bnl.gov/~maslov/Sandpile.htm• [4]:http://math.hws.edu/xJava/CA/EdgeOfChaos.html• [5]:http://www.calresco.org/perturb.htm• [6]:http://www.wfu.edu/~petrejh4/PhaseTransition.htm• [7]:http://www.theory.org/complexity/cdpt/html/

node5.html• [8]:http://delta.cs.cinvestav.mx/~mcintosh/newweb/

what/node8.html


References

• [9]:Carlson, Doyle: Highly Optimized Tolerance: Robustness and Design in Complex Systems 1999

• [10]:Doyle: HOT-intro Powerpoint presentation, John Doyle www-pages: http://www.cds.caltech.edu/~doyle/CmplxNets/

• [11]:http://www.lassp.cornell.edu/sethna/OrderParameters/TopologicalDefects.html

• [12]:http://www.exploratorium.edu/complexity/CompLexicon/catastrophe.html

• [13]:http://necsi.org/guide/concepts/• [14]:http://www.neci.nec.com/homepages/tang/sand/

sand.html


References

• [15]:http://pespmc1.vub.ac.be/COMPLEXI.html • [16]:http://pespmc1.vub.ac.be/CAS.html

Date post:	26-Dec-2015
Category:	Documents
Upload:	augustus-williams
View:	214 times
Download:	0 times

12.3.2003AS-74.330 Complex Systems1 Towards new order Lasse Eriksson 12.3.2003.

Documents