AN INFORMATION-THEORETIC PRIMER ONCOMPLEXITY,

SELF-ORGANISATION &EMERGENCE

Mikhail ProkopenkoCSIRO

Fabio BoschettiCSIRO

Alex RyanDSTO

a Complex System “story”• many, but not too many, components interact in a non trivial fashion

• the system is open (receives energy/information/matter from the environment)

• interactions → symmetry breaking → coordinated behaviour arises

• no central director/template → the system ‘self-organises’

• coordination as patterns detectable by an external observer; or structures convey new properties to the systems itself

• new behaviours ‘emerge’ from the system

• coordination and emergence may arise from response to environment → adaptation

• when adaptation occurs across generations at a population level we say that the system evolved

• now, at new scale, the system can be identified as a novel unit

• this becomes the building block → new cycle at a new scale

Aim

A framework in which concepts like

• complexity, • emergence,

• self-organisation, • adaptation and evolution

can be:

• described,• distinguished

• defined consistently

Framework

We chose an information-theoretic framework:

1. we borrow from work pioneered by the Santa Fe Institute

2. it provides a well developed theory;

3. definitions can be formulated mathematically;

4. some computational tools are readily available.

Complexity

Predictive Information = diversity (signal) - non-assortativeness (diversity between past and future);

Excess Entropy = Richness of structure;

Statistical Complexity = minimum Memory for optimal predictions;

Complexity

Predictive Information =Excess Entropy ≤ Statistical Complexity

Predictive information = amount of structure ≤ memory for optimal predictions

The memory needed for optimal prediction cannot be lower than the structure contained in the data, i.e., the mutual information between the past and future.

Self-Organisation

The more a system organises →the more behaviours it can display →the more effort is needed to describe its dynamics.

Organisation= increase in complexity:

Self-Organisation

Self = spontaneous

the amount of information flowing from the outside is strictly less than the change in statistical complexity

IOutside < C(t+∆t)- C(t)

Increases in organisation > Information received.

Or

Energystimulus < Energyresponse

Emergence

There are classes of phenomena, which when observed at different levels, display behaviours which appear fundamentally different.

What level should we choose for our analysis?

The level at which it is easier or more efficient to construct a workable model.

Efficiency of Prediction = Excess Entropy / Statistical Complexity(Shalizi)

= How much can be predicted / How difficult it is to predict

Complexity lStatistica

Entropy Excess

Complexity lStatistica

Entropy Excess

Complexity lStatistica

Entropy Excess

Complexity lStatistica

Entropy Excess

Complexity lStatistica

Entropy Excess

Complexity lStatistica

Entropy Excess

Complexity lStatistica

Entropy Excess

Complexity lStatistica

Entropy Excess

Complexity lStatistica

Entropy Excess

Adaptation and Evolution

Adaptation = increase in the mutual information between the system and the environment.

“Evolution increases the amount of information a population harbors about its niche" (Adami)

I (Environment, Population) = Entropy (Population) – Entropy (Population | Environment) =

entropy in the absence of selection (Max Population Entropy) - diversity tolerated by selection in the given environment =

how much data can be stored in the population - how much data irrelevant to environment is stored

Concept Plain English Information Theory

Complexity The amount of information needed to describe a process / system / object.

PI= diversity - non-assortativeness;

PI = E ≤ C

Self-Organisation Spontaneous increase in complexity InfOutside < C(t+∆t)- C(t)

Emergence Presence of behaviours which appear fundamentally different when observed at different levels.

Efficiency of Prediction

e=E/C

Adaptation /

Evolution

Increase in the mutual information between the system and its environment.

I (Pop, Env) = H (Pop) – H (Pop |

Env)

a Complex System “story”• many, but not too many, components interact in a non trivial fashion

• the system is open (receives energy/information/matter from the environment)

• interactions → symmetry breaking → coordinated behaviour arises

• no central director/template → the system ‘self-organises’

• coordination as patterns detectable by an external observer; or structures convey new properties to the systems itself

• new behaviours ‘emerge’ from the system

• coordination and emergence may arise from response to environment → adaptation

• when adaptation occurs across generations at a population level we say that the system evolved

• now, at new scale, the system can be identified as a novel unit

• this becomes the building block → new cycle at a new scale