Mandelbrot’s Eyes and 1/f Noise

Nick Watkins [MPI PKS, Dresden]

NickWatkins@mykolab.com

• Given as a seminar

at Open University

on 1st December 2014.

• Minor errors

corrected.

Outline

Ionosphere

Magnetosphere

• How Thomson and Tait distinguished between formula and fact, and howat least for complexity science, this remains a relevant issue

• How facts continually led Mandelbrot to new formulas: the history of his journeyfrom heavy tails to long range dependence to multifractals

• but also how he warned us against putting too much weight on one property, self-similarity when we should be using our eyes as well.

• This has led me to some much less well known work of his in the mid 60s, andto wondering about why he neglected it … as well as some new work with collaborators that I will (briefly) advertise.

• And has also stimulated some digging on the broader topic of image versus words in science.

I will mainly be surveying ideas today, the detail is here:

[1] Watkins, “Bunched black (and grouped grey) swans”, Frontiers review article, GRL, 2013, http://onlinelibrary.wiley.com/doi/10.1002/grl.50103/full[2] Franzke et al, Phil. Trans. A, 2012, doi:10.1098/rsta.2011.0349[3] Graves et al, “A brief history of long memory”, submitted American Statistican, 2014 arXiv:1406.6018v1 [stat.OT][4] Graves et al, “Efficient Bayesian inference for long memory processes”, submitted CSDA, 2014, arXiv:1403.2940v1 [stat.ME]

and is being developed in:

[5] Franzke et al, submitted Scientific Reports.[6] Watkins and Franzke, in prep.

“THE FORMULA & NOT THE FACT”

Thomson (Kelvin) and Tait

Ionosphere

Magnetosphere

“I am myself a good example of the want of such a book as we

contemplate, having got all my information bit by bit from

scattered sources, which often contained more error than truth.

The next generation will thank us.”-Tait to Thomson, 1861.

Thompson and Tait

Ionosphere

Magnetosphere

We believe that the mathematical reader will

especially profit by a perusal of the large type

portion of this volume; as he will thus be

forced to think out for himself what he has

been too often accustomed to reach by a

mere mechanical application of analysis.

Nothing can be more fatal to progress than a

too confident reliance on mathematical

symbols; for the student is only too apt to

take the easier course, and consider the

formula and not the fact as the physical

reality. Thomson &Tait, 1890 edition, page

viii.

Not an obsolete point …

Ionosphere

Magnetosphere

“… for the student is only too apt to take the easier course, and consider the

formula and not the fact as the physical reality. “

I accept this is a19th Century view, and 20th century physics has illuminated this

question from both sides. We will hear from Dirac later but QM might be an

example where perhaps the formula is the fact ?

For me the “fact vs formula” question really became important as a result of the

paradigm of self-organised criticality. I and some others in space plasma physics

were attracted to SOC in mid 1990s, and worked both on forward problem of

what sorts of plasma physics would map on to SOC, and also inverse problem of

unambiguous identification of SOC in data.

WILD AND SLOW FLUCTUATIONS-MANDELBROT’S JOURNEY

Wild and slow ionospheric fluctuations

Ionosphere

Magnetosphere

Ground-based Magnetometerssense ionosphericcurrents

0.5 1 1.5 2 2.5 3 3.5 4 4.5 5

x 105

0

500

1000

1500

2000

2500

3000

1978 AE data: threshold percentile=99

Time, t [minutes]

AE

[nano T

esla

]

AE data

threshold

AE power spectrum

Tsurutani et al, GRL, 1991

Benoit B. Mandelbrot

Ionosphere

Magnetosphere

• Born Warsaw, November 20th, 1924

• Graduated Ecole Polytechnique 1947

• Masters in Aeronautics Caltech 1949 (turbulence and Kolmogorov)

• PhD University of Paris 1952 (heavy tails and Zipf)

• CNRS 1949-58: IAS Princeton, Geneva, Lille, …

• IBM 1958-87, then Yale (gained tenure aged 75 …)

• Died, 14th October 2010• ”When I listen to the list of my previous jobs, I wonder if I

exist. The intersection of such sets is surely empty”-BBM quoted in Gleick’s “Chaos”

“It's very strange that in high school I never knew, I never felt that I

had this very particular gift, but in that year in that special cramming

school it became more and more pronounced, and in fact in many

ways saved me. In the fourth week again I understood nothing, but

after five or six weeks of this game it became established that I

could spontaneously just listen to the problem and do one

geometric solution, then a second and a third. Whilst the professor

was checking whether they were the same, I would provide other

problems having the same structure. It went on. I didn't learn much

algebra. I just learned how better to think in pictures because I

knew how to do it. I would see them in my mind's eye, intersecting,

moving around, or not intersecting, having this and that property,

and could describe what I saw in my eye. Having described it, I

could write two or three lines of algebra, which is much easier if you

know the results than if you don't”

---Mandelbrot, interviewed in 1998, www.webofstories.com

Mandelbrot’s eyes

Fact: Wild Fluctuations

Ionosphere

Magnetosphere

1963

[S&P 500] Mantegna &

Stanley, Nature, 1996

Formula: Heavy tails

Light tailed exampleGaussian

Heavy tailed example, alpha stable distribution which has a power law tail.

Note that the distribution flattens out at small x, & can have a finite mean for some alpha values.

Variance is however infinite

1(1 )( ) ~ p xx

2 2~ exp(( ) / )2xp x

Pdf p(x).

x

(1 )) ~ , 1p( / x x H

Fact: Hurst’s growth of range

Ionosphere

Magnetosphere600 700 800 900 1000 1100 1200 13009

10

11

12

13

14

15Annual minimum level of Nile: 622-1284

Annual m

inim

um

:

Time in years

“I heard about the … Nile … in '64, ... the variance doesn't draw like time span as you take bigger and bigger integration intervals; it goes like time to a certain power different from one. … Hurst …was getting results that were incomprehensible”. – BBM, op. cit

Nile minima, 622-1284Hurst, Nature, 1957

1/f spectra

Ionosphere

Magnetosphere

“This was very much noticed and the literature grew about it …it was viewed as a major puzzle,this thing which didn't work out”BBM initially thought he couldexplain Hurst’s observations witha heavy tailed model like his financial one. But when saw data realised it wasn’t heavy tailed in amplitude ! Instead abstracted out property of self similarity, but in spectral rather than amplitude domain --- i.e. a model with a heavy tailed power spectrum of 1/f shape, even down to lowestfrequency !

SpectralDensity S(f).

Frequency f.

S(f) ~ f

BBM: Joseph Effect

long range serial dependence in time

... there came seven years of great plenty throughout the land of Egypt. And there shall arise after them seven years of famine ... Genesis: 41, 29-30.

BBM: Joseph Effect

long range serial dependence in time

... there came seven years of great plenty throughout the land of Egypt. And there shall arise after them seven years of famine ... Genesis: 41, 29-30.

BBM, van Ness & Wallis 1965-68

Ionosphere

Magnetosphere

• Proposed use of fractional Brownian motion, a nonstationary self similar model which generalises the familiar Wiener process and has a spectral index between -1 and -3.

• … and its increment process, fractional Gaussian noise, which is stationary

1 12 2

2 2, ( ) ( ) ( )~ ) (H

H H

RX t t s s dL s

Memory kernel: Joseph effect

Gaussian jump

BBM, van Ness & Wallis 1965-68

Ionosphere

Magnetosphere

• Unlike the stable amplitude distribution we just saw, the power spectra of fBm and fGn are singular at zero frequency.

• In fBm case this is a symptom of its nonstationarity, but fGn shows this singularity even in a stationary model.

• One diagnostic for long range dependence has since thus been a singular S(f) (or an ACF whose summed lags blow up) --- but interpretation asLRD then presupposes stationarity.

• Modelled the Joseph effect (LRD in a stationary model), Mandelbrot’s favourite explanation for the Hurst effect (empirical growth of rescaled range).

• Long range kernel and lowfrequency blowup means that meaning of long range dependence has always been controversial

Ionosphere

Magnetosphere

When it became quite clear that to analyse something like the River Nile over a thousand years, or some other data over many thousands of years, or IBM stock over thirty-five years - which still represents a very large number of thousands of data, if infinite dependence is necessary it does not mean that IBM's details of ten years ago influence IBM today, because there's no mechanism within IBM for this dependence. However, IBM is not alone. The River Nile is not alone. They're just one-dimensional corners of immensely big systems.

The behaviour of IBM stock ten years ago does not influence its stock today through IBM, but IBM the enormous corporation has changed the environment very strongly. The way its price varied, went up or went up and fluctuated, had discontinuities, had effects upon all kinds of other quantities, and they in turn affect us. And so my argument has always been that each of these causal chains is totally incomprehensible in detail, probably exponentially decaying. There are so many of them that a very strong dependence may be perfectly compatible. Now I would like to mention that this is precisely the reason why infinite dependence exists, for example, in physics. In a magnet- because two parts far away have very minor dependence along any path of actual dependence. There are so many different paths that they all combine to create a global structure. In other words, there is no global structure in one dimension, but there's one in two and three dimensions etc. for magnets -the basis of Onsager's work and the whole theory. And in economics there is nothing comparable to these calculations, but the intuition of what they represent is the same – BBM, op cit

“1/F … IS SEVERELY UNDERDETERMINED”

Ionosphere

Magnetosphere

• So is that it ? No. Late in his life, Mandelbrot was

keen to emphasise that the formula wasn’t the fact,

and the property of self-similarity seen in his most famous models wasn’t the whole story. “Reducing the notion of “1/f noise" to self-affinity ... shows it to be very severely underspecified”-BBM in Selecta volume 1998.

Why was he saying this ? Because his eyes told him to: “Like the ear, the eye is very sensitive to features that the spectrum does not reflect. Seen side by side, different 1/f noises, Gaussian, dustborne and multifractal, obviously

differ from one another”- Selecta, op cit.

Common to above effects is self similarity.

Dilating power law tail of stable amplitude pdf by scale factor leaves it a power law.

Similarly dilating a power law ACF or PSD leaves it power law.

However a single self-similarity exponent may not be enough and by early 1970s Mandelbrot was thinking about multifractals i.e. a spectrum of exponents prompted by turbulence. In 1990s applied to finance.

BBM & Wallis 1969:

Ionosphere

MagnetosphereMandelbrot’s first attempt to unify the long range memory kernel of fractional Brownian motion with heavy tailed amplitudefluctuations - he called it a fractional hyperbolic model

Antecedent of today’s linear fractional stable noise, but it didn’t satisfy him.

Multifractals and volatility clustering

Correlations between the absolute value of the time series- or here, in AE, its first differences.

0 0.5 1 1.5 2 2.5 3 3.5 4

x 104

-600

-400

-200

0

200

400

600

incre

ments

, r

First differences of AE index January-June 1979

-100 -80 -60 -40 -20 0 20 40 60 80 100-0.1

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

lag

acf

AE data: acf of returns

-100 -80 -60 -40 -20 0 20 40 60 80 100-0.1

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

lag

acf

AE data: acf of squared returns

BBM’s other 1/f model:

Ionosphere

Magnetosphere

• Very different from the stationary fGn-in an important sense, nonstationary. • model intermittency ``off" periods of no activity interrupted by jumps to a negative or positive ``on" active state. Used long tailed distributions of the waiting timesbetween transitions between on and off states. • Key finding was that traditional measures such as the PSD and ACF would return a ``1/f'' spectrum for this model, but that this empirical spectrum could be decomposed into a stationary part, and one depending on the time series' length.

• Rather than representing a true singularity in power at the lowest frequencies, in this model he described the infrared catastrophe in the power spectral density as a ``mirage" resulting from the fact that the moments of the model varied in time in a step-like fashion: In other words, the existence of [1/f] noises challenges the mathematician to reinterpret spectral measurements otherwise than in “Wiener-Khinchin" terms.... operations meant to measure the Wiener-Khinchin spectrum may unvoluntarily [sic] measure something else, to be referred to as the ”conditional spectrum" of a ”conditionally covariance stationary" random function. – Mandelbrot 1967

Distinct from fBm and fGN:

Ionosphere

Magnetosphere

Prepared in the same period as Mandelbrot and Wallis on FGM and FGN, which it cites as ``to be published". In it a contrast is clearly drawn between Mandelbrot's conditionally stationary, non-Gaussian 1/f model and his stationary, Gaussian (FGN) model:

“Section VI [... of this paper... ] showed that some [1/f] noises have a very erraticsampling behavior. Some other [1/f] noises are Gaussian and, therefore, perfectly

``well-behaved;“ an example is provided by the ``fractional white noise" [i.e. FGN] which is the formal derivative of the process of Mandelbrot and Van Ness [i.e. FBM]”– Mandelbrot 1967

The ugly duckling ?

Ionosphere

Magnetosphere

Received far less contemporary attention than did his papers on heavy tails in finance inthe early 1960s or the series with Van Ness and Wallis in 1968-69 on stationary fractionalGaussian models for LRD, gaining only about 20 citations in its first 20 years.

It was for example apparently unknown to his critic Vit Klemes [1974], and seems to be still relatively little known, being for example not cited by Beran [2013], and while listed in the citations of Beran [1994] I can find no mention in the text.

Although he revisited the paper with a new commentary in the volume of his Selecta that deals with multifractals and 1/f' noise, Mandelbrot himself neglected to mention it

explicitly in his popular and historical accounts of the genesis ofLRD such as Mandelbrot [2008], and clearly saw it as a different strand of his work to that on fractional Brownian motion collected in another Selecta volume Mandelbrot [2002].

Why ? Because it wasn’t as popular as fBm/fGn ? Or because it wasn’t as beautiful ?

Should we pay more attention to it ? As the AFRP, or weak ergodicty breaking, we do !

SO HOW DO WE PROCEED ?

Formula: Bayesian inference. In his PhD Tim Graves developed method: tested on α-stable ARFIMA(0,d,0) where heavy tails &

LRD co-exist

Graves, Gramacy, Franzke & Watkins, submitted CSDA 2014; and in prep.

1.5 0.15 d

Fact: Hurst effect from state changes

Ionosphere

Magnetosphere

Franzke et al, submitted, Sci. Rep., 2014

THE MIND’S EYE AND SCIENCE

Importance can be seen from simple thought experiment,

and the fact we don’t know the answer to it ...

If we could not only see and hear my talk, but also see the

pictures drawn (or not) in individual audience

member’s minds by it---how much variety would we see ?

Many steps made over millions of years to allow human

beings to be convinced that we are communicating about

same thing. Language itself, cave painting [c.f. Herzog’s film

The Cave of Forgotten Dreams], maths, blackboards,

visualisation ….

Cognitive diversity:

Tensions :

Solar wind

Magnetosphere

Ionosphere

I am fascinated by the the tension (metaphorical, and by Mandelbrot’s

own account sometimes literal !) between the very visual thinking of

BBM and the non-visual, formal proofs of e.g. the Bourbaki, of which

his uncle, Szolem Mandelbrojt was a founder member ...

“Should I go to the École Normale or to the École Polytechnique? And a

kind of family council got together to, in a certain sense, really fight for

my soul … one of the reasons why my life has been so complicated, with

lurches, if you will, towards very pure mathematics at one time, then

lurches away from pure mathematics to very practical things, was very

much this fight between these two men when I was approximately

twenty.” BBM, on Web of Stories

And by various reports about mathematical and scientific thought in the

19th and 20th centuries

“It's very strange that in high school I never knew, I never felt that I

had this very particular gift, but in that year in that special cramming

school it became more and more pronounced, and in fact in many

ways saved me. In the fourth week again I understood nothing, but

after five or six weeks of this game it became established that I

could spontaneously just listen to the problem and do one

geometric solution, then a second and a third. Whilst the professor

was checking whether they were the same, I would provide other

problems having the same structure. It went on. I didn't learn much

algebra. I just learned how better to think in pictures because I

knew how to do it. I would see them in my mind's eye,

intersecting, moving around, or not intersecting, having this

and that property, and could describe what I saw in my eye.

Having described it, I could write two or three lines of algebra,

which is much easier if you know the results than if you don't”

---Mandelbrot, at www.webofstories.com

Mandelbrot

Dirac

• “Her fundamental laws do not govern the world as it appears in our mental picture in any very direct way, but instead they control a substratum of which we cannot form a mental picture without introducing irrelevancies."

--- Preface to The Principles of Quantum Mechanics [1930]

Euclid

Solar wind

Magnetosphere

Ionosphere

“In Euclid, you find some drawings but it is knownthat most of them were added after Euclid, in

later editions. Most of the drawings in the original are abstract drawings. You make some reasoning about some proportions and you draw some segments, but they are not intended to be geometrical segments, just representations ofsome abstract notions.”

The Continuing Silence of Bourbaki—An Interview with Pierre Cartier [Bourbaki1955-83], June 18, 1997

by Marjorie Senechal in The Mathematical Intelligencer, № 1 (1998) · pp.22–28

Lagrange & Russell

Solar wind

Magnetosphere

Ionosphere

Also Lagrange proudly stated, in his textbookon mechanics, "You will not find any drawing

in my book!“

The analytical spirit was part of the French traditionand part of the German tradition. And I suppose itwas also due to the influence of people like Russell,

who claimed that they could prove everything formally—that so-called geometrical intuitionwas not reliable in proof.

Senechal , op cit

Bourbaki

Solar wind

Magnetosphere

Ionosphere

“Again Bourbaki's abstractions and disdain forvisualization were part of a global fashion,

as illustrated by the abstract tendencies in the music and the paintings of that period.“

Senechal op cit in The Mathematical Intelligencer, № 1 (1998) · pp.22–28

http://www.ega-math.narod.ru/Bbaki/Cartier.htm

19th Century Scientists

Solar wind

Magnetosphere

Ionosphere

“The earliest results of my inquiry amazed me. I had begun by questioning friends in the scientific world, as they were the most likely class of men to give accurate answers concerning this faculty of visualizing, to which novelists and poets continually allude, which has left an abiding mark on the vocabularies of every language, and which supplies the material out of which dreams and the well-known hallucinations of sick people are built.”

“To my astonishment, I found that, the great majority of the men of science to whom I first applied protested that mental imagery was unknown to them, and they looked on me as fanciful and fantastic in supposing that the words 'mental imagery' really expressed what I believed everybody supposed them to mean. They had no more notion of its true nature than a color-blind man, ... has of the nature of color.” - Francis Galton quoted in James, op cit

No mind’s eye ?

Solar wind

Magnetosphere

Ionosphere

“They [...] naturally enough supposed that those who affirmed they

possessed [visual images] were romancing. To illustrate their mental attitude it will be sufficient to quote a few lines from the letter of one of my correspondents, who writes:

"These questions presuppose assent to some sort of a proposition regarding the "mind's eye," and the "images" which it sees. . . . This points to some initial fallacy. . . . It is only by a figure of speech that I can describe my recollection of a scene as a "mental image" which I can "see" with my "mind's eye. " . . . I do not see it . . . any more than a man sees the thousand lines of Sophocles which under due pressure he is ready to repeat. The memory possesses it,' etc”.”

James, The Principles of Psychology, Volume 2.

How can we (will we) make further leaps towards this collective

communication while discovering and appreciating why (is it

evolutionary advantage ?) that we don’t all see the same thing ?

Cognitive diversity:

CONCLUSIONS

Recap

Ionosphere

Magnetosphere

• How Thomson and Tait distinguished between formula and fact, and howat least for complexity science, this remains a relevant issue

• How facts continually led Mandelbrot to new formulas: the history of his journeyfrom heavy tails to long range dependence to multifractals

• but also how he warned us against putting too much weight on one property, self-similarity when we should be using our eyes as well.

• This has led me to some much less well known work of his in the mid 60s, andto wondering about why he neglected it … as well as some new work with collaborators that I (briefly) advertised.

• And has also stimulated some digging on the broader topic of image versus words in science.

I was mainly surveying ideas today, the detail is here:

[1] Watkins, “Bunched black (and grouped grey) swans”, Frontiers review article, GRL, 2013, http://onlinelibrary.wiley.com/doi/10.1002/grl.50103/full[2] Franzke et al, Phil. Trans. A, 2012, doi:10.1098/rsta.2011.0349[3] Graves et al, “A brief history of long memory”, submitted American Statistican, 2014 arXiv:1406.6018v1 [stat.OT][4] Graves et al, “Efficient Bayesian inference for long memory processes”, submitted CSDA, 2014, arXiv:1403.2940v1 [stat.ME]

and is being developed in:

[5] Franzke et al, submitted Scientific Reports.[6] Watkins and Franzke, in prep.