Mandelbrot’s Other Model of 1/f Noise: Can History Help Us With

Today’s Topics Like Weak ErgodicityBreaking ?

Nick Watkins [CATS, LSE]

NickWatkins@mykolab.com

• Given as an invited talk at the 4th Workshop on Anomalous Diffusion, Hugo Steinhaus Centre, Wroclaw, Poland on 6th December 2014.

Thanks

• Alex, Marcin and all at Wroclaw for inviting me

• Holger Kantz for hosting me in Dresden since September 2013

• Co-authors Tim Graves, Christian Franzke, Bobby Gramacy

• Discussions with all of the above and Eli Barkai, Igor Sokolov, Rainer Klages and AlekseiChechkin among others

Outline

Ionosphere

Magnetosphere

• How Thomson and Tait differentiated formula from fact, and why I think this remains a relevant topic and a stimulating distinction.

• How facts led Mandelbrot to formulae: brief history of his relatively well known journey from heavy tails to long range dependence [Graves et al, arXiv, 2014]

• His warnings in his Selecta [Volume N, 1999] against putting too much weight on one property, self-affinity

• His advocacy of instead using ones eyes to distinguish between models, including hisfractional hyperbolic [1969] & multifractal [1972] examples ….

• … and his much less well known work [IEEE Boulder Conference 1965, IEEE TransInformation Theory 1967] (collected and reviewed in Selecta N) on renewal processes with heavy tailed waiting times, which discussed ergodicity breaking, time dependent power spectra, and the meaning of 1/f in such processes. Work whose time has come ?

Thomson (Kelvin) and Tait

Ionosphere

Magnetosphere

“Nothing can be more fatal to progress than a too confident

reliance on mathematical symbols; for the student is only too apt

to … consider the formula and not the fact as the physical

reality”. Thomson & Tait, The Treatise on Natural

Philosophy, page viii. of 1890 edition.

An out of date view?

Ionosphere

Magnetosphere

“… and consider the formula and not the fact as the physical reality.”

I accept this is a 19th Century viewpoint, and 20th century physics has

offered arguments for and against it. Is Quantum Mechanics best

example where formula is the fact ?

For me it became important as result of self-organised criticality. I

worked on SOC in mid 1990s, both on forward problem of what sorts of

plasma physics could map on to SOC, and also inverse problem of

unambiguous identification of SOC in data.

Wild and slow ionospheric fluctuations

Ionosphere

Magnetosphere

Ground-based Magnetometerssense ionosphericcurrents

0.5 1 1.5 2 2.5 3 3.5 4 4.5 5

x 105

0

500

1000

1500

2000

2500

3000

1978 AE data: threshold percentile=99

Time, t [minutes]

AE

[nano T

esla

]

AE data

threshold

AE power spectrum

Tsurutani et al, GRL, 1991

Fact: Wild Fluctuations

Ionosphere

Magnetosphere

J. Business,

1963

[S&P 500] Mantegna &

Stanley, Nature, 1996

Formula: Heavy tails

Light tailed exampleGaussian

Heavy tailed example, alpha stable distribution which has a power law tail.

1(1 )( ) ~ p xx

2 2~ exp(( ) / )2xp x

Pdf p(x).

x

(1 )) ~ , 1p( / x x H

Another fact: Hurst’s growth of range

Ionosphere

Magnetosphere600 700 800 900 1000 1100 1200 13009

10

11

12

13

14

15Annual minimum level of Nile: 622-1284

Annual m

inim

um

:

Time in years

“I heard about the … Nile … in '64, ... the variance doesn't draw like time span as you take bigger and bigger integration intervals; it goes like time to a certain power different from one. … Hurst …was getting results that were incomprehensible”. – Mandelbrot, 1998

Nile minima, 622-1284Hurst, Nature, 1957

But what’s the formula ?

Ionosphere

Magnetosphere

“This was very much noticed and the literature grew about it …it was viewed as a major puzzle,this thing which didn't work out”-Mandelbrot, 1998.He initially thought (Selecta) he couldexplain Hurst’s observations witha heavy tailed model like his financial one. But when saw data realised it wasn’t heavy tailed in amplitude ! Instead abstracted out property of self similarity, but in spectral rather than amplitude domain --- i.e. a model with a heavy tailed power spectrum of 1/f shape, even down to lowest frequency.

SpectralDensity S(f).

Frequency f.

S(f) ~ f

Formula: fBm, 1965-68

Ionosphere

Magnetosphere

1 12 2

2 2, ( ) ( ) ( )~ ) (H

H H

RX t t s s dL s

Memory kernel Gaussian

Kolmogorov’sWiener Spiral

LRD and its controversies

Ionosphere

Magnetosphere

• Mandelbrot [1965]; M and Van Ness[1968] proposed use of fractional Brownian motion. Non stationary self similar model which generalises Wiener process, has spectral index between -1 and -3.

• … and its derivative, fractional Gaussian noise, which is stationary, and long range dependent.

• Unlike the stable amplitude distribution we just saw, the power spectra of fBm and fGn are singular at zero frequency. In Bm (and fBm)this is a result of its nonstationarity … [Selecta, N2, 1999]

LRD and its controversies

Ionosphere

Magnetosphere

• But fGn shows singularity in a stationary model.

• Most diagnostics for long range dependence have thus been based on singular S(f) (or an ACF whose summed lags blow up) --- but their interpretation as LRD presupposes stationarity.

• Mandelbrot’s “Joseph effect” (LRD in a stationary model), seems tohave been his favourite explanation for the Hurst effect (empirical growth of rescaled range).

• But long range kernel and low frequency blowup means that physical meaning of long range dependence has always been controversial

Compatible with the emerging fractional Langevinpicture of fBm e.g. Metzler et al, PCCP, 2014; Taloniet al, 2010; Kupferman, 2004; Lutz, 2001 and others.

Ionosphere

Magnetosphere

…, if infinite dependence is necessary it does not mean that IBM's details of ten years ago influence IBM today, because there's no mechanism within IBM for this dependence. However, IBM is not alone. The River Nile is [not] alone. They're just one-dimensional corners of immensely big systems. The behaviour of IBM stock ten years ago does not influence its stock today through IBM, but IBM the enormous corporation has changed the environment very strongly. The way its price varied, went up or went up and fluctuated, had discontinuities, had effects upon all kinds of other quantities, and they in turn affect us. –Mandelbrot, interviewed in 1998 for Web of Stories

So what did Mandelbrot think LRD is ?

Many faces of “1/f”

Ionosphere

Magnetosphere

• Late in his life though, Mandelbrot emphasised that the formula wasn’t the fact, and the property of self-similarity seen in his most famous models wasn’t the whole story. “Reducing the notion of “1/f noise" to self-affinity ... shows it to be very severely underspecified”- Selecta volume N, 1999.

• Why was he saying this ? Because his eyes told him to: “Like the ear, the eye is very sensitive to features that the spectrum does not reflect. Seen side by side, different 1/f noises, Gaussian, dustborne and multifractal, obviously differ from one another”- Selecta, op cit.

So what were the other models ?

• Additive models extending fGn like fractional hyperbolic model of Mandelbrot & Wallis [1969].

• Multiplicative, multifractal models exhibiting volatility bunching as well as 1/f spectra and fat tails-1972 (turbulence), 1990s (finance).

• And the class he referred to as “dustborne”: the least known of his papers, from 1965-67, though closely related to the Alternating Fractional Renewal Process, the CTRW and modern work on weak ergodicity breaking.

Ionosphere

MagnetosphereMandelbrot & Wallis [1969] first attempt to unify long range memory kernel of fGn with heavy tailed amplitude fluctuations - called it “fractional hyperbolic” model because of its power law tails.

Anticipated the versatile linear fractional stable noises, but it didn’t satisfy him completely for problems he was looking at.

Additive fractional stable class

Multiplicative multifractal cascades

Many systems have aggregation, but not by an additive route. Classic example is turbulence.

One indicator: lognormal or stretched exponential pdf …

Selecta Volume N1999

Multifractals and volatility clustering

another: correlations between the absolute value of the time series- or here, in ionospheric data, the first differences.

0 0.5 1 1.5 2 2.5 3 3.5 4

x 104

-600

-400

-200

0

200

400

600

incre

ments

, r

First differences of AE index January-June 1979

-100 -80 -60 -40 -20 0 20 40 60 80 100-0.1

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

lag

acf

AE data: acf of returns

-100 -80 -60 -40 -20 0 20 40 60 80 100-0.1

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

lag

acf

AE data: acf of squared returns

BBM’s other 1/f models:

Ionosphere

Magnetosphere

• 1965 IEEE Boulder conference paper [N8 in Selecta 1999] and 1967 journal paper [N9].

• Not same as stationary fGn-he called them conditionally stationary.

• Abrupt state changes• Fat distributions of waiting

times including E[t] = ∞ case.

The conditional spectrum:

Magnetosphere

• First key finding was that Wiener-Khinchine inspired measures like PSD would return a ``1/f'' spectrum for such models, but that the power spectrum S(f) could be factored into two parts, one of which depending on the time series' length.

Reviewed in N2, Selecta, 1999

The infrared catastrophe as mirage:

Ionosphere

Magnetosphere

• Rather than representing a true singularity in power at the lowest frequencies, in this model he described the infrared catastrophe in the power spectral density as a ``mirage“:

Reviewed in N2, Selecta, 1999

Ionosphere

Magnetosphere

(Fifth Berkeley Symposium onProbability, 1965)

Non-ergodicity:

Ionosphere

Magnetosphere

• M 1967 illustrated this first in case of single jump, infinite interval

Non-ergodicity:

Ionosphere

Magnetosphere

• Mandelbrot 1967 then discussed case of many state changes with power law waiting times

Distinct from fBm and fGN:

Ionosphere

Magnetosphere

Mandelbrot 1967 was prepared in the same period as Mandelbrot and van Ness on fBm and fGn, which it cites as ``to be published". In it contrast is clearly drawn between the conditionally stationary, non-Gaussian renewal process as a 1/f model and his stationary, Gaussian (fGn) model:

CONCLUSIONS

A neglected paper … ?

Ionosphere

Magnetosphere

Mandelbrot 1967 received far less attention than either papers on heavy tails in finance in early 1960s or the series with Van Ness and Wallis in 1968-69 on stationary fractional Gaussian models for LRD. Only about 20 citations in its first 20 years !

Was apparently unknown to Vit Klemes [Water Resources Research, 1974], who essentially reinvented it to criticise fBm. Still seems a relatively little known paper. Not cited by Beran et al [2013], and while listed in the citations of Beran [1994] I can’t find in the text. Some exceptions, e.g. Grigolini et al, Physica A, 2009.

Although he revisited the paper with new commentary in Selecta Volume N [1999] dealing with multifractals and 1/f noise, Mandelbrot neglected to mention it explicitly in his popular and historical accounts of genesis of LRD such as Mandelbrot and Hudson [2008].

Why ? Because it wasn’t as popular as fBm/fGn ? Or because it wasn’t as “beautiful” ?

… Whose time has come ?

Ionosphere

Magnetosphere

• Should we pay more attention to this class of models ? As the AFRP [Lowen and Teich’s book], and as models for weak ergodicity breaking [c.f. Niemann et al, 2013], we already are ! Also links to the CTRW.

• Particular value of looking back nearly 50 years to how Mandelbrot saw these models is to see how they fit into “the panorama of grid-bound self-affine variability” as he later put it [Selecta, 1999, N1].

Helps link maths and physics, the formula & the fact.And inform future work.

Fact: Hurst effect from state changes

Ionosphere

Magnetosphere

Franzke et al, submitted, Sci. Rep., 2014

I have mainly been surveying ideas today, the detail is here:

[1] Watkins, “Bunched black (and grouped grey) swans”, Frontiers review article, GRL, 2013, http://onlinelibrary.wiley.com/doi/10.1002/grl.50103/full[2] Franzke et al, Phil. Trans. A, 2012, doi:10.1098/rsta.2011.0349[3] Graves et al, “A brief history of long memory”, submitted American Statistician, 2014 arXiv:1406.6018v1 [stat.OT][4] Graves et al, “Efficient Bayesian inference for long memory processes”, submitted CSDA, 2014, arXiv:1403.2940v1 [stat.ME]

and is being developed in:

[5] Franzke et al, submitted Scientific Reports.[6] Watkins and Franzke, in prep.

SPARES

Formula: Bayesian inference. In his PhD Tim Graves developed method: tested on α-stable ARFIMA(0,d,0) where heavy tails &

LRD co-exist

Graves, Gramacy, Franzke & Watkins, submitted CSDA 2014; and in prep.

1.5 0.15 d

Ionosphere

Magnetosphere

And so my argument has always [sic] been that each of these causal chains is totally incomprehensible in detail, probably exponentially decaying. There are so many of them that a very strong dependence may be perfectly compatible. Now I would like to mention that this is precisely the reason why infinite dependence exists, for example, in physics. In a magnet- because two parts far away have very minor dependence along any path of actual dependence. There are so many different paths that they all combine to create a global structure. In other words, there is no global structure in one dimension, but there's one in two and three dimensions etc. for magnets -the basis of Onsager's work and the whole theory. And in economics there is nothing comparable to these calculations, but the intuition of what they represent is the same – BBM, op cit

A critical phenomenon ?